Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and design appropriate solutions.
•
Work closely with IT and infrastructure teams to deploy and maintain data storage and processing systems, including databases, data warehouses, and cloud infrastructure.
•
Design, build, and maintain robust, scalable data pipelines to ingest, process, and transform data from various sources, including internal databases and external APIs.
•
Optimize data pipelines for performance, scalability, and reliability, ensuring timely and accurate delivery of data.
•
Implement data validation and quality checks to identify and rectify any issues in data pipelines, ensuring data integrity.
•
Explore and evaluate new technologies, tools, and frameworks to improve data engineering processes and capabilities.
•
Perform data modeling and schema design to support analytical and reporting requirements.
•
Design and build data marts and business layers to facilitate efficient data access and analysis.
Requirements
•
Bachelor’s degree in Computer Science, Information Systems, or other related technical field or equivalent work.
•
Proven experience as a Data engineer or similar role, with at least 4 years of experience.
•
Advanced SQL knowledge/experience - complex queries and query optimization.
•
Proficiency in programming languages such as Python.
•
Experienced in the development of data warehouses and proficient in data modeling.
•
Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform.
•
Experience in building data streams using technologies such as Kafka, Flink, or Spark Streaming is considered a plus.
•
Strong problem-solving skills and attention to detail.
•
The ability and willingness to learn new technologies on the job.
•
Self driven, highly motivated, fast learner, ambitious and creative.