Job Purpose
As a Data Engineer your primary role is to support and enhance our current ETL processes ensuring the efficient ingestion cleansing and transformation of data from various sources. You will also play a key role in the development and maintenance of our data lake and data pipelines ensuring that our organization can access and analyze data effectively. Additionally your responsibility includes meticulous documentation of data processes to ensure transparency and compliance with best practices.
Key Accountabilities:
Data Ingestion & Integration
- Integrate data from multiple sources including databases APIs and external data providers.
- Design build and maintain robust data pipelines to ingest process and transform data from various sources into usable formats.
- ETL (Extract Transform Load): Create ETL processes to clean transform and load data into our data warehouse.
- Implement data quality checks and validation processes to ensure data accuracy and consistency.
Data Warehousing
- Develop and manage data warehousing solutions to store and organize large datasets efficiently.
- Design and implement data models that facilitate efficient querying and reporting.
- Monitor and optimize data pipelines and databases for performance and scalability.
Documentation
- Maintain thorough documentation of data processes schemas and workflows
Collaboration
- Collaborate with data scientists analysts and other stakeholders to understand data requirements and provide data solutions to meet their needs.
Requirements
- Degree in Engineering or Information Technology.
- Proven experience (minimum 3 years) in data engineering
- Proficiency in SQL or Python
- Proficiency in QlikSense or Power BI.