Main Roles and Responsibilities
Experience: 6.5 to 10 years
- Design develop and implement ETL pipelines for the continuous deployment and integration.
- Collaborate with other data engineers and data scientists to understand data requirements and optimize data solution processes.
- Assure that data is cleansed mapped transformed and otherwise optimised for storage and use according to business and technical requirements
- The ability to automate tasks and deploy production standard code (with unit testing continuous integration versioning etc.)
- Load transformed data into storage and reporting structures in destinations including data warehouse high speed indexes realtime reporting systems and analytics applications
- Build data pipelines to collectively bring data together
- Other responsibilities include extracting data troubleshooting and maintaining the data warehouse
- Implement best practices for pipeline building and governance.
- Troubleshoot and resolve issues related to pipeline deployment and performance.
- Ensure compliance with security and data privacy standards
- Solution design using Microsoft Azure services and other tools
Required skills and experience
- 23 years of experience in Azure Data Analytics Synapse DevOps.
- Proficiency in programming languages such as Python SQL PySpark SparkSQL.
- Experience with Azure: ADLS Synapse Stream Analytics SQL DW Analysis Services Serverless Architecture ARM Templates
- Experience with cloud platforms like Azure AWS or and their respective services.
- Knowledge of CI/CD pipelines automation tools and version control systems like Git.
- Strong problemsolving skills and ability to troubleshoot complex issues.
- Experience with monitoring tools.
- Ability to work collaboratively in crossfunctional teams.
Remote Work :
No