Data engineer Azure
Data engineer Azure
Posted on :
20-02-2025
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Data engineer (Azure) Synapse and Pyspark Python Datawarehouse and Power BI Azure Devops
Skills & Experience
Bachelor s Degree in Computer Science or Engineering with 35 years of experience in Azure Data engineering Python Pyspark or Big Data development
Sound Knowledge of Azure Synapse analytics for pipelines orchestration set up
12 experience in Visualization design and development with Power BI. Knowledge on rowlevel security access control
Sound experience in SQL Datawarehouse data marts data ingestion with Pyspark and Python
Expertise in developing and maintaining ETL processing pipelines in cloudbased platforms such as AWS Azure etc. (Azure Synapse or data factory preferred)
Team player with good interpersonal communication and problemsolving skills.
Job Scope
Design review and development of Pyspark scripts. Testing troubleshooting of data pipelines orchestration
Designing and developing reports and dashboards in Power BI setting up access control with rowlevel security DAX query experience
Establishing connections to source data systems including Sembcorp internal systems e.g. SAP Historians Data Lake etc. as well as external systems such as Web APIs etc
Managing the collected data in appropriate storage/database solutions e.g. file systems SQL servers Big Data platforms such as Hadoop HANA etc. as required by the specific project requirements
Design development of data marts and relevant data pipelines using pyspark data copy activities for batch ingestion
Deployment of pipeline artifacts from one environment to the other using Azure Devops
Performing data integration e.g. using database table joins or other mechanisms at an appropriate level as required by the analysis requirements of the project.
Good to have
Data catalog with Purview enabling effective metadata management lineage tracking and data discovery
Candidates should demonstrate the ability to leverage Purview to ensure data governance compliance and efficient data exploration within Azure environments.
Others
Able to work independently on assignment according to agreed schedule without much supervision
Own assignment and take initiative to resolve issues hinder completion of assignment Proactively reach out for help/guidance whenever required.
Data engineer (Azure)
Education
Degree / diploma
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
