Azure Data Engineer
Azure Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
About Company:
It is a global consulting service and systems integrator that operates in over 90 countries delivering solutions with a unique blend of digital innovation and robust industrystrong processes. With our promise to help our customers scale at speed we design future state solutions for industry leaders and offer innovative digital experiences that enable them to transform and scale at speed. Our aim is to enable enterprises associates and society to Rise for a more equal world future readiness and value creation.
Job Description:
Job Overview:
We are seeking a talented Azure Data Engineer to join our team. The ideal candidate will have solid experience in designing implementing and maintaining data solutions on the Azure cloud platform with a particular focus on Azure Databricks Python/PySpark and SQL. You will be responsible for building scalable highperformance data pipelines and working with large datasets to derive valuable insights.
Key Responsibilities:
Data Pipeline Design and Implementation:
Build test and optimize data pipelines and ETL processes using Azure Databricks Azure Data Factory (ADF) and other Azure data services like Azure SQL Database Azure Blob Storage and Azure Synapse Analytics.
Big Data Processing:
Work with largescale datasets using PySpark in Azure Databricks to implement data transformations aggregations and analyses.
Data Integration & ETL:
Design and implement ETL solutions to extract data from various sources transform it for analytical purposes and load it into cloud data platforms like Azure Data Lake Azure SQL and Azure Synapse Analytics.
Database and Data Warehouse Management:
Optimize SQL queries and implement best practices for database and data warehouse solutions in Azure SQL Database and Azure Synapse Analytics.
Collaboration and Requirement Gathering:
Work with crossfunctional teams including data scientists analysts and business stakeholders to understand requirements and ensure that data solutions meet business needs.
Optimization & Performance Tuning:
Focus on optimizing data processes for performance cost and scalability using best practices in Databricks and Azure cloud services.
Security and Data Governance:
Implement security measures for data ensure compliance with data governance policies and use Azure Purview or similar tools for data cataloging and lineage.
Automation and Monitoring:
Automate workflows and ensure data pipeline monitoring and alerting to maintain the health and reliability of the data infrastructure.
Required Skills & Qualifications:
Experience:
5 years of experience as a Data Engineer or in a similar role with handson experience working on Azure cloud technologies especially Azure Databricks and ETL pipelines.
Azure Technologies:
Azure Databricks: Deep experience with Databricks for processing and transforming large datasets with PySpark.
Azure Data Factory (ADF): Experience building and managing data integration pipelines in ADF.
Azure SQL Database and Azure Synapse Analytics: Experience designing and optimizing data solutions in these Azure services.
Azure Blob Storage Azure Data Lake: Experience working with cloud storage solutions in Azure.
Programming and Querying:
Python and PySpark: Strong knowledge of Python for ETL tasks data transformations and automation. Familiarity with PySpark for largescale data processing.
SQL: Proficiency in writing complex SQL queries optimizing query performance and working with relational databases.
Data Modeling and Design:
Experience with designing data models schemas and partitions for big data environments.
Version Control and CI/CD:
Familiarity with version control tools like Git and experience with CI/CD pipelines for automated deployment of data engineering solutions.
Data Integration:
Experience with integrating diverse data sources including relational databases APIs flat files and streaming data sources.
Preferred Qualifications:
Certifications:
Microsoft Certified: Azure Data Engineer Associate or Azure Solutions Architect certification is highly desirable.
Big Data Tools:
Familiarity with other big data technologies such as Apache Kafka Apache Spark or Hadoop is a plus.
Machine Learning Frameworks:
Experience in working with machine learning pipelines in Azure Databricks or other ML/AI platforms is a bonus.
Data Governance:
Familiarity with Azure Purview data cataloging and data lineage tools for data governance.
Agile Methodologies:
Experience working in Agile teams and using tools like JIRA Confluence or Azure DevOps.
Soft Skills:
Strong communication skills ability to collaborate with business users to gather requirements and experience presenting technical concepts to nontechnical stakeholders.
Job Role: Azure Data Engineer
Job Type: Full Time
Work Mode: Hybrid Mode
Shift Timings: 12PM to 9PM
Experience: 7 Years to 10 Years
CTC Range: 25 LPA to 27 LPA
Location: PAN India
Notice Period :Immediate to 30 Days & Serving NP
Employment Type
Full Time
