Azure Data Engineer
Azure Data Engineer
Posted on :
19-12-2024
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Job Description:
We are looking for a highly skilled Azure Data Engineer to join our team. The ideal candidate will have strong expertise in implementing data solutions on Azure using technologies like Azure Data Lake Storage Gen2 (ADLS2) Azure Data Factory (ADF) Azure Databricks and Azure SQL/other RDBMS. The role involves building robust and scalable data pipelines transforming raw data into actionable insights and supporting advanced analytics initiatives.
Key Responsibilities:
- Design and Develop Data Solutions
- Build and optimize data pipelines using Azure Data Factory and Azure Databricks.
- Implement solutions for structured and unstructured data storage using ADLS Gen2.
- Integrate data from various sources into Azure SQL or other relational databases.
- Build and optimize data pipelines using Azure Data Factory and Azure Databricks.
- Data Transformation and Processing
- Design ETL/ELT workflows for data ingestion cleansing transformation and aggregation.
- Process and analyze large datasets using Databricks (PySpark/Scala/Spark SQL).
- Design ETL/ELT workflows for data ingestion cleansing transformation and aggregation.
- Performance Optimization
- Optimize data storage and retrieval in ADLS2 and relational databases to ensure high performance.
- Implement partitioning indexing and caching strategies for efficient data operations.
- Optimize data storage and retrieval in ADLS2 and relational databases to ensure high performance.
- Collaboration and Support
- Collaborate with data scientists analysts and business stakeholders to gather requirements and deliver solutions.
- Ensure data security governance and compliance with organizational standards.
- Collaborate with data scientists analysts and business stakeholders to gather requirements and deliver solutions.
- Monitoring and Maintenance
- Monitor data pipelines for reliability and scalability.
- Troubleshoot and resolve issues related to data processing and integration.
- Monitor data pipelines for reliability and scalability.
Requirements
Required Skills and Qualifications:
- Technical Expertise:
- Strong experience with Azure Data Lake Storage Gen2 (ADLS2) for data storage and management.
- Proficiency in building data pipelines with Azure Data Factory (ADF).
- Handson experience in Databricks (including PySpark Scala or Spark SQL).
- Advanced knowledge of Azure SQL or any other RDBMS (e.g. SQL Server PostgreSQL MySQL).
- Strong experience with Azure Data Lake Storage Gen2 (ADLS2) for data storage and management.
- Programming Skills:
- Proficiency in Python SQL Pyspark for data engineering tasks.
- Proficiency in Python SQL Pyspark for data engineering tasks.
- Cloud Knowledge:
- Solid understanding of Azure ecosystem including services like Azure Synapse Analytics Azure Blob Storage and Azure Key Vault (optional but preferred).
- Solid understanding of Azure ecosystem including services like Azure Synapse Analytics Azure Blob Storage and Azure Key Vault (optional but preferred).
- ProblemSolving and Analytical Skills:
- Ability to design and implement efficient solutions for largescale data processing and analytics.
- Ability to design and implement efficient solutions for largescale data processing and analytics.
- Soft Skills:
- Strong communication and collaboration skills.
- Ability to work in agile fastpaced environments
- Strong communication and collaboration skills.
Preferred Qualifications:
- Azure certifications (e.g. Microsoft Certified: Azure Data Engineer Associate).
- Familiarity with data governance frameworks and tools.
- Knowledge of distributed systems and big data frameworks like Apache Kafka (optional).
Experience Level:
- Minimum 3 5 years in data engineering or related roles.
Benefits
What a Consulting role at Thoucentric will offer you
- Opportunity to define your career path and not as enforced by a manager
- A great consulting environment with a chance to work with Fortune 500 companies and startups alike.
- A dynamic but relaxed and supportive working environment that encourages personal development.
- Be part of One Extended Family. We bond beyond work sports gettogethers common interests etc. Work in a very enriching environment with Open Culture Flat Organization and Excellent Peer Group.
- Be part of the exciting Growth Story of Thoucentric!
Azure Data Lake Storage Gen2 (ADLS2), Azure Data Factory (ADF), Azure Databricks, and Azure SQL/other RDBMS. Python, SQL, ETL, PySpark
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
