We are looking to hire Azure Data Engineers for a client who is a leader in the AI and Analytics solution. You will work on a broad range of cuttingedge data analytics problems.
Roles & Responsibilities:
- Design and build scalable & metadata driven data ingestion pipelines (For Batch and Streaming Datasets)
- Execute highperformance data processing for structured and unstructured data and data harmonization
- Schedule orchestrate and validate pipelines
- Design exception handling and log monitoring for debugging
- Make tech stack and tools related decisions
- Collaborate with business consultants data scientists and application developers to develop analytics solutions
Required Experience Skills & Competencies:
- 4 to 8 years of total IT experience with 2 years on big data engineering and Microsoft Azure.
- Strong Handson experience in implementing Data Lake with technologies like:
- Mandatory: Azure Data Factory (ADF) PySpark Databricks ADLS Azure SQL Database
- Strong programming unit testing & debugging skills in SQL Python or Scala/Java.
- Some experience of using big data technologies like Hadoop Spark Airflow NiFi Kafka Hive Neo4J Elastic Search.
- Good Understanding of different file formats like Delta Lake Avro Parquet JSON and CSV.
- Experience of working in Agile projects and following DevOps processes with technologies like Git Jenkins & Azure DevOps.
Location: Delhi/ NCR/ Gurgaon/ Pune
Availability: Available to Join within 30 to 45 Days
Remote Work :
No