Data Engineer Azure Pyspark
Data Engineer Azure Pyspark
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
5years
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
Location: Specify Location or Remote
Responsibilities: Design build and maintain scalable data pipelines using PySpark on Data bricks. Work with Delta Lake to ensure data reliability consistency and scalability. Utilize Azure Synapse for integrated analytics querying and big data processing. Develop and deploy data solutions on Azure including data lakes data warehouses and data pipelines. Collaborate with data scientists analysts and other stakeholders to understand data requirements and deliver solutions. Monitor and optimize data pipelines for performance and cost efficiency. Requirements: Experience with PySpark: Handson experience in writing PySpark scripts and managing Spark clusters. Delta Lake Expertise: Understanding of Delta Lake features such as ACID transactions and schema evolution.
Experience with PySpark: Hands-on experience in writing PySpark scripts and managing Spark clusters. Delta Lake Expertise: Understanding of Delta Lake features, such as ACID transactions and schema evolution.
Employment Type
Remote
Company Industry
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
