Data Engineer Azure Pyspark
Data Engineer Azure Pyspark
Posted on :
18-09-2024
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Job Title: Data Engineer
Location: Specify Location or Remote
Responsibilities: Design build and maintain scalable data pipelines using PySpark on Data bricks. Work with Delta Lake to ensure data reliability consistency and scalability. Utilize Azure Synapse for integrated analytics querying and big data processing. Develop and deploy data solutions on Azure including data lakes data warehouses and data pipelines. Collaborate with data scientists analysts and other stakeholders to understand data requirements and deliver solutions. Monitor and optimize data pipelines for performance and cost efficiency. Requirements: Experience with PySpark: Handson experience in writing PySpark scripts and managing Spark clusters. Delta Lake Expertise: Understanding of Delta Lake features such as ACID transactions and schema evolution.
Experience with PySpark: Hands-on experience in writing PySpark scripts and managing Spark clusters. Delta Lake Expertise: Understanding of Delta Lake features, such as ACID transactions and schema evolution.
Location: Specify Location or Remote
Responsibilities: Design build and maintain scalable data pipelines using PySpark on Data bricks. Work with Delta Lake to ensure data reliability consistency and scalability. Utilize Azure Synapse for integrated analytics querying and big data processing. Develop and deploy data solutions on Azure including data lakes data warehouses and data pipelines. Collaborate with data scientists analysts and other stakeholders to understand data requirements and deliver solutions. Monitor and optimize data pipelines for performance and cost efficiency. Requirements: Experience with PySpark: Handson experience in writing PySpark scripts and managing Spark clusters. Delta Lake Expertise: Understanding of Delta Lake features such as ACID transactions and schema evolution.
Experience with PySpark: Hands-on experience in writing PySpark scripts and managing Spark clusters. Delta Lake Expertise: Understanding of Delta Lake features, such as ACID transactions and schema evolution.
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
