685 - Sr Data Engineer
685 - Sr Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Job Summary:
We are looking for a Principal Data Engineer with strong programming expertise in Python and experience in building largescale data processing pipelines using technologies such as Databricks Spark Python and Postgres SQL. Exceptional communication skills and a proactive approach to challenges are essential. Candidates with experience in the CPG or Retail industry will have an advantage though this is not a requirement.
Responsibilities
Programming Expertise: Design code and maintain largescale data processing pipelines using Databricks Spark Python and SQL.
Data Processing: Architect and optimize data pipelines to ensure high efficiency scalability and reliability.
Cloud Platform Management: Develop and deploy data solutions on cloud platforms with a preference for Azure.
Quality Assurance: Implement processes to maintain data accuracy consistency and reliability.
Data Integration: Seamlessly integrate data from diverse sources and formats into processing pipelines.
Data Governance: Collaborate with data governance teams to establish and enforce best practices and quality standards.
Requirements
Programming Skills: Advanced proficiency in Python.
Databricks Experience: Minimum of 2 years with Databricks (4 years preferred).
Apache Spark: Expertise in using Spark for efficient data processing.
SQL Knowledge: Advanced skills in SQL for data analysis and transformations.
Cloud Expertise: Strong knowledge of at least one cloud platform preferably Azure.
Communication: Ability to effectively communicate and challenge assumptions to drive solutions.
Industry Knowledge: Experience in the CPG or Retail industry is advantageous but not required.
Core Skills
Programming in Python and SQL.
Building and optimizing pipelines using Databricks and Spark.
Strong understanding of cloud platforms particularly Azure.
Quality assurance and data governance best practices.
Preferred Skills/Experience
Familiarity with other datacentric technologies outside of Databricks such as Data Warehousing ETL Analytics and Reporting.
Previous experience in working with multiple collaborative teams especially Data Science and Engineering delivery teams.
Experience in the Consumer Packaged Goods (CPG) or Retail industry.
A bachelors or masters degree in computer science data engineering or related fields.
46 years of experience in data engineering roles focusing on largescale data processing pipelines.
Employment Type
Full Time
