Data Engineer with Databricks Experience

Job Description:
We are seeking an experienced Data Engineer with strong expertise in Databricks to join our team in Boston MA. The ideal candidate will have 8 years of experience in designing implementing and optimizing data solutions including building robust ETL pipelines and advanced analytics platforms. You will work closely with crossfunctional teams to manage and process largescale datasets using Databricks and cloud technologies.
Key Responsibilities:
Design develop and maintain ETL pipelines for data ingestion processing and analytics using Databricks.
Collaborate with data scientists analysts and business stakeholders to deliver endtoend data solutions.
Optimize data workflows and performance ensuring scalability and efficiency.
Develop test and deploy SQL and Python scripts for data processing and transformation.
Work with cloud platforms primarily Azure or AWS to manage largescale datasets.
Implement data governance and security best practices.
Perform data validation cleansing and quality assurance to ensure accurate results.
Troubleshoot and resolve datarelated issues.
Qualifications:
8 years of handson experience in data engineering with expertise in Databricks.
Proficiency in Apache Spark PySpark and Scala for largescale data processing.
Strong experience with SQL for querying and manipulating databases.
Proven experience with cloud platforms like Azure AWS or GCP.
Experience with Azure Data Lake Azure Synapse or similar tools is a plus.
Familiarity with data warehousing concepts and ETL processes.
Excellent problemsolving skills and ability to work in a collaborative environment.
Strong communication skills and ability to work with crossfunctional teams