Data Engineer with Databricks Experience
Data Engineer with Databricks Experience
Posted on :
12-10-2024
Employer Active
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Job Description:
We are seeking an experienced Data Engineer with strong expertise in Databricks to join our team in Boston MA. The ideal candidate will have 8 years of experience in designing implementing and optimizing data solutions including building robust ETL pipelines and advanced analytics platforms. You will work closely with crossfunctional teams to manage and process largescale datasets using Databricks and cloud technologies.
Key Responsibilities:
Design develop and maintain ETL pipelines for data ingestion processing and analytics using Databricks.
Collaborate with data scientists analysts and business stakeholders to deliver endtoend data solutions.
Optimize data workflows and performance ensuring scalability and efficiency.
Develop test and deploy SQL and Python scripts for data processing and transformation.
Work with cloud platforms primarily Azure or AWS to manage largescale datasets.
Implement data governance and security best practices.
Perform data validation cleansing and quality assurance to ensure accurate results.
Troubleshoot and resolve datarelated issues.
Qualifications:
8 years of handson experience in data engineering with expertise in Databricks.
Proficiency in Apache Spark PySpark and Scala for largescale data processing.
Strong experience with SQL for querying and manipulating databases.
Proven experience with cloud platforms like Azure AWS or GCP.
Experience with Azure Data Lake Azure Synapse or similar tools is a plus.
Familiarity with data warehousing concepts and ETL processes.
Excellent problemsolving skills and ability to work in a collaborative environment.
Strong communication skills and ability to work with crossfunctional teams
We are seeking an experienced Data Engineer with strong expertise in Databricks to join our team in Boston MA. The ideal candidate will have 8 years of experience in designing implementing and optimizing data solutions including building robust ETL pipelines and advanced analytics platforms. You will work closely with crossfunctional teams to manage and process largescale datasets using Databricks and cloud technologies.
Key Responsibilities:
Design develop and maintain ETL pipelines for data ingestion processing and analytics using Databricks.
Collaborate with data scientists analysts and business stakeholders to deliver endtoend data solutions.
Optimize data workflows and performance ensuring scalability and efficiency.
Develop test and deploy SQL and Python scripts for data processing and transformation.
Work with cloud platforms primarily Azure or AWS to manage largescale datasets.
Implement data governance and security best practices.
Perform data validation cleansing and quality assurance to ensure accurate results.
Troubleshoot and resolve datarelated issues.
Qualifications:
8 years of handson experience in data engineering with expertise in Databricks.
Proficiency in Apache Spark PySpark and Scala for largescale data processing.
Strong experience with SQL for querying and manipulating databases.
Proven experience with cloud platforms like Azure AWS or GCP.
Experience with Azure Data Lake Azure Synapse or similar tools is a plus.
Familiarity with data warehousing concepts and ETL processes.
Excellent problemsolving skills and ability to work in a collaborative environment.
Strong communication skills and ability to work with crossfunctional teams
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
