Databricks

Request ID:

Work Location: Chicago IL 60601(ONSITE)

Job Description:

Minimum 8 yrs experience on Databricks Python Spark Java SQL Mulesoft Develop and optimize Databricks notebooks and workflows.

Utilize Databricks Delta Lake for reliable data management and ACID transactions.

Leverage Databricks MLflow for machine learning lifecycle management.

Write clean efficient and maintainable Python code for data processing analysis and machine learning.

Develop scalable and distributed data processing applications using Apache Spark.

Optimize Spark performance and resource utilization.

Develop and deploy machine learning models on Databricks.

Having Knowledge on MuleSoft platform.

Design and implement ETL (Extract Transform Load) processes using Talend components.

Integrate with cloud platforms (e.g. AWS Azure GCP) for cloudnative data pipelines.

Work closely with business analysts data architects and other team members to understand requirements and deliver solutions.

Contribute to the development and maintenance of team standards and best practices.

Relevant required 8 years of experience as lead developer with Big data and Hadoop.

Participate in design and implementation of the analytics architecture.

Experience in working on Hadoop Distribution good understanding of core concepts and best practices Good experience in building/tuning Spark pipelines in Python/Java/Scala Good experience in writing complex Hive queries to drive business critical insights Understanding of Data Lake vs Data Warehousing concepts Experience with Spark Scala & Hive technologies.

Must Have Skills:

As Lead developer provide resolve technical queries develop queries work with QA and other team for final implementation Design and development using Databricks Python Spark Java SQL Mulesoft Deliver as per scrum/Agile methodology Onsite/offshore coordination

Regards