Databricks Support Engineer
Databricks Support Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
LOCATION Chicago IL Hybrid
DURATION 6 months
Need a Databricks SME!
This needs to be a go to person for all things databricks platform
Need someone who is HEAVILY INGRAINED in databricks
(Mandatory) Supporting day to day databricks jobs troubleshooting databricks pipelines debugging Python/SQL code.
(basic knowledge) Databricks admin: Cluster management cluster policies
(add on) Machine learning pipelines using mlFlow in Databricks
Regularly monitor the health and performance of Databricks jobs clusters and pipelines. Ensure that clusters are properly sized running efficiently and scaling as needed.
Diagnose and resolve issues related to jobs notebooks and clusters including performance bottlenecks failures and resource constraints. This often involves debugging both Python and SQL code used in Databricks environments.
Provide support to data scientists data engineers and other users by helping them with their Databricks workspaces job configurations code issues and cluster configurations.
Respond to and resolve support tickets related to Databricks failures performance degradation connectivity issues etc. Keep stakeholders informed about the status of incidents.
Regularly analyze the performance of Databricks clusters and jobs using Databricks builtin monitoring tools (e.g. Spark UI Ganglia Databricks Metrics) and suggest improvements.
Work with data teams to ensure that Spark jobs running in Databricks are optimized for speed resource usage and cost.
Monitor and optimize costs by managing cluster scaling idle time and instance selection for both Databricks jobs and clusters.
Employment Type
Full Time
