Sr Data Engineer Databricks
Sr Data Engineer Databricks
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
- Design and build data pipelines using SparkSQL and PySpark in Azure Databricks
- Design and build ETL pipelines using ADF
- Build and maintain a Lakehouse architecture in ADLS / Databricks.
- Perform data preparation tasks including data cleaning normalization deduplication type conversion etc.
- Work with DevOps team to deploy solutions in production environments.
- Control data processes and take corrective action when errors are identified. Corrective action may include executing a work around process and then identifying the cause and solution for data errors.
- Participate as a full member of the global Analytics team providing solutions for and insights into data related items.
- Collaborate with your Data Science and Business Intelligence colleagues across the world to share key learnings leverage ideas and solutions and to propagate best practices.
- You will lead projects that include other team members and participate in projects led by other team members.
- Apply change management tools including training communication and documentation to manage upgrades changes and data migrations.
documentation,data preparation,etl,data normalization,spark-sql,adf,azure databricks,devops,deduplication,etl pipelines,data cleaning,change management,pyspark,lakehouse architecture,sql,communication,azure,adls,type conversion
Employment Type
Full Time
Company Industry
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
