DataBricks Engineer
DataBricks Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Role description:
Develop and provide solutions on Big Data;
Develop hightraffic flawless web applications using Python Cloud platform and PySpark;
Code with performance scalability and usability in mind;
Work on new tools in leading industry trends with new and emerging technologies prototypes andengineering process improvements;
Migrate existing pipelines developed in ADF and python scripts from Onpremise to Cloud using ADFand Databricks;
Develop new pipelines using ADF for data extraction from multiple sources (MySQL databases;
Oracle databases CSV files) and Databricks (PySpark & SparkSQL) for curation validation andapplying required business transformation logic;
Work on client projects to deliver Microsoft Azure based Data engineering & Analytics solutions;
Engineer and implement scalable analytics solutions on Azure.
Qualifications:
Bachelors degree in computer science or related field
5 years of experience in a similar role; leading fellow data engineers within the data engineeringarea;
Extensive experience working in agile project environments;
Working experience with Python or PySpark is required;
Working experience and understanding of Azure Databricks Azure Data Factory Azure Data LakeAzure SQL DW and Azure SQL is required;
The ability to apply such methods to solve business problems using one or more Azure Data andAnalytics services in combination with building data pipelines data streamsand systemintegration;
Experience in driving new data engineering developments (e.g. applying new cuttingedge dataengineering methods to improve the performance of data integration using new tools to improve.
Employment Type
Full Time
