Job Title: Azure Databricks Developer
Location: Canonsburg PA Remote
Duration: 12 Months
Must have 14 Years of total IT experience.
Note: This will be remote opportunity But occasional travel like once a month to client location will be required.
Need CST or EST candidates.
Responsibilities
- 10 years of experience in implementing large data and analytics platforms.
- 5 years of experience architecting and building data platforms on Azure.
- 5 years of experience architecting and building data platforms on Databricks from scratch.
- Good Knowledge in Databricks engineering services like Unity Catalog Spark Jobs Delta Live Table (DLT) DLT Meta Databricks Workflow Auto Loader.
- Hands on in Databricks SQL PySpark
- Understanding of data modelling data mappings understand the data model changes and propose the best practices and guidelines.
- Creating technical design based on endtoend data architecture and frameworks for meta data driven data ingestion transformation logging and monitoring.
- Deploying data pipelines/applications to higher environments using CICD pipelines
- Experience in writing technical user stories.
- Experience in Agile methodology and scrums.
- Minimum Qualifications
- Bachelors degree or masters Degree in the field of Computer Science or Information Systems or a related field.
- Preferred Qualifications
- Design Azure Databricks platform architecture from scratch.
- Design and build ingestion framework data processing framework logging & monitoring framework and data sharing framework.
- Share Databricks best practices.
- Develop data models in different layers of Databricks.
- Design build and orchestrate data pipelines in Azure and Databricks
- Define strategy for configuration management and CI/CD