Azure Databricks Engineer

Title: Azure Databricks Engineer

Location: Weehawken NJ (Day 1 Onsite)

Job Description:

Primary: Databricks Spark (batch streaming solutions ( delta lake and lake house ) Python

Good to have: Perl SSIS SSAS with java and oracle

bachelors and/or Masters degree in Computer Science Computer Engineering or related technical discipline

At least 10 years of experience in working in large global teams experience in working in large global teams especially for highperformance largescale systems through various project lifecycle stages like design build testing etc.

understanding of dev ops tools and engineering practices using microservices ci/cd pipeline and test automation

strong understanding of the software development lifecycle

experience in one or more programming languages: Java 8 Python

Experience of Tomcat & Spring framework is a plus

Good understanding of relational (Oracle SQL Server)

Cloud technologies and patterns

building and optimizing big data data pipelines architectures and data sets

proven skills in performance tuning and quality improvements

strong in Algorithms and Data Structures

interpersonal and communication skills and an ability to work as part of a global team

passion for learning and innovation

keen interest in understanding how the business works

Agile Software delivery Methodology (Scrum XP Kanban etc.)

Additional Experience Advantageous

Microsoft Azure experience

Infrastructureascode using tools such as Terraform ARM Bicep or CloudFormation

Azure CLI setting up ADO pipelines Terraform Enterprise etc

Spark/Databricks batch and streaming solutions (Delta Lake Lakehouse)

Knowledge of Azure Data Factory

Knowledge of Kafka Event hubs and ADLS Gen2 on Microsoft Azure

Containers and container orchestration using Docker Kubernetes and Helm