Position Title: Azure Data Engineer
Length of Contract: 3 Months Contract to Hire
Location: Onsite Louisville KY
Only W2 USC/GC
Healthcare is Must
PBM experience is required
Skills must have: Azure SynapseDatabricks SnowFlake Python PySpark(ETL)
Job Details:
Considering Python as the ETL tool for reading data from the source handling datatype and data transformations managing error handling logging and loading data into Databricksmanaged Delta tables the candidate should have strong expertise in Python along with solid knowledge of Databricks and PySpark enabling them to evaluate and perform data transformations within Databricks. They should be proficient in all aspects of Python including classes objects methods dictionaries loops Wheel files and packages and should be able to connect to various data sources including databases and streaming data
-
Expertise in Azure services such as Azure SQL Database Azure Data Factory Azure Synapse Analytics and Azure Data Lake is also required.
-
The ideal candidate has extensive experience in data engineering working with largescale data infrastructures and optimizing ETL processes
-
Expertise in Azure services such as Azure SQL Database Azure Data Factory Azure Synapse Analytics and Azure Data Lake
-
At least 6 years of experience in data engineering with a focus on building data pipelines
-
Proven experience with Spark Scala and other related technologies
-
Familiarity with Could Warehouses data lakes storage formats.
-
Strong knowledge of data modeling and ETL processes
-
Experience leading data engineering projects.
-
Proven experience with Databricks and Apache Spark
-
Strong proficiency in Python
-
Experience in Unix shell scripting and scheduling tools