Senior Databricks Engineer

Position: Senior Databricks Engineer

Location: Raleigh NC / Charlotte NC (Onsite / Hybrid)

Job Description

Skillsets for data engineer:

• Azure:::Azure Resource Group ADLS Gen 2 storage Container event hub IOT hub stream analytics Power BI/ Tableau.
• Databricks: : Databricks lakehouse architecture (Bronze Silver and Gold) using Delta Lake (ACID schema enforcement timetravel schema evolution) databricks workspace pyspark auto loader python SQL spark cluster( all purpose job) driver/worker workflow/job/task dataframe dataset azure blob file system secure( ABFSS)/DBFS transaction log delta table delta live table materialized view metastore databricks access connector unity catalog schema DBFS Databrick CLI databricks performance tuning.
• Data Modeling: Bigdata Modeling 3NF Dimensional Modeling (Fact Dim)
• Other: GitHub CD/CI (Azure Dev/Ops)

Roles & Responsibility

• Design develop test deploy and maintain enterpriselevel applications using the Snowflake platform
• Work with a variety of stakeholders to understand requirements and deliver solutions
• Take ownership of a project and see it through to completion
• Experience with the Snowflake platform
• Experience with enterpriselevel applications
• Ability to work independently as well as part of a team
• Ability to take ownership of a project and see it through to completion
• Strong analytical and problemsolving skills
• Working experience on Snowflake cloud data warehouse
• Able to write advanced SQL queries
• ETL IICS knowledge
• Have working experience on DATA Warehousing Projects
• Understanding of Dimensions Facts and Datamart Terminologies.
• Must have minimum 5 years experience in Informatica PowerCenter using various transformations ETL Framework and Design/Development.
• Develop ETL Solutions using IICS Data Integration/Application Integration handling large data volumes
• Must have working knowledge in handling various source/target systems including XML/Flat files/Databases using PowerCenter/Informatica MFT/ IICS File Ingestion.
• Experience in working Agile/Sprint projects and exposure to Azure DevOps is a plus
• Experience in change data capture Data warehousing SQL SQL server and Unix shell scripting
• Prior DataStage and Azure cloud platform experience is an advantage.

BIGDATA , SQL SERVER , UNIX SHELL , XML , SQL , DIMENSIONAL MODELING , PYTHON , DATA WAREHOUSING