Position: Senior Databricks Engineer
Location: Raleigh NC / Charlotte NC (Onsite / Hybrid)
Job Description
Skillsets for data engineer:
- Azure:::Azure Resource Group ADLS Gen 2 storage Container event hub IOT hub stream analytics Power BI/ Tableau.
- Databricks: : Databricks lakehouse architecture (Bronze Silver and Gold) using Delta Lake (ACID schema enforcement timetravel schema evolution) databricks workspace pyspark auto loader python SQL spark cluster( all purpose job) driver/worker workflow/job/task dataframe dataset azure blob file system secure( ABFSS)/DBFS transaction log delta table delta live table materialized view metastore databricks access connector unity catalog schema DBFS Databrick CLI databricks performance tuning.
- Data Modeling: Bigdata Modeling 3NF Dimensional Modeling (Fact Dim)
- Other: GitHub CD/CI (Azure Dev/Ops)
Roles & Responsibility
- Design develop test deploy and maintain enterpriselevel applications using the Snowflake platform
- Work with a variety of stakeholders to understand requirements and deliver solutions
- Take ownership of a project and see it through to completion
- Experience with the Snowflake platform
- Experience with enterpriselevel applications
- Ability to work independently as well as part of a team
- Ability to take ownership of a project and see it through to completion
- Strong analytical and problemsolving skills
- Working experience on Snowflake cloud data warehouse
- Able to write advanced SQL queries
- ETL IICS knowledge
- Have working experience on DATA Warehousing Projects
- Understanding of Dimensions Facts and Datamart Terminologies.
- Must have minimum 5 years experience in Informatica PowerCenter using various transformations ETL Framework and Design/Development.
- Develop ETL Solutions using IICS Data Integration/Application Integration handling large data volumes
- Must have working knowledge in handling various source/target systems including XML/Flat files/Databases using PowerCenter/Informatica MFT/ IICS File Ingestion.
- Experience in working Agile/Sprint projects and exposure to Azure DevOps is a plus
- Experience in change data capture Data warehousing SQL SQL server and Unix shell scripting
- Prior DataStage and Azure cloud platform experience is an advantage.
BIGDATA , SQL SERVER , UNIX SHELL , XML , SQL , DIMENSIONAL MODELING , PYTHON , DATA WAREHOUSING