Lead Data Engineer
Posted on :
14-05-2024
Employer Active
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
It is very Urgent role
Title Lead Data Engineer
Location Remote however if someone is nearby to Rahway NJ that candidate will have to go onsite.
Duration 12 month
Lead experience required 23 years
Must have total 10 exp. needed
MustHave:
Databricks expert need an experienced resource who can provide solutions and give recommendations to client.
Pyspark Super important
SQL Super important
Airflow good experience
Experience
Role
- Responsible for defining technical architecture and application landscape.
- Responsible for authoring SQL and Python scripts on Databricks and DBT (Data Build tool) to create data pipelines to create Operational Data Mart.
- Responsible for creation of Data Pipelines for Data processing of Delta files into ODM format for downstream data consumption
- Responsible for identifying data set relationship join criteria and implement it in code for ODM model development.
- Responsible for creation of Delta Lake for ODM model and setup of consumption pattern using Databricks Unity catalog
- Responsible for creation of Airflow DAGs for job orchestration and scheduling of data pipeline jobs
Employment Type
Remote
Company Industry
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
