Big Data Engineer - Azure Databricks
Big Data Engineer - Azure Databricks
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
In this role you will work with our Business Analytics & Insights (BAI) team collaborating with crossfunctional teams to deliver highquality data solutions in areas such as Supply Chain Finance Operations Customer Experience HR Risk Management and Global IT.
Responsibilities:
- Lead the technical planning for data migration including data ingestion transformation storage and access control within Azure Data Factory and Azure Data Lake.
- Design and implement scalable efficient data pipelines to ensure smooth data movement from multiple sources using Azure Databricks.
- Develop reusable frameworks for the ingestion of large datasets.
- Ensure data quality and integrity by implementing robust validation and cleansing mechanisms throughout the data pipeline.
- Work with eventbased/streaming technologies to ingest and process data in realtime.
- Provide technical support to the team resolving challenges during the migration and postmigration phases.
- Stay current with the latest advancements in cloud computing data engineering and analytics technologies; recommend best practices and industry standards for data lake solutions.
Qualifications :
- Min. 4 years of IT experience.
- Min. of 4 years working with Azure Databricks.
- Proficiency in Data Modeling and Source System Analysis.
- Strong knowledge of PySpark and SQL.
- Experience with Azure components: Data Factory Data Lake SQL Data Warehouse (DW) and Azure SQL.
- Experience with Python for data engineering purposes.
- Ability to conduct data profiling cataloging and mapping for technical design and construction of data flows.Familiarity with data visualization/exploration tools.
Remote Work :
Yes
Employment Type :
Fulltime
Employment Type
Remote
Company Industry
Department / Functional Area
Engineering
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
