Data Engineer
Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Job Overview:
To work on development monitoring and maintenance of Data pipelines across AE clusters.
Primary responsibilities:
Develop Monitor and Maintain data pipeline for various AE plants.
Create and maintain optimal data pipeline architecture.
Assemble large complex data sets that meet functional / nonfunctional business requirements.
Identify design and implement internal process improvements: automating manual processes optimizing data delivery redesigning infrastructure for greater scalability.
Work with stakeholders including the Data officers and stewards to assist with datarelated technical issues and support their data infrastructure needs.
Work on incidents highlighted by the data officers.
Incident diagnosis routing evaluation & resolution.
Analyze the root cause of incidents.
Create incident closure report.
Qualifications :
Qualifications
- Bachelors degree in Computer Science Electronics & Communication Engineering a related technical field or equivalent practical experience.
- 34 years of experience in Java or Scala software development.
- Experience in largescale software development.
- Excellent software engineering skills (i.e. data structures algorithms software design).
- Excellent problemsolving investigative and troubleshooting skills.
Additional Information :
Skills
- Selfstarter and empowered professional with strong execution and project management capabilities
- Ability to collaborate effectively well developed inter personal relationships with all levels in the organization and outside contacts.
- Outstanding written and verbal communication skills.
- High Collaboration & a perseverance to drive performance & change
Additional information
Key Competencies
- Distributed computing systems
- Experience with CI/CD tools such as Jenkins or Github Actions
- Experience with Python programming
- Working knowledge of Docker & Kubernetes
- Experience in developing data pipelines using spark & scala.
- Experience in debugging pipeline issues.
- Experience in writing python and shell scripts.
- InDepth Knowledge of SQL and Other Database Solutions
- Having a strong understanding of Apache Hadoopbased analytics
- Hands on experience on InteliJ Github /Bitbucket HUE.
Soft Skills:
Good Communication Skills
Ability to use extensively the latest IT communication tools
( Skype /MS TEAMS etc.)
Decent Level in English as Business Language
Remote Work :
No
Employment Type :
Fulltime
Employment Type
Full-time
