Lead Data Engineer
Lead Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Our data team has expertise across engineering analysis architecture modeling machine learning artificial intelligence and data science. This discipline is responsible for transforming raw data into actionable insights building robust data infrastructures and enabling datadriven decisionmaking and innovation through advanced analytics and predictive modeling.
A Lead Data Engineer designs implements and optimises scalable data pipelines and architectures. This role bridges raw data and actionable insights ensuring robustness performance and data governance. Collaboration with analysts and scientists is central to delivering highquality solutions aligned with business objectives.
Key Responsibilities
- Data Pipeline Development
- Architect and maintain realtime and batch data pipelines to handle large datasets efficiently.
- Employ frameworks such as Apache Spark Databricks or Airflow to automate ingestion transformation and delivery.
- Data Integration & Transformation
- Work with Lead Data Analysts to understand sourcetotarget mappings and quality requirements.
- Build ETL/ELT workflows validation checks and cleaning steps for data reliability.
- Automation & Process Optimisation
- Automate data reconciliation metadata management and errorhandling procedures.
- Continuously refine pipeline performance scalability and costefficiency.
- Collaboration & Leadership
- Coordinate with Data Scientists Data Architects and Analysts to ensure alignment with business goals.
- Mentor junior engineers and enforce best practices (version control CI/CD for data pipelines).
- Governance & Compliance
- Apply robust security measures (RBAC encryption) and ensure regulatory compliance (GDPR).
- Document data lineage and recommend improvements for data ownership and stewardship.
Qualifications :
Key Skills & Competencies
- Technical Skills
- Programming: Python SQL Scala Java.
- Big Data: Apache Spark Hadoop Databricks Dask.
- Cloud: AWS (Glue Redshift) Azure (Synapse Data Factory) GCP (BigQuery Dataflow).
- Data Modelling & Storage: Relational (PostgreSQL SQL Server) NoSQL (MongoDB Cassandra) Dimensional modelling.
- DevOps & Automation: Docker Kubernetes Terraform CI/CD pipelines for data flows.
- Architectural Competencies
- Data Modelling: Designing dimensional relational and hierarchical data models.
- Scalability & Performance: Building faulttolerant highly available data architectures.
- Security & Compliance: Enforcing rolebased access control (RBAC) encryption and auditing.
- Soft Skills
- Problem Solving: Ability to debug and resolve data pipeline failures efficiently.
- Collaboration: Works crossfunctionally with business and technology teams.
- Leadership: Guides junior engineers enforces best practices in data engineering.
Additional Information :
Discover some of the global benefits that empower our people to become the best version of themselves:
- Finance: Competitive salary package share plan company performance bonuses valuebased recognition awards referral bonus;
- Career Development: Career coaching global career opportunities nonlinear career paths internal development programmes for management and technical leadership;
- Learning Opportunities: Complex projects rotations internal tech communities training certifications coaching online learning platforms subscriptions passiton sessions workshops conferences;
- WorkLife Balance: Hybrid work and flexible working hours employee assistance programme;
- Health: Global internal wellbeing programme access to wellbeing apps;
- Community: Global internal tech communities hobby clubs and interest groups inclusion and diversity programmes events and celebrations.
Our diversity makes us stronger it drives meaningful change and enables us to build innovative technology solutions. We are committed to creating an inclusive community where all of us regardless of background identity or personal characteristics feels valued respected and free from discrimination. As an equal opportunity employer we welcome applications from all individuals and base hiring decisions on merit skills qualifications and potential.
Remote Work :
No
Employment Type :
Fulltime
Employment Type
Full-time
