Enter a job title or keyword

Lead Data Engineer


Job Location:

Bedford - UK

Yearly Salary: GBP 81500 - 81500
Posted: 20 November 2026 (16 hours ago)
Application Deadline: 17 February 2027
Vacancies: 1 Vacancy

Job Summary

About us

Lead Data Engineer
81500 per annum
Bedford
Permanent
Hybrid working

Join us at the Money and Pensions Service (MaPS) as a Lead Data Engineer. The Lead Data Engineer provides technical leadership for the architecture development and operation of the MaPS data platform. Working with the Principal Data Engineer you will translate strategic platform goals into practical architectural decisions engineering approaches and a prioritised technical roadmap enabling secure reliable and reusable data products across MaPS.

You will lead the technical evolution of the platform including migration from Azure Synapse to Microsoft Fabric and support for expanding data needs. You will own data engineering technical standards and solution patterns ensuring that platform development balances delivery needs with security maintainability performance and cost.

As part of a larger data engineering team you will manage lead and mentor Data Engineers building capability and shared ownership of engineering quality. This is a hands-on leadership role: you will guide technical delivery resolve complex engineering problems and develop reusable platform capabilities rather than focusing solely on individual project solutions.

You will work with architecture governance security DataOps analysts and delivery stakeholders to turn business requirements into sustainable data solutions. You will challenge constructively explain technical choices and trade-offs clearly and ensure that architectural decisions and engineering practices support MaPS services and the people who use them.

The role
  • Lead data platform architecture and detailed technical design working within agreed enterprise architecture and governance.
  • Translate strategic goals into implementation options technical decisions delivery sequencing and a platform development roadmap agreed with the Principal Data Engineer.
  • Own document and maintain data engineering technical standards reference architectures and reusable patterns including data layering modelling naming ingestion transformation testing and deployment. Embed their use through design reviews code reviews and technical assurance.
  • Develop the platforms technology and shared capabilities. Evaluate new features through proportionate investigation and proofs of concept recommending adoption based on business value security supportability performance and cost rather than technology alone.
  • Lead technical delivery across multiple workstreams resolving cross-project dependencies and ensuring solutions contribute to the shared platform rather than creating unnecessary duplication.
  • Line manage allocated Data Engineers setting clear objectives supporting development and wellbeing and managing workload and delivery expectations. Coach and mentor engineers across the wider team delegate meaningful technical ownership and build shared capability.
  • Remain hands-on in designing and developing complex pipelines integrations and data models. Translate requirements into reusable tested production-ready solutions and provide technical direction and constructive review for other engineers.
  • Work with data owners suppliers analysts and governance and security colleagues to define data contracts quality controls access models metadata lineage and retention requirements. Implement appropriate engineering controls for secure and ethical data use and sharing applying open standards and open-by-default principles where appropriate.
  • Embed disciplined engineering practices including source control automated testing CI/CD release management and environment configuration. Maintain clear architectural decisions technical documentation and reusable implementation guidance.
  • Design for operational ownership with DataOps including monitoring structured logging alerting recovery and support runbooks. Lead resolution of complex technical issues address root causes and manage technical debt reliability capacity and cost improvements.
Key accountabilities
  • Strong experience designing building and operating Microsoft cloud data platforms including practical Microsoft Fabric experience and an understanding of Azure Synapse and Azure Data Factory in migration and integration scenarios.
  • Experience taking responsibility for data platform architecture and technical design translating strategic objectives into implementation roadmaps and explaining decisions and trade-offs to technical and non-technical stakeholders.
  • Experience defining and embedding engineering standards reusable frameworks and solution patterns across multiple engineers and delivery workstreams.
  • Experience managing leading and mentoring Data Engineers including setting objectives supporting professional development delegating responsibility and building a collaborative engineering culture.
  • Advanced SQL and strong Python/PySpark skills with practical experience of Spark lakehouse and warehouse architectures layered data processing and analytical data modelling.
  • Experience delivering robust ingestion integration and transformation pipelines using APIs files and databases including incremental processing schema change error handling and automated data quality validation.
  • Experience of platform migration or significant modernisation including dependency assessment reconciliation testing and controlled transition to production.
  • Practical understanding of cloud identity and access management data protection governance metadata lineage retention and secure data sharing including how to implement these through engineering controls.
  • Experience of source control code review automated testing and CI/CD with clear technical documentation and effective agile delivery practices.
  • Experience designing for production support including observability incident investigation recovery and operational handover and improving platform reliability performance and cost.
  • Ability to work with business stakeholders architects governance security and operational teams to clarify requirements challenge assumptions and agree sustainable solutions.
  • Ability to evaluate emerging platform capabilities assess risks and benefits and make proportionate evidence-based technology recommendations.
  • Ability to coordinate technical delivery across a larger team balancing immediate delivery commitments with platform development technical debt and longer-term goals.
Other

About Us
The Money and Pensions Service (MaPS) is based in Bedford in a recently renovated modern office. This is a wonderful opportunity for you to become an integral part of a dynamic organisation working to help people across the UK.

At the heart of the Money and Pensions Service are our values caring connecting and transforming which are the foundation of our success. They permeate every area of our work and define all our business relationships and the way we work with each other. Were not only looking for the best people to come and work for us but we need people who align themselves with our values:

  • Caring
    We care about our colleagues and the people whose lives we are here to transform.
  • Connecting
    We will transform lives through our ability to make positive connections.
  • Transforming
    We are committed to transforming lives and making a positive societal impact.

Our Inclusive Working Environment
By fostering our values we are immensely proud of the inclusive working environment that we have created. The diversity of our people is a strength that we embrace and wish to build upon so we are committed to attracting people of all backgrounds. We work hard to ensure that we have a progressive approach to inclusion equity and belonging. We really do want our colleagues to bring their whole selves to work.

Our colleague and ally networks encompass LGBTQ neurodiversity womens health mens health ethnicity and diversity.

What We Offer

  • Generous Annual Leave 30 days plus Bank Holidays
  • Pension scheme contributions matched 2 to 1 (employer contribution up to 10% of your salary)
  • Interest-free loan for season tickets for buses and trains
  • Cycle to work Scheme
  • Subsidised eye tests & flu jabs
  • Life assurance scheme
  • Give as you earn scheme
  • Employee assistance programme (EAP)
  • PAM Assist and PAM Life scheme (Wellbeing)
  • Enhanced family and sick pay
  • Paid volunteering (2 days a year)
  • Recognition Scheme
  • Discounts portal to numerous retailers

Flexible Working
At MaPS we take pride in our flexible approach to work. Hybrid working is a voluntary non-contractual arrangement and our headquarters in Bedford will be your contractual place of work. The number of days that anyone will be able to work at home will be determined primarily by business needs but personal and other relevant circumstances will also be considered. If you are successful any opportunities for hybrid working including whether a hybrid working arrangement is suitable for you will be discussed with you prior to you taking up your post.

Compliance

Recruitment Process:

  1. Initial Review:Our recruiter will review your CV. If it looks like a good fit theyll reach out to arrange a call.
  2. Hiring Manager Review:If the recruiter thinks youre suitable your CV will be passed to the hiring manager who will decide whether to invite you for an interview.
  3. Interview:If selected youll participate in a virtual interview. This single-stage process allows you to demonstrate your skills and experience through competency-based questions.

Career Development
In MaPS we take career development seriously. We actively encourage and support applications from our existing MaPS colleagues. However we do follow the Civil Service Commissioner recruitment principles which means that you will be required to participate in a full open and fair process.

Reserve List
If you are successful at interview we operate a reserve list where your details will be held for up to 6 months. Should a vacancy come available in that time with the same essential criteria reserve list candidates will be offered that position with no further assessment required.

Application Process
The law requires that selection for appointment to the Civil Service is on merit on the basis of fair and open competition as outlined in the Civil Service Commissions Recruitment Principles. If you feel your application has not been treated in accordance with the Recruitment Principles and you wish to make a complaint in the first instance you should contact the Money and Pensions Service via email: If you are not satisfied with the response you receive from the Department you can contact the Civil Service Commission: Visit the Civil Service Commission website.

Job Reference: MaPS01405
Close Date: 1/12/2026


Required Experience:

Senior IC