Data Engineer with DevOps expertise
Data Engineer with DevOps expertise
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
We now have a new need and are therefore looking for a consulting colleague with the qualifications below. The position is part of our consulting services which means you will be employed by us and work with clients or internal projects and assignments.
Job Description
We are looking for a skilled Data Engineer with DevOps expertise to support the design development and automation of data products on our Azurebased Data Hub. This role will focus on building and maintaining infrastructure using Terraform automating workflows with Azure DevOps and developing data pipelines with Python and SQL. This consultant will work closely with crossfunctional teams to ensure the smooth operation and scalability of our data infrastructure.
Objectives:
Develop and maintain data pipelines and infrastructure in Azure.
Automate deployment processes using Azure DevOps and Terraform.
Ensure data products are scalable secure and optimized for performance.
Tasks and Responsibilities:
Pipeline Development:
Collaborate with data product designers to implement and optimize data models and pipelines using SQL and Python.
Programming and Code Quality: Work independently and alongside data product designers to write test and optimize code (SQL and Python) locally before committing to source control. Ensure that code meets high standards of quality and efficiency with a proactive approach to troubleshooting and debugging.
Pipeline Implementation: Collaborate with data modeling and designfocused engineers to implement and optimize data models and pipelines on the Databricks platform ensuring seamless integration and alignment with business needs.
Ensure data pipelines are efficient reliable and scalable on the Databricks platform.
Infrastructure as Code (IaC):
Implement infrastructure as code using Terraform to manage and provision Azure resources.
Ensure infrastructure is consistent reliable and capable of supporting largescale data operations.
CI/CD Automation:
Automate and manage CI/CD pipelines through Azure DevOps to ensure smooth deployments and minimal downtime.
Monitor data workflows and troubleshoot any issues that arise in the deployment or pipeline processes.
CrossFunctional Collaboration:
Work with development operations and data teams to ensure alignment on data infrastructure and operations.
Collaborative Development: Work closely with data product designers modelers and other engineers focused on specific areas such as data modeling design or architecture. Share ideas best practices and solutions to ensure alignment on data infrastructure and operations.
Contribute to best practices around cloud architecture security and data governance.
Deliverables:
Fully functional data pipelines on the Databricks platform.
Infrastructureascode solutions using Terraform.
Automated CI/CD pipelines through Azure DevOps.
Documentation of the infrastructure and deployment processes for crossteam visibility and ongoing maintenance.
Your Profile (Musthave requirements)
Handson experience with Python and SQL for building and managing data pipelines.
Proficiency in Terraform for infrastructure as code particularly with Azure resources.
Strong knowledge of Azure DevOps for managing CI/CD pipelines and automating deployments.
Familiarity with Databricks and the Azure ecosystem.
Solid understanding of cloudnative architectures and data processing frameworks.
Experience with collaborative work in multidisciplinary teams particularly with data modelers and designers.
Strong problemsolving skills and attention to detail.
Excellent communication and teamwork abilities.
Application deadline:
About Us
We currently employ experienced consultants in areas such as System Development AI Machine Learning Testing Cloud Infra DevOps and IT Project Management.
Working at Deploja is something entirely different from working at an ordinary consulting companyand we are really proud of that. So what makes us unique
We simply offer more than just a high salary and greater freedom; thats just the beginning. With us youll find the work environment and opportunities you deserve. Welcome to Deploja where the grass is indeed greener!
Vi erbjuder dig fljande Valet r ditt!
Exciting assignments with our many clients. We focus on longterm projects which gives you the opportunity to create value for the client while also developing yourself.
A secure monthly salary together well agree on a base salary depending on your assignments.
Company car we encourage you to choose an electric car for a more sustainable future.
Occupational pension and great opportunities for salary conversion.
Vacation Would you like more than 30 days of vacation perhaps 60 days
Private health insurance accident insurance and life insurance for increased security.
Skills development you set your own budget and choose the training you wish to attend.
Quality of life Are you struggling to balance everyday life and wish to work a little less while still earning as much if not more
We handle applications continuously and due to the high workload its especially important to include a short motivation for the position explaining how you meet the requirements.
Employment Type
Full Time
