DataOps Engineer
DataOps Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
The main responsibilities are:
Data modeling and pipelines development with Spark on Scala in order to ingest and transform data from several sources (Kafka topics APIs HDFS structured databases).
Data transformation and quality to ensure data consistency and accuracy.
Set up CI/CD pipelines to automate deployments unit testing and development management.
The implementation of different orchestrators and scheduling processes to automate the data pipeline execution (Airflow as a service).
Modifying the existing code as per business requirements and continuously improvement to achieve a better performance and maintainability.
Ensuring the performance and security of the data infrastructure and following the best practice of Data engineering.
Contributing to production support incident and anomaly corrections and to implement functional and technical evolutions to ensure the stability of production processes
Writing technical documentation to ensure knowledge capitalization.
Qualifications :
Good knowledge of
- Spark on Scala
- CI/CD tools (Gitlab Jenkins)
- HDFS and structured databases (SQL)
Full understanding of
- Apache Airflow
- Streaming process (Kafka event steam)
- S3 storage
- Shell script
Some knowledge of *
- Kubernetes
Optionally/ as a plus *
- Elasticsearch and Kibana
- HVault
- Dremio as tool to virtualize data
- Dataiku
Additional Information :
Qu podemos ofrecerte
- Formars parte de un gran equipo de profesionales con inquietud y motivacin por el desarrollo y la programacin.
- Trabajars a jornada completa con un horario flexible bajo un modelo de trabajo remoto.
- 22 das de vacaciones 2 de asuntos propios.
- Formacin por parte de la empresa para que puedas seguir desarrollndote y promocionar dentro del plan de carrera que existe para ti.
- Podrs asistir a eventos y conferencias relevantes del sector.
- Contrato indefinido.
- Estabilidad y buen clima laboral.
- Salario Competitivo.
- Acceso a ventajas del grupo de empresa
- Retribucin flexible y ms beneficios
Remote Work :
No
Employment Type :
Fulltime
Employment Type
Full-time
