Data engineer

Job Description
We are seeking an experienced Software Engineer to help develop the next generation data ingest infrastructure. You will be working on exciting technologies like Kafka Zookeeper Scala and event driven architecture. Your work will help keep data up to date.

Responsibilities
Design and develop a platform that allows users to configure and run complex non linear data pipelines.
Develop primitives to allow users easily leverage multiple compute paradigms (Scala Kafka SQL etc.) in a single workflow.
Develop dashboards to monitor pause and terminate the data pipelines in flight.
Maintain and manage the operations of the pipeline platform.

Minimum qualifications
BS in computer science or equivalent.
5 years developing infrastructure for asynchronous batch processing pipelines.
5 years of proficiency in Java or Scala.
Excellent analytical and problem solving skills; excellent communication skills.

Preferred qualifications
Track record of building automation infrastructure to orchestrate hundreds of concurrent data pipelines spanning SQL NoSQL and inmemory subsystems.
Fluent in Scala development frameworks such as Play Akka Docker and Kubernetes.
Working knowledge of NoSQL frameworks like Kafka Cassandra Solr or Elastic Search.
Expertise in a scripting language such as Python.
Application development experience with relational databases (e.g. Postgres) and NoSQL databases (e.g. Cassandra).