Data Engineer--Scala development
Data Engineer--Scala development
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Job Overview:
As a Scala Developer in our team you work with large scale manufacturing data coming from our globally distributed plants. You will focus on building efficient scalable & datadriven applications that among other use cases connect IoT devices preprocess standardize or enrich data feed ML models or generate alerts for shopfloor operators.
The data sets produced by these applications whether data streams or data at rest need to be highly available reliable consistent and qualityassured so that they can serve as input to downstream applications such as shopfloor reporting alerting apps ML apps / models and a wide range of users spanning analysis and data scientists.
We run these applications on a Kubernetes based edge data platform in our plants. The platform is currently in rampup phase so apart from building applications you will also contribute to scaling the platform including topics such as automation and observability.
Finally you are expected to interact with customers and other technical teams e.g. for requirements clarification & definition of data models.
Primary responsibilities:
- Develop deploy and operate data processing applications running on Kubernetes written in Scala (we leverage Kafka for messaging Spark KStreams and ZIO for data processing PostgreSQL and S3 for storage)
- Contribute to rampup of our edge data processing platform incl. topics such as deployment automation building CI/CD pipelines (we use Github Actions ArgoCD) and evaluation of platform extensions
- Discuss requirements with stakeholders such as customers or up and downstream development teams
- Derive design proposals incl. meaningful data models
- Engage in design discussions with team members architects & technical leadership
- Review code contributed by other team members
- Depending on experience mentor junior team members
Qualifications :
Qualifications
- Bachelors degree in computer science Computer Engineering relevant technical field or equivalent
- 35 years of experience in data engineering and / or backend development
Additional Information :
Key Competencies
- Experience developing software in a JVMbased language. Scala preferred but Java Kotlin or Clojure also accepted.
- Experience with datadriven backend software development
- Experience with objectoriented & functional programming principles
- Deep level of understanding in distributed systems for data storage and processing (e.g. Kafka ecosystem Spark Flink HDFS S3)
- Experience with RDBMS (e.g. Postgres)
- (optional) prior experience with functional stream processing libraries such as fs2 ziostreams or akka streams
- Excellent software engineering skills (i.e. data structures & algorithms software design)
- Excellent problemsolving investigative and troubleshooting skills
- Experience with CI/CD tools such as Jenkins or Github Actions
- Comfortable with Linux and scripting languages for workflow automation
Soft Skills:
Good Communication Skills
Ability to use extensively the latest IT communication tools
( Skype /MS TEAMS etc.)
Decent Level in English as Business Languagew
Remote Work :
No
Employment Type :
Fulltime
Employment Type
Full-time
