Tips: Provide a summary of the role what success in the position looks like and how this role fits into the organization overall.
Responsibilities
Be specific when describing each of the responsibilities. Use genderneutral inclusive language.
Example: Determine and develop user requirements for systems in production to ensure maximum usability
Qualifications
Requirements
- Graduate degree in Computer Science Statistics Informatics Information Systems or another quantitative field.
- 24 Years experience as a data engineer.
- 2 Years experience using Azure Services like Data Lake Data Factory Databricks etc.
- Experience with big data tools: Hadoop Spark Kafka etc.
- Experience with relational SQL and NoSQL databases including Postgres MongoDB Cassandra etc.
- Experience with data pipeline and workflow management tools: Azkaban Luigi Airflow etc.
- Experience with streamprocessing systems: Storm SparkStreaming etc.
- Experience with objectoriented/object function scripting languages: Python Java Scala etc.
- Understanding of infrastructure (including hosting containerbased deployments and storage architectures) would be advantageous.
- Understanding of Lambda and Kappa architecture patterns preferred.
data,storm,hadoop,scala,mongodb,python,databases,azure,nosql,azkaban,kappa architecture,architecture,azure services,data lake,spark-streaming,sql,data factory,java,luigi,lambda,postgres,databricks,data engineering,airflow,cassandra,kafka,big data,infrastructure,spark,sql azure