Data Engineer

Role Title: Data Engineer

Job Family Software Engineer

Skill Type Python/SQL/AWS

Minimum Requirements:

Bachelors degree in computer science information systems software engineering or a related field (foreign equivalent degree acceptable) plus

three (3) years of experience in: software development technology using Java and SQL to write data pipelines/ETL/ELT designing OLAP and S3 object store data architecture

Two(2) years of experience with/in: AWS DevOps PythonSpark

Some experience with/in: Test Automation CICD Pipeline Development

Job Description:

Design develop test and support highly complex data engineering software solutions for analytical data storage (AWS Data Lake Data Warehouse) by providing necessary design and technical expertise.

Understand the big picture in an EndtoEnd data environment and understand upstream system data and downstream consumption requirements.

Adopt standard platform tools for source code management CICD Orchestration monitoring and alerting (gitlab airflow jenkins etc).

Work on AWS cloud infrastructure in a DevOps capacity including automating the deployment and scaling of EMR EC2 S3.

Participate in technical planning & requirements gathering phases including discovery design code test and documentation.

Demonstrate development best practices to guarantee the quality and sustainability of software deliverables.