Senior Data Engineer

Senior Data Engineer (Remote)

We are looking for a Senior Data Engineer to join a team to work on data pipeline development. You would work closely with small highly collaborative Operations and Development teams to architect and deploy highimpact security solutions.

You must be able to work US time zones (UTC8 to UTC5/UTC7 to UTC4).

Job Responsibilities

• Build data management pipelines from heterogeneous data sources into Data Marts and other repositories for subsequent ysis and data mining
• Code test and support Pythonbased services implementing ETL/ELT data cleansing etc.
• Design and use RESTful APIs develop shippable code doentation and unit test new features for Cloudbased products
• Collaborate with Quality Product and other Engineering teams
• Provide code reviews design feedback demos technical requirements & doentation
• Scope projects provide accurate estimates for reliable delivery of projects
• Work closely with the DevOps team on the IAC setup for BI

Key Applicant Requirements

• Professional development experience using Python including an understanding of functional programming: 5 years
• Cloud infrastructure experience in AWS Docker: 3 years
• Webscale data engineering experience using enterprise relational databases: 3 years
• Experience with Apache Spark especially SparkCore and SparkSQL: 3 years
• CI/CD experience and DevSecOps: 2 years
• Infrastructure As Code (Terraform) experience: 1 years
• Experience in Apache Airflow to orchestrate Spark jobs
• Strong proficiency in Python for Data Science applications
• Proficient with relational databases such as Oracle MySQL PostgreSQL
• Experience with AWS services such as RDS Redshift S3 EMR
• Experience with designing and building Data Marts and ETL/ELT
• Proficient with Jira Confluence and GIT toolset; IDEs like PyCharm or Visual Studio Code
• Experience with CI/CD solutions message queue systems and background task management
• Sed in Git and use it for version control
• Handson experience with Agile/Scrum
• Excellent written & verbal communication ss
• Ability to coordinate between teams
• Bachelors degree or higher in Computer Science/related field or equivalent experience

Pluses

• Design implementation and/or maintenance of CICD Bash scripts
• Experience with using AWS EMR Serverless
• AWS Certified Data ytics Specialty
• Databricks Certified ociate Developer for Apache Spark
• Experience with serverless technology such as Amazon Lambda or Google Compute Engine
• Experience with TeamCity and Octopus

Interview Process

We will ist you with preparation including mock interviews and coaching to succeed! The steps typically are:

• Prescreen with recruiters
• Technical interview with the development team