AWS Big Data Engineer

Job Title: AWS Big Data Engineer

Job Location: Portland OR (Onsite)
Contractor Bill rate: $65/h

Responsibilities:
(The primary tasks functions and deliverables of the role)
Design and build reusable components frameworks and libraries at scale to support analytics products
Design and implement product features in collaboration with business and Technology stakeholders
Identify and solve issues concerning data management to improve data quality
Clean prepare and optimize data for ingestion and consumption
Collaborate on the implementation of new data management projects and restructure of the current data architecture
Implement automated workflows and routines using workflow scheduling tools
Build continuous integration testdriven development and production deployment frameworks
Analyze and profile data for designing scalable solutions
Troubleshoot data issues and perform root cause analysis to proactively resolve product and operational issues

Requirements:

Experience:
Strong understanding of data structures and algorithms
Strong understanding of solution and technical design
Has a strong problem solving and analytical mindset.
Able to influence and communicate effectively both verbally and written with team members and business stakeholders
Able to quickly pick up new programming languages technologies and frameworks
Experience building cloud scalable real time and highperformance data lake solutions
Fair understanding of developing complex data solutions
Experience working on endtoend solution design
Willing to learn new skills and technologies
Has a passion for data solutions


Required and Preferred Skill Sets:
Hands on experience in Databricks and AWS EMR Hive Pyspark S3 Athena.
Familiarity with Spark Structured Streaming
Minimum 4 years experience working experience with Hadoop stack dealing huge volumes of data in a scalable fashion
10 years of handson experience with SQL ETL data transformation and analytics functions
8 years of handson Python experience including Batch scripting data manipulation distributable packages
4 years experience working with batch orchestration tools such as Apache Airflow or equivalent preferable Airflow
4 years working with code versioning tools such as GitHub or BitBucket; expert level understanding of repo design and best practices
Familiarity with deployment automation tools such as Jenkins
8 years of handson experience designing and building ETL pipelines; expert with data ingest change data capture data quality; hand on experience with API development
4 years designing and developing relational database objects; knowledgeable on logical and physical data modelling concepts; some experience with Snowflake
Familiarity with Tableau or Cognos use cases
Familiarity with Agile; working experience preferred