Data Engineer

Position : Data Engineer

Location: Bay Area CA

Note : Look for candidate form Tier1 company only who works on large Data Sets.

Strong experience building robust and scalable data processes and pipelines for modeling, analysis, and reporting

Experience with Enterprise Data Warehousing systems including Snowflake and Teradata (or equivalent)

Fluency in SQL and Python, including data wrangling and schema design

Experience working with pipeline tools like Airflow and dbt

Experience with BI processes and some experience with dashboard tools like Tableau

Some experience with CI/CD and containerization tools (e.g. Jenkins, Docker, Kubernetes)

Ability to initiate, refine, and complete projects with minimal guidance and some experience working in a scrum or release cycle environment

Ability to clearly communicate technical concepts, definitions, logic, and processes to a non-technical audience

Ability to think critically and collaborate cross-functionally with other data engineering, data science and analytics stakeholders distilling business requirements into clear data products

Experience designing data ingestion processes and working with unstructured data a plus

Project Description

Design, create, refine, and maintain data processes and pipelines used for modeling, analysis, and reporting

Operationalize data products with detailed documentation, automated data quality checks and change alerts

Support data access through various sharing platforms, including dashboard tools

Troubleshoot failures in data processes, pipelines, and products

Communicate and educate consumers on data access and usage, managing transparency in metric and logic definitions

Collaborate with other data scientists, analysts, and engineers to build full-service data solutions

Develop and communicate architectures, code patterns and data structure design choices to team of data scientists, analysts and engineers laying out tradeoffs

Support codebase compatibility with Snowflake by designing, creating, and driving adoption of templates, packages, and best practices

Optimize query and database performance through designing, creating, refining, and maintaining performance management system

Work with cross-functional business partners and vendors to acquire and transform raw data sources

Design, create, refine, and maintain data ingestion process

Provide weekly updates to the team on progress and status of planned work