Employer Active
Job Alert
You will be updated with latest job alerts via emailJob Alert
You will be updated with latest job alerts via email
Position - Data Engineer -
Location -USA, PA (Up to 2 months remote, later on-site is required)
Experience-9+ Years
C2H
Must Haves:
2+ years of recent experience with Data bricks / Spark / Snowflake and a total of 6+ years in data engineering role.
Designing and implementing highly performant data ingestion pipelines from multiple sources using spark and databricks.
Extensive working knowledge of Spark and Databricks
Demonstrable experience designing and implementing modern data warehouse/data lake solutions with an understanding of best practices.
Hands-on development experience with Snowflake data platform including Snowpipes, SnowSQL,tasks, stored procedures, streams, resource monitors, RBAC controls, virtual warehouse sizing, query performance tuning, cloning, time travel, data sharing and understanding how to use these features.
Advanced proficiency in writing complex SQL statements and manipulating large structured and semi-structured datasets.
Data loading/Unloading and Data sharing
Strog hands-on Experience on SNOWSQL queries, script preparation and stored procedures and performance tunning
Knowledge of SnowPipe implementation
Create Spark jobs for data transformation and aggregation
Produce unit tests for Spark transformations and helper methods
Security design and implementation on Databricks
Build processes supporting data transformation, data structures, metadata, dependency and workload management.
A successful history of manipulating, processing and extracting value from large disconnected datasets.
Working knowledge of message queuing, stream processing, and scalable 'big data' data stores.
Full Time