Data Onboarding Engineer

Role: Data Onboarding Engineer

Locations: Jersey City NJ

Duration: Fulltime

Note: You need to look for ETL Developers. 78 Years of experience is good to onboard data into a Data Lake.

Skills: ETL Data Engineer Data Lake Hadoop

Essential Skills & Experience

Spark & Scala Expertise:

Strong foundation in Scala: The primary language for Spark development so a solid understanding of its syntax data structures functional programming and common libraries is critical.

Extensive Spark experience: Deep knowledge of Spark core concepts (RDDs DataFrames Datasets) Spark SQL and Spark Streaming for building robust and performant data pipelines.

Familiarity with Spark cluster deployment: Understanding how to configure and run Spark applications in a cluster environment (e.g. YARN Mesos).

ETL Development:

Data Extraction: Experience with various data sources (relational databases APIs flat files etc.) and techniques for extracting data from them.

Data Transformation: Mastery of data manipulation techniques using Spark (aggregation filtering joining etc.) to prepare data for analysis and loading.

Data Loading: Experience with loading data into various target systems including Hive tables databases and data lakes.