Data Engineer

Job Description

The role requires data ingestion processing and extraction of data to & from Hadoop Cluster only
using Informatica BDM. It requires extensive experience in ETL SQL Scoop Spark Hive Hadoop &
KAFKA. Knowledge of PySpark will be preferable

Job Title Data Engineer

Experience Minimum 45 years of Experience in Development in
Informatica Big Data Management/ DEI & Informatica Power
Center
Extensive experience in delivering data pipelines using
Informatica on Big Data Clusters
Hands on experience with extraction of unstructured and semi
structured data using data processor transformation
Knowledge of PySpark will be high preferable
Full knowledge of designing and creating complex data models

Education Skillsets &
Qualifications

Bachelors degree in Computer Science or equivalent; Masters
preferred Programming
Solid programming skills in SQL
Deep Knowledge in data architecture designing workflows
building ingestion framework
Expertise of working on Enterprise Big data distribution like
Cloudera. Exposure to big data components HDFS HIVEQL
Spark HBase Impala is essential

Main Duties & Responsibilities Building and optimizing data pipelines architectures and data sets;
both batchoriented and real time using Informatica BDM from
both relational & non relational DBs
Design develop and customize mappings/workflows for different
types of loads using Informatica BDM
Leverage various transformations in Informatica BDM tool to work
on data ingestion data profiling & data quality
Leverage Hadoop ecosystem knowledge to design and develop
capabilities to deliver our solutions
Support and play a pivotal role as part of data engineering team to
build the data lake on Cloudera for Enterprise analytics.
Work hands on with Spark & KAFKA in addition to Informatica
platform
Design recommend best approach suited for data movement from
different sources to HDFS
Ensure optimum performance of the Informatica mappings and
workflow
Work with dynamic cross functional teams including business
Solution architects & Data engineers