Ab Initio Data Engineer

Job Description:
Work with business and technical leadership to understand requirements
Design to the requirements and document the designs
Ability to write product-grade performant code for data extraction, transformations and loading using Spark, Py-Spark, Ab Initio
Do data modeling as needed for the requirements
Write performant queries using Teradata SQL, Hive SQL and Spark SQL against Teradata and Hive
Implementing dev-ops pipelines to deploy code artifacts on to the designated platform/servers like AWS or Hadoop Edge Nodes
Implement Hadoop job orchestration using Shell scripting, Apache Oozie, CA7 Enterprise Scheduler and Airflow
Troubleshooting the issues, providing effective solutions and jobs monitoring in the production environment
Participate in sprint planning sessions, refinement/story-grooming sessions, daily scrums, demos and retrospectives
Skills:
Strong development experience in Spark, Py-Spark, Shell scripting, Teradata, Hive and Hadoop
Proven Experience of Ab Initio