SPARK Data Reconciliation Engineer

Job Position: SPARK Data Reconciliation Engineer
Location: Jersey City NJ
Type of Employment: Fulltime
Minimum Years of Experience: 6
Job Role: Developing and implementing PySparkbased applications to perform complex data reconciliations in financial systems.
Key Responsibilities:
Designing developing and testing PySparkbased applications for data reconciliation.
Implementing efficient data transformation and matching algorithms.
Developing error handling and exception management systems within Spark jobs.
Collaborating with business analysts and data architects to understand data requirements.
Analyzing and interpreting data structures formats and relationships.
Working with distributed datasets in Spark.
Integrating PySpark applications with rules engines.
Collaborating with crossfunctional teams to identify and analyze data gaps.
Designing and developing PySparkbased solutions to address data integration challenges.
Contributing to the development of data governance and quality frameworks.
Qualifications and Skills:
Bachelors degree in Computer Science or a related field.
5 years of experience in big data development.
Proficiency in PySpark Apache Spark and related big data technologies.
Experience with rules engine integration and development.
Strong analytical and problemsolving skills.
Excellent communication and collaboration skills.
Familiarity with data streaming platforms and big data technologies.