Data Engineer (PythonPySpark)
Job Summary
Job Summary
Synechron is seeking a Data Engineer with 5 years of experience building data marts ETL pipelines and large-scale data processing solutions. The role will develop reliable data products using Python PySpark SQL and data warehousing technologies while supporting the full software development lifecycle.
Software Requirements
Required
- Python: 5 years of development experience.
- PySpark/Spark: 3 years of experience.
- Oracle SQL: 3 years including query optimization and data analysis.
- Pandas and Jupyter Notebook.
- ETL development and data warehousing.
- Hadoop Hive and MapReduce.
- SQL and NoSQL databases.
- Git CI/CD tools automated testing and data validation.
- Experience supporting development UAT production deployment and production issues.
Preferred
- Cloud-based data platforms.
- Workflow orchestration tools.
- Data quality and monitoring tools.
- Banking or financial services data experience.
Overall Responsibilities
- Design develop and maintain scalable ETL pipelines using Python and PySpark.
- Build and optimize data marts and transformation processes.
- Process structured semi-structured and unstructured data.
- Analyze and resolve data issues using Oracle SQL PySpark and data analysis techniques.
- Support development testing UAT deployment and production operations.
- Collaborate with engineering QA business and operations teams.
- Apply coding standards testing practices and CI/CD processes.
- Improve data quality pipeline performance reliability and maintainability.
Technical Skills (By Category)
Programming Languages
- Essential: Python PySpark Spark SQL and Pandas.
- Preferred: Scala or another data-processing language.
Databases and Data Management
- Essential: Oracle SQL and NoSQL databases data marts data warehousing ETL data validation and query optimization.
- Preferred: Data modeling partitioning indexing metadata management and data quality frameworks.
Cloud Technologies
- Essential: Understanding of distributed data environments and production data platforms.
- Preferred: Experience with cloud storage compute managed Spark services and cloud data warehouses.
Frameworks and Libraries
- Essential: PySpark Apache Spark Pandas Hadoop Hive MapReduce and Jupyter Notebook.
- Preferred: Workflow orchestration streaming and data quality frameworks.
Development Tools and Methodologies
- Essential: Git CI/CD Agile delivery code reviews automated testing debugging deployment and production support.
- Preferred: Infrastructure as code monitoring alerting and automated release management.
Security Protocols
- Essential: Secure handling of data access controls secrets management encryption and data privacy practices.
- Preferred: Experience with audit controls and regulated data
- environments.
Experience Requirements
- 5 years of experience in data engineering or software development.
- Proven experience developing ETL pipelines data marts and large-scale data processing solutions.
- Experience with Python PySpark Spark SQL Oracle Hadoop and Hive.
- Experience supporting the complete SDLC including UAT deployment bug fixes and production support.
- Banking or financial services experience is preferred.
- Equivalent experience in data development analytics engineering platform engineering or production data support may be considered.
Day-to-Day Activities
- Develop and enhance Python and PySpark data pipelines.
- Build data marts and optimize transformations and SQL queries.
- Investigate data quality issues and pipeline failures.
- Review logs job results database records and performance metrics.
- Create unit integration and data validation tests.
- Support UAT deployments releases and production incidents.
- Participate in planning requirements discussions code reviews defect triage and retrospectives.
- Document data flows technical decisions support procedures and known issues.
- Recommend improvements to pipeline design performance testing and reliability.
Qualifications
- Bachelors degree in Computer Science Information Technology Engineering or a related field or equivalent practical experience.
- Certifications in data engineering cloud database or big-data technologies are preferred.
- Complete Synechron training on security data protection SDLC and operational procedures.
- Maintain current knowledge of Python Spark data architecture cloud platforms data quality and engineering practices.
Professional Competencies
- Structured problem-solving and data investigation.
- Ability to analyze complex data issues and identify root causes.
- Clear communication with technical and business stakeholders.
- Effective collaboration across development QA product and operations teams.
- Ability to prioritize development support and production issues.
- Commitment to quality documentation testing and maintainable code.
- Willingness to learn new data technologies and improve existing processes.
SYNECHRONS DIVERSITY & INCLUSION STATEMENT
Diversity & Inclusion are fundamental to our culture and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity Equity and Inclusion (DEI) initiative Same Difference is committed to fostering an inclusive culture promoting equality diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger successful businesses as a global company. We encourage applicants from across diverse backgrounds race ethnicities religion age marital status gender sexual orientations or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements mentoring internal mobility learning and development programs and more.
All employment decisions at Synechron are based on business needs job requirements and individual qualifications without regard to the applicants gender gender identity sexual orientation race ethnicity disabled or veteran status or any other characteristic protected by law.
Required Experience:
IC
About Company
Chez Synechron, nous croyons en la puissance du numérique pour transformer les entreprises en mieux. Notre cabinet de conseil mondial combine la créativité et la technologie innovante pour offrir des solutions numériques de premier plan. Les technologies progressistes et les stratégie ... View more