Enter a job title or keyword

Python, AWS Services, Pyspark Developer

CGI


Job Location:

Hyderabad - India

Monthly Salary: Not provided by the employer
Posted: 21 November 2026 (8 hours ago)
Application Deadline: 18 February 2027
Vacancies: 1 Vacancy

Job Summary

Position Description:

Position: Python AWS Services Pyspark Developer
Experience: 5 Years
Technology: Python AWS Services Pyspark Developer
Work Location: Hyderabad/Bangalore/Chennai
Employment Type: Full-Time
Job ID : J1026-1382
#LI-SP40

Position Description
CGI is seeking an experienced Python AWS Services Pyspark Developer to design develop optimize and maintain scalable data engineering solutions on cloud platforms. The ideal candidate will have strong hands-on experience in Python programming distributed data processing using PySpark and building data pipelines using AWS services.
The candidate will be responsible for developing robust ETL/ELT pipelines processing large volumes of structured and unstructured data implementing data transformations and ensuring data quality reliability and performance. The role requires close collaboration with Data Engineers Data Architects Business Analysts and cross-functional teams to deliver enterprise-grade data solutions.

Your future duties and responsibilities:

Design develop test and maintain scalable data processing applications using Python and PySpark.
Build and maintain ETL/ELT pipelines for data ingestion transformation validation and integration.
Develop distributed data processing solutions using Apache Spark and PySpark to process large datasets efficiently.
Design implement and manage cloud-based data pipelines using AWS services.
Work with AWS services such as Amazon S3 AWS Glue AWS Lambda Amazon EMR Amazon Redshift AWS Step Functions and Amazon CloudWatch as applicable to the project.
Develop data extraction transformation and loading processes from multiple data sources into data lakes and data warehouses.
Implement data transformation logic business rules aggregations joins and data cleansing using Python and PySpark.
Optimize Spark jobs through partitioning caching join strategies and resource tuning to improve performance and reduce processing time.
Develop and maintain reusable Python modules libraries utilities and automation scripts.
Implement data validation reconciliation error handling logging and exception management within data pipelines.
Monitor pipeline execution troubleshoot failures and resolve performance bottlenecks in production environments.
Work with relational databases and data platforms using SQL for data extraction analysis validation and reconciliation.
Implement secure data processing practices using appropriate AWS IAM roles policies encryption and access controls.
Support CI/CD pipelines automated deployments version control and release management using tools such as Git and AWS-native or other DevOps solutions.
Collaborate with Data Architects Data Engineers DevOps teams and business stakeholders to translate requirements into technical solutions.
Participate in code reviews unit testing integration testing production deployments and technical documentation.
Perform root cause analysis and implement corrective actions for data pipeline failures and production incidents.
Ensure adherence to coding standards data governance requirements security policies and enterprise development practices.

Required qualifications to be successful in this role:

Design develop test and maintain scalable data processing applications using Python and PySpark.
Build and maintain ETL/ELT pipelines for data ingestion transformation validation and integration.
Develop distributed data processing solutions using Apache Spark and PySpark to process large datasets efficiently.
Design implement and manage cloud-based data pipelines using AWS services.
Work with AWS services such as Amazon S3 AWS Glue AWS Lambda Amazon EMR Amazon Redshift AWS Step Functions and Amazon CloudWatch as applicable to the project.
Develop data extraction transformation and loading processes from multiple data sources into data lakes and data warehouses.
Implement data transformation logic business rules aggregations joins and data cleansing using Python and PySpark.
Optimize Spark jobs through partitioning caching join strategies and resource tuning to improve performance and reduce processing time.
Develop and maintain reusable Python modules libraries utilities and automation scripts.
Implement data validation reconciliation error handling logging and exception management within data pipelines.
Monitor pipeline execution troubleshoot failures and resolve performance bottlenecks in production environments.
Work with relational databases and data platforms using SQL for data extraction analysis validation and reconciliation.
Implement secure data processing practices using appropriate AWS IAM roles policies encryption and access controls.
Support CI/CD pipelines automated deployments version control and release management using tools such as Git and AWS-native or other DevOps solutions.
Collaborate with Data Architects Data Engineers DevOps teams and business stakeholders to translate requirements into technical solutions.
Participate in code reviews unit testing integration testing production deployments and technical documentation.
Perform root cause analysis and implement corrective actions for data pipeline failures and production incidents.
Ensure adherence to coding standards data governance requirements security policies and enterprise development practices.

Skills:
  • Agile
  • AWS Cloudformation
  • AWS Glue
  • Communication
  • Python
  • SQLite
What you can expect from us:

Together as owners lets turn meaningful insights into action.

Life at CGI is rooted in ownership teamwork respect and belonging. Here youll reach your full potential because

You are invited to be an owner from day 1 as we work together to bring our Dream to life. Thats why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our companys strategy and direction.

Your work creates value. Youll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas embrace new opportunities and benefit from expansive industry and technology expertise.

Youll shape your career by joining a company built to grow and last. Youll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons.

That same commitment to fairness extends to how we use technology. To support our recruitment team AI tools may be used to help assess applications though they never replace human judgement. All hiring decisions remain entirely in the hands of our recruitment professionals.

Come join our teamone of the largest IT and business consulting services firms in the world.


Required Experience:

IC


About Company

Company Logo

The COMPANY is one of the few end-to-end consulting firms with the scale, reach, capabilities and commitment to meet clients’ enterprise digital transformation needs. Our 77,500 consultants and professionals work side-by-side with clients in 10 industries across more than 400 location ... View more

View Profile View Profile