AWS Data Engineer Lead
AWS Data Engineer Lead
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
About Alphanext
Welcome to Alphanext a leading provider of talent solutions headquartered in London with operational bases in Pune and Indore. With a strong focus on the global technology community we are dedicated to connecting exceptional talent with innovative organizations around the world.
Role Overview
We are seeking a highly skilled and experienced Lead AWS Data Engineer to join our team. The ideal candidate will have handson expertise in AWS services such as Lambda Glue SNS SQS Step Functions and S3. Strong proficiency in Python PySpark SQL and experience with various data file formats is essential. Candidates should also have a background in building scalable and efficient data pipelines coupled with experience in data visualization tools like Looker or Power BI.
Key Responsibilities
- Data Pipeline Development: Design develop and maintain robust ETL pipelines using AWS Glue and PySpark.
- Serverless Architecture: Implement and manage serverless solutions using AWS Lambda Step Functions and related services.
- Messaging Systems: Utilize SNS and SQS for messaging and eventdriven architecture.
- Data Storage & Management: Manage data storage retrieval and optimization using AWS S3 and Delta Lake.
- Query Optimization: Use AWS Athena for querying large datasets efficiently.
- File Format Handling: Work with diverse file formats such as JSON XML CSV TSV and Parquet.
- Monitoring & Logging: Monitor resources and troubleshoot issues using AWS CloudWatch.
- Visualization: Create effective visualizations using Looker or Power BI.
- CI/CD Implementation: Implement CI/CD pipelines using tools like Jenkins GitLab/GitHub Jira and Confluence.
- Version Control: Utilize Delta Lake for efficient data versioning and management.
Required Qualifications
- Technical Skills:
- Expert in Python PySpark SQL and AWS services.
- Proficient in AWS Glue Lambda SNS SQS Step Functions Athena CloudWatch and S3.
- Strong knowledge of data file formats: JSON XML CSV TSV and Parquet.
- Handson experience with visualization tools (Looker Power BI).
- Familiarity with CI/CD tools (Jenkins GitLab/GitHub) and agile tools like Jira Confluence.
- Experience:
- Minimum 10 years as a Data Engineer with a focus on AWS services.
- Prior experience in consumer finance or equivalent industries is a plus.
- Soft Skills:
- Strong organizational skills with the ability to handle multiple priorities.
- Excellent communication skills capable of presenting complex data insights to nontechnical audiences.
AWS,Data Engineering,Lambda
Employment Type
Full Time
