AWS Cloud Data Engineer

Heres a detailed job description based on the provided requirements:

Job Title: Data Engineer

Location: Insert Location

Job Type: Fulltime

About the Role:

We are seeking an experienced Data Engineer to join our team and help us build and maintain highly reliable data pipelines and data platforms. You will work on a wide range of data engineering tasks from data discovery and profiling to data governance and compliance. The ideal candidate will have strong technical skills particularly in Python Java SQL and AWS Data and Analytics Tools.

Key Responsibilities:

• Data Pipeline and Platform Development:
  

  • Develop highly reliable data pipelines and data platforms with comprehensive test coverage under limited supervision.
    
  • Ensure the solutions are scalable secure and aligned with industry best practices.
    

• Technical Design and Enhancements:
  

  • Contribute to the technical design for system enhancements and new features.
    
  • Ensure designs meet business requirements and adhere to architectural principles.
    

• CI/CD Pipelines and Release Management:
  

  • Implement and manage CI/CD pipelines for automated deployment and release management on data platforms.
    
  • Apply deployment mechanisms that ensure smooth and efficient release processes.
    

• Technical Support and Collaboration:
  

  • Provide technical assistance and guidance to other team members.
    
  • Collaborate with stakeholders to deliver robust secure and highly available data solutions.
    

• Peer Reviews and Continuous Improvement:
  

  • Actively participate in peer reviews to maintain high code quality.
    
  • Identify and drive opportunities for continuous improvement within the team and in the delivery of products.
    

• Data Engineering Duties:
  

  • Engage in core data engineering tasks including data design preparation transformation and load processes.
    
  • Perform data discovery profiling analysis and impact assessments.
    
  • Design build and maintain data solutions that integrate data from various sources including thirdparty APIs and transactional databases.
    
  • Ensure data governance security and compliance across all platforms.
    
  • Create and maintain comprehensive data documentation and reporting.
    

Technical Skills:

• Programming Languages:
  

  • Proficient in Python Java and SQL for data processing and analysis.
    

• Database Management:
  

  • Experience in managing databases and working with large datasets.
    

• AWS Data and Analytics Tools:
  

  • Expertise in AWS Glue ETL Glue Catalogue Lake Formation EMR S3 Athena Lambda Step Functions and EventBridge.
    

• Data Governance and Compliance:
  

  • Knowledge of best practices in data governance security and compliance.
    

Required Qualifications:

• Bachelor s degree in Computer Science Data Science or a related field.
  
• 5 years of experience in data engineering or a related role.
  
• Strong experience with data pipeline and platform development.
  
• Proven experience in deploying and managing CI/CD pipelines.
  
• Solid understanding of data governance security and compliance requirements.
  
• Excellent problemsolving skills and attention to detail.
  
• Strong communication skills and ability to work collaboratively in a team environment.