AWS Data Engineer
AWS Data Engineer
Posted on :
24-10-2024
Employer Active
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
NOTE:
Only W2.
Only USCGC.
Title: AWS Data Engineer
Location: Onsite in Torrance CA
Duration: 612 months contract
Job Description:
Daily Tasks Performed:
Develop and Maintain Data Integration Solutions:
- Design and implement data integration workflows using AWS Glue/EMR Lambda Redshift
- Demonstrate proficiency in Pyspark Apache Spark and Python for data processing large datasets
- Ensure data is accurately and efficiently extracted transformed and loaded into target systems.
Ensure Data Quality and Integrity:
- Validate and cleanse data to maintain high data quality.
- Ensure data quality and integrity by implementing monitoring validation and error handling mechanisms within data pipelines
Optimize Data Integration Processes:
- Enhance the performance optimization of data workflows to meet SLAs scalability of data integration processes and costefficiency on AWS cloud infrastructure.
- Identify and resolve performance bottlenecks finetuning queries and optimizing data processing to enhance Redshifts performance
- Regularly review and refine integration processes to improve efficiency.
Support Business Intelligence and Analytics:
- Translate business requirements to technical specifications and coded data pipelines
- Ensure timely availability of integrated data for business intelligence and analytics.
- Collaborate with data analysts and business stakeholders to meet their data requirements.
Maintain Documentation and Compliance:
- Document all data integration processes workflows and technical system specifications.
- Ensure compliance with data governance policies industry standards and regulatory requirements.
What will this person be working on:
- The IT Data Integration Engineer / Developer is tasked with the design development and management of data integration processes to ensure seamless data flow and accessibility across the organization. This role is pivotal in integrating data from diverse sources transforming it to meet business requirements and loading it into target systems such as data warehouses or data lakes. The aim is to support the organizations datadriven decisionmaking by providing highquality consistent and accessible data.
Position Success Criteria Desired WANTS
- Bachelors degree in computer science information technology or a related field. A masters degree can be advantageous.
- 710 years of experience in data engineering database design ETL processes
- 5 in programming languages such as PySpark Python
- 5 years of experience with AWS tools and technologies S3 EMR Glue Athena RedShift Postgres RDS Lambda PySpark
- 3 years of experience of working with databases/ data marts/data warehouses
- Proven experience in ETL development system integration and CI/CD implementation.
- Experience in complex database objects to move the changed data across multiple environments
- Solid understanding of data security privacy and compliance.
- Excellent problemsolving and communication skills.
- Display good communication skills to effectively collaborate with multifunctional teams
- Participate in agile development processes including sprint planning standups and retrospectives
- Provide technical guidance and mentorship to junior developers
- Attention to detail and a commitment to data quality.
- Continuous learning mindset to keep up with evolving technologies and best practices in data engineering.
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
