Big Data Engineer
Big Data Engineer
Posted on :
08-10-2024
Employer Active
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Monthly Salary
Not Disclosed
Salary Not Disclosed
Posted on :
08-10-2024
Job Description
Responsibilities
- Writes ETL (Extract / Transform / Load) processes designs database systems and develops tools for realtime and offline analytic processing.
- Troubleshoots software and processes for data consistency and integrity.
- Integrates complex and large scale data from a variety of sources for business partners to generate insight and make decisions.
- Translates business specifications into design specifications and code.
- Responsible for writing complex programs ad hoc queries and reports.
- Ensures that all code is well structured includes sufficient documentation and is easy to maintain and reuse.
- Partners with internal clients to gain an expert understanding of business functions and informational needs. Works closely with other technical and data analytics experts across the business to implement data solutions.
- Leads all phases of solution development.
- Explains technical considerations at related meetings including those with internal clients and less experienced team members.
- Assesses data quality and tests code thoroughly for accuracy of intended purpose.
- Provides data analysis guidance and serves as a technical consultant for the client.
- Educates and develops junior data engineers on the team while applying quality control to their work.
- Develops data engineering standards and contributes expertise to other data expert teams across Vanguard.
- Tests and implements new software releases through regression testing.
- Identifies issues and engages with vendors to resolve and elevate software into production.
- Participates in special projects and performs other duties as assigned.
Qualifications
- Mandatory Python CI/CD pipeline (Bitbucket Bamboo GitHub) AWS Services S3 Glue Athena EMR Cluster Redshift Pandas Spark Lambda CloudFormation IAM Service Catalog SQL ETL Processes
- Good to have Collibra Dremio Tableau Data Modeling
Required Skills : Data WarehousePython
Basic Qualification :
Additional Skills : Applications ArchitectBig Data
Background Check : No
Drug Screen : No
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
