ADSK7989-Data Engineer EDM III - INR
ADSK7989-Data Engineer EDM III - INR
Posted on :
03-12-2024
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Description:
Title: Sr Data Engineer Analytics & Insights
Location: Bangalore (Hybrid)
Responsibilities:
Location: Bangalore (Hybrid)
Responsibilities:
- Deploy and configure Apache Spark clusters on AWS (particularly EKS) optimizing for performance resource allocation and scalability.
- Develop automate and support robust and reliable Spark data pipelines focusing on high performance and low latency.
- Design and implement highly optimized queries to improve data processing efficiency streamline analysis minimize latency and enhance overall system performance.
- Collaborate to make the Data Platform an effective scalable and resilient system.
- Gather requirements from stakeholders prioritize work and document technical solutions clearly and effectively.
- Engage with data engineers data scientists product managers and internal stakeholders to align project goals and implementations.
- Contribute to a team culture that values quality robustness and scalability while fostering initiatives and innovation
Minimum Qualifications:
- 5 years of data processing experience in large cloudbased infrastructure (AWS is must)
- Handson software development experience in Python with strong proficiency in Py Spark for data engineering tasks and data pipeline development
- Expert understanding of SQL dimensional modeling and analytical data warehouses such as Snowflake
- Understanding of Data Engineering best practices for medium to large scale production workloads
- Expertise with data pipeline orchestration tools such as Airflow
- Familiar with processing semistructured file formats such as json or parquet
- Team player with good communication skills
- Problem solver with excellent written and interpersonal skills
- Strong problemsolving skills
- Ability to learn new technical skills
- Bachelors degree in Computer Science Data Science or related fields
Preferred Qualifications:
- Experience with Jinja Shell scripting DBT
- Developing in Cloud platform using serverless technologies such as glue lambda EMR is a plus.
- Experience with remote development using AWS SDK is a plus
- Experience with ELT pipelines DBT
- REST API design and implementation
- Familiarity with containers and infrastructureascode principles
- Experience with automation frameworks Git Jenkins and Terraform
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
