Data Engineer
Data Engineer
Posted on :
20-12-2024
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Job Title: Data Engineer
Interview: Virtual
Job Duration: 6 Months
Job Location: Hopkins MN
Job Description:
We are seeking a Data Engineer with expertise in building scalable data pipelines and managing big data solutions in Azure. This role involves collaborating with crossfunctional teams to design develop and optimize data workflows while ensuring alignment with industry standards and best practices.
Key Responsibilities:
- Design and develop data pipelines in Azure using Spark technologies to process and manage large datasets.
- Build and maintain Zafin and AC360 data pipelines to ensure seamless integration and highperformance data processing.
- Develop Airflow DAGs for automated scheduling and orchestration of workflows.
- Collaborate with product owners business stakeholders and engineering teams to gather and define technical requirements.
- Apply a DevOps mindset to production systems incorporating automation active alerting and selfhealing techniques.
- Optimize big data solutions to ensure scalability reliability and efficiency.
- Perform peer code reviews and define best engineering practices to maintain highquality deliverables.
- Document technical designs workflows and engineering artifacts to support team knowledge sharing and maintenance.
- Partner with key stakeholders to ensure endtoend testing and smooth deployment of software products.
Qualifications:
- Big Data Expertise: 10 years of experience managing largescale datasets and distributed systems.
- Spark (Scala SQL Py): 5 years of handson experience.
- Cloud Technologies: 4 years of experience with Azure Kubernetes and PostGres.
- DevOps and CI/CD: Proficiency in Maven Git Jenkins Docker and Kubernetes.
- Programming Skills: Strong knowledge of Python Java and GoLang (3 years).
- Airflow and Elastic Stack: 3 years of experience with tools like Kibana and ELK.
- Soft Skills: Excellent verbal and written communication skills (10 years).
- Analytical and ProblemSolving Skills: Strong technical and logical capabilities (10 years).
Preferred Skills:
- Experience with Spark Scala Python and the Azure technology stack.
- Familiarity with microservicesbased applications and best practices for data engineering.
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
