Associate Staff Engineer - Data Engineering
Associate Staff Engineer - Data Engineering
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
We are seeking an experienced Data Engineer who can assist with managing scheduled DAGs and data workflows. The ideal candidate should have expertise in modern data engineering tools and technologies particularly in the Azure ecosystem and be comfortable working in a virtual machine environment via terminal.
Key Responsibilities:
- Design develop and manage workflows using Apache Airflow ensuring scheduled DAGs run efficiently and reliably.
- Work with Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS) for data integration and storage solutions.
- Develop and maintain data pipelines and ETL processes using Databricks PySpark and Python.
- Monitor troubleshoot and optimize data workflows and pipelines to ensure high availability and performance.
- Collaborate with crossfunctional teams to understand data requirements and deliver scalable solutions.
- Manage and operate within virtual machine environments using terminal commands for development and debugging.
Qualifications :
Must have Skills: Azure Data Factory Azure Pipeline Management (Strong) Databricks Airflow.
- Proficiency in Apache Airflow for orchestrating workflows and managing DAGs.
- Strong experience with Azure cloud services including ADF and ADLS.
- Handson experience with Databricks and PySpark for big data processing and analytics.
- Advanced programming skills in Python with a focus on data manipulation and automation.
- Comfort with terminal operations and commands in VM environments.
- Solid understanding of data engineering principles including data modeling ETL processes and data pipeline optimization.
- Strong problemsolving skills and the ability to work independently in a dynamic environment.
- Experience with other Azure services or similar cloud platforms is a plus Familiarity with DevOps practices and tools for CI/CD in data engineering is a plus.
- Knowledge of database technologies such as SQL and NoSQL systems is a plus.
- Understanding of data security and compliance standards is a plus.
Remote Work :
Yes
Employment Type :
Fulltime
Employment Type
Remote
Company Industry
Department / Functional Area
Engineering
Key Skills
- Computer Science
- Docker
- Kubernetes
- Python
- VMware
- C/C++
- Go
- System Architecture
- gRPC
- OS Kernels
- Perl
- Distributed Systems
