Job Description: Data Engineer L2 Support
Job Overview: We are looking for a Data Engineer with expertise in SQL and Python who can independently trace and resolve data issues follow standard operating procedures (SOP) or create new ones as needed. The role involves monitoring and troubleshooting data pipelines documenting findings and escalating issues to the core Data Engineering team when required. This position includes availability for night shift monitoring during emergency scenarios.
Key Responsibilities:
- Gather and analyze relevant information to diagnose reported data issues.
- Trace issues within data pipelines identifying root causes.
- Provide initial feedback to the issue reporter as appropriate.
- Document issues thoroughly including actions taken timeline time spent and any impediments encountered.
- Summarize findings and escalate to the core Data Engineering team when necessary for further investigation.
- Develop or follow existing SOPs to address recurring issues and streamline processes.
- Make intuitive connections between data inputs and outputs even when working with partially opaque ETL pipelines (e.g. in Customer Success Management).
Required Skills:
- SQL Strong proficiency in writing complex queries and troubleshooting.
- Python Experience with scripting and automating data processes.
- Data Issue Troubleshooting Ability to trace and communicate data issues and propose solutions independently.
- SOP Management Ability to follow existing standard operating procedures or develop new ones.
Preferred/Optional Skills:
- Apache Airflow Knowledge of task orchestration and workflow management.
- BigQuery (BQ) Experience with Googles BigQuery platform.
- HANA Studio Familiarity with SAP HANA Studio for data management.
- PySpark Experience with distributed data processing.
Work Conditions:
- Available for night shift monitoring particularly in emergency scenarios to ensure smooth data pipeline operations.
Education and Experience:
- Bachelor s degree in Computer Science Information Technology or a related field.
- 35 years of experience in a similar role with a proven track record in data issue resolution.
hana studio,data issue troubleshooting,bigquery (bq),airflow,big data,spark,python,pyspark,hana,troubleshooting,core data,sop management,data,data engineering,apache airflow,trace,extract, transform, load (etl),sop development,sql,pipelines,python scripting,management