MW Partners is currently seeking a Data Engineer to work for our client who is a global leader in multimedia and creativity software products.
Responsibilities and duties:
Data Asset Inventory and Assessment
- Inventory of existing data assets; tables dashboards and pipelines
- Follow requirements to pull required data to measure the current state of these assets set up usage metrics for internal and external stakeholders.
Table Metadata
- to improve documentation coverage for tables including table descriptions column definitions and data lineage.
- Implement a centralized metadata management system to maintain and access asset documentation.
- Ensure that all existing and new data assets are properly documented according to established standards.
- Pipeline Cleanup and Consolidation
- Consolidate and streamline pipelines by eliminating redundancies and unnecessary elements according to the set of provided rules.
- Clean up and restructure data tables ensuring consistent naming conventions data types and schema definitions.
- Retire or archive obsolete dashboards and workflows.
- Implement monitoring and alerting mechanisms for critical workflows to ensure timely issue detection and resolution.
Set up a foundation for scalable Data Model for the Stock Business
- Implement and build performant data models to solve common analytics usecases at Stock.
- Knowledge Transfer and Documentation
- Thoroughly document the work performed including methodologies decisions and any scripts or tools developed.
- Provide comprehensive knowledge transfer to the data team ensuring a smooth transition and the ability to maintain the optimized data environment.
Requirements:
- 4 7 Years of Experience
- Proven experience in data engineering and data asset management.
- Proficiency in SQL Python and other relevant data processing languages and tools.
- Expertise in data modeling ETL processes and workflow orchestration (e.g. Airflow Databricks).
- Strong analytical and problemsolving skills.
- Excellent communication and documentation abilities.
- Familiarity with cloud data platforms (e.g. Azure AWS GCP) is a plus.
For a confidential discussion or to find out more contact Nitin Kuntiya on (909)or apply now.