Lead Data Engineer AWS
Posted on :
20-09-2024
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
This is a remote position.
We are currently seeking a Lead AWS Platform Data Engineer to drive the evolution of our data infrastructure and unlock the full potential of our meter data. Currently we leverage a suite of proprietary tools along with Git and Snowflake for our data warehousing needs.
Responsibilities:
- Leading the transition from proprietary tools to AWSnative technologies for meter data ingestion integrating seamlessly with Snowflake. Develop solutions to facilitate easier data handling and identify data issues.
- Implementing processes and technologies to enable realtime or halfhour meter data ingestion improving data timeliness and accuracy.
- Oversee the creation and management of a selfservice data catalogues. Enable analysts to access and utilise data sources effectively for realtime reporting and analysis.
- Utilise AWS services (e.g. Lambda Glue Kinesis) to build and optimise data pipelines and infrastructure.
- Continuously monitor and enhance data quality and streamline data processes to ensure accuracy and efficiency.
- Work closely with stakeholders to understand their data needs deliver tailored solutions and support business objectives.
- Stay updated on the latest AWS technologies and data engineering best practices. Drive innovation in data processing and reporting solutions.
Requirements
- Extensive experience with AWS services relevant to data ingestion and processing (e.g. Lambda Glue Kinesis) and data warehousing with Snowflake.
- Strong background in developing and optimising data pipelines and data management systems.
- Proven experience in developing and managing PowerBI reports and selfservice data catalogues.
- Proficiency in programming languages such as Python and SQL.
- Experience with realtime data ingestion and processing solutions.
- Proven ability to lead and mentor a team of data engineers with strong project management skills.
- Expertise in improving data quality and optimising data workflows.
Desirable
- Experience with meter data ingestion systems and familiarity with industryspecific requirements.
- Knowledge of advanced analytics techniques and tools.
- Experience with integrating cloud data solutions with various data sources and systems.
- AWS Professional Certification or Data Analytics/Machine Learning.
Benefits
- Work Location: Remote
- 5 days working
Extensive experience with AWS services relevant to data ingestion and processing (e.g., Lambda, Glue, Kinesis) and data warehousing with Snowflake. Strong background in developing and optimising data pipelines and data management systems. Proven experience in developing and managing PowerBI reports and self-service data catalogues. Proficiency in programming languages such as Python and SQL. Experience with real-time data ingestion and processing solutions. Proven ability to lead and mentor a team of data engineers, with strong project management skills. Expertise in improving data quality and optimising data workflows
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
