Employer Active
Job Alert
You will be updated with latest job alerts via emailJob Alert
You will be updated with latest job alerts via emailJob Summary
We are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate must have a minimum of 4 years of experience managing data engineering jobs in big data environment e.g. Cloudera Data Platform. Overall experience can be 4 to 10 years. Consultant/ Senior Consultant preferably. The successful candidate will be responsible for designing developing and maintaining the data ingestion and processing jobs as per business requirements. Candidate will also be integrating data sets to provide seamless data access to users.
Key Responsibilities
Analyze the Authoritys data needs and document the requirements.
Refine data collection/consumption by migrating data collection to more efficient channels.
Plan design and implement data engineering jobs and reporting solutions to meet the analytical needs.
Develop test plan and scripts for system testing support user acceptance testing.
Work with the Authoritys technical teams to ensure smooth deployment and adoption of new solution.
Ensure the smooth operations and service level of IT solutions.
Support production issues Track record in implementing systems with high availability high performance high security.
What we are looking for
Good understanding and completion of projects using waterfall/Agile methodology.
Good understanding of analytics and data warehouse implementations.
Ability to troubleshoot complex issues ranging from system resource to application stack traces.
Strong SQL data modelling and data analysis skills are a must.
Handson experience in big data engineering jobs using Python PySpark Linux and ETL tools like Informatica.
Track record in implementing systems using Hive Impala and Cloudera Data Platform will be preferred.
Handson experience in DevOps deployment and data virtualization tools like Denodo will be preferred.
Understanding/ Hands on experience of reporting or visualization tool like SAP BO and Tableau will be beneficial but not required.
Passion for automation standardization and best practices
Good written and verbal communication and interpersonal skills ability to understand the business requirement communicate confidently with stakeholders.
Remote Work :
No
Full Time