Data Engineer
Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Job Tittle: Data engineer
Duration: LongtermData engineer responsibilities:
Regardless of the focus data engineers have similar responsibilities. This is mostly a technical position that combines knowledge and skills in computer science engineering and databases.
Architecture design. At its core data engineering entails designing the architecture of a data platform (though in big companies with complex data processes there might be a separate role for a data architect).
Development of datarelated instruments. As a data engineer is a developer role in the first place these specialists use programming skills to build customize and manage integration tools databases warehouses and analytical systems.
Data pipeline maintenance/testing. During the development phase data engineers would test the reliability and performance of each part of a system. Or they can cooperate with the testing team.
Machine learning algorithm deployment. Machine learning models are designed by data scientists. Data engineers are responsible for deploying those into production environments (but again in some companies this is in the ML engineers area of competence). This entails providing the model with data stored in a warehouse or coming directly from sources configuring data attributes managing computing resources setting up monitoring tools etc.
Manage data and metadata. Information in a data repository is stored either in a structured or unstructured form. In addition theres metadata or information about the data itself. Data engineers are in charge of managing all kinds of data especially in case theres no separate role of metadata manager.
Provide data access tools. In some cases data engineers take on the role of BI developers and set up tools to view data generate reports and create visuals. Well talk more about the differences between these and other datarelated roles in one of the next sections.
Track pipeline stability. Monitoring the overall performance and stability of the system is really important as long as the warehouse needs to be cleaned from time to time. The automated parts of a pipeline should also be monitored and modified since data/models/requirements can change.
Data engineer skills and toolset:
Skills for any specialist correlate with the responsibilities theyre in charge of. The required skill set would vary for every project or organization as there is a wide range of things data engineers could do. But generally their activities can be sorted into three main areas: engineering database/warehouse management and data science.
Employment Type
Full Time
