Data Engineer

Job Description:

• Create and manage data pipelines for diverse sources in our data lake/warehouse.
• Collaborate with analysts to implement necessary data transformations. Research relevant data sources for property development.
• Develop web scraping scripts for data acquisition.
• Automate deployment and testing of data pipelines using CI/CD.
• Utilize Azure AWS and other technologies for data solutions.
• Implement data integration between systems.
• Monitor and optimize pipeline performance.
• Ensure data security compliance and quality.
• Stay updated on data engineering trends.

Requirements:

• Bachelors or Masters degree in Computer Science Engineering or related field.
• Proven experience as a Data Engineer with strong understanding of data engineering concepts.
• Handson experience with Azure (Azure Function Azure Data Factory Azure Databricks) and/or AWS (AWS Glue Lambda Amazon Redshift).
• Proficiency in Python Java or Scala and experience with data processing frameworks (e.g. Spark Hadoop).
• Knowledge of data modelling warehousing and database technologies.

Remote Work :

No