Overview:
The role of a Data Engineer is crucial to our organization as it involves building optimizing and maintaining conceptual and logical database models. The Data Engineer oversees the development of data set processes for data modeling mining and production. They work closely with the Data Science and Analytics teams to select and implement the most efficient data solutions for our organization.
Key Responsibilities:
- Design and implement efficient data pipelines
- Develop and maintain data ingestion and deployment processes
- Collaborate with Data Scientists to enable more advanced analytics
- Optimize data delivery and redesign infrastructure for greater scalability
- Implement industry best practices for data management and governance
- Support and enhance data warehousing solutions
- Conduct performance tuning for large scale data sets
- Produce and maintain documentation on data flow and data infrastructure
- Diagnose and resolve datarelated issues
- Stay current with technological trends and advancements in the field
Required Qualifications:
- Bachelor s degree in Computer Science or a related field
- Proven experience in data engineering
- Proficiency in Snowflake Azure (ADLS ADF Synapse) and/or AWS (Redshift EMR Glue)
- Strong understanding of DataStage
- Expertise in data modeling ETL development and data warehousing
- Knowledge of programming languages such as Python Java or Scala
- Experience with big data technologies such as Hadoop and Spark
- Ability to work with structured semistructured and unstructured data
- Excellent problemsolving and analytical skills
- Effective communication and collaboration abilities
snowflake,azure,aws,datastage,data engineering