Data Engineer Big Data Migration Specialist
Data Engineer Big Data Migration Specialist
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
We are looking for a Data Engineer to support the migration of a Big Data project from Google Cloud Platform (GCP) to Microsoft Azure. The role involves ensuring performance optimization adherence to best practices and comprehensive documentation throughout the process.
Key Responsibilities:
- Migration Tasks:
- Migrate existing processes implemented in Scala on GCP to PySpark on Azure.
- Ensure the consistency and integrity of data and processes during migration.
- Optimization and Best Practices:
- Identify and implement performance improvements.
- Apply best practices in data engineering and cloudbased solutions.
- Documentation:
- Document business rules and processes implemented in the system.
- Provide clear and organized technical documentation to ensure process transparency and reproducibility.
Requirements:
- Proven experience with:
- Google Cloud Platform (GCP) and Microsoft Azure.
- Scala and PySpark for Big Data processing.
- Tools like Apache Spark and Big Data pipelines.
- Strong ability to document complex systems and identify optimization opportunities.
- Excellent communication and organizational skills.
Details:
- Work Location: Hybrid (12 days per week at the client office in UTL Lisbon).
Take on this exciting challenge to leverage your expertise in Big Data and cloud migration in a dynamic and innovative environment!
Employment Type
Full Time
Company Industry
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
