Data Engineer
Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
Hello
We are from Mavinsys Talent Acquisition team based on One World Trade Centre New York. We are specializing in IT services and staffing majorly in lateral hiring/contract. Below is one of our requirement to fill immediately if youre interested please share your candidature to
Job Title: Data Engineer
Location: San Francisco CA
Duration: 12months
Job Description;
- 59 years of relevant industry experience with a BS/Masters or 2 years with a PhD
- Experience with distributed processing technologies and frameworks such as Hadoop Spark Kafka and distributed storage systems (e.g.
HDFS S3)
- Demonstrated ability to analyze large data sets to identify gaps and inconsistencies provide data insights and advance effective product
solutions.
- Expertise with ETL schedulers such as Apache Airflow Luigi Oozie AWS Glue or similar frameworks
- Solid understanding of data warehousing concepts and handson experience with relational databases (e.g. PostgreSQL MySQL) and columnar
databases (e.g. Redshift BigQuery HBase ClickHouse)
- Excellent written and verbal communication skills
- Design build and maintain robust and efficient data pipelines that collect process and store data from various sources including user
interactions financial details and external data feeds.
- Develop data models that enable the efficient analysis and manipulation of data for merchandising optimization. Ensure data quality consistency
and accuracy.
- Build scalable data pipelines (SparkSQL & Scala) leveraging Airflow scheduler/executor framework
- Collaborate with crossfunctional teams including Data Scientists Product Managers and Software Engineers to define data requirements and
deliver data solutions that drive merchandising and sales improvements.
- Contribute to the broader Data Engineering community to influence tooling and standards to improve culture and productivity
- Improve code and data quality by leveraging and contributing to internal tools to automatically detect and mitigate issues.
- Skill Sets Python SQL (expert level) Spark and Scala (intermediate).
Python, SQL (expert level), Spark and Scala
Employment Type
Full Time
