Data Engineer
San Francisco CA
Required Skills : SQL Python Spark or PySpark and Scala. Databricks. ETL Tools preferably Datastage but others like FiveTran or Informatica are ok. BI Reporting Tools SASS Cubes Tableau ThoughtSpot Power BI etc. Platform Migration experience. Strong communication and crossfunctional support skills.
- 8 10 years of experience in data engineering and designing scalable ETL pipelines
- Advanced Proficiency in Hadoop Scala Spark Kafka and SQL
- 6 years of experience with cloud platforms like Azure AWS Google
- Be skilled at architecting large and complex data pipelines.
- Experience performing data analysis and data exploration.
- Experience working with Cloud technologies Azure Databricks ADLS Spark Cosmos DB and other big data technologies.
- Experience working in an agile delivery environment.
- Experience working in multideveloper environments using version control.
- Excellent knowledge of data structures and design patterns
- Preferred experience with data integration tools
- Work with crossfunctional agile teams to drive projects through full development cycle.
- Help the team improve with the usage of data engineering best practices.
- Write SQL for processing raw data data validation and QA
- Knowledge working with APIs to collect or ingest data
- Strong Database knowledge SQL & NoSQL preferred
- Communication Skills Data Engineers are part of a team working with database administrators data analysts and management and need to be effective communicators.
- ProblemSolving Skills Data Engineers look at an issue that needs to be solved and come up with solutions quickly.
- Experience integrating and building data platforms in support of BI Analytics Data Science and realtime applications.
- Strong communication skills with the ability to initiate and drive projects proactively and accurately
- Experience with Datastage or other ETL tools experience with reporting tools like Cognos SSAS Cubes Tableau Thoughtspot.