Lead Data Engineer with using GCP cloud Platform
Lead Data Engineer with using GCP cloud Platform
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Designation : Senior Technical Lead
Location : New Jersey
City : Newark
Job Description (Posting). : Should have overall 15 years with Lead skillAbility to design and develop a high performance data pipeline framework from scratch Data ingestion across systemsData quality and curationData transformation and efficient data storageData reconciliation monitoring and controlsSupport reporting model and other downstream application needsSkill in technical design documentation data modeling and performance tuning applicationsLead and manage a team of data engineers contribute towards code reviews and guide the team in designing and developing convoluted data pipelines adhering to the defined standards.Be hands on performs POCs on the open sourcelicensed tools in the market and share recommendations.Provide technical leadership and contribute to the definition development integration test documentation and support across multiple platforms (GCP Python HANA)Establish a consistent project management framework and develop processes to deliver high quality software in rapid iterations for the business partners in multiple geographiesParticipate in a team that designs develops troubleshoots and debugs software programs for databases applications tools etc.Experience in balancing production platform stability feature delivery and reduction of technical debt across a broad landscape of technologies.Skill in the following platform tools and technologiesGCP cloud platform GCS Big Query Streaming (pubsub) data proc and data flowPython PYSpark Kafka SQL shell scripting Stored procsData warehouse distributed data platforms and data lakeDatabase definition schema design Looker Views ModelsCICD pipelineProven track record in scripting code in Python PySpark and SQLExcellent structured thinking skills with the ability to break down multidimensional problemsAbility to navigate ambiguity and work in a fastmoving environment with multiple stakeholdersGood communication skills and ability to coordinate and work with cross functional teams. (1.) To be responsible for providing technical guidance to a team of developers enhancing their technical capabilities and increasing productivity. (2.) To conduct comprehensive code reviews establish and oversee quality assurance processes performance optimization implementation of best practices and coding standards to ensure succeful delivery of complex projects. (3.) To ensure process compliance in the assigned module and participate in technical discussionsorreview as a technical consultant for feasibility study (technical alternatives best packages supporting architecture best practices technical risks breakdown into components estimations). (4.) To collaborate with stakeholders to define project scope objectives deliverables and accordingly prepare and submit status reports for minimizing exposure and closure of escalations.
Skill (Primary): :
Modern Application DevelopmentGCPGCP Application Development
Job Family : Development
Band : E2
Job : Senior Technical Lead
Other Requirement : Location: Remote : Preferrable Remote
JD: Lead Data Engineer with using GCP cloud Platform.
GCP cloud platform GCS Big Query Streaming (pub/sub) data proc and data flow
o Python PYSpark Kafka SQL shell scripting & Stored procs
o Data warehouse distributed data platforms and data lake
o Database definition schema design Looker Views Models
o CI/CD pipeline
Proven track record in scripting code in Python PySpark and SQL
Employment Type
Full Time
