Senior Data Engineer - W2

SENIOR Data Engineer

Location: NYC NY

US Citizens and GC Holders

3 days a. week onsite

Pyspark expertise Scala Databricks and AWS are a must have.

Qualifications:
Expertlevel understanding of Agile SDLC CI/CD DevOps GitOps and ProdOps is required.
Expertlevel expertise in SQL Python Scala or similar; other objectoriented programming experience a plus
Expertlevel experience in data architecture data types/formats optimizing code and ETL jobs data architecture data modeling data contracts a key requirement for successful collaboration with data architects and visualization developers.
Expertlevel professional handson experience in extensible data pipeline development using Databricks Alteryx AWS Glue and Lambda functions.
Expertlevel professional handson experience with RDBMS ETL scripting tools and IDEs data pipeline development and management using orchestration/monitoring tools.
Expertlevel knowledge and experience in implementing software engineering security best practices and contributing to design/engineering for DevSecOps.
Expertlevel experience with master data management strategies rulesbased engines and tools for multistage data correlation on large data sets.
Must have experience managing deployment and leading the release of code to production.
Expertlevel understanding of data visualization tools and requirements for business customers.

Responsibilities:
Work on migrating applications from an onpremises location to the cloud service providers.
Develop products and services on the latest technologies through contributions in development enhancements testing and implementation.
Develop modify extend code for building cloud infrastructure and automate using CI/CD pipeline.
Partners with business and peers in the pursuit of solutions that achieve business goals through an agile software development methodology.
Perform problem analysis data analysis reporting and communication.
Work with peers across the system to define and implement best practices and standards.
Assess applications and help determine the appropriate application infrastructure patterns.
Use best practices and knowledge of internal or external drivers to improve products or services.

Additional Skills & Qualifications

Expertlevel experience working in agile setting or being an agile bestpractice mentor.
Expertlevel experience working in Jira confluence and contributing to engineering team documentation/playbooks.
Proven experience influencing and partnering with product management Product Owners in translating business requirements into data and engineering specifications.