Virtual Networx is looking for Lead Data Engineer for our client located in Mt. Laurel, NJ (On-site from Day 1)
Job Type: Full-time
Experience: 10-12+ Years
Must have skills: Databricks, Python, RDBMS, Powershell scripting, datawarehouse
Detailed JD:
- Experience in ETL/Pipeline Development using tools such as Azure Databricks/Apache Spark and Azure
- Data Factory with development expertise on batch and real-time data integration
- Experience in programming using Python
- RDBMS knowledge and experience in writing the Store Procedures
- Experience in writing bash and Power shell scripting.
- Experience in data ingestion, preparation, integration, and operationalization techniques in optimally addressing the data requirements
- Experience in Cloud data warehouse like Azure Synapse, Snowflake analytical warehouse
- Experience with Orchestration tools, Azure DevOps, and GitHub
- Experience in building end to end architecture for Data Lakes, Data Warehouses and Data Marts
- Experience in relational data processing technology like MS SQL, Delta Lake, Spark SQL, SQL Server
- Experience to own end-to-end development, including coding, testing, debugging and deployment
- Extensive knowledge of ETL and Data Warehousing concepts, strategies, methodologies
- Experience working with structured and unstructured data
- Familiarity with Azure services like Azure functions, Azure Data Lake Store, Azure Cosmos
- Ability to provide solutions that are forward-thinking in data and analytics