Lead Data Engineer

Title: Lead Data Engineer

Location: Frisco TX (Permanent remote)

Type: Contract

Note : Please look for 12 years experienced lead level candidates. Recent experience (multiple years) in healthcare domain is highly preferred.

Key Responsibilities:

• Work in an agile environment with a development team to:
• Design develop and maintain scalable data pipelines using software development patterns.
• Implement data processing solutions using Python and PySpark on a cloudnative Lakehouse data platform.
• Write efficient SQL queries to extract transform and load data.
• Collaborate with product management and analysts to understand data requirements and deliver solutions.
• Optimize and troubleshoot data pipelines for performance and reliability.
• Ensure data quality and integrity through comprehensive testing and validation processes.
• Follow DevOps principles and use CI/CD to deploy and operate data pipelines.

Required Skills:

• Proficiency in Python and PySpark.
• Strong experience with SQL and database management.
• Knowledge of software development patterns and best practices in data engineering.
• Experience with ETL/ELT processes and data pipeline orchestration.
• Proficiency developing using version control automated testing and deployments using gitbased tools like GitHub and GitHub Actions.

Preferred Skills:

• Understanding of testing methodologies for data pipelines including unit testing integration testing and endtoend testing.
• Knowledge of data governance and data security best practices.
• Familiarity with data warehousing concepts and tools.
• Experience with cloud platforms (e.g. Azure AWS GCP) with Azure preferred.
• Knowledge of big data technologies (e.g. Microsoft Fabric Azure Synapse Lakehouse Databricks).
• Familiarity with advanced data orchestration tooling and development frameworks like dbt or Airflow.
• Experience working in a healthcare related industry.

Qualifications:

• Intense intellectual curiosity and an ability to view old problems with a fresh perspective.
• Bachelors degree in Computer Science Engineering or a related field.
• 8 years of experience in data engineering or a related role.
• Strong problemsolving skills and attention to detail.
• Excellent communication and teamwork abilities.
• Specific vision abilities including close vision operate computer screen for extending periods of time
• May be required to sit or stand for extended periods of time
• Ability to read write and speak the English language fluently