Lead Data Engineer
Lead Data Engineer
Posted on :
13-10-2024
Employer Active
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Title: Lead Data Engineer
Location: Frisco TX (Permanent remote)
Type: Contract
Note : Please look for 12 years experienced lead level candidates. Recent experience (multiple years) in healthcare domain is highly preferred.
Key Responsibilities:
- Work in an agile environment with a development team to:
- Design develop and maintain scalable data pipelines using software development patterns.
- Implement data processing solutions using Python and PySpark on a cloudnative Lakehouse data platform.
- Write efficient SQL queries to extract transform and load data.
- Collaborate with product management and analysts to understand data requirements and deliver solutions.
- Optimize and troubleshoot data pipelines for performance and reliability.
- Ensure data quality and integrity through comprehensive testing and validation processes.
- Follow DevOps principles and use CI/CD to deploy and operate data pipelines.
Required Skills:
- Proficiency in Python and PySpark.
- Strong experience with SQL and database management.
- Knowledge of software development patterns and best practices in data engineering.
- Experience with ETL/ELT processes and data pipeline orchestration.
- Proficiency developing using version control automated testing and deployments using gitbased tools like GitHub and GitHub Actions.
Preferred Skills:
- Understanding of testing methodologies for data pipelines including unit testing integration testing and endtoend testing.
- Knowledge of data governance and data security best practices.
- Familiarity with data warehousing concepts and tools.
- Experience with cloud platforms (e.g. Azure AWS GCP) with Azure preferred.
- Knowledge of big data technologies (e.g. Microsoft Fabric Azure Synapse Lakehouse Databricks).
- Familiarity with advanced data orchestration tooling and development frameworks like dbt or Airflow.
- Experience working in a healthcare related industry.
Qualifications:
- Intense intellectual curiosity and an ability to view old problems with a fresh perspective.
- Bachelors degree in Computer Science Engineering or a related field.
- 8 years of experience in data engineering or a related role.
- Strong problemsolving skills and attention to detail.
- Excellent communication and teamwork abilities.
- Specific vision abilities including close vision operate computer screen for extending periods of time
- May be required to sit or stand for extended periods of time
- Ability to read write and speak the English language fluently
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
