Job Title: Data Infrastructure Engineer
Location: Seattle WA (Onsite)
Duration: 6 Months
Implementation Partner: Blue.cloud
End Client: To be disclosed
JD:
- Capacity planning configure deploy and maintain Databricks clusters workspaces and Snowflake infrastructure on Azure cloud
- Use Terraform to automate provisioning and deploy Databricks clusters workspaces Snowflake and associated Azure resources. Ensure consistency and repeatability by treating infrastructure as code
- Monitor and optimize Databricks cluster performance and Snowflake resource utilization troubleshoot issues to ensure optimal performance and costeffectiveness.
- Implement and manage access controls and security policies to protect sensitive data.
- Develop environment strategies across the technology stack and governance based on best practices
- Provide technical support to Databricks and Snowflake users including troubleshooting and issue resolution.
- Implement and enforce security policies RBAC access controls and encryption mechanisms.
- Develop and maintain backup and disaster recovery strategies to ensure data integrity and availability.
- Collaborate with crossfunctional teams including data scientists data engineers and business analysts to understand their requirements and provide technical solutions
- Data Governance and Quality Management: Create and enforce data governance standards ensuring robust data quality and compliance through tools such as Databricks Unity Catalog Collibra and Snowflake Polaris.
- Enforce data governance data quality and enterprise standards supporting a robust production environment
Required Experience:
- Experience in Data Platform Engineering: Proven track record in architecting and delivering cloudnative data solutions on Azure using Terraform Infrastructure as Code.
- Proficiency in Azure Databricks and Snowflake: Strong skills in data warehousing and lakehouse technologies with handson experience in Azure Databricks Delta Lake and Snowflake
- Tooling Knowledge: Experience with version control (GitHub) CI/CD pipelines (Azure DevOps GitHub Actions) data orchestration and dashboarding tools