Data Architect

Hello People

Job Title: Data Architect

Location: Louisville KY

Responsibilities:

We are seeking an experienced Data Architect with handson expertise in Azure Databricks Azure Data Lakehouse and Medallion Architecture. The ideal candidate will have strong implementation experience including the integration of Delta Lake tables and a background in working with Customer Data Platforms (CDP). This role requires a deep understanding of cloudnative data solutions and the ability to architect and deliver scalable highperforming data platforms that can support advanced analytics machine learning and business intelligence solutions.

Key Responsibilities:

• Design and Implement Data Architecture: Lead the design and development of scalable cloudbased data architectures leveraging Azure Data Lakehouse and Medallion Architecture principles.
• Azure Databricks and Delta Lake: Architect and implement data pipelines and ETL processes using Azure Databricks ensuring seamless integration with Delta Lake to enable ACID transactions time travel and optimized data storage.
• Medallion Architecture: Implement the Medallion Architecture pattern building Bronze Silver and Gold layers to ensure efficient data processing aggregation and enrichment for analytics and reporting.
• Customer Data Platform (CDP) Integration: Design and manage the data architecture to effectively handle customer data ensuring the Customer Data Platform (CDP) is integrated with the Azure data ecosystem enabling personalized and customercentric analytics.
• Data Lakehouse Management: Lead the implementation and optimization of Azure Data Lake Storage (ADLS) to support structured semistructured and unstructured data storage ensuring efficient querying and data retrieval.
• Data Governance and Security: Define and implement robust data governance security and compliance practices using tools like Azure Data Catalog Azure Purview and Azure Key Vault. Ensure data is securely stored and accessed in compliance with data privacy regulations (GDPR CCPA).
• Performance Optimization: Optimize data pipelines for performance scalability and costefficiency ensuring that data is processed efficiently and meets SLAs for downstream analytics and reporting systems.
• Collaboration with Data Engineers and Analysts: Work closely with data engineers business analysts and data scientists to define data requirements ensuring that data pipelines are designed to meet business needs.
• RealTime Data Integration: Architect realtime data ingestion pipelines integrating with various data sources (APIs event hubs etc.) and streamlining realtime analytics using Azure Databricks and Delta Lake.
• Documentation and Best Practices: Establish best practices for data architecture provide documentation for data pipelines and architecture and ensure knowledge sharing across the organization.

Required Qualifications:

• 5 years of experience in data architecture and engineering with expertise in Azure Databricks Azure Data Lakehouse and Delta Lake.
• Handson experience implementing Medallion Architecture building and managing Bronze Silver and Gold data layers.
• Strong experience with Azure Data Lake Storage (ADLS) and integration with Delta Lake for efficient data storage and querying.
• Customer Data Platform (CDP) experience with the ability to design and integrate customercentric data architectures that drive personalized analytics.
• Experience in data governance security and compliance using Azure tools like Azure Purview Data Catalog and Key Vault.
• Proven experience in ETL/ELT development and realtime data ingestion pipelines working with largescale datasets.
• Strong knowledge of SQL Python and Spark for data processing analysis and pipeline development.
• Familiarity with data modeling techniques big data technologies and data warehousing in a cloud environment.
• Excellent communication and collaboration skills with the ability to work crossfunctionally and lead datadriven initiatives.
• Preferred Qualifications:
• Experience with machine learning and advanced analytics workflows using Azure Databricks.
• Familiarity with API integration and realtime data streaming solutions like Azure Event Hubs or Azure Stream Analytics.
• Knowledge of DevOps for data including automated CI/CD pipelines for data deployments.

Education:

• Bachelors or Masters degree in Computer Science Information Systems or a related field.