Enter a job title or keyword

Lead Data & AI Engineer – Databricks Data Analytics & Agent Bricks

Orbia


Job Location:

Pune - India

Salary: Not provided by the employer
Posted: 17 November 2026 (Yesterday)
Application Deadline: 14 February 2027
Vacancies: 1 Vacancy

Job Summary

About Orbia:

Orbia Advance Corporation is a Purpose-led company with big aspirations. We are out to advance life around the world while maximising value to our shareholders customers and employees. The Company is passionate about the topics that define how people will live and thrive tomorrow: the future of cities buildings agriculture and materials. Orbia Advance Corporation has five business groups which offer innovative solutions across multiple industries including building and infrastructure data communications chemicals and 2018 Orbia Advance Corporation bought a majority stake in Israeli-based Netafim the worlds leader in drip irrigation and is helping the world grow more with less as it helps to solve food and water scarcity. Orbia Advance Corporation has operations in 41 countries with more than 22000 employees.

We started as a producer of commodities and have evolved to become a provider of innovative solutions that address the global issues of rapid urbanisation water and food scarcity and a growing and aging population. Were already a global leader in Polymers Fluor Water & Climate Solutions Datacom and Precision Irrigation. We have embarked on a CEO-led transformation as part of our journey to become a truly purpose-led future fit company.


Key Responsibilities:

1. MAIN PURPOSE

The Lead Data Engineer Databricks Data Analytics & Agent Bricks will be responsible for designing building governing and operating enterprise-grade data engineering and AI-agent solutions on the Databricks Lakehouse Platform (Delta Lake Unity Catalog Delta Live Tables Workflows SQL Warehouses MLflow Model Serving) and Databricks Agent Bricks / Mosaic AI Agent Framework within Orbias global Data Analytics & AI landscape.

The role exists to ensure that data products AI/BI dashboards Genie spaces RAG and Agent bricks and analytical workloads are designed using secure scalable compliant cost-aware and reusable engineering patterns aligned with Orbias Lakehouse governance and Responsible AI standards.

The Lead Data Engineer will provide hands-on technical leadership to a team of data and analytics engineers partner with business groups platform owners security infrastructure integration governance and external partners and translate business and AI use cases into production-ready Databricks solutions.

2. KEY RESPONSIBILITIES

The Lead Data & AI Engineer Databricks Data Analytics & Agent Bricks will be responsible for:

Strategy & Planning:

  • Define and govern Databricks Lakehouse and Agent Bricks architecture principles design standards and reusable reference patterns for enterprise data and AI-agent solutions.

  • Design end-to-end data engineering patterns covering ingestion bronze/silver/gold medallion architecture Delta Live Tables streaming batch and data product publishing on Unity Catalog.

  • Design Agent Bricks / Mosaic AI Agent Framework patterns for analytical agents RAG applications Genie spaces AI/BI dashboards and business assistants grounded on governed Lakehouse data.

  • Translate approved business and AI use cases into solution design documents data contracts platform-fit decisions integration patterns and production-readiness plans.

  • Define model and tool selection guidance covering accuracy latency cost (DBU/token) privacy security data sensitivity and business purpose.

  • Establish engineering guardrails for prompt engineering RAG grounding tool calling human-in-the-loop approvals write-back controls and auditability for Agent Bricks solutions.

  • Support Orbia AI/data governance ARB submissions decision records architecture exceptions and platform standards for Databricks solutions.

  • Partner with Data Analytics & AI leadership to align Databricks engineering roadmap with Orbias enablement strategy and scalable operating model.

.

Acquisition & Deployment:

  • Evaluate Databricks capabilities including Unity Catalog Delta Live Tables Workflows SQL Warehouses MLflow Model Serving Vector Search Genie AI/BI Dashboards and Agent Bricks for enterprise fit.

  • Collaborate with cloud infrastructure network security integration and AI architecture teams to design DEV / QA / PROD-ready deployment patterns using Databricks Asset Bundles and CI/CD.

  • Define secure ingestion and integration patterns from SAP SharePoint ServiceNow Coupa Salesforce REST APIs file shares and event sources using Azure Data Factory Auto Loader and partner connectors.

  • Design API-led integration patterns using Azure API Management Entra ID OAuth managed identity service principals and approved enterprise gateways for Databricks-hosted agents and endpoints.

  • Lead technical design reviews for data pipelines Unity Catalog models agent tools vector indexes model endpoints evaluation workflows monitoring and release gates.

  • Build and review hands-on production-grade code in PySpark Python and SQL; enforce coding standards peer reviews unit/integration testing and automated quality checks.

  • Support partner and vendor technical reviews to validate delivery approach platform compatibility security posture and architecture compliance.

Operational Management:

  • Define operational runbooks for Databricks workspaces Unity Catalog jobs DLT pipelines Agent Bricks lifecycle prompt/version control evaluations monitoring telemetry and incident support.

  • Ensure Databricks solutions follow least-privilege RBAC/ABAC access reviews environment segregation audit logging data protection lineage and Responsible AI controls.

  • Provide technical leadership mentoring and guidance to data engineers analytics engineers platform admins business analysts and implementation partners on approved design and implementation patterns.

  • Monitor and optimize solution design for DBU consumption storage SQL Warehouse usage vector search cost model endpoint usage latency quality security and scalability.

  • Review delivered solutions against engineering principles Orbia governance expectations SOX/ITGC controls and production acceptance criteria before go-live.

  • Continuously improve Databricks engineering standards based on lessons learned platform roadmap changes business adoption and operational feedback.

  • Lead and mentor a small team of data/analytics engineers; perform code reviews sprint planning capacity planning performance feedback and skill development.

3. COMPLEXITY OF THE JOB

  • Complexity of enterprise data and AI engineering across Databricks Lakehouse Unity Catalog Delta Live Tables MLflow Vector Search Agent Bricks / Mosaic AI Agent Framework Azure Data Factory ADLS Gen2 APIM Entra ID Purview SharePoint SAP ServiceNow and Power BI.

  • Complexity of selecting the right delivery pattern across Databricks-led Foundry-led Copilot Studio Power Platform and hybrid Foundry Databricks solutions.

  • Complexity of safely grounding analytical and agent answers using structured and unstructured enterprise data while maintaining data protection lineage citations and auditability.

  • Complexity of GenAI and agent risks including hallucination prompt misuse sensitive data exposure model and tool selection cost escalation and business process impact.

  • Complexity of supporting multiple Orbia business groups with diverse data sources processes and regional requirements.

  • Complexity of cost and performance optimization across DBU storage SQL Warehouses model serving and vector search while meeting SLAs and FinOps guardrails.

  • Collaboration with multiple global teams / departments for technical alignment and decision-making such as Infrastructure Network Security Enterprise Architecture AI Architecture Integration Privacy Compliance and business groups.

  • Requirement to balance speed of data and AI experimentation with production readiness lifecycle management Responsible AI FinOps and enterprise governance (including SOX / ITGC)

  • EXPERIENCE LANGUAGE & PHYSICAL REQUIREMENTS (To perform satisfactorily in this job what type of knowledge and experience is required)

    • Minimum 12-14 years of hands-on experience on the Databricks Lakehouse Platform including Apache Spark PySpark Delta Lake Unity Catalog Delta Live Tables (DLT) Workflows SQL Warehouses MLflow and Model Serving.

    • Minimum 2 years of hands-on or strong architecture experience with Databricks Agent Bricks / Mosaic AI Agent Framework Vector Search Genie spaces AI/BI Dashboards and RAG-based agentic applications grounded on Lakehouse data.

    • Strong programming skills in Python PySpark and advanced SQL; deep understanding of distributed computing performance tuning partitioning Z-Ordering file optimization and Delta best practices.

    • Experience with CI/CD for Databricks using Databricks Asset Bundles Azure DevOps and/or GitHub Actions infrastructure-as-code (Terraform) and environment promotion patterns across DEV / QA / PROD.

    • Experience with the Azure data ecosystem including ADLS Gen2 Azure Data Factory Azure Key Vault Entra ID Microsoft Purview Azure Monitor Azure API Management and Azure OpenAI services.

    • Experience designing and operating ingestion pipelines from SAP (ECC/S4 HANA BW BTP) SharePoint ServiceNow Coupa Salesforce REST APIs files and streaming sources using Auto Loader ADF and partner connectors.

    • Experience designing RAG and agent solutions using SharePoint enterprise documents semantic indexes embeddings vector search citations chunking grounding evidence and answer evaluation.

    • Experience designing secure enterprise integrations using REST APIs OAuth 2.0 managed identities service principals APIM event-driven patterns SAP BTP ServiceNow APIs or Microsoft Graph APIs.

    • Strong understanding of governance controls including Unity Catalog (catalogs/schemas/tables row/column-level security ABAC) RBAC least privilege segregation of duties access reviews private endpoints audit logging DLP Purview data classification lineage and SOX / ITGC expectations.

    • Experience in AI/data governance Responsible AI model and agent evaluation prompt engineering hallucination control content safety human approval gates and production release criteria.

    • Demonstrated experience leading a team of data engineers / analytics engineers including hands-on technical leadership mentoring code reviews sprint planning and performance development.

    • Experience preparing architecture review documents solution and low-level design documents technical standards operating procedures decision records and ARB-ready materials.

    • Strong understanding of cost management for Databricks and AI solutions including DBU usage SQL Warehouse sizing storage vector search cost model endpoints monitoring and FinOps guardrails.

    • Experience working in an international environment at a global company with cross-functional teams and business stakeholders across multiple regions.

    • Excellent communication skills in English both verbally as well as in writing.

    • Capabilities to build trustworthy relationships with business users platform teams security architecture vendors and implementation partners.

    • Self-starter with the ability to provide technical leadership mobilize business partners analyze requirements challenge solution assumptions push back when required and remain result-oriented.

    • Physical requirements for the position such as mobility vision strength etc. (applicable in line with local laws):






















Requirements:

















Our Offer:














Our Global brands: Dura-Line Koura Global Vestolit Netafim Alphagary Wavin.

They offer a broad range of value-added solutions and finished products that contribute to customers success and ultimate improve the quality of life for people around the world. Along its commitment to good citizenship Orbia Advance Corporation delivers Total Value to customers employees and investors worldwide every day.


About Company

Company Logo

Discover how Orbia is addressing the challenges that define how people live today and preparing them to thrive in the future.

View Profile View Profile