drjobs Azure Data Engineering Architect العربية

Azure Data Engineering Architect

Employer Active

1 Vacancy
drjobs

Job Alert

You will be updated with latest job alerts via email
Valid email field required
Send jobs
Send me jobs like this
drjobs

Job Alert

You will be updated with latest job alerts via email

Valid email field required
Send jobs
Job Location drjobs

Brent - USA

Monthly Salary drjobs

Not Disclosed

drjobs

Salary Not Disclosed

Vacancy

1 Vacancy

Job Description

Description
Lead Azure Data Engineer Architect role is to strategically design develop and implement Azure Cloud based Data Warehouse Data Lake Data Lakehouse Data Marts & BI solution using modern data engineering techniques and advanced analytics methods to provide information & insights to end users. The role requires modeling collecting harmonizing aggregating storing and reconciling data from variety of data sources for our petroleum refineries IT/OT technology platform including time series Database SCADA data batch real time enterprise applications Rest API s streaming & IOT data. The role will help design and build data pipelines streams data warehouse data lake reporting application data generators and a whole range of tools to provide information and insight.
The ideal candidate will be responsible for designing developing and maintaining scalable data processing pipelines in a cloudbased environment. The candidate should have a deep understanding of Azure ADF Databricks streaming analytics data engineering best practices and a proven track record of delivering highquality real time solutions.


Responsibilities

Strategy & Planning
  • Assess and cultivate longterm strategic goals for data warehouse data lake BI & advance analytics development in conjunction with end users managers clients and other stakeholders.
  • Coordinate and work with other technical staff to develop Azure Data & BI architecture coding standards and quality assurance policies and procedures.
  • Promote a culture of selfserve data analytics by minimizing technical barriers to data access and understanding.
  • Is responsible for data collaboration and communication with various stakeholders.
Operational Management
  • Lead the integration efforts for Azure data warehouse data lakehouse sourcing data from disparate enterprise systems including applications both on premises & cloud IOT devices Data Service (Rest API s) timeseries database semistructure unstructured data & external data sources.
  • Organize and lead projects in the implementation and use of BI software tools and systems.
  • Design code test and document all new or modified BI systems applications and programs.
  • Develop the semantic layer metadata reports and report definitions.
  • Assist in the design of databases to ensure interoperability with DW & BI solutions.
  • Produce ELT/ETL design guidelines to ensure a manageable ELT/ETL infrastructure for the analytics system.
  • Participate in design discussions about new features and ensure realtime technology integrates seamlessly with other pieces of the platform
  • Applying technical knowledge to architect solutions that meet business needs AI/ML roadmaps and contributing to architectures that enable the ability to scale to support additional modelling use cases (e.g. Azure ML Azure Databricks Azure EventHub/IoT Hub etc)
  • Build processes supporting data transformation data structures metadata dependency and workload management.
  • Work with project managers to ensure that data entry retrieval change and delete functions meet business requirements for project completion.
  • Act as the primary owner and maintainer of new and existing Lakehouse/DW/BI data models
  • Design implement and maintain CI/CD pipelines using Azure DevOps to integrate analytical solutions into cloud based custom developed applications
  • Support Cloud strategy team to integrate analytical capabilities into an overall cloud architecture and business case development
  • Facilitate Master/ Reference Data Management and overall Data Governance process
  • Conduct job duties and responsibilities according to the organization s business systems development methodology and its Systems Development Life Cycle (SDLC) methodology.
Design and deliver enduser training and training materials; provide technical support as necessary.
  • Troubleshoot Data Integration & BI tools systems and software; performancetune these applications as necessary.
  • Defines and lead proof of concept activity associated with cloud data & analytics technology assessment.
  • Act as evangelist for Azure Cloud data & analytics benefits across the organization; increase usage to relevant departments.
  • Act as a subject matter expert on all things cloud data architecture and pipeline development

Position Requirements

Knowledge & Experience
  • Minimum of 12 to 15 years of Data Engineering Data Warehouse/ Business intelligence with handson experience leading and delivering multiple full life cycle implementations.
  • Comprehensive data warehouse analysis and design experience with full knowledge of data warehouse methodologies and data modeling.
  • Handson experience & deep understanding in designing development and implementation of Azure Data & Analytics architecture data modeling transformations and development.
  • Indepth knowledge and minimum 5 to 7 years experience developing & implementing data engineering solutions using Azure services for Data Warehouse Data Lakehouse & Advance analytics Primarily Azure Data Factory ADLS Azure Databricks Azure Functions Logic Apps IoT Hub Event Hub & Azure SQL
  • Experience with Data pipeline and workflow management tools & Azure DevOps processes.
  • Strong proficiency in programming languages such as Python PySpark or Scala
  • End to End Implementations expertise in Azure Data Warehouse & Data Lakehouse
  • Solid understanding of database structure principles Master and Reference Data management
  • Experience & expertise in optimizing the ETL/ELT data pipeline technology options to minimize the ongoing operational cost of Azure data engineering & storage solutions.
  • 8 years experience in agile Data warehouse development methodology.
  • 8 years experience designing and developing dimensional models for data warehousing
  • 3 years experience in MSBI suite primarily SSIS
  • 3 years of implementing operational data model is plus not essential
  • 3 years of experience with big data batch/streaming processing.
  • 5 years experience using Pyspark Python or Scala
  • Experience delivering reporting solution using visualization tools like PowerBI
  • Experience in implementing SAP data analytics solution in Azure is plus not essential
  • Experience using data modeling tool like ERWin
  • Ability to lead and work with remote & offshore teams.
  • Experience integrating & analyzing data from timeseries databases SCADA data IOT voice to text images & videos for creating Industrial Intelligence solution.
  • Knowledge/expertise on Azure ML/AI predictive modeling capabilities is a plus
  • Experience using proven methods to solve business problems using Azure Data and Analytics services in combination with building data pipelines data streams and system integration
  • Handson experience with analytics and big data technologies within Microsoft Azure Cloud platform in tools such as Azure Machine Learning & Azure Cognitive Services is plus not essential
  • Experience with data profiling cataloguing and data mapping for technical design using a use casebased approach that drives the construction of technical data flows
  • Experience with Azure Administration & Security (Connection setup User setup Object security) is plus
  • Understanding of statistical and machine learning modeling with experience applying these modeling techniques to business problems is a plus not essential.
  • Ability to set up data and experimental platforms using Azure cloud service
  • Ability to work independently with business stakeholders to collect and understand requirements for reporting & analytics.
  • Full project management and software development life cycle experience.
  • Experience with data process flowcharting techniques.
  • Oil & Gas industry experience preferred but not essential.
  • Experience & knowledge of SAP S4 and SAP BW4HANA is a plus not essential.
  • Excellent quantitative analytical & mathematical skills is a plus
  • Strong understanding of the organization s goals and objectives
Formal Education & Certification
  • Bachelor s degree in computer science information systems or a related field or equivalent work experience.
  • Certifications in Azure Data Engineering and Analytics space is a plus
Personal Attributes
  • Ability to work independently and collaboratively in a fastpaced environment
  • Strong written and oral communication skills.
  • Strong presentation and interpersonal skills.
  • Exceptional analytical conceptual and problemsolving abilities.
  • Able to prioritize and execute tasks in a highpressure environment.
Experience working in a teamoriented collaborative environment.

data lake,data governance,pyspark,iot,azure data factory,scada data,bi,azure databricks,data warehouse,power bi,azure devops,ci/cd,data process flowcharting,azure ml,statistical modeling,data mapping,microsoft azure cloud platform,analytics,iot hub,azure cognitive services,sap bw4hana,sap s4,data profiling,etl,streaming analytics,powerbi,azure sql,data engineering,python,azure eventhub,big data technologies,metadata,machine learning modeling,azure,data lakes,data modeling

Employment Type

Full Time

Company Industry

Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.