Azure Data Engineers AWS Data Engineers

Requirements

Experience in Developing Data Pipelines that process large volumes of data using Python PySpark Pandas etc on AWS / Azure

Experience in developing ETL OLAP based and Analytical Applications.

Experience in ingesting batch and streaming data from various data sources.

Strong Experience in writing complex SQL using any RDBMS (Oracle PostgreSQL SQL Server etc.)

Ability to quickly learn and develop expertise in existing highly complex applications and architectures.

Exposure to AWS platforms data services (AWS Lambda Glue Athena Redshift Kinesis etc.)

Proficiency in Azure technologies

such as Azure Data Factory (ADF) Azure Data Bricks (ADB)Azure Synapse Analytics Azure Active Directory Azure Storage Azure data Lake Services (ADLS) Azure key vault Azure SQL DB Azure HD Insight.

Experience in Airflow DAGS AWS EMR S3 IAM and other services

Snowflake or Redshift data warehouses

Experience of DevOps and CD/CD tools.

Familiarity with Rest APIs

.

Clear and precise communication skills

.

Experience with CI/CD pipelines branching strategies & GIT for code management

.

Comfortable working in Agile projects

Benefits

Best in the industry

This is a remote position.

Experience in Developing Data Pipelines that process large volumes of data using Python, PySpark, Pandas etc, on AWS / Azure Experience in developing ETL, OLAP based and Analytical Applications. Experience in ingesting batch and streaming data from various data sources. Strong Experience in writing complex SQL using any RDBMS (Oracle, PostgreSQL, SQL Server etc.) Ability to quickly learn and develop expertise in existing highly complex applications and architectures. Exposure to AWS platform's data services (AWS Lambda, Glue, Athena, Redshift, Kinesis etc.) Proficiency in Azure technologies such as Azure Data Factory (ADF), Azure Data Bricks (ADB),Azure Synapse Analytics, Azure Active Directory, Azure Storage, Azure data Lake Services (ADLS), Azure key vault, Azure SQL DB, Azure HD Insight. Experience in Airflow DAGS, AWS EMR, S3, IAM and other services Snowflake or Redshift data warehouses Experience of DevOps and CD/CD tools. Familiarity with Rest APIs Clear and precise communication skills Experience with CI/CD pipelines, branching strategies, & GIT for code management Comfortable working in Agile projects