Data Engineer
Data Engineer
Posted on :
20-12-2024
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Job Description :
Role : Sr. Data Engineer
Location : Hyderabad
Client : Microsoft
Certainly! Anticipating You to join the company within short Notice.
Please Acknowledge on below details and will keep you posted on further Process.
About the Role
We are searching for a highly skilled and experienced Senior Data Engineer to join our growing team. In this role you will play a critical part in designing developing and maintaining largescale data pipelines on the Microsoft Azure cloud platform. You will leverage your expertise in big data technologies and cloud computing to ensure our organization has access to the clean reliable data it needs to make informed decisions.
Responsibilities:
- Design develop and implement robust data pipelines using Azure Data Factory (ADF) and Apache Airflow.
- Utilize Azure Data Bricks (ADB) and Spark (PySpark) to process and transform large datasets efficiently.
- Collaborate with data scientists and analysts to understand data requirements and develop solutions that meet their needs.
- Implement data quality checks and ensure data integrity throughout the data pipeline.
- Design and implement data storage solutions using Azure Data Lake Storage (ADLS) and Azure SQL Database.
- Work with various data querying technologies like Trino and Azure Data Explorer for efficient data retrieval.
- Manage and optimize data pipelines for performance and scalability.
- Participate in code reviews ensure code quality and adhere to best practices.
- Stay uptodate on the latest advancements in big data technologies and cloud computing.
- Document data pipelines and procedures effectively.
Qualifications:
- Bachelors degree in Computer Science Information Technology or a related field (or equivalent experience).
- 5 years of experience as a Data Engineer or similar role.
- Extensive experience with Azure Data Factory (ADF) for building data pipelines.
- Indepth knowledge of Apache Spark and PySpark for big data processing.
- Familiarity with Apache Airflow for data orchestration.
- Experience with SQL coding and relational database management systems.
- Experience with cloud platforms preferably Microsoft Azure.
- Strong understanding of data modeling and schema design principles.
- Excellent problemsolving and analytical skills.
- Effective communication and collaboration skills.
- Ability to work independently and as part of a team
cloud computing,design,schema design,sql coding,pyspark,big data technologies,cloud,data modeling,microsoft,data,azure data bricks,apache airflow,azure,spark,azure sql database,azure data factory,data lake storage
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
