Data Engineer
Data Engineer
Posted on :
20-12-2024
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
As Senior Associate L2 in Data Engineering you will translate client requirements into technical design and implement components for data engineering solution. Utilize deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to insure the necessary health of the overall solution
The role requires a handson technologist who has strong programming background like Java / Scala / Python should have experience in Data Ingestion Integration and data Wrangling Computation Analytics pipelines and exposure to Hadoop ecosystem components. You are also required to have handson knowledge on at least one of AWS GCP Azure cloud platforms.
Role & Responsibilities:
Your role is focused on Design Development and delivery of solutions involving:
- Data Integration Processing & Governance
- Data Storage and Computation Frameworks Performance Optimizations
- Analytics & Visualizations
- Infrastructure & Cloud Computing
- Data Management Platforms
- Implement scalable architectural models for data processing and storage
- Build functionality for data ingestion from multiple heterogeneous sources in batch & realtime mode
- Build functionality for data analytics search and aggregation
Experience Guidelines:
Mandatory Experience and Competencies:
#
Competency
Overall 5 years of IT experience with 3 years in Data related technologies
Minimum 2.5 years of experience in Big Data technologies and working exposure in at least one cloud platform on related data services (AWS / Azure / GCP)
Handson experience with the Hadoop stack HDFS sqoop kafka Pulsar NiFi Spark Spark Streaming Flink Storm hive oozie airflow and other components required in building end to end data pipeline.
Strong experience in at least of the programming language Java Scala Python. Java preferable
Handson working knowledge of NoSQL and MPP data platforms like Hbase MongoDb Cassandra AWS Redshift Azure SQLDW GCP BigQuery etc
Wellversed and working knowledge with data platform related services on at least 1 cloud platform IAM and data security
Preferred Experience and Knowledge (Good to Have):
#
Competency
Good knowledge of traditional ETL tools (Informatica Talend etc) and database technologies (Oracle MySQL SQL Server Postgres) with hands on experience
Knowledge on data governance processes (security lineage catalog) and tools like Collibra Alation etc
Knowledge on distributed messaging frameworks like ActiveMQ / RabbiMQ / Solace search & indexing and Micro services architectures
Performance tuning and optimization of data pipelines
CI/CD Infra provisioning on cloud auto build & deployment pipelines code quality
Cloud data specialty and other related Big data technology certifications
Personal Attributes:
- Strong written and verbal communication skills
- Articulation skills
- Good team player
- Selfstarter who requires minimal oversight
- Ability to prioritize and manage multiple tasks
- Process orientation and the ability to define and set up processes
mpp data platforms,postgres,kafka,cloud data specialty,rabbitmq,storm,hadoop ecosystem,activemq,data integration,gcp,hbase,alation,spark streaming,pulsar,mongodb,solace,aws,ci/cd,oozie,microservices architectures,oracle,collibra,aws redshift,spark,etl tools,data ingestion,azure,analytics pipelines,hdfs,nosql,big data,hive,azure sqldw,scala,data wrangling,sql server,java,sqoop,python,nifi,flink,mysql,data governance,cassandra,airflow,gcp bigquery
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
