Senior Big Data Engineer
Senior Big Data Engineer
Posted on :
06-09-2024
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Overview:
The Senior Big Data Engineer plays a crucial role in our organization responsible for designing developing and maintaining scalable big data solutions. This role is essential in harnessing the power of data to drive informed business decisions and strategic initiatives.
Key Responsibilities:
- Design and implement big data solutions using technologies such as Hadoop and Spark.
- Design develop deploy and maintain scalable software solutions
- Build and manage large data lakes and data warehouses (e.g. Snowflake Redshift Synapse)
- Develop and optimize data pipelines for processing high volumes of data
- Work with AWS resources including Kinesis EMR Glue SQS Lambda etc.
- Collaborate with crossfunctional teams within an Agile framework (Scrum)
- Ensure the performance scalability and reliability of data solutions
- Lead medium to largescale projects that align with strategic business objectives
- Develop and maintain ETL processes to acquire and process large volumes of data.
- Optimize and tune the performance of big data applications and platforms.
- Collaborate with data scientists and analysts to understand data requirements and deliver appropriate solutions.
- Conduct data modeling to ensure efficient storage and retrieval of data.
- Implement and manage data security and privacy measures.
- Identify and address data quality issues and ensure data accuracy.
- Contribute to the development of data governance practices and standards.
- Stay updated on emerging big data technologies and recommend potential solutions.
- Provide technical leadership and mentorship to junior team members.
Required Qualifications:
- Bachelors or Masters degree in Computer Science Engineering or a related field.
- Proven 7 experience in developing and maintaining big data solutions
- Strong experience with SQL and NoSQL databases
- Proficient in data warehousing and data lake architecture
- Experience with Snowflake or similar data warehousing technologies
- At least 2 years of experience in deploying and maintaining software in public cloud environments like AWS or Azure
- Proficiency in programming languages such as Python Java or Scala.
- Strong understanding of Hadoop ecosystem including HDFS MapReduce YARN and Hive.
- Experience with realtime data processing frameworks like Kafka or Storm.
- Ability to design and optimize data storage and retrieval processes.
- Expertise in ETL tools and processes.
- Familiarity with cloudbased big data solutions such as AWS EMR or Azure HDInsight.
- Excellent problemsolving and analytical skills.
- Effective communication and teamwork abilities.
kinesis,emr,mapreduce,nosql,lambda,synapse,aws,java,spark,glue,hive,big data,yarn,hadoop,data modeling,cloud,design,snowflake,scala,sql,scrum,hdfs,aws emr,azure hdinsight,data warehousing,etl tools,azure,data,storm,kafka,data solutions,etl,sqs,python,redshift
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
