Big Data Engineer Java or Python

Job Description:

Required Skills:

6 years of software development experience and leading teams of engineers and scrum teams

3 years of handson experience of working with MapReduce Hive Spark (core SQL and PySpark) Solid Datawarehousing conceptsKnowledge of Financial reporting ecosystem will be a plus

Expert on Distributed ecosystem Handson experience with programming using Core Java or Python/ScalaExpert on Hadoop and Spark Architecture and its working principle Handson experience on writing and understanding complex SQL(Hive/PySparkdataframes) optimizing joins while processing huge amount of data Experience in UNIX shell scripting Ability to design and develop optimized Data pipelines for batch and real time data processing. Should have experience in analysis design development testing and implementation of system application. Demonstrated ability to develop and document technical and functional specifications and analyze software and system processing flows.

Preferred Qualifications: Knowledge of cloud platforms like GCP/AWS building Microservices and scalable solutions will be an advantage1 years of experience in designing and building solutions using Kafka streams or queues. Experience with GitHub/Bitbucket and leveraging CI/CD pipelines Experience with NoSQL i.e. HBase Couchbase MongoDB is good to have Excellent technical and analytical aptitudeGood communication skills Excellent Project management skills. Results driven