Spark Developer Lead

We are searching for a Spark Developer for a Direct Placement assignment with one of our premier clients in the USA



Role: Spark Developer
Location: Tampa FL (Hybrid)
Duration: Full Time
Experience: Should have Lead Exp

MUST HAVE: SPARK, SCALA, HBASE, JSON with BANKING EXPERIENCE

Job Responsibilities
Significant experience with big data ETL pipeline development with Spark and related technologies
Significant experience with programming language such as Python or Scala
Experienced with Spark framework and related tools (PySpark, SparkR, Spark SQL, Spark UI)
Experienced with Spark-job server and fine tuning the job server configurations
Solid understanding of performance tuning concepts for Apache Spark jobs and handling large data volumes data from various sources/formats
Optimize Spark code, identify the bottlenecks, understand the scalability requirements
Experienced with Data wrangling and creating workable datasets and work on different file formats like Parquet, ORC, Sequence files, and different serialization formats like Avro
Solid experience with AWS EMR cluster and cluster configurations such as auto-scaling etc..
Comfortable with deploying and managing Spark applications with Yarn
Comfortable with using Spark cluster monitoring tools and integrations
Comfortable in Shell-Scripts, Cron Automation, and Regular Expressions
Comfortable with Cloud systems and AWS bigdata services

Skills : JSON , HBASE , SPARK SQL , SHELL , SPARK , AWS , BANKING , SCALA , PYTHON