drjobs Big Data Engineer Various Levels العربية

Big Data Engineer Various Levels

Employer Active

drjobs

Job Alert

You will be updated with latest job alerts via email
Valid email field required
Send jobs
Send me jobs like this
drjobs

Job Alert

You will be updated with latest job alerts via email

Valid email field required
Send jobs
Job Location drjobs

Ro - Italy

Monthly Salary drjobs

Not Disclosed

drjobs

Salary Not Disclosed

Job Description

Big Data Engineer: Various levels (Principal Lead Senior Intermediate Junior)

Job Posting Title: Big Data Engineer

Company

Client Corporation is Fortune300 transportation company specializing in freight railroading. We operate approximately 21000 route miles in 22 states and the District of Columbia serve every major container port in the eastern United States and provide efficient connections to other rail carriers. Client has the most extensive intermodal network in the East and is a major transporter of coal and industrial products.

Job Description

Client Corporation is currently seeking an experienced Data Engineer Big Data individual for their Midtown office in Atlanta GA. The successful candidate must have Big Data engineering experience and must demonstrate an affinity for working with others to create successful solutions. Join a smart highly skilled team with a passion for technology where you will work on our state of the art Big Data Platforms. They must be a very good communicator both written and verbal and have some experience working with business areas to translate their business data needs and data questions into project requirements. The candidate will participate in all phases of the Data Engineering life cycle and will independently and collaboratively write project requirements architect solutions and perform data ingestion development and support duties.

Skills and Experience:

Required:

  • 6 years of overall IT experience
  • 3 years of experience with highvelocity highvolume stream processing: Apache Kafka and Spark Streaming
    • Experience with realtime data processing and streaming techniques using Spark structured streaming and Kafka
    • Deep knowledge of troubleshooting and tuning Spark applications
  • 3 years of experience with data ingestion from Message Queues (Tibco IBM etc.) and different file formats across different platforms like JSON XML CSV
  • 3 years of experience with Big Data tools/technologies like Hadoop Spark Spark SQL Kafka Sqoop Hive S3 HDFS or
  • 3 years of experience building testing and optimizing Big Data data ingestion pipelines architectures and data sets
  • 2 years of experience with Python (and/or Scala) and PySpark/ScalaSpark
  • 3 years of experience with Cloud platforms e.g. AWS GCP etc.
  • 3 years of experience with database solutions like Kudu/Impala or Delta Lake or Snowflake or BigQuery
  • 2 years of experience with NoSQL databases including HBASE and/or Cassandra
  • Experience in successfully building and deploying a new data platform on Azure/ AWS
  • Experience in Azure / AWS Serverless technologies like S3 Kinesis/MSK lambda and Glue
  • Strong knowledge of Messaging Platforms like Kafka Amazon MSK & TIBCO EMS or IBM MQ Series
  • Experience with Databricks UI Managing Databricks Notebooks Delta Lake with Python Delta Lake with Spark SQL Delta Live Tables Unity Catalog
  • Knowledge of Unix/Linux platform and shell scripting is a must
  • Strong analytical and problemsolving skills

Preferred (Not Required):

  • Strong SQL skills with ability to write intermediate complexity queries
  • Strong understanding of Relational & Dimensional modeling
  • Experience with GIT code versioning software
  • Experience with REST API and Web Services
  • Good business analyst and requirements gathering/writing skills

Education

Bachelors Degree required. Preferably in Information Systems Computer Science Computer Information Systems or related field


Required Skills : Data Warehouse
Additional Skills : Big DataThis is a high PRIORITY requisition. This is a PROACTIVE requisition

Employment Type

Full Time

Company Industry

Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.