Big Data Engineer
Job Summary
4-7 years of experience delivering enterprise software solutions
Proficient with Java Spark Scala Python AWS Cloud technologies
2 years of experience across multiple Spark technologies such as MapReduce Hive Kafka Scala
Flair for data schema data model how to bring efficiency in big data related life cycle.
Experience with Agile Development methodologies.
Experience with data ingestion and transformation
Have understanding for secure application development methodologies.
Experience in with Airflow and Python will be preferred.
Understanding of automated QA needs related to Big data technology.
Strong object-oriented design and analysis skills
Excellent written and verbal communication skills
Responsibilities
Utilize your software engineering skills including Spark Python Scala to Analyze disparate complex systems and collaboratively design new products and services
Integrate new data sources and tools
Implement scalable and reliable distributed data replication strategies
Responsibilities
Collaborate with other teams to design and develop and deploy data tools that support both operations and product use cases
Perform analysis of large data sets using components in AWS Cloud
Own product features from the development testing through to production deployment
Evaluate big data technologies and prototype solutions to improve our data processing architecture
Automate different pipelines
Qualifications
Knowledge and hands-on experience of using AI technologies for development Testing and automation.
Knowledge of Cloud Technologies is a must such as AWS and its services
Required Experience:
IC
About Company
Bristlecone is the leading provider of connected logistics solutions, including AI applications, SAP integration, and more.