Senior Data Engineer
Senior Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
- Understand the factories manufacturing process data availability and avenues for
- improvement
- Brainstorm together with engineering manufacturing and quality problems that can be solved using the acquired data in the data lake platform.
- Define what data is required to create a solution and work with connectivity engineers users to collect the data
- Create and maintain optimal data pipeline architecture.
- Assemble large complex data sets that meet functional / nonfunctional business requirements.
- Identify design and implement internal process improvements: automating manual processes optimizing data delivery for greater scalability
- Work on data preparation data deep dive help engineering process and quality to understand the process/ machine behavior more closely using available data Deploy and monitor the solution
- Work with data and analytics experts to strive for greater functionality in our data systems.
- Work together with Data Architects and data modeling teams.
- Good knowledge of the business vertical with prior experience in solving different use cases in the manufacturing or similar industry
- Ability to bring cross industry learning to benefit the use cases aimed at improving manufacturing process
- Experience in problem scoping solving quantification
- Strong analytic skills related to working with unstructured datasets.
- Build processes supporting data transformation data structures metadata dependency and workload management.
- Working knowledge of message queuing stream processing and highly scalable big data data stores
- Ability to foresee and identify all right data required to solve the problem
- Strong skill in data mining data wrangling techniques for creating the required analytical dataset
- Experience building and optimizing big data data pipelines architectures and data sets
- Adaptive mindset to improvise on the data challenges and employ techniques to drive desired outcomes
- Experience with big data tools: Spark Delta CDC NiFi Kafka etc
- Experience with relational SQL NoSQL databases and query languages including oracle Hive sparkQL.
- Experience with objectoriented languages: Scala Java C etc.
- Know how of any visualization tools such as PowerBI Tableau
- Good storytelling skills to present the data in simple and meaningful manner.
- Strong skill in data analysis techniques to generate finding and insights by means of exploratory data analysis
- Good understanding of how to transform and connect the data of various types and form
- Great numerical and analytical skills
- Identify opportunities for data acquisition
- Explore ways to enhance data quality and reliability
- Build algorithms and prototypes
- Reformulating existing frameworks to optimize their functioning.
- Good understanding of optimization techniques to make the system performant for requirements.
data visualization (powerbi, tableau),data analysis techniques,kafka,sparql,optimization techniques,object-oriented programming (scala, java, c++),cdc,problem scoping and definition,data pipeline architecture,spark,data wrangling,dwh,relational sql and nosql databases,big data tools (spark, delta, cdc, nifi, kafka),scala
Employment Type
Full Time
Company Industry
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
