Data Engineer
Data Engineer
Posted on :
09-01-2025
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Roles and Responsibilities:
- 0
- LLM LLM
- My main role is to build the pipeline and infrastructure for data collection and I collect unstructured data on the internet cleanse it model it and make it available for products. All data collection systems are frameworkbased and instead of creating them from scratch each time I have built a system where data cleansing and modeling run automatically just by combining input and output
- Recently the size of data has increased and the data engineering team is building data platforms and introducing distributed systems to conduct analysis so if you wish you can also get involved in the fields of data analytics and science. We are particularly focused on the use of LLM and there are projects to verify how LLM can be used to process difficult unstructured data
:
:
- TypeScript React
- Storybook jest
- Amplify
Frontend:
- Language: TypeScript React
- Library: Storybook jest
- Hosting: Amplify
:
- AWS ElasticBeanstalk
- DB Aurora ElasticSearch
- Node.js Python
- Express
- DataDog
- AWS Lambda AWS Batch AWS API GateWay AWS Glue
Server side:
- Infrastructure: AWS ElasticBeanstalk
- DB: Aurora ElasticSearch
- Language: Node.js Python
- Framework: Express
- Monitoring: DataDog
- Other: AWS Lambda AWS Batch AWS API GateWay AWS Glue
:
- OpenAI
- Amazon Bedrock
- OpenSearch
- SageMaker
- Athena
- Glue
Requirements
:
-
-
-
- ETL
- MYSQL MongoDB
-
-
:
-
-
-
- OSS
-
- Tableau Power BI D3.js
- OpenAI LLM
Required Skills:
- Web sing experience
- Data cleansing experience
- Data modeling experience
- ETL experience
- Experience with databases (MYSQL MongoDB)
- Experience in extracting data from unstructured data
- Business level English
Welcome Skills:
- Natural Language Processing Machine Learning
- Streaming processing experience
- Experience in building and operating data platforms
- Development experience outside of work OSS activities etc.
- Native Japanese
- Experience using data visualization tools (Tableau Power BI D3.js etc.)
- Experience using LLM products such as OpenAI
Roles and Responsibilities:
- 0
- LLM LLM
- My main role is to build the pipeline and infrastructure for data collection, and I collect unstructured data on the internet, cleanse it, model it, and make it available for products. All data collection systems are framework-based, and instead of creating them from scratch each time, I have built a system where data cleansing and modeling run automatically just by combining input and output
- Recently, the size of data has increased, and the data engineering team is building data platforms and introducing distributed systems to conduct analysis, so if you wish, you can also get involved in the fields of data analytics and science. We are particularly focused on the use of LLM, and there are projects to verify how LLM can be used to process difficult unstructured data
:
[ ]:
- TypeScript, React
- Storybook, jest
- Amplify
[Frontend]:
- Language: TypeScript, React
- Library: Storybook, jest
- Hosting: Amplify
[ ]:
- AWS, ElasticBeanstalk
- DB Aurora, ElasticSearch
- Node.js, Python
- Express
- DataDog
- AWS Lambda, AWS Batch, AWS API GateWay, AWS Glue
[Server side]:
- Infrastructure: AWS, ElasticBeanstalk
- DB: Aurora, ElasticSearch
- Language: Node.js, Python
- Framework: Express
- Monitoring: DataDog
- Other: AWS Lambda, AWS Batch, AWS API GateWay, AWS Glue
[ ]:
- OpenAI
- Amazon Bedrock
- OpenSearch
- SageMaker
- Athena
- Glue
Requirements
:
-
-
-
- ETL
- MYSQL MongoDB
-
-
:
-
-
-
- OSS
-
- Tableau Power BI D3.js
- OpenAI LLM
Required Skills:
- Web sing experience
- Data cleansing experience
- Data modeling experience
- ETL experience
- Experience with databases (MYSQL, MongoDB)
- Experience in extracting data from unstructured data
- Business level English
Welcome Skills:
- Natural Language Processing, Machine Learning
- Streaming processing experience
- Experience in building and operating data platforms
- Development experience outside of work, OSS activities, etc.
- Native Japanese
- Experience using data visualization tools (Tableau, Power BI, D3.js, etc.)
- Experience using LLM products such as OpenAI
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
