Machine Learning Research Engineer, Pretraining Data — Foundation Models
Cupertino, CA - USA
Job Summary
As a Research Engineer on the Pretraining Data team you will own technically challenging projects spanning research algorithm development experimentation and production will investigate new approaches to improving training data quality diversity and efficiency while building robust and scalable infrastructure to support the processing of massive datasets. You will collaborate closely with model training teams to understand the relationship between data and model capabilities design controlled experiments and continuously improve pretraining strategies.
Research design and implement novel algorithms for large-scale pretraining data curation including quality filtering deduplication data selection and mixture and optimize scalable data processing systems capable of handling trillions of tokens across diverse data machine learning-based approaches for data quality assessment synthetic data generation and data efficiency with foundation model researchers to design and execute training experiments analyze model performance and identify opportunities for improving model the relationship between training data characteristics model scaling training dynamics and downstream evaluation research prototypes into efficient reliable and production-ready data current with advances in foundation model research and apply emerging techniques to practical data challenges.
BS MS or PhD in Computer Science Machine Learning or a related technical field or 5 years of equivalent practical programming skills in Python and proficiency in developing production-quality with large-scale data processing distributed computing or machine learning understanding of machine learning fundamentals and modern deep learning analytical and problem-solving skills with the ability to independently investigate complex technical challenges.
Experience with LLM pretraining training data research or foundation model designing and implementing scalable data processing pipelines using Spark Ray or similar distributed computing -on experience with PyTorch model training and experimental with data filtering deduplication data mixture optimization and learned data quality of scaling laws training dynamics and the relationship between data and model with synthetic data generation data selection or model-based data ability to translate research ideas into scalable production or meaningful contributions to machine learning research or open-source projects.
Required Experience:
IC
About Company
Ask Siri to name the most successful company in the world and it might respond: Apple. And it's not just out of familial pride. Apple consistently ranks highly in profit, revenue, market capitalization, and consumer cachet. In 2018, the company became the first reach a trillion dollar ... View more