Senior Software Developer YT Platform

The company

Nebius AI is an AIcentric public cloud platform specifically crafted to serve AI models for training and inference.

Our mission is to help ML practitioners concentrate on their core jobs while DevOps MLOps and infrastructurerelated tasks are handled by us. The idea is to build an MLspecific cloud platform covering the entire ML lifecycle from A to Z: from data preparation and labeling to ML training and inference.

We recognize the potential of ML and AI technologies and aim to provide our future users with the perfect environment to train and finetune their models. We are committed to delivering the best user experience and excellent customer support.

Four development hubs:
Nebius is headquartered in the Netherlands with hubs in Finland Serbia and Israel.

Data center in Europe:
Our own data center in Finland features server racks designed inhouse for MLspecific high load with powerefficient solutions including a freecooling system.

500 professionals:
Our mature team of engineers has a proven track record in developing sophisticated cloud and ML solutions and designing cuttingedge hardware.

Our team

Nebius was founded by a core team engineers and business professionals with a proven track record of using cloud technologies to create value for other businesses. We know from experience that cuttingedge technologies can only make an impact if their innovation is matched by the level of the experts managing them so as Nebius expands our core priority is to attract the most qualified enthusiasticand driven individuals we can find to join our growing team.

The role

We are looking for a Software Engineer to join our YTsaurus team.

We are building a Nebius YTsaurus platform based on YTsaurus a mature Big Data platform which recently went opensource. It includes distributed storage able to store exabytes of data a scalable MapReduce framework builtin OLTP keyvalue storage and a message queue with a streamprocessing API. It also employs various highlevel computational engines such as YQL CHYT (powered by ClickHouse) and SPYT (powered by Apache Spark).

We are working on making YTsaurus a successful cloud product in both the Managed and Serverless domains. This requires developing various integrations with Nebius Cloud infrastructure such as Identity and Access Management or the Cloud Public API. We are also developing core YTsaurus parts in order to integrate YTsaurus with cloudnative technologies such as introducing user workload isolation via virtualization (a la FireCracker) or developing an autoscaling mechanics for YTsaurus compute nodes based on the amount of enqueued jobs.

At the same time various Nebius teams are aiming to use YTsaurus as a primary storage and compute engine including Billing and Data Office data warehouse and Machine Learning Platform workloads e.g. learning private LLMs. Providing a reliable infrastructure for these teams requires developing a number of automation and operations tools around the largest YTsaurus clusters many of which will later be bundled into the cloud YTsaurus product.

Core server parts of YTsaurus are written in modern C with versatile asynchronous and multithreaded primitives while other parts of the system are written in Go and Python. We tend to write new microservices in Go and Python and we actively use Python for utilities and integration tests.

Our team is a small group of core YTsaurus developers with many years of experience in building distributed systems. We work on system architecture software engineering and operations making sure our code is not only committed and welltested but also works in production without disruptions. Join us and help us build the best BigData platform both outside and inside Nebius!

In this position your responsibility will be to:

• Design and implement new functionalities in the YTsaurus core.
• Work on integrations between YTsaurus and Nebius Cloud platform.
• Support internal and external clients including oncall support.
• Carry out existing YTsaurus cluster operations including deploying new releases monitoring the condition of clusters reading logs investigating issues and fixing bugs.

Some of the projects we offer:

• Design and implement cloud integrations such as Identity and Access Management or Nebius YTsaurus Public API layer.
• Design and implement various autoscaling functionality such as compute node autoscaling or full utilization of unused Nebius GPU nodes.
• Make YTsaurus truly cloudnative e.g. implement S3 support as a cold external medium for YTsaurus.
• Contribute to the opensource YTsaurus repository and collaborate with the YTsaurus developer community.

We expect you to have:

• 5 years of experience as a software engineer.
• Practical knowledge of Python Go Java or C.
• Ready to write code in at least one of the languages above and read code in the rest of them when necessary.
• Knowledge of classical algorithms and data structures.
• Ability to use SSH and read logs of your code.
• Experience developing or operating highload/distributed services.

Does all that sound like your kind of challenge Then join us!