Software Developement Manager, Apple Services Engineering
Seattle, WA - USA
Job Summary
Today updating a nodes software means choosing between speed and workload impact. This teams job is to remove that tradeoff so we can patch the fleet faster and more often with minimal impact on users. We have not chosen an approach. Live migration snapshot and restore kexec-based host updates live patching and in-place upgrade of the runtime and VMM are all options and a different approach may fit better than any of them. We are looking for an expert to decide and guide: which approach or combination gets us there what it costs in workload impact and operational risk and how we measure the will:nn* Define the fleet patching strategy: evaluate the candidate approaches (including ones we havent listed) choose and make the tradeoffs explicit grounded in a model of how each one fails.n* Set the metrics that matter such as guest-visible downtime total migration time network amplification abortability and time to patch the fleet.n* Design and drive the first implementation building on Cloud Hypervisor and our Linux microVM runtimes on x86 and arm64.n* Hire and grow the team. Depending on your background and interests this is a technical-lead role or a people-management role and we will decide that together.n* Work with the SRE Compute and security organizations on rollout safety and fleet
Bachelors Degree in Computer Science or equivalent related least 8 years of software design and development experience in infrastructure engineering (compute storage and network virtualization).nDeep demonstrated experience in one of the following and working fluency in the other:n * Operating live migration live patching or kexec-style host updates across a production fleet including the failures: non-convergence aborts and rollback and fleet-wide rollout safety.n * Building live migration snapshot/restore or device-state serialization inside a hypervisor or VMM (for example QEMU/KVM Cloud Hypervisor Firecracker Xen or Hyper-V).nAdvanced programming language proficiency in C C Rust Swift or Go with a track record of designing and developing high performance secured scalable and reliable record of owning technical design end-to-end from design docs and architecture tradeoffs through implementation and setting technical direction for a group of engineers with or without formal management responsibility.
Experience with Linux kernel and system programming (KVM userfaultfd VFIO kexec kernel live patching).nUnderstanding of dirty-page tracking memory and device state handling and time and CPU-model compatibility across with live migration of devices that carry hardware state (VFIO vDPA SR-IOV).nExperience designing benchmarks and load tests to characterize downtime and performance impact during an contributing to or maintaining open source virtualization projects (Cloud Hypervisor rust-vmm QEMU KVM).nExperience with Kubernetes node lifecycle draining and fleet rollout building a new team from zero including hiring.
Required Experience:
Manager
About Company
Ask Siri to name the most successful company in the world and it might respond: Apple. And it's not just out of familial pride. Apple consistently ranks highly in profit, revenue, market capitalization, and consumer cachet. In 2018, the company became the first reach a trillion dollar ... View more