Junior/Senior/Staff Software Engineer, Inference / Compute Infrastructure Engineering

ph3bAbout the Role /b /h3pWe're looking for a Software Engineer to build the systems that treat infrastructure as software. This role owns the software state machines that provision hardware, bring it into service, and manage its full lifecycle — turning racks of GPUs into brunning inference clusters /b without a human touching a runbook. The Research and Inference team is your customer: today they file tickets and wait; the target state is that they issue a single API call to stand up, scale, or tear down a cluster, and the system takes care of the rest. The platform is manifest-driven such that teams declare the desired state of a cluster or host — shape, topology, software stack — and the system is responsible for reconciling reality to that manifest, continuously, through every stage of its lifecycle. You will design the engines that manifest the schema, the engines that execute against it, and the workflows that carry a piece of hardware or a cluster from one state to the next—taking it from bare metal to a fully functioning AI cluster for training or inference. /ppYou'll write production code which is typed, tested, versioned, and deployed through CI/CD that models infrastructure state and reconciles it, the same way a Kubernetes controller reconciles a cluster's desired state. Success looks like eliminating manual provisioning work, not documenting it better. /ppA product mindset - you've built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship.bYou build it, you own it. /b You are not only responsible for delivering the software but also for operating and supporting it in production. /ph3bResponsibilities /b /h3ullibBuild the provisioning state machine: /bdesign and implement the software that models the full lifecycle of a physical host from discovery, inference bring-up to GPU driver/CUDA stack, health validation, and decommission/RMA — as explicit, versioned states and transitions. /lilibBuild the self-service API: /bdesign declarative APIs and a control plane so the inference team can request, scale, and tear down inference clusters with one API call — no ticket, no human in the loop. /lilibAutomate self-healing: /bdetect degraded or failed nodes, drain them safely, trigger repair or replacement, and reintroduce healthy capacity into the pool automatically. /lilibOwn reliability of the pipeline: /bidempotency, retries, rollback, and drift detection so the provisioning system is as dependable as any other production service. /lilibPartner with the inference/ML platform team: /bunderstand the cluster shapes they need — topology, interconnect, scheduling constraints — and encode them as first-class abstractions in the platform. /lilibEngineer it like software: /bstrong typing, automated tests, code review, versioning, and CI/CD for infrastructure code — this is a product, not a collection of Ansible playbooks. /li /ulh3bRequirements /b /h3pCore requirements (all levels): /pulliStrong software engineering background in Go, Python, Rust, or similar — you write and test real software for a living. /liliExperience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent to run long-lived, manifest-driven workflows that survive failures and resume mid-execution. /liliExperience building software control planes or orchestration systems that model state and reconcile it over time (e.g., Kubernetes controllers/operators, custom reconciliation loops, workflow engines). /liliExperience with event-driven systems — designing and building software around message queues, event streams, or pub/sub (e.g., Kafka, NATS, SQS) rather than polling or cron-driven scripts. /liliA product mindset. You’ve built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship. /li /ulpNice to have: /pulliExposure to bare-metal provisioning (PXE/iPXE, Redfish/IPMI, BMC) and/or networking fundamentals (VLANs, BGP, fabric design), or GPU/accelerator infrastructure. /liliExperience with GPU cluster software stacks (NCCL, CUDA, InfiniBand/RoCE). /liliPrior work at a hyperscaler, GPU cloud, or datacenter-scale infrastructure organization. /liliSystems programming in Rust or Go. /li /ulh3bAbout Together AI /b /h3pTogether AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month. /ph3bEqual Opportunity /b /h3pTogether AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more. /ppPlease see our privacy policy at /p /p #J-18808-Ljbffr