Staff Engineer, Crusoe Managed Data Transfer (Distributed Systems)
Crusoe · San Francisco, California
- Location
- San Francisco, California, USA
- Workplace
- Remote
- Employment
- Full time
- Level
- Staff
- Posted
- 28 October
About this role
Crusoe is on a mission to accelerate the abundance of energy and intelligence. As the only vertically integrated AI infrastructure company built from the ground up, we own and operate each layer of the stack - from electrons to tokens - to power the world's most ambitious AI workloads. When you join Crusoe, you join a team that is building the future, faster.
We're in the midst of the greatest industrial revolution of our time. The demand for AI compute is boundless, and power is a bottleneck. We're solving that - with an energy-first approach that makes AI infrastructure better for the world and faster for the people innovating with AI.
We're looking for problem-solving, opportunity-finding teammates with a sense of urgency, who believe in the scale of our ambition and thrive on a path not fully paved - people who want to grow their careers alongside a team of experts across energy, manufacturing, data center construction, and cloud services.
If you want to do the most meaningful work of your career, help our customers and partners advance their AI strategies, and be part of a high-performing team that believes in each other, come build with us at Crusoe.
About the Role:
We are looking for a Staff Engineer to be the technical lead and systems architect for Crusoe's Managed Data Transfer (MDT) service, the person who owns how its parts fit together and how MDT fits into Crusoe Cloud. MDT moves petabyte-scale training data between hyperscaler storage and Crusoe storage as a managed service, so AI labs and Physical AI companies can train on Crusoe GPUs without building their own transfer pipelines. You will design the control plane, the transfer worker fleet and the integration with Crusoe's storage and network, and set the path for adding other clouds and Crusoe-internal transfers.
This is an engineering leadership role on a new team. Managed Data Transfer starts from a working prototype that already serves customers in production, so your first decisions are about what to keep, harden or replace, and how to sequence that work without disrupting the customers who depend on it. You will lead the design through Crusoe's architecture review, mentor the engineers who build it, and report to the Engineering Manager of Managed Data Transfer within Managed platform services.
What You'll Be Working On:
Architecture Ownership: Own the end-to-end design of MDT: the job API and scheduler, the transfer workers on Kubernetes, storage integration and network paths. You decide what the system looks like and explain why.
Prototype Assessment and Migration: Evaluate the production prototype, decide what gets refactored, extended or replaced, and move its current users onto the managed service without interruption.
Throughput and Correctness: Set the data-plane strategy (sharding, concurrency, straggler handling, checksum verification) and the benchmarks that prove it, aiming for line-rate transfers over dedicated interconnects.
Cross-Team Contracts: Own the interfaces between MDT and Storage, Network, Kubernetes, IAM, Console, Billing and Monitoring. You define what partner teams build to and hold the line on it.
Extensibility and Build-versus-Buy: Design for new clouds and Crusoe-internal transfers with a pluggable transfer engine and regional worker pools, and evaluate open-source and vendor options before the team builds.
Operational Excellence: Set the bar for reliability, observability and on-call for every system the team ships.
Product Partnership: Work with Product on scope, pricing trade-offs and milestones, and surface edge cases early enough to change the plan.
Technical Mentorship: Raise the team's level through design reviews, RFC feedback and direct coaching, so the team runs well without depending on you.
What You'll Bring to the Team:
Distributed Systems Depth: Hands-on experience designing and operating data-intensive distributed systems at scale, including the failure modes that only appear in production.
Professional Experience: 8+ years shipping production software, including leading a service from prototype to general availability.
Data Movement and Storage Performance: Understanding of object storage semantics, file systems and TCP throughput, and the ability to profile and fix a transfer that is slower than it should be.
Systems Integration Judgment: Proven ability to evolve a live system toward a better design while keeping risk to current customers low.
Cross-Team Influence: Experience defining API boundaries and schema contracts with partner teams, and negotiating them to agreement.
Operational Ownership: You have carried customer-facing on-call and built systems that are maintainable, observable and predictable.
Software Engineering Craft: Proficiency in Go or another modern compiled language (Go strongly preferred).
Communication: You can explain an architectural decision to an engineer, a PM and an EM, and you know which parts matter to each.
Bonus Points
Experience building or operating bulk data transfer systems, such as AWS DataSync, Google Storage Transfer Service, or rclone at petabyte scale.
Familiarity with cloud networking and interconnects, such as AWS Direct Connect or GCP Interconnect.
Experience with cross-account cloud IAM and threat modeling for services that hold customer credentials.
Approximate nearest-neighbor (ANN) indexing in depth: HNSW, IVF, product and scalar quantization, DiskANN-style indexes that live on SSD.
Experience with AI, HPC or Physical AI data pipelines.
Benefits:
Competitive compensation and equity packages
Restricted Stock Units
Paid time off, paid holidays & leave of absence programs
Comprehensive health, dental & vision insurance
Employer contributions to HSA account
Paid parental leave
Paid life insurance, short-term and long-term disability
Professional development & tuition reimbursement
Mental health & wellness support
Commuter benefits (parking & transit)
Cell phone stipend
401(k) Retirement plan with company match up to 4% of salary
Volunteer time off
Global travel insurance & emergency assistance
Daily meals allowance
Additional perks & programs specific to location
Compensation Range
Compensation will be paid in the range of up to $215,000 -$260,000 + Bonus. Restricted Stock Units are included in all offers. Compensation to be determined by the applicant's knowledge, education, and abilities, as well as internal equity and alignment with market data.
Crusoe is an Equal Opportunity Employer. Employment decisions are made without regard to race, color, religion, disability, genetic information, pregnancy, citizenship, marital status, sex/gender, sexual preference/ orientation, gender identity, age, veteran status, national origin, or any other status protected by law or regulation.
As published by Crusoe. Applications are handled on their site.
Skills this posting mentions
About Crusoe
Crusoe is building the World’s Favorite AI-first Cloud infrastructure company. We’re pioneering vertically integrated, purpose-built AI infrastructure solutions trusted by Fortune 500 companies to power their most advanced AI applications. Crusoe is redefining AI cloud infrastructure, with a mission to align the future of computing with the future of the climate. Our AI platform is recognized as the “gold standard” for reliability and performance. Our data centers are optimized for AI workloads and are powered by clean, renewable energy.
All 352 openings at CrusoeOne click, then it is written
Apply to Crusoe with a resume written for this role.
Queue Staff Engineer, Crusoe Managed Data Transfer (Distributed Systems) and I read the posting, rewrite your resume against it, draft the cover letter, and score the fit. Then you press send, or press one button and I fill in Crusoe’s form for you.
01Drop your resume
A PDF or a LinkedIn URL. About a minute, once.
02I rank the openings
Every weekday morning, the live catalog scored against your history. Up to 20 worth your time, not two hundred links.
03Each one is written up
Resume rewritten for the posting, a cover letter, a fit score. Press send, or let me fill in the form.
- 25 sent a week, free
- No card
- Nothing sent until you say so
More roles at Crusoe
See all- 29 October
- 29 October
- 29 October
- 29 October
Associate / Senior Associate, Corporate Development
San Francisco, California
$130k - $160k/yrSeniorSales - 29 October
- 29 October
Similar roles elsewhere
See more- 29 October
- 29 October
- 29 October
Staff Software Engineer, Service Configuration Deployments
PinterestSan Francisco, CA +1Remote
StaffEngineering - 29 October
Sr. Software Engineer, AI Platform & Infrastructure
OktaSan Francisco, California
$165k - $227k/yrSeniorEngineering
Put this to work
Paste your career in once. Every application after that is written for you.
Drop a resume or a LinkedIn URL. I rank the live openings against it, rewrite the resume and write a cover letter for the best of them, and fill in the employer's form when you press the button. You read, you decide what goes out.
01Drop your resume
A PDF or a LinkedIn URL. About a minute, once.
02I rank the openings
Every weekday morning, the live catalog scored against your history. Up to 20 worth your time, not two hundred links.
03Each one is written up
Resume rewritten for the posting, a cover letter, a fit score. Press send, or let me fill in the form.
- New matches ranked and written before you are up.
- Every bullet stays inside what your history supports. Nothing invented.
- Queued, submitted, interviewing, offer: one screen, not a spreadsheet.
Free, no card. Up to 25 applications sent a week, and nothing is sent until you say so.
Listed from the job board Crusoe publishes. Refolk is not the employer and does not handle their hiring. Applications go to Crusoe directly.
Hiring at Crusoe? Post this role to the Refolk talent pool for $299, once. No commission, refunded if I find nobody. Post a role for $299