Principal Engineer – Distributed Systems (GPU Edge + Inference)
Read straight from Elloe AI's careers page, verified 3d ago.Seekless can tailor your résumé to it.
Seekless can write your résumé for this posting first.
Before you apply
SalaryNo pay range posted
LocationAustin, Texas, San Francisco, United States - Remote
Work moderemote
Typefull-time
SeniorityLead
DepartmentEngineering
EquityIncluded
Company size11–50 people
PostedJul 21, 2025 · 1y ago
Verified live3d ago
May no longer be actively hiring
Posted by the company on Jul 21, 2025100% of this company's open roles have been listed for 90+ daysNo pay range listed, though California requires one
At a glanceSummarised by Seekless from the posting.
Senior systems engineer with GPU fleet experience (KubeRay, Istio, Envoy)
Operated real-time AI infra with 10M+ QPS loads
Comfortable with compliance observability and infra governance
Skills
KubeRay
Istio
Envoy
SHAP
VaultChain
Benefits
Equity
Full-time | Remote | Infrastructure | Reports to CTO
About Elloe
Elloe is the trust layer for AI.
We sit between the world’s most powerful language models and the institutions that can’t afford to get it wrong — hospitals, banks, regulators. We trace and block failures in real time. That’s not marketing — we’re deployed at the European Commission, with NIH clinical trials, and inside a Top-5 EU bank catching GDPR violations live.
This is the enforcement layer GenAI has been missing. We’re not visualizing problems — we’re fixing them.
About the Role
You’ll lead our GPU-edge inference systems. From chaos-resilient deployment to SHAP-driven compliance metrics, you’ll own global infra that makes AI safe and performant.
What You’ll Own
1.
Global Edge Routing
•
Design zone-routing that ensures <50ms SLA in 10+ regions
•
Build fallback orchestration to handle compliance-aware rollbacks
2.
GPU Infra Ops
•
Maximize utilization across 100K+ GPUs via mesh & load prediction
•
Integrate compliance overlays with VaultChain and SHAP triggers
3.
Reliability Telemetry
•
Ship /vault/audit, /inference/predict, /compliance/log endpoints
•
Trace every edge request across governance and model layers
Who You Are
•
Senior systems engineer with GPU fleet experience (KubeRay, Istio, Envoy)
•
Operated real-time AI infra with 10M+ QPS loads
•
Comfortable with compliance observability and infra governance
Why This Matters
Our competitive edge isn’t just AI — it’s defensible enforcement. This role turns that into product.
You’ll Leave This Role With
•
Referenceable contributions to enforcement infra that’s live in EU and US institutions
•
First-hand product work across legal, engineering, and GTM teams
•
Influence over how regulatory primitives become systems people trust
Logistics & Application
•
Start Date: Flexible (Q3–Q4 ideal)
•
Location: Remote-first; timezone overlap with NY or EU preferred
•
Compensation: Top of market salary + equity
•
To Apply: Send your resume and a sentence on the hardest infra problem you’d want to own at scale.
More roles at Elloe AI
Member of Technical Staff – AGI Governance Researcher (Constitutional Safety)Member of Technical Staff – ML Security Engineer (Red Teaming + Patch Risk Forecasting)LLM Red Team Intern (Evaluation Systems)Product Manager – Enforcement Infrastructure (LLM + Vault)Member of Technical Staff – Explainability Engineer (Alignment + SHAP)