Frontline Evaluator - Frontier AI Systems
Read straight from Transluce's careers page, verified 2d ago.Seekless can tailor your résumé to it.
Seekless can write your résumé for this posting first.
Before you apply
Salary$250k – $500k
LocationSan Francisco, United States
Work modeon-site
Typefull-time
DepartmentEngineering & Research
Company size51–200 people
PostedSep 24, 2026 · 1mo ago
Verified live2d ago
At a glanceSummarised by Seekless from the posting.
Strong empirical judgment to extract meaningful findings from data, design follow-up experiments, and identify anomalies
Proficiency in Python to implement data analysis, experiments, and evaluation tooling
Ability to turn an ambiguous concern into a testable question
Operational resourcefulness, adaptability, and sound prioritization in the face of incomplete information
Ability to iterate quickly and balance between scrappiness and thoroughness based on the impact needs of a project
Strong communication skills, including the ability to clearly explain technical findings to various readers
Collaborative orientation, low ego, and openness to both giving and receiving feedback
Experience with evaluation (e.g. model-based judges or agent evaluations), red-teaming, or rapid-response and incident-analysis work on frontier systems
A track record of insightful empirical investigations shared through reports, blog posts, research, or independent projects
Practical understanding of how model training and deployment can affect behavior and evaluation results
Experience in customer-facing, consulting, or forward-deployed roles translating ambiguous partner needs into concrete deliverables
Experience delivering technical work under tight deadlines or in unfamiliar or constrained environments
We are open to sponsoring international visas
Skills
Python
Salary range: $270,000 - $600,000/year + benefits
About the company
Description: Transluce is a fast-moving nonprofit research lab building the public tech stack for AI evaluation and oversight. We have contributed foundational research to the study of AI agents and their behaviors, and we put the results where they can change decisions: in the hands of labs, policymakers, and the public.
About the role
About the role: We are looking for a frontline evaluator to investigate the honesty and alignment of frontier AI systems. You will develop and run rigorous automated evaluations, conduct novel analyses of massive datasets, and surface behaviors of interest, writing up your results for technical, policy, and lab audiences. Example behaviors of interest include misreporting results, falsely claiming success, evaluation awareness, and memetic effects within AI swarms. Your work will uncover risks that might otherwise go unnoticed, turning observations into evidence for the public to decide how AI is built, deployed, and governed.
As an early member of a highly collaborative team, you will learn and grow quickly, and work with our governance and infrastructure teams to scale your impact and technical reach. Your work will be with frontier labs, with governments, and on key topics of public interest — for example, rapid-response investigations of major incidents, public reports on frontier model behavior, and serving as an independent evaluator for governments such as the EU. It may include embedded evals within frontier AI labs as those opportunities arise. As we further develop this approach to evaluating frontier AI systems, we expect the role to evolve.
Core responsibilities:
•
Develop and conduct evaluations to investigate misalignment and unexpected behaviors in AI systems
•
Write code to quickly build and run evaluation and analysis workflows, including data-science tools, LLM-as-a-judge pipelines, and evaluation environments
•
Analyze agent transcripts, datasets, and evaluation results, finding notable patterns or unexpected behaviors to investigate
•
Perform investigations under time and access constraints, iterating quickly while validating conclusions
•
Translate findings into clear, rigorous written reports
•
Collaborate with teammates and external technical stakeholders to conduct evaluations, communicate progress, and share relevant findings
What we’re looking for:
•
Strong empirical judgment to extract meaningful findings from data, design follow-up experiments, and identify anomalies.
•
Proficiency in Python to implement data analysis, experiments, and evaluation tooling.
•
Ability to turn an ambiguous concern into a testable question.
•
Operational resourcefulness, adaptability, and sound prioritization in the face of incomplete information.
•
Ability to iterate quickly and balance between scrappiness and thoroughness based on the impact needs of a project.
•
Strong communication skills, including the ability to clearly explain technical findings to various readers.
•
Collaborative orientation, low ego, and openness to both giving and receiving feedback.
Preferred qualifications (nice to have):
•
Experience with evaluation (e.g. model-based judges or agent evaluations), red-teaming, or rapid-response and incident-analysis work on frontier systems
•
A track record of insightful empirical investigations shared through reports, blog posts, research, or independent projects.
•
Practical understanding of how model training and deployment can affect behavior and evaluation results.
•
Experience in customer-facing, consulting, or forward-deployed roles translating ambiguous partner needs into concrete deliverables.
•
Experience delivering technical work under tight deadlines or in unfamiliar or constrained environments.
We are hiring at all levels of experience and would encourage those enthusiastic about the role who do not meet all of the qualifications to apply. We are located in San Francisco and excited to work together in-person. We are open to sponsoring international visas.