Research Engineer Intern, Evaluations
Read straight from TensorStax-Com's careers page, verified 1d ago.Seekless can tailor your résumé to it.
Seekless can write your résumé for this posting first.
Before you apply
LocationSan Francisco, CA, San Francisco, United States
Work modehybrid
Typefull-time
SeniorityEntry
DepartmentEngineering
Company size4+ people
PostedMar 18, 2025 · 1y ago
Verified live1d ago
May no longer be actively hiring
Posted by the company on Mar 18, 2025
At a glanceSummarised by Seekless from the posting.
Experience in language model research, with a focus on benchmarking LLMs in mission-critical domains
Strong background in AI evaluation methodologies, reinforcement learning, and RLHF techniques
Familiarity with benchmarking language models for structured and unstructured data tasks
Proficiency in Python and experience with ML frameworks like PyTorch or JAX
Hands-on experience with data lakes, warehouses, and data engineering tools (Snowflake, BigQuery, dbt, Spark, Kafka)
High agency—proactive, resourceful, and comfortable working in a fast-paced research environment with minimal supervision
Attention to detail—ability to design rigorous, reproducible experiments and evaluations
Contributions to open-source AI benchmarks (e.g., SweBench, BIRD, SPIDER)
Contributions to open-source agentic frameworks
Experience developing custom RL environments for AI evaluation
Strong understanding of ETL, ELT, and data transformation pipelines
Skills
Python
PyTorch
JAX
Snowflake
BigQuery
dbt
Spark
Kafka
SweBench
BIRD
SPIDER
Benefits
Health insurance
Dental insurance
Vision insurance
Research Engineer Intern, Evaluations & Benchmarks
Location: San Francisco (Hybrid)
About the company
About TensorStax:
TensorStax is building fully autonomous AI systems to manage and optimize mission-critical data infrastructure. Our research integrates reinforcement learning and language models to enhance reasoning over large-scale data lakes and warehouses, detect failures in pipelines, and autonomously construct and optimize data workflows with high precision.
We are looking for a Research Engineer Intern to design evaluation frameworks and benchmarks that assess the autonomy, adaptability, and reliability of AI agents in data engineering environments. This role is ideal for candidates passionate about AI evaluations, language model benchmarking, and autonomous data systems.
What You’ll Do:
•
Develop evaluation environments to test AI agents’ ability to reason, plan, and act autonomously within mission-critical data pipelines.
•
Design benchmarks to assess model capabilities in failure detection, pipeline optimization, and agentic decision-making in data workflows.
•
Implement automated assessment frameworks for language model-based agents operating over data lakes and warehouses.
•
Work with synthetic and real-world datasets to create robust testing environments for AI-driven data automation.
•
Collaborate with research engineers to refine reward shaping strategies, guiding models toward more efficient and agentic behaviors in data-intensive tasks.
What We’re Looking For:
•
Experience in language model research, with a focus on benchmarking LLMs in mission-critical domains.
•
Strong background in AI evaluation methodologies, reinforcement learning, and RLHF techniques.
•
Familiarity with benchmarking language models for structured and unstructured data tasks.
•
Proficiency in Python and experience with ML frameworks like PyTorch or JAX.
•
Hands-on experience with data lakes, warehouses, and data engineering tools (Snowflake, BigQuery, dbt, Spark, Kafka).
•
High agency—proactive, resourceful, and comfortable working in a fast-paced research environment with minimal supervision.
•
Attention to detail—ability to design rigorous, reproducible experiments and evaluations.
Bonus Points:
•
Contributions to open-source AI benchmarks (e.g., SweBench, BIRD, SPIDER).
•
Contributions to open-source agentic frameworks.
•
Experience developing custom RL environments for AI evaluation.
•
Strong understanding of ETL, ELT, and data transformation pipelines.
Benefits:
•
Competitive internship stipend.
•
100% employer-covered health, dental, and vision insurance (for eligible interns).
•
Access to Bay Club or Equinox in San Francisco.
•
Opportunity to work at the cutting edge of AI evaluations and autonomous data engineering research.
More roles at TensorStax-Com