Software Engineer, Benchmarking
Epoch AIAbout the role
Please do not include a cover letter, photograph, or headshot of yourself, or any personal information that is not relevant to the role for which you're applying (including marital status, age, identity traits, etc.).
We are looking for a Software Engineer to help us expand and develop our AI Benchmarking Hub. You will work closely with the rest of the benchmarking team to run and maintain benchmarks, integrate with AI providers, set up existing benchmarks to run on our infrastructure, help design and develop brand new benchmarks, and facilitate internal experiments.
This role is fully remote, and we are able to hire in many countries. We invite anyone who is interested to apply, regardless of background, experience, or credentials.
Applications are rolling.
Key Responsibilities
- Implement benchmarks: Implement AI benchmarks within our evaluation infrastructure (primarily using the Inspect library) to expand the suite of capabilities we track. Develop our existing suite of benchmarks so we can quickly and painlessly evaluate new model releases.
- Develop new benchmarks: Contribute to the development of brand new benchmarks. You will have the opportunity to pitch and prototype your own ideas in addition to helping out with existing projects.
- Collaborate: Work closely with researchers, analysts, and other engineers at Epoch AI to ensure evaluation data and outputs are accurate, insightful, and effectively integrated into our research products and publications.
What we are looking for
- Solid engineering skills: A strong software engineering background with more than two years of professional experience building and maintaining complex systems. You are expected to regularly contribute high-quality, robust, and maintainable code and be comfortable diving deep into existing codebases and infrastructure.
- Ideas and creativity:Candidates should be able to generate their own ideas for new benchmarks, experiments, novel things to try, and other projects.
- Mission-driven: You’re motivated by Epoch AI’s mission to provide rigorous, independent insight into key trends in AI. You want to deliver public, trustworthy evaluations of AI capabilities on challenging benchmarks, empowering researchers, policymakers, and the wid
Opens the company's application page
Listed via
Himalayas
himalayas.app
Similar roles
Design & Tech
Related reads from TCHNX
Why Do AI-Generated Algorithmic Interfaces Feel Wrong?
AI-optimized interfaces may be mathematically efficient, but they often violate the psychological principles humans expect. I examine why algorithmic design creates friction, even when the data suggests it shouldn't.
Why AI Design Tools Are Quietly Replacing Junior Designers and What Actually Comes Next
AI tools promise efficiency, but London studios are discovering an unexpected paradox: automation creates new bottlenecks requiring precisely the expertise being eliminated. We investigate what's actually happening to entry-level design work.
How Passive Data Collection is Reshaping UX Research
As users grow weary of surveys and interviews, researchers are turning to ambient behavioural signals from keystroke dynamics to micro-interactions to understand product experience without asking a single question.