Repository Software Engineering Expert
Help us find where advanced coding agents break in repository software engineering. Build or solve realistic tasks, stress-test generated code, and explain failures with reproducible evidence. Remote project-based work can pay up to $45/hour, depending on the project and demonstrated expertise.
Apply for this role →What you'll do
- Create, solve, or review realistic tasks involving real multi-file repositories, issue diagnosis, code changes, regression tests, and maintainability.
- Test AI-generated code and agent traces for correctness, edge cases, regressions, unsafe assumptions, and incomplete fixes.
- Write strong tests, reference patches, expected outcomes, and concise explanations of the failure mode.
- Review peer-authored tasks for realism, reproducibility, difficulty, and clean evaluation criteria.
What we look for
- Strong practical depth in real multi-file repositories, issue diagnosis, code changes, regression tests, and maintainability.
- Ability to navigate unfamiliar code or specifications and communicate technical decisions in clear written English.
- Verifiable evidence such as meaningful GitHub work, shipped software, a competition profile, technical writing, or a portfolio. Confidential industry work can be described in the application without exposing proprietary material.
- Prior AI-evaluation experience is useful but not required; strong engineering judgement and testing discipline matter more.