Research Stream Lead

About METR

We are a nonprofit research organization that develops scientific methods to assess AI capabilities, risks and mitigations, with a specific focus on threats related to autonomy, AI R&D automation, and alignment.

We believe it is robustly good for civilization to have a clearer understanding of what dangers AI systems pose, and we are extremely excited to find ambitious, excellent people to join our team and tackle one of the most important challenges of our time.

We evaluate candidates primarily through (paid) work tests. We usually do an in-person trial as well but can be flexible about this.

• How can we tell models aren’t undermining our evaluations by sandbagging or alignment faking?
• How much cognition can models do without revealing it in their reasoning traces?
• How close are models to being able to sabotage research at AI companies or at METR?
• How reliable are safety claims made by AI developers based on new techniques, e.g. activation steering?
• What egregiously misaligned behaviors do models display? How good are monitors at picking up on this, or do they tend to collude with (or get exploited by) the agents?
• Build model organisms or red-teaming approaches to test the robustness of METR’s or external evaluations and safety measures

• Collaborate with other research streams to identify the questions that need to be answered for METR to be able to accurately assess catastrophic risk from models in the near and longer-term future.
• Lead research to answer these questions as cheaply and effectively as possible, trading off between being scrappy and careful in the right places. 
• Significantly improve METR's risk reports based on the research your team has done.
• Publish 'fundamental' research that’s similarly impactful to the time horizon methodology. Your team's work makes foundational progress, improves collective understanding of the relevant phenomena, and becomes the standard way to think and talk about it, for METR and our target audience (highly informed and engaged people but who may be skeptical of AI risk or of METR).
• Maintain METR’s high-integrity culture - communicate research accurately and don’t overhype - even critics or skeptics generally praise the work's quality.
• Build a strong team, identify talent needs and be an effective hiring manager, maintain high standards of performance on your team, grow and empower top performers.
• Lead team effectively, get people excited about and bought into goals, maintain motivation and momentum. 
• Maintain high research velocity: we're generally learning meaningful new things every week or two (unless we're explicitly investing in a large high-payoff project we've derisked).

• Experience as a research manager in an ML related area (or a technical non-ML field like quantitative trading while also keeping up with relevant ML / AI safety literature)
• Experience as a technical engineering manager, including with some experience with fast-moving/scrappy research workflows
• Track record of high-quality ML research, with clear evidence of multiple impactful research outputs (such as papers, blog-posts, etc.) where you are a key contributor. These works feature well-designed methodology or experiments, are well-written, and clearly and carefully communicated without overstating results.
• High quality output that's highly relevant to METR's work. You have public research, writing, code or some other artifact that demonstrates your careful thinking, deep understanding of and ability to make progress on METR’s research directions.
• Evidence of outstanding achievement. You have some other impressive achievement that demonstrates you have the relevant skills or can learn very quickly, such as founding a successful startup, having widely-used open-source projects, or having made scientific contributions outside your area of expertise. These are not exhaustive and we'd be excited to hear about other evidence of outstanding achievements!

Our Culture

METR is a mission-driven organization. We believe our work can meaningfully shape humanity's future for the better, and we want to be the best people in the world doing this work. We have a tight-knit, collaborative research culture rooted in truth-seeking and integrity. We're fiercely committed to producing high-quality, trustworthy science. We're honest and transparent about our results, especially when they may go against the grain. We've earned trust as reliable partners who handle confidential information with care. We maintain a low-ego, drama-free environment focused on what matters.

Hybrid Requirements: Our technical team members are in our office in Berkeley 3-5 days/week. Please let us know in your application if this is a constraint. If you lack US work authorization and would like to work in-person (strongly preferred), we can likely sponsor a cap-exempt H-1B visa for this role.

We encourage you to apply even if your background may not seem like the perfect fit! We would rather review a larger pool of applications than risk missing out on a promising candidate for the position.

We are committed to diversity and equal opportunity in all aspects of our hiring process. We do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. We welcome and encourage all qualified candidates to apply for our open positions.