Anthropic's performance optimization team has been using a take‑home test for job applicants since 2024. As AI coding assistants like Claude have improved, the company has repeatedly redesigned the test to keep human candidates distinguishable. Team lead Tristan Hume noted that each new Claude model, from Opus 4 to Opus 4.5, has forced a redesign because the AI began matching or exceeding top human performers. While candidates are permitted to use AI tools, the situation raises questions about the test's ability to assess human talent. Hume ultimately created a new, hardware‑agnostic test intended to stump current AI tools and invited the community to propose better challenges.
Read more →