AI researchers have used all sorts of video games, from Street Fighter to Pictionary, to test new models — often more for amusement than utility. But Anthropic said that Pokémon proved to be a useful benchmark for Claude 3.7 Sonnet, which can effectively “think” through the sorts of puzzles the game contains.

A new Twitch channel called ClaudePlaysPokemon uses Claude's LLM capabilities to see if the AI can become a true Pokemon Master.

Claude 3.7 Sonnet is the latest AI model to play video games successfully. Last March, researchers used ChatGPT to play classic first-person shooter Doom, managing to get to the last room in the game once.