Task set v1.0 · Core tasks
ARTB-018 · v1.0
What changed in the latest release?
The situation
An app I use just updated. I want to know what changed, from the official source.
What we say to the agent
What's the latest version of {software} and what changed? Link the official release notes.
Changes every round
{software}
Prompt templates are public. The concrete values are published with each round's results.
How it is judged
Pass
- Correct latest version, at least 2 correct changes, official link.
Partial
- Correct version, but changes or link incomplete.
Fail
- Outdated version or invented changes.
Evidence we keep
screen_recording, changelog_screenshot
Results
| Doubao | Not tested |
| Kimi | Not tested |
| Yuanbao | Not tested |
| Coze Space | Not tested |
| AutoGLM | Not tested |
| ChatGPT agent | Not tested |
| Gemini | Not tested |
| Manus | Not tested |
| OpenClaw | Not tested |
| Hermes Agent | Not tested |