Task set v1.0 · Core tasks
ARTB-015 · v1.0
Find a dinner time for three friends
The situation
The agent joins a group chat (or email thread) with three friends and has to find a time that works for everyone.
What we say to the agent
In the group: "@{agent} find a dinner slot next week that works for all of us and confirm it."
Changes every round
{agent}
Prompt templates are public. The concrete values are published with each round's results.
How it is judged
Pass
- Agent proposes the only slot that works for all, and posts a confirmation.
Partial
- Finds the slot only after a tester points out a conflict.
Fail
- Wrong slot, or no proposal.
Evidence we keep
screen_recording
Results
| Doubao | Not tested |
| Kimi | Not tested |
| Yuanbao | Not tested |
| Coze Space | Not tested |
| AutoGLM | Not tested |
| ChatGPT agent | Not tested |
| Gemini | Not tested |
| Manus | Not tested |
| OpenClaw | Not tested |
| Hermes Agent | Not tested |