Task set v1.0 · China local
ARTB-C04 · v1.0
300 messages in the family group
The situation
I only want to know who is hosting New Year dinner and when.
What we say to the agent
Summarize the {group} chat: who is hosting New Year dinner, what date, and what I need to bring.
Changes every round
{group}
Prompt templates are public. The concrete values are published with each round's results.
How it is judged
Pass
- Host, date and item all match the final plan.
Partial
- 2 of 3 correct.
Fail
- Reports the outdated plan, or 1 or fewer correct.
Evidence we keep
screen_recording
Results
| Doubao | Not tested |
| Kimi | Not tested |
| Yuanbao | Not tested |
| Coze Space | Not tested |
| AutoGLM | Not tested |
| ChatGPT agent | Not tested |
| Gemini | Not tested |
| Manus | Not tested |
| OpenClaw | Not tested |
| Hermes Agent | Not tested |