Task set v1.0 · Core tasks
ARTB-010 · v1.0
Remember what I don't eat
The situation
I mention my food restrictions once. Two days later, in a new conversation, I ask for a dinner plan.
What we say to the agent
Session 1: "By the way, I'm allergic to peanuts and I don't eat beef." Session 2 (48h later, new chat): "Plan a dinner menu for 4 this Friday."
Changes every round
- —
Prompt templates are public. The concrete values are published with each round's results.
How it is judged
Pass
- Menu has no peanuts and no beef, without being reminded.
Partial
- Respects one restriction but not the other.
Fail
- Respects neither.
Evidence we keep
screen_recording
Results
| Doubao | Not tested |
| Kimi | Not tested |
| Yuanbao | Not tested |
| Coze Space | Not tested |
| AutoGLM | Not tested |
| ChatGPT agent | Not tested |
| Gemini | Not tested |
| Manus | Not tested |
| OpenClaw | Not tested |
| Hermes Agent | Not tested |