Task set v1.0 · Core tasks
ARTB-007 · v1.0
Mom's 60th birthday dinner
The situation
Eight of us, a quiet private room, near a specific area, within a budget.
What we say to the agent
Recommend 3 restaurants near {area} for my mom's 60th birthday. 8 people, private room, about {budget} per person. Tell me why each one fits.
Changes every round
{area}{budget}
Prompt templates are public. The concrete values are published with each round's results.
How it is judged
Pass
- All 3 exist and are open, have private rooms (verified by phone or listing), and are within 30% of the budget.
Partial
- 2 of 3 meet every constraint.
Fail
- Fewer than 2 meet every constraint, or any restaurant is invented.
Evidence we keep
screen_recording, verification_notes
Results
| Doubao | Not tested |
| Kimi | Not tested |
| Yuanbao | Not tested |
| Coze Space | Not tested |
| AutoGLM | Not tested |
| ChatGPT agent | Not tested |
| Gemini | Not tested |
| Manus | Not tested |
| OpenClaw | Not tested |
| Hermes Agent | Not tested |