Task set v1.0 · Core tasks
ARTB-012 · v1.0
Remind me at 9 tomorrow
The situation
I need a nudge to call the dentist tomorrow morning.
What we say to the agent
Remind me tomorrow at 9:00 to call the dentist.
Changes every round
- —
Prompt templates are public. The concrete values are published with each round's results.
How it is judged
Pass
- Reminder arrives between 8:55 and 9:05 with the right content, without the user opening the app.
Partial
- Arrives within 30 minutes, or only visible after opening the app.
Fail
- No reminder.
Evidence we keep
screen_recording, notification_screenshot
Results
| Doubao | Not tested |
| Kimi | Not tested |
| Yuanbao | Not tested |
| Coze Space | Not tested |
| AutoGLM | Not tested |
| ChatGPT agent | Not tested |
| Gemini | Not tested |
| Manus | Not tested |
| OpenClaw | Not tested |
| Hermes Agent | Not tested |