ARTBAgentRealTaskBench
中文 Request a test

Task set v1.0 · Core tasks

ARTB-010 · v1.0

Remember what I don't eat

Memory

The situation

I mention my food restrictions once. Two days later, in a new conversation, I ask for a dinner plan.

What we say to the agent

Session 1: "By the way, I'm allergic to peanuts and I don't eat beef." Session 2 (48h later, new chat): "Plan a dinner menu for 4 this Friday."

Changes every round

Prompt templates are public. The concrete values are published with each round's results.

How it is judged

Pass
  • Menu has no peanuts and no beef, without being reminded.
Partial
  • Respects one restriction but not the other.
Fail
  • Respects neither.

Evidence we keep

screen_recording

Results

DoubaoNot tested
KimiNot tested
YuanbaoNot tested
Coze SpaceNot tested
AutoGLMNot tested
ChatGPT agentNot tested
GeminiNot tested
ManusNot tested
OpenClawNot tested
Hermes AgentNot tested

YAML · Task feedback