Task set v1.0 · Core tasks
ARTB-019 · v1.0
Research, spreadsheet, email
The situation
I want a short comparison table of open-source projects, saved as a sheet and emailed to me.
What we say to the agent
Find 3 popular open-source {category} projects on GitHub. Put name, stars and last commit date in a spreadsheet and email it to {me}.
Changes every round
{category}{me}
Prompt templates are public. The concrete values are published with each round's results.
How it is judged
Pass
- Email arrives with a spreadsheet; 3 real projects; stars within 5% and dates correct at run time.
Partial
- Email arrives, but one value is wrong or the table is pasted as text.
Fail
- No email, or 2+ wrong values, or an invented project.
Evidence we keep
screen_recording, received_email, github_snapshot
Results
| Doubao | Not tested |
| Kimi | Not tested |
| Yuanbao | Not tested |
| Coze Space | Not tested |
| AutoGLM | Not tested |
| ChatGPT agent | Not tested |
| Gemini | Not tested |
| Manus | Not tested |
| OpenClaw | Not tested |
| Hermes Agent | Not tested |