Activity Frames compiles screen activity for agent memory
The arXiv paper describes a deterministic pipeline for auditable computer-use agent memory.
Why it matters
The work suggests that deterministic activity logs could make agent memory more auditable and cheaper than repeatedly using frontier inference to reconstruct user routines. Its evidence is limited to one single-user corpus, so broader reliability and privacy implications remain open.
The key points
- 1.Deterministic compiler creates typed activity frames from screen activity.
- 2.Single-user evaluation reports 98.4% recall-question accuracy.
- 3.The approach targets agent memory, replay, auditability, and cost modeling.
A new arXiv paper, also listed by HF Daily Papers, proposes Activity Frames, a deterministic, zero-model pipeline that compiles passively captured screen activity into typed frames for computer-use agent memory and replay. On a single professional's corpus of 128,756 frames over 51 active days, the system reduced a day of raw capture into a prompt-ready context block 86 times smaller in 68 ms. An agent using that block answered day-recall questions at 98.4% accuracy against an independent oracle, compared with 66-80% for an LLM summary of the same capture.
⚡ Try this today
Read the paper before building screen-memory systems, especially if auditability, caching, or cost controls matter.
Sources & original reporting
This brief summarizes and links to reporting from the publishers below.
Enjoyed this brief? Get the next one in your inbox.
More in Research
New studies probe spatial reasoning in vision models
SpaRRTa, SMA and PinpointQA target gaps in model spatial understanding for embodied AI.
CW-BASS v2 targets pseudo-label filtering with DINOv2 teachers
The arXiv paper proposes a saturation-aware method for semi-supervised semantic segmentation.
Study tracks ChatGPT Enterprise use across organizations
The paper links account records to roles, tasks and public-company data through March 2026.