watch-and-learn-computer-use-videos

Convert internet tutorial videos into executable UI action trajectories using inverse dynamics models trained on 600K synthetic screen transitions. Generate 53K high-quality demonstrations spanning 69 applications, achieving state-of-the-art 7B agent performance on WindowsAgentArena via both in-context learning and supervised fine-tuning.

Independent GitHub snapshot · Automated classification · Original documentation governs