ocap

High-performance desktop recorder for Windows. Captures screen, audio, keyboard, mouse, and window events.

This project was first introduced and developed for the D2E project. For more details, see D2E: Scaling Vision-Action Pretraining on Desktop Data for Transfer to Embodied AI If you find this work useful, please cite our paper.

What is ocap?

ocap (Omnimodal CAPture) captures all essential desktop signals in synchronized format. Records screen video, audio, keyboard/mouse input, and window events. Built for the open-world-agents project but works for any desktop recording needs.

TL;DR: Complete, high-performance desktop recording tool for Windows. Captures everything in one command.

demo.mp4

📊 Working with recorded data? See the OWAMcap Format Guide for analysis, processing, and ML integration.

Key Features

Feature	ocap	OBS	wcap	pillow/mss
Advanced data formats (OWAMcap)	✅ Yes	❌ No	❌ No	❌ No
Timestamp aligned logging	✅ Yes	❌ No	❌ No	❌ No
Customizable event definition & Listener	✅ Yes	❌ No	❌ No	❌ No
Single python file	✅ Yes	❌ No	❌ No	❌ No
Audio + Window + Keyboard + Mouse	✅ Yes	⚠️ Partial	❌ No	❌ No
Hardware-accelerated encoder	✅ Yes	✅ Yes	✅ Yes	❌ No
Supports latest Windows APIs	✅ Yes	✅ Yes	✅ Yes	❌ No (legacy APIs only)
Optional mouse cursor capture	✅ Yes	✅ Yes	✅ Yes	❌ No

• Complete desktop recording: Video, audio, keyboard/mouse events, window events
• High performance: Hardware-accelerated with Windows APIs and GStreamer
• Efficient encoding: H265/HEVC for high quality and small file size
• Simple operation: ocap FILE_LOCATION (stop with Ctrl+C)
• Clean architecture: Core logic in a single 400-line recorder.py
• Modern formats: MKV with embedded timestamps, OWAMcap format for events (built on MCAP)

System Requirements

Based on OBS Studio recommended specs + NVIDIA GPU requirements:

Component	Specification
OS	Windows 11 (64-bit)
Processor	Intel i7 8700K / AMD Ryzen 1600X
Memory	8 GB RAM
Graphics	NVIDIA GeForce 10 Series or newer ⚠️
DirectX	Version 11
Storage	600 MB + ~100MB per minute recording

⚠️ NVIDIA GPU Required: Currently only supports NVIDIA GPUs for hardware acceleration. AMD/Intel GPU support possible through GStreamer framework - contributions welcome!

🖥️ OS Support: Currently only supports Windows. However, support for other operating systems (Linux, macOS) can be relatively easily extended due to the presence of GStreamer. Simply using different GStreamer pipelines can enable capture on other platforms - contributions welcome!

Installation & Usage

Option 1: Download Release

1. Download ocap.zip from releases
2. Unzip and run:
   • Double-click run.bat (opens terminal with virtual environment)
   • Or in CLI: run.bat --help

Option 2: Package Install

All OWA packages are available on PyPI:

# Install GStreamer dependencies first (for video recording)
$ conda install open-world-agents::gstreamer-bundle

# Install ocap
$ pip install ocap

Basic Usage

# Start recording (stop with Ctrl+C)
$ ocap my-recording

# Show all options
$ ocap --help

# Advanced options
$ ocap FILENAME --window-name "App"   # Record specific window
$ ocap FILENAME --monitor-idx 1       # Record specific monitor
$ ocap FILENAME --fps 60              # Set framerate
$ ocap FILENAME --no-record-audio     # Disable audio

Output Files

• .mcap — Event log (keyboard, mouse, windows) in OWAMcap format
• .mkv — Video/audio with embedded timestamps

Your recording files will be ready immediately!

Technical Architecture

Built on GStreamer with clean, maintainable design:

flowchart TD
    %% Input Sources
    A[owa.env.desktop] --> B[Keyboard Events]
    A --> C[Mouse Events] 
    A --> D[Window Events]
    E[owa.env.gst] --> F[Screen Capture]
    E --> G[Audio Capture]
    
    %% Core Processing
    B --> H[Event Queue]
    C --> H
    D --> H
    F --> H
    F --> I[Video/Audio Pipeline]
    G --> I
    
    %% Outputs
    H --> J[MCAP Writer]
    I --> K[MKV Pipeline]
    
    %% Files
    J --> L[📄 events.mcap]
    K --> M[🎥 video.mkv]
    
    style A fill:#e1f5fe
    style E fill:#e1f5fe
    style H fill:#fff3e0
    style L fill:#e8f5e8
    style M fill:#e8f5e8

Loading

• Easy to verify: Extensive OWA's Env design enables customizable recorder.py
• Native performance: Direct Windows API integration (DXGI/WGC, WASAPI)

Troubleshooting

• Record terminates right after start? Re-run the same command a few times. This is due to an intermittent GStreamer crash with an unknown cause.
• GStreamer error message box appears on first run? This is a known issue where GStreamer may show error dialogs the first time you run ocap. These messages do not affect recording—simply close the dialogs and continue. ocap will function normally.
• Audio not recording? By default, only audio from the target process is recorded. To change this, manually edit the GStreamer pipeline.
• Large file sizes? Reduce file size by adjusting the gop-size parameter in the nvd3d11h265enc element. See pipeline.py.
• Performance tips: Close unnecessary applications before recording, use SSD storage for better write performance, and record to a different drive than your OS drive.

FAQ

• How much disk space do recordings use? ~100MB per minute for 1080p H265 recording.
• Will ocap slow down my computer? Minimal impact with hardware acceleration. Designed for low overhead.
• What is OWAMcap format? A specialized format that stores screen video (.mkv) + synchronized events (.mcap) for AI training. Contains keyboard, mouse, window events with nanosecond precision. Learn more →
• Can I save recording in other formats? Yes sure, all the source code you must edit is single recorder.py. You can implement JSONL, Parquet, CSV, anything you want easily.

When to Use ocap

• AI Agent Training: Capture desktop interactions for training multimodal models
• Workflow Documentation: Record exact steps with precise timing
• Performance Testing: Low-overhead recording during intensive tasks
• Research & Datasets: Generate standardized OWAMcap data for the community (HuggingFace Hub)

Citation

If you find this work useful, please cite our paper:

@article{choi2025d2e,
  title={D2E: Scaling Vision-Action Pretraining on Desktop Data for Transfer to Embodied AI},
  author={Choi, Suwhan and Jung, Jaeyoon and Seong, Haebin and Kim, Minchan and Kim, Minyeong and Cho, Yongjun and Kim, Yoonshik and Park, Yubeen and Yu, Youngjae and Lee, Yunsung},
  journal={arXiv preprint arXiv:2510.05684},
  year={2025}
}