AI SDK Computer Use Demo

An open-source AI chatbot demonstrating computer use capabilities with Anthropic Claude Sonnet 4.5, Vercel Sandboxes, and the AI SDK by Vercel.

Features · How It Works · Deploy Your Own · Running Locally

Features

• Streaming text responses powered by the AI SDK.
• Anthropic Claude Sonnet 4.5 with computer use and bash tool capabilities.
• Remote desktop environment running in a Vercel Sandbox with Chrome, a window manager, and VNC streaming.
• shadcn/ui components for a modern, responsive UI powered by Tailwind CSS.
• Built with the latest Next.js App Router.

How It Works

The app spins up a Vercel Sandbox from a pre-built snapshot that includes:

• Xvnc — a virtual X11 display server
• openbox — a lightweight window manager
• noVNC + websockify — streams the desktop to the browser via WebSocket
• Google Chrome — auto-launched so the AI agent has a browser ready
• xdotool + ImageMagick — for mouse/keyboard control and screenshots

When a user sends a message, Claude uses the computer tool (screenshot, click, type, scroll) and the bash tool (run shell commands) to interact with the sandbox desktop. The noVNC stream is displayed in a resizable iframe alongside the chat.

Architecture

User ↔ Next.js Chat UI ↔ AI SDK ↔ Claude Sonnet 4.5
                                        ↓
                                  Vercel Sandbox
                              ┌─────────────────────┐
                              │  Xvnc (:99)         │
                              │  openbox             │
                              │  Chrome              │
                              │  websockify → noVNC  │
                              └─────────────────────┘
                                        ↓
                              noVNC iframe in browser

Deploy Your Own

You can deploy your own version to Vercel by clicking the button below:

Running Locally

Prerequisites

• Node.js 18+
• A Vercel account (for Sandbox access)
• An Anthropic API key

1. Install dependencies

pnpm install

2. Set up Vercel credentials

Install the Vercel CLI and link your project:

pnpm install -g vercel
vercel link
vercel env pull

This creates a .env.local file with VERCEL_OIDC_TOKEN for Sandbox authentication.

Alternatively, set VERCEL_TOKEN, VERCEL_TEAM_ID, and VERCEL_PROJECT_ID manually in your .env.local.

3. Create a sandbox snapshot

The snapshot pre-installs the desktop environment (Xvnc, Chrome, openbox, noVNC, xdotool, ImageMagick) so sandboxes boot in seconds.

npx tsx lib/sandbox/create-snapshot.ts

This takes ~10 minutes. When done, it outputs a snapshot ID. Add it to your .env.local:

SANDBOX_SNAPSHOT_ID=snap_xxxxxxxxxxxxx

4. Add your Anthropic API key

ANTHROPIC_API_KEY=sk-ant-...

5. Start the dev server

pnpm dev

Open http://localhost:3000 to use the computer use agent.

Environment Variables

Variable	Required	Description
ANTHROPIC_API_KEY	Yes	Anthropic API key for Claude
SANDBOX_SNAPSHOT_ID	Yes	Vercel Sandbox snapshot with the desktop environment
VERCEL_OIDC_TOKEN	Yes*	Auto-set by vercel env pull for Sandbox auth
VERCEL_TOKEN	Alt*	Alternative to OIDC — a Vercel personal access token
VERCEL_TEAM_ID	Alt*	Required with VERCEL_TOKEN
VERCEL_PROJECT_ID	Alt*	Required with VERCEL_TOKEN

* Either VERCEL_OIDC_TOKEN (via vercel env pull) or the VERCEL_TOKEN + team/project IDs are required for Sandbox authentication.