Nyandsign — Hand Sign Con

What is NyandSign?

NyandSign turns your webcam into a hands-free media controller. Using Google MediaPipe's on-device hand tracking AI, it detects hand signs in real time and maps them to media playback actions — all processed locally in your browser with zero data sent to any server.

Open the side panel, start your camera, and control YouTube, Spotify, Netflix, or any browser media with intuitive hand gestures. No extra hardware required — just your built-in webcam.

KEY FEATURES

🖐️ 8 Recognizable Hand Signs
Fist, Peace, OK, Aloha (shaka), Point Right, Point Left, Thumbs Up, and Thumbs Down — each mappable to a media action of your choice.

🎬 Full Media Control
▶⏸ Play / Pause / Toggle
🔊🔉 Volume Up / Down
🔇 Mute
⏩⏪ Seek Forward / Backward (10 seconds)
⏭⏮ Next / Previous Track

⚡ Fully Customizable
Remap any gesture to any action
Adjustable gesture hold time (prevents accidental triggers)
Repeatable actions (hold a sign to keep adjusting volume)
Wake gesture mode — require an "activation" sign before commands are accepted
Toggle gesture — enable/disable control with a two-hand sign

📌 PiP (Picture-in-Picture) Mode
When fullscreen video makes the side panel intrusive, switch to a compact floating popup window. Hand gesture recognition continues seamlessly in the small overlay without blocking your content.

🎯 Target Tab Control
Direct media commands to a specific tab — control background music while browsing other pages.

🔒 100% Local & Private
All hand tracking runs on-device using WebAssembly. Your camera feed never leaves your browser. No cloud processing, no data collection, no accounts required.

🌐 Bilingual
Full English and Japanese (日本語) interface support.

BENEFITS OF INSTALLING NyandSign

Hands-free convenience — Control media while cooking, exercising, eating, or when your hands are occupied.

Accessibility — Provides an alternative input method for users who have difficulty using a keyboard or mouse.

AR smart glasses integration — Designed to work with XREAL and similar AR glasses that have built-in cameras, enabling gesture control while wearing your glasses.

Distraction-free viewing — PiP mode lets you enjoy fullscreen content without a visible side panel while maintaining gesture control.

Zero learning curve — Intuitive hand signs (thumbs up = volume up, fist = pause) make the controls immediately understandable.

No additional software — Everything runs as a lightweight Chrome extension. No desktop apps, drivers, or accounts needed.

WHY THESE PERMISSIONS ARE NEEDED

NyandSign requests the minimum permissions required to function. Here is exactly what each permission does and why it is necessary:

activeTab
Allows the extension to interact with the currently active tab when you click the extension icon. This is needed to inject the media control script into the page you are viewing, so that play/pause/volume commands can reach the video or audio player on that page. Without this permission, the extension cannot control any media.

tabs
Allows the extension to read tab titles and detect which tabs contain media. This is used for the "Target Tab" feature — it lets you see a list of open tabs and choose which one receives your gesture commands. Tab information is only used locally to display tab names in the side panel; it is never transmitted anywhere.

scripting
Allows the extension to inject a small content script into web pages. This script is the bridge between the extension and the media player on the page — it listens for gesture commands (play, pause, volume, seek) and executes them on the page's video or audio elements. Without this permission, gestures would be detected but could not actually control any media.

storage
Allows the extension to save your settings (gesture mappings, volume preferences, display options) locally in Chrome's built-in storage. Your configuration persists across browser restarts. All data stays on your device — nothing is synced to external servers.

sidePanel
Allows the extension to display its interface as a Chrome side panel. The side panel is the primary UI where you see the camera feed, hand tracking visualization, gesture status, and all settings. This is the core interface of the extension.

Host permission: all URLs
The content script needs to run on any page because media can be played on any website — YouTube, Spotify, Netflix, Twitch, SoundCloud, podcast sites, and countless others. Restricting to specific domains would break media control on unlisted sites. The content script only interacts with media elements and does nothing else on the page.

TECHNICAL DETAILS

Uses Google MediaPipe Hand Landmarker (on-device, WebAssembly)
Runs at configurable FPS (default 15 fps) to balance accuracy and CPU usage
Distance-based gesture recognition with palm-size normalization
Temporal majority voting (200ms window) + hysteresis for stable detection
All processing happens locally — camera feed is never transmitted
Manifest V3 compliant

SUPPORT

If you encounter any issues or have feature requests, please visit our GitHub repository.