← Sign Chat

Connect your hands to ChatGPT

Sign Chat is an experimental ASL camera plugin. It sends a short sequence of images to ChatGPT, which attempts to interpret your signing and reply. It does not contain a trained ASL recognition model.

  1. Open ChatGPT Plugins.
  2. Choose + → Add custom MCP server. Account and workspace availability may vary.
  3. Name it Sign Chat. Use this connection URL:
    https://sign.zentimental.org/mcp
  4. Choose No authentication, review the connection details, and create the plugin.
  5. Start a chat with Sign Chat enabled and say “Open Sign Chat so I can sign in ASL.”
  6. Press Turn on camera, or choose Use a short video to import a ½–6 second clip (up to 20 MB). Video is decoded locally with audio excluded. Sign a short phrase, optionally Pause to review, then press Send this phrase. Check the English reading in ChatGPT.
Live mode

Enable the switch to send one phrase on a pause or after 5.5 seconds of movement. Capture then pauses until you read the reply and press Reply read — start next phrase. This is phrase-by-phrase input. Background movement can trigger capture. The app cannot tell when ChatGPT finishes answering.

Response time

ChatGPT chooses the model. Pro took 79 seconds for a short sample in our initial test. Choose a faster model setting for testing shorter waits, and check its interpretation carefully. The plugin cannot switch models for you or promise a response time.

If the camera is blocked

The ChatGPT host must grant camera access to the plugin, in addition to your browser and OS permissions. If it does not, open the camera preview, capture a phrase, choose Save phrase image, and attach the downloaded image to ChatGPT manually. Ask: “Attempt to interpret this ordered ASL sequence. Tell me if it is unclear; do not guess.” The preview does not translate by itself. Sampled images can miss important movement.

What has been tested

See the supplied source package and validation notes for automated tests of capture, transport, and failure behavior. Those tests do not establish ASL comprehension. A small photo test found clear, tentative, and partial readings. Live-camera recognition with fluent signers still needs validation. Read the current results and limitations.

Support: chaollapark@gmail.com. Describe the problem and browser version; do not email camera recordings or private chat content.

Privacy · Official ChatGPT setup documentation