Fixing Unicode Rendering in xterm.js: It's Not the Frontend

Building a web-based terminal using xterm.js, I encountered a frustrating issue: Unicode characters like » and CJK characters (你好) rendered as underscores or blank boxes. After hours of debugging font loading, WebGL renderers, and CSS font-face declarations, the fix was a single line change in the backend.

This post documents the debugging journey and the actual solution.

The Symptom

In a web terminal powered by xterm.js connecting to Docker containers via WebSocket:

• ASCII characters rendered fine
• Unicode symbols like », →, ← showed as underscores
• CJK characters (Chinese, Japanese, Korean) showed as blank boxes
• The PS1 prompt %F{63}» %f displayed as _ instead of »

The Wrong Direction: Frontend Debugging

I spent hours trying to fix this on the frontend:

1. Font loading attempts:

// Tried loading multiple fonts
fontFamily: '"JetBrains Mono", "Noto Sans Mono", "PingFang SC", monospace'

// Waited for fonts to load
await document.fonts.load('13px "JetBrains Mono"');
await document.fonts.ready;

2. WebGL renderer suspicion:

// Disabled WebGL thinking it was the issue
// const webglAddon = new WebglAddon();
// terminal.loadAddon(webglAddon);

3. CSS @font-face with unicode-range:

@font-face {
  font-family: 'TerminalFont';
  src: local('PingFang SC'), local('Microsoft YaHei');
  unicode-range: U+4E00-9FFF; /* CJK characters */
}

4. Unicode11 addon:

const unicode11Addon = new Unicode11Addon();
terminal.loadAddon(unicode11Addon);
terminal.unicode.activeVersion = '11';

None of these worked.

The Breakthrough: A Self-Contained Test

I created a standalone xterm.js demo that wrote Unicode directly to the terminal:

terminal.writeln('Chinese: 你好世界 中文测试');
terminal.writeln('Unicode: » « → ← ↑ ↓');

Result: Everything rendered perfectly.

This proved xterm.js and WebGL were not the problem. The issue was somewhere in the data pipeline from the Docker container to the browser.

Finding the Real Cause

I tested the Docker container directly:

docker run --rm --entrypoint /bin/sh worker:latest -c '
  echo "Direct: » 你好"
  tmux new-session -d -s test
  tmux send-keys "echo » 你好" Enter
  tmux capture-pane -p
'

Output showed Unicode correctly. But the container's locale was set:

LANG=C.UTF-8
LC_ALL=C.UTF-8

Then I checked how the shell was being spawned. The docker exec command was:

const dockerCmd = [
  "/usr/bin/docker",
  "exec",
  "-it",
  "-e", "TERM=xterm-256color",
  containerName,
  "/bin/sh", "-c", tmuxCmd,
];

The problem: TERM was set, but LANG and LC_ALL were not passed to the exec environment. Even though the container had these set in the Dockerfile, docker exec creates a new environment that doesn't inherit all container environment variables.

The Fix

Add LANG and LC_ALL to the docker exec command:

const dockerCmd = [
  "/usr/bin/docker",
  "exec",
  "-it",
  "-e", "TERM=xterm-256color",
  "-e", "LANG=C.UTF-8",
  "-e", "LC_ALL=C.UTF-8",
  containerName,
  "/bin/sh", "-c", tmuxCmd,
];

That's it. Unicode rendering worked immediately.

Why This Happens

1. Container environment ≠ exec environment: Setting ENV LANG=C.UTF-8 in Dockerfile sets the variable for processes started by the container's entrypoint. But docker exec creates a new process with a minimal environment.

2. Shell needs locale for Unicode: Without LANG/LC_ALL, the shell (zsh/bash) may not properly handle multi-byte characters. It might output escape sequences or replacement characters instead.

3. The browser receives corrupted data: By the time the data reaches xterm.js via WebSocket, the Unicode characters are already mangled at the source.

Diagnostic Checklist

When debugging Unicode issues in web terminals:

1. Test xterm.js in isolation - Write Unicode directly to terminal. If it renders, the frontend is fine.

2. Test the container directly - Run echo "你好" via docker exec in a regular terminal.

3. Check the exec environment - Print $LANG and $LC_ALL in the exec session, not just docker inspect.

4. Verify the data pipeline - Log the raw bytes at each stage (container → API → WebSocket → browser).

Key Takeaway

When Unicode doesn't render in a web terminal:

• It's probably not xterm.js
• It's probably not font loading
• It's probably not the WebGL renderer

Check how the shell process is spawned. Specifically, verify that LANG and LC_ALL are set to a UTF-8 locale in the exec environment, not just in the container's Dockerfile.