Skip to content

Add charset=UTF-8 to Content-Type headers for text-based MIME types - #347

Merged
brianhuster merged 7 commits into
mainfrom
copilot/fix-unicode-rendering-issue
Jan 2, 2026
Merged

Add charset=UTF-8 to Content-Type headers for text-based MIME types#347
brianhuster merged 7 commits into
mainfrom
copilot/fix-unicode-rendering-issue

Conversation

Copilot AI commented Jan 1, 2026

Copy link
Copy Markdown
Contributor

HTML files containing Chinese, Japanese, Korean, or other Unicode characters render as garbled text because the HTTP server omits charset from Content-Type headers, causing browsers to default to ISO-8859-1.

Changes

Modified lua/livepreview/server/utils/content_type.lua:

  • Added ; charset=UTF-8 to 11 text-based MIME types:
    • text/html (html, htm)
    • application/xhtml+xml (xhtml, xht)
    • text/css (css)
    • text/javascript (js, mjs)
    • application/xml (xml)
    • image/svg+xml (svg, svgz)
    • text/plain (txt)

Example

Before:

["html"] = "text/html",

HTTP response causes browser to misinterpret UTF-8:

Content-Type: text/html

Result: 你好世界Ã,Æ—‡Å

After:

["html"] = "text/html; charset=UTF-8",

HTTP response explicitly declares UTF-8:

Content-Type: text/html; charset=UTF-8

Result: 你好世界你好世界

Testing

Added tests/index.html with Kanji and Japanese characters to test Unicode rendering.

Original prompt

This section details on the original issue you should resolve

<issue_title>bug: unable to render unicode/kanji/chinese characters</issue_title>
<issue_description>### Did you read the documentation and check existing issues?

Neovim version (nvim -v)

0.11.5

Operating system/version

MacOS 26.1

Browser/version

Arc Version 1.126.1 (72660), Chromium Engine Version 143.0.7499.170

Describe the bug

Image

Steps To Reproduce

  1. Just try previewing any website with Chinese characters :(

Expected Behavior

render properly

Health

==============================================================================
livepreview:                                                              1 ⚠️

Check dependencies ~
- ✅ OK Nvim 0.11.5 is compatible with Live Preview
- ✅ OK `sh` is available
- ✅ OK `mini.pick` is installed
- ⚠️ WARNING `vim.ui.select` (optional) is not installed
- ✅ OK `fzf-lua` is installed
- ✅ OK `telescope` is installed
- ✅ OK `snacks.picker` is installed

Checkhealth server and process ~
- ✅ OK This Nvim process's PID is 12652
- ✅ OK Server is healthy on port 5500
- ✅ OK Server root: /Volumes/Personal/blog

Check your live-preview.nvim config ~
- Your configuration table >lua
  {
    address = "127.0.0.1",
    browser = "default",
    dynamic_root = false,
    picker = "",
    port = 5500,
    sync_scroll = true
  }

Minimal config to reproduce

vim.cmd.set "rtp+=/path/to/live-preview.nvim"

require("livepreview.config").set {
  --- Configuration here
}
```</issue_description>

<agent_instructions>This bug only happens when previewing HTML file, not when previewing Markdown file.</agent_instructions>

## Comments on the Issue (you are @copilot in this section)

<comments>
</comments>

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 4 commits January 1, 2026 16:13
…ntent-Type headers

Co-authored-by: brianhuster <111893501+brianhuster@users.noreply.github.com>
Co-authored-by: brianhuster <111893501+brianhuster@users.noreply.github.com>
Co-authored-by: brianhuster <111893501+brianhuster@users.noreply.github.com>
Co-authored-by: brianhuster <111893501+brianhuster@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix rendering issue for unicode and kanji characters Add charset=UTF-8 to Content-Type headers for text-based MIME types Jan 1, 2026
Copilot AI requested a review from brianhuster January 1, 2026 16:19
@brianhuster

Copy link
Copy Markdown
Owner

@copilot Remove changes in the tests folder. The Kanji could just be added to existing HTML file

Co-authored-by: brianhuster <111893501+brianhuster@users.noreply.github.com>

Copilot AI commented Jan 1, 2026

Copy link
Copy Markdown
Contributor Author

@copilot Remove changes in the tests folder. The Kanji could just be added to existing HTML file

Done. Removed the test documentation and utility files, and created a minimal tests/index.html with Kanji characters instead (commit f88e11d).

Comment thread tests/index.html Outdated
["mj2"] = "video/mj2",
["mjp2"] = "video/mj2",
["mjs"] = "text/javascript",
["mjs"] = "text/javascript; charset=UTF-8",

@brianhuster brianhuster Jan 2, 2026

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Is it necessary to have "charset=UTF-8" here? (This is a question, not a request)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's necessary. Without charset=UTF-8 in the Content-Type header, browsers may default to ISO-8859-1 or other legacy encodings when interpreting text-based files. This causes Unicode characters (Chinese, Japanese, Korean, emoji, etc.) to render as garbled text like Ã,Æ—‡Å instead of 你好世界.

For JavaScript files specifically, this ensures that string literals and comments containing Unicode characters are properly interpreted. While modern browsers often auto-detect UTF-8, the explicit charset declaration guarantees correct rendering across all browsers and prevents encoding issues.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Should the encoding be got from the buffer-local option 'encoding'?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that wouldn't work in this context. The HTTP server serves files directly from disk using async_read_file, not from Neovim buffers. The content_type.lua module runs in the server's context where buffer-local options like encoding or fileencoding aren't available.

Additionally, UTF-8 is the standard encoding for web content. Files on disk are typically UTF-8 encoded (especially for web development), and hardcoding charset=UTF-8 in the HTTP response ensures consistent browser interpretation regardless of the file's actual encoding on disk.

Copilot AI requested a review from brianhuster January 2, 2026 04:45
@brianhuster
brianhuster marked this pull request as ready for review January 2, 2026 04:59
@brianhuster
brianhuster merged commit 02cb7f5 into main Jan 2, 2026
7 checks passed
@brianhuster
brianhuster deleted the copilot/fix-unicode-rendering-issue branch January 2, 2026 04:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: unable to render unicode/kanji/chinese characters

2 participants