Change text encoding option

[slickrcbd]

I just got a report on another board that a story on the site fanficauthors.net was showing weird formatting and characters. It seems something is wrong with the site as I also got a few errors before getting the story.
However, when I did get the story I recognized one of the problems. Firefox was recognizing the wrong text encoding.
I haven't seen this issue in a while, but in the past I used to use the "text encoding" option from the view menu to choose the correct format. However, I found it was dimmed in the latest version:
TextEncoding by slickrcbd, on Flickr

Any idea how I'd clean up the text in the latest version of Firefox? Since Netscape or Mozilla I'd have used the dimmed "text encoding", but now I have no clue.

[dickvl]

This is not a problem with Firefox, but is a problem with the website.
The server sends text encoding UTF-8 as you can see via "Tools -> Page Info -> General" and in the page source (via a meta tag), but there are individual UTF-8 combining characters send.
So it looks that UTF-8 encoded text is interpreted as 8-bit code and the result is send as UTF-8.
There is not much you can do about this, the server should do a proper conversion to UTF-8.

See:
https://www.i18nqa.com/debug/utf8-debug.html

Code: Select all

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>

[slickrcbd]

Something like this happened to the server a couple times before over the last 14 years (can't believe it's been that long), and in the past I handled it by messing with the "text encoding" options. They seem to have been removed with the latest release of Firefox.

[dickvl]

It is possible that that server was configured in the past to use an eight bit Western or Windows encoding and has changed to UTF-8 without updating their files properly to Unicode (i.e. the server thinks that they are 8-bit encoding and not Unicode).
Since these Unicode files are now
This looks like files from quite some time ago (2004-2006) that are likely no longer maintained.

[slickrcbd]

Hence why I asked how to change the text encoding from Unicode to Western or Windows.
It used to be a cinch using the view menu.

I just remembered it used to be possible to add text encoding to the pop-up menu (that no longer seems to exist), but it too shows the options as dimmed except for Unicode.
(Do I need to post a screenshot?)

[dickvl]

The server sends the content as you see it (i.e. with Unicode characters that aren't encoded properly), so there is not much that can be done apart from stripping out the Unicode control characters.
The files on the server are Unicode files, but are interpreted as 8-bit encoding and are automatically re-encoded by the server once again to Unicode before they are send to you.
This needs to be fixed on the server.

How do other browsers handle these pages ?