From: pjb@informatimago.com (Pascal J. Bourguignon)
To: help-gnu-emacs@gnu.org
Subject: Re: How to determine encoding for file?
Date: Sun, 24 Jan 2010 22:59:46 +0100 [thread overview]
Message-ID: <87y6jn2mgd.fsf@hubble.lan.informatimago.com> (raw)
In-Reply-To: hjie4c$e4v$1@reader1.panix.com
kj <no.email@please.post> writes:
> I've downloaded a large file that is supposed to contain a mixture
> of Japanese and English (it's basically a learner's dictionary).
> The English is displayed correctly, but not so for the Japanese.
>
> I've tried setting the buffer's coding system to utf-8,
> japanese-shift-jis, japanese-shift-jis-mac, japanese-shift-jis-dos
> (just guessing). None worked.
>
> In fact, I'm not even sure that any of these changes of the coding
> system achieved *anything*, since the buffer's appearance remained
> unchanged throughout all this mucking around. I used the command
> set-buffer-file-coding-system to do this. Should I need to do
> anything besides re-setting the coding system to see a change in
> how the file is displayed?
>
> More importantly, is there a better way to determine a file's
> correct coding system besides trial and error?
No, there is no better way.
From the sequences of bytes you can find inside the files, you may
eliminate some encodings (eg. some byte sequences are invalid in
UTF-8), but there are several encodings that use all the byte values
and where any sequence of byte is valid, so you cannot choose between
these encoding without knowing the meaning of the file.
For example, the following file:
$ od -t x1 -c /tmp/a.txt
0000000 50 72 69 63 65 20 6f 66 20 74 68 65 20 69 74 65
P r i c e o f t h e i t e
0000020 6d 3a 20 31 34 20 a4 0a
m : 1 4 244 \n
0000030
The encoding cannot be determined without knowing what item is
referenced, and what price ranges are probable for this item. It
could contain:
Price of the item: 14 €
or:
Price of the item: 14 ¤
(It could also contain another character, but it would be less probable).
--
__Pascal Bourguignon__ http://www.informatimago.com/
next prev parent reply other threads:[~2010-01-24 21:59 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-01-24 21:29 How to determine encoding for file? kj
2010-01-24 21:59 ` Pascal J. Bourguignon [this message]
2010-01-25 5:57 ` tomas
[not found] ` <mailman.146.1264399148.14305.help-gnu-emacs@gnu.org>
2010-01-25 14:55 ` kj
2010-01-26 9:01 ` Thien-Thi Nguyen
2010-01-26 2:58 ` Kevin Rodgers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87y6jn2mgd.fsf@hubble.lan.informatimago.com \
--to=pjb@informatimago.com \
--cc=help-gnu-emacs@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.