Re: Filename encoding

[John Darrington <john@darrington.wattle.id.au>]

On Thu, Jan 16, 2014 at 02:03:05PM +0100, Ludovic Court??s wrote:
     Eli Zaretskii <eliz@gnu.org> skribis:
     
     >> From: ludo@gnu.org (Ludovic Court??s)
     >> Date: Thu, 16 Jan 2014 00:29:06 +0100
     >> 
     >> Does anyone know of systems where the file name encoding is commonly
     >> different from locale encoding?  Is it the case on Windows?
     >
     > Windows stores file names on disk encoded in UTF-16, but converts them
     > to the current codepage if you use Posix-style interfaces like 'open'
     > and 'rename'.
     
     So in practice, given that Guile uses the POSIX interfaces, the
     assumption that file names are in the locale encoding is valid on
     Windows.
     

If you know that the filename was always obtained using the Guile's 
interface then the issue is never pertinent.    The problem comes when a function
is aske to open a non-ascii named file, without any information about where that
filename came from.


There is no answer to this general problem.  We've encountered it over the years
in PSPP what we are doing now, is to pass the filename around in a structure along
with a variable indicating the encoding in which that filename should be interpreted.

This works up to a point, but eventually there comes an interface where the crucial 
information is missing.  For example, what happens if the filename is in a text file.
We have heuristics which can guess the encoding of a file, but that is of course not
completely reliable.

One has to decide on an approach which will give the lowest probability of surprises.

J'




     

-- 
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.