bug#15426: 24.3.50; Multibyte filenames and directory-files in unibyte buffer

[Eli Zaretskii <eliz@gnu.org>]

> From: Andreas Politz <politza@hochschule-trier.de>
> Cc: monnier@iro.umontreal.ca,  15426@debbugs.gnu.org
> Date: Sat, 21 Sep 2013 11:35:52 +0200
> 
> If I save this in mb-dir/foo.el, where mb-dir is a directory containing
> multi-bytes, the results (d1 and d2) of the same calls to
> `directory-list' are different in the uni-byte and multi-byte buffer.

For the record, a much simpler test case is this:

  M-: (multibyte-string-p (car (directory-files default-directory t))) RET

invoked from the unibyte buffer that visits your mb-dir/foo.el.  Note
that default-directory is a multibyte string, as shown by calling
multibyte-string-p on it.  So the problem happens inside the
directory-files call.

(You should never trust what the echo area shows when potentially
unibyte strings are involved, always use multibyte-string-p to tell if
a string is multibyte or unibyte.)

The bug that caused this should be fixed now in revision 114421 on the
trunk.

> It seems that the 2 byte sequences of the UTF-8 characters are
> replaced by some 4 bytes.

That's how Emacs represents raw bytes internally in a multibyte
buffer, so that "expansion" is a clear sign of a unibyte string.

> Anyway, the resulting filename d2 names a non-existent file.

Because encoding a unibyte string with raw bytes in their internal
representation will never get you the right file name.

> > "The minibuffer displays" is the key point here:[...]
> 
> No, the key is that the file's existence depends on the buffer's
> multi-byte status, in which the code is evaluated.

The truth is neither (although what the minibuffer displays in these
cases can easily fool you, so don't trust it).  The truth was that the
code in a subroutine of directory-files, when it is called with its
second argument non-nil, incorrectly marked the full file name it
produced as a unibyte string.  The rest was the consequence of that.

Thanks.