bug#49723: 28.0.50; Test in coding.c for NUL bytes in filenames is not reliable

[Federico Tedin <federicotedin@gmail.com>]

I'm interested in looking into this one since I want to learn more about
the C side of the codebase. However, I wasn't able to find a call to
expand-file-name in encode_file_name or encode_file_name_1. I did find
the null byte check though (CHECK_TYPE + memchr). Maybe I am missing
something out.

I assume that a similar check on expand-file-name should be applied to
both input arguments, NAME and DEFAULT-DIRECTORY?

Thanks!

Eli Zaretskii <eliz@gnu.org> writes:

> From: eliz@HOME-C4E4A596F7.i-did-not-set--mail-host-address--so-tickle-me
> --text follows this line--
> Emacs currently tries to catch null bytes in file names in
> encode_file_name, after encoding the file name.  But that is not
> reliable enough, because it assumes that the call to expand-file-name
> at the beginning of encode_file_name leaves those null bytes intact.
> That is not guaranteed, and in fact doesn't happen at least on
> MS-Windows: the file name gets chopped after the first null byte, and
> doesn't trigger the test in encode_file_name.
>
> So for a more reliable test we need to include such a check inside
> expand-file-name (and then we could probably drop the test in
> encode_file_name, because Emacs always runs file names through
> expand-file-name before using them).
>
> In GNU Emacs 28.0.50 (build 1573, i686-pc-mingw32)
>  of 2021-07-24 built on HOME-C4E4A596F7
> Repository revision: 7c83e605ab84e8b62254c55f347abc8aa9c6057b
> Repository branch: master
> Windowing system distributor 'Microsoft Corp.', version 5.1.2600
> System Description: Microsoft Windows XP Service Pack 3 (v5.1.0.2600)
>
> Configured using:
>  'configure -C --prefix=/d/usr --with-wide-int --with-modules
>  --enable-checking=yes,glyphs 'CFLAGS=-O0 -gdwarf-4 -g3''
>
> Configured features:
> ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NOTIFY
> W32NOTIFY PDUMPER PNG RSVG SOUND THREADS TIFF TOOLKIT_SCROLL_BARS XPM
> ZLIB
>
> Important settings:
>   value of $LANG: ENU
>   locale-coding-system: cp1255
>
> Major mode: Lisp Interaction
>
> Minor modes in effect:
>   tooltip-mode: t
>   global-eldoc-mode: t
>   eldoc-mode: t
>   electric-indent-mode: t
>   mouse-wheel-mode: t
>   tool-bar-mode: t
>   menu-bar-mode: t
>   file-name-shadow-mode: t
>   global-font-lock-mode: t
>   font-lock-mode: t
>   blink-cursor-mode: t
>   auto-composition-mode: t
>   auto-encryption-mode: t
>   auto-compression-mode: t
>   line-number-mode: t
>   indent-tabs-mode: t
>   transient-mark-mode: t
>
> Load-path shadows:
> None found.
>
> Features:
> (shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
> rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
> rmail-loaddefs auth-source cl-seq eieio eieio-core cl-macs
> eieio-loaddefs password-cache json map text-property-search time-date
> subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
> mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
> cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
> iso-transl tooltip eldoc electric uniquify ediff-hook vc-hooks
> lisp-float-type mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win
> w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe
> tabulated-list replace newcomment text-mode elisp-mode lisp-mode
> prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
> timer select scroll-bar mouse jit-lock font-lock syntax font-core
> term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
> misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
> cp51932 hebrew greek romanian slovak czech european ethiopic indian
> cyrillic chinese composite charscript charprop case-table epa-hook
> jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button
> loaddefs faces cus-face macroexp files window text-properties overlay
> sha1 md5 base64 format env code-pages mule custom widget
> hashtable-print-readable backquote threads w32notify w32 lcms2 multi-tty
> make-network-process emacs)
>
> Memory information:
> ((conses 16 56888 7103)
>  (symbols 48 7785 1)
>  (strings 16 21560 2884)
>  (string-bytes 1 630636)
>  (vectors 16 13320)
>  (vector-slots 8 173301 9720)
>  (floats 8 23 47)
>  (intervals 40 265 116)
>  (buffers 888 10))