bug#19701: 24.3; directory-files platform-dependent behaviour when directory doesn't exist

bug#19701: 24.3; directory-files platform-dependent behaviour when directory doesn't exist

[VIDA Gábor]

Hi!

On Linux when I specify a non-existing directory to directory-files, it signals an
error:

  (directory-files "abcd")
  eval: Opening directory: no such file or directory, /home/evidgbo/tmp/abcd

But on Windows, it returns nil:

  (directory-files "abcd")
  => nil

I think they should work tha same way regardless of the underlying system, and it
ends up in a Gnus error, what I will report separately once I understand that problem
better.

I used the official Windows version:

  GNU EMacs 24.3.1 (i386-mingw-nt6.1.7601) of 2013-03-17 on MARVIN

--
Thanks and regards,
Gábor Vida

In GNU Emacs 24.3.1 (x86_64-pc-linux-gnu, GTK+ Version 3.10.7)
 of 2014-03-07 on lamiak, modified by Debian
Windowing system distributor `The X.Org Foundation', version 11.0.11501000
System Description:	Ubuntu 14.04.1 LTS

Configured using:
 `configure '--build' 'x86_64-linux-gnu' '--build' 'x86_64-linux-gnu'
 '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib'
 '--localstatedir=/var/lib' '--infodir=/usr/share/info'
 '--mandir=/usr/share/man' '--with-pop=yes'
 '--enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.3/site-lisp:/usr/share/emacs/site-lisp'
 '--with-crt-dir=/usr/lib/x86_64-linux-gnu' '--with-x=yes'
 '--with-x-toolkit=gtk3' '--with-toolkit-scroll-bars'
 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fstack-protector
 --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall'
 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro'
 'CPPFLAGS=-D_FORTIFY_SOURCE=2''

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<help-echo> <help-echo> <help-echo> M-x r e p o r t 
- e m a c - <backspace> s - u g <backspace> <backspace> 
<tab> <return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment lisp-mode register page menu-bar
rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
make-network-process dbusbind dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty emacs)

bug#19701: 24.3; directory-files platform-dependent behaviour when directory doesn't exist

[Eli Zaretskii]

> From: VIDA Gábor <vidagabor@gmail.com>
> Date: Tue, 27 Jan 2015 13:55:35 +0100
> 
> On Linux when I specify a non-existing directory to directory-files, it signals an
> error:
> 
>   (directory-files "abcd")
>   eval: Opening directory: no such file or directory, /home/evidgbo/tmp/abcd
> 
> But on Windows, it returns nil:
> 
>   (directory-files "abcd")
>   => nil

That is correct.

> I think they should work tha same way regardless of the underlying
> system, and it ends up in a Gnus error, what I will report
> separately once I understand that problem better.

The Windows implementation of directory-files always worked like that;
I looked as far back as Emacs 21.4.  (Technically, this happened
because the Windows implementation of 'opendir' doesn't actually open
the directory, so it doesn't know.)

This was never a problem, though.  What exactly does Gnus do that this
subtlety gets in the way?

In any case, I fixed this in commit 9664def on the emacs-24 branch.

Thanks.

bug#19701: 24.3; directory-files platform-dependent behaviour when directory doesn't exist

[Drew Adams]

> The Windows implementation of directory-files always worked like that;
> I looked as far back as Emacs 21.4.

Same thing for Emacs 20.7.   ;-)

bug#19701: 24.3; directory-files platform-dependent behaviour when directory doesn't exist

[Vida Gábor]

> This was never a problem, though.  What exactly does Gnus do that this
> subtlety gets in the way?

I'm migrating from Windows to GNU/Linux, and while I'm doing this Gnus didn't start
properly with my old configuration. I use %F in gnus-group-line-format, which ends up
calling gnus-total-fetched-for. This function calculates the total amount of disk
space used by cache and agent. I use Gnus agent but don't use cache (gnus-use-cache
is nil).

Later, gnus-cache-update-file-total-fetched-for iterates over the files that belong
to the cahe of the group. This list in nil on Windows, so I had no problem, but on
GNU/Linux, it raises an error, because I have no cache directory.

I replaced some parts of the call stack below with ellipsis.

Debugger entered--Lisp error: (file-error "Opening directory" "no such file or directory" "/home/evidgbo/config/emacs.d/gnus-cache/nntp+gmane:gmane.announce")
  directory-files("/home/evidgbo/config/emacs.d/gnus-cache/nntp+gmane:gmane.announce" t nil t)
  gnus-cache-update-file-total-fetched-for("nntp+gmane:gmane.announce" nil)
  gnus-cache-total-fetched-for("nntp+gmane:gmane.announce")
  gnus-total-fetched-for("nntp+gmane:gmane.announce")
  (format " %5s U, %3d T, %3d D,  %s  L:%d  %s  %s\n" gnus-tmp-number-of-unread (gnus-range-length (cdr (assq (quote tick) gnus-tmp-marked))) (gnus-range-length (cdr (assq (quote dormant) gnus-tmp-marked))) (gnus-total-fetched-for gnus-tmp-group) gnus-tmp-level (gnus-user-format-function-t gnus-tmp-header) (if (boundp (quote gnus-tmp-decoded-group)) gnus-tmp-decoded-group gnus-tmp-group))
  (insert (format " %5s U, %3d T, %3d D,  %s  L:%d  %s  %s\n" gnus-tmp-number-of-unread (gnus-range-length (cdr (assq (quote tick) gnus-tmp-marked))) (gnus-range-length (cdr (assq (quote dormant) gnus-tmp-marked))) (gnus-total-fetched-for gnus-tmp-group) gnus-tmp-level (gnus-user-format-function-t gnus-tmp-header) (if (boundp (quote gnus-tmp-decoded-group)) gnus-tmp-decoded-group gnus-tmp-group)))
  (let (gnus-position) (insert (format "%c%s %c%c %5s " gnus-tmp-process-marked gnus-group-indentation gnus-tmp-summary-live (gnus-group-new-mail gnus-tmp-group) (if (gnus-active gnus-tmp-group) (gnus-number-of-unseen-articles-in-group gnus-tmp-group) "*"))) (setq gnus-position (point)) (gnus-put-text-property (point) (progn (insert gnus-tmp-qualified-group) (point)) gnus-mouse-face-prop gnus-mouse-face) (if (> (current-column) 35) (let ((end (point))) (if (= (move-to-column 35) 35) (delete-region (point) end) (delete-region (1- (point)) end) (insert " "))) (insert-char 32 (max (- 35 (current-column)) 0))) (insert (format " %5s U, %3d T, %3d D,  %s  L:%d  %s  %s\n" gnus-tmp-number-of-unread (gnus-range-length (cdr (assq (quote tick) gnus-tmp-marked))) (gnus-range-length (cdr (assq (quote dormant) gnus-tmp-marked))) (gnus-total-fetched-for gnus-tmp-group) gnus-tmp-level (gnus-user-format-function-t gnus-tmp-header) (if (boundp (quote gnus-tmp-decoded-group)) gnus-tmp-decoded-group gnus-tmp-group))) (if gnus-position (gnus-put-text-property gnus-position (1+ gnus-position) (quote gnus-position) t)))
  eval((let (gnus-position) (insert (format "%c%s %c%c %5s " gnus-tmp-process-marked gnus-group-indentation gnus-tmp-summary-live (gnus-group-new-mail gnus-tmp-group) (if (gnus-active gnus-tmp-group) (gnus-number-of-unseen-articles-in-group gnus-tmp-group) "*"))) (setq gnus-position (point)) (gnus-put-text-property (point) (progn (insert gnus-tmp-qualified-group) (point)) gnus-mouse-face-prop gnus-mouse-face) (if (> (current-column) 35) (let ((end (point))) (if (= (move-to-column 35) 35) (delete-region (point) end) (delete-region (1- (point)) end) (insert " "))) (insert-char 32 (max (- 35 (current-column)) 0))) (insert (format " %5s U, %3d T, %3d D,  %s  L:%d  %s  %s\n" gnus-tmp-number-of-unread (gnus-range-length (cdr (assq (quote tick) gnus-tmp-marked))) (gnus-range-length (cdr (assq (quote dormant) gnus-tmp-marked))) (gnus-total-fetched-for gnus-tmp-group) gnus-tmp-level (gnus-user-format-function-t gnus-tmp-header) (if (boundp (quote gnus-tmp-decoded-group)) gnus-tmp-decoded-group gnus-tmp-group))) (if gnus-position (gnus-put-text-property gnus-position (1+ gnus-position) (quote gnus-position) t))))
  gnus-group-insert-group-line("nntp+gmane:gmane.announce" 3 ((seen (1 . 18742) (18916 . 19115))) 3948 "nntp:gmane")
  gnus-topic-prepare-topic((("announcements" visible nil nil)) 2 5 nil nil 1 nil)
  gnus-topic-prepare-topic((("gmane" visible nil ((subscribe . "^nntp\\+gmane:") (gcc-self . "nnimap+nedu:incoming") (agent-predicate and (not my-article-old-p)))) (("announcements" visible nil nil))) 1 5 nil nil 1 nil)
  gnus-topic-prepare-topic(...)
  gnus-group-prepare-topics(5 nil nil)
  gnus-group-list-groups(nil)
  byte-code("..." [dont-connect did-connect gnus-startup-file gnus-current-startup-file gnus-slave gnus-use-dribble-file gnus-group-quit gnus-run-hooks gnus-startup-hook gnus-make-newsrc-file gnus-dribble-read-file gnus-setup-news nil gnus-setup-news-hook gnus-request-create-group "queue" (nndraft "") gnus-start-draft-setup gnus-group-list-groups gnus-group-first-unread-group gnus-configure-windows group gnus-group-set-mode-line gnus-started-hook level gnus-agent] 4)
  gnus-1(nil nil nil)
  gnus(nil)
  call-interactively(gnus record nil)
  command-execute(gnus record)
  execute-extended-command(nil "gnus")
  call-interactively(execute-extended-command nil nil)

But I had no time so far to do a clean reproduction. As a workaround, I removed %F
from gnus-group-line-format temporary.

bug#19701: 24.3; directory-files platform-dependent behaviour when directory doesn't exist

[Eli Zaretskii]

> From: Vida Gábor <vidagabor@gmail.com>
> Cc: 19701-done@debbugs.gnu.org
> Date: Wed, 28 Jan 2015 12:45:53 +0100
> 
> > This was never a problem, though.  What exactly does Gnus do that this
> > subtlety gets in the way?
> 
> I'm migrating from Windows to GNU/Linux, and while I'm doing this Gnus didn't start
> properly with my old configuration. I use %F in gnus-group-line-format, which ends up
> calling gnus-total-fetched-for. This function calculates the total amount of disk
> space used by cache and agent. I use Gnus agent but don't use cache (gnus-use-cache
> is nil).
> 
> Later, gnus-cache-update-file-total-fetched-for iterates over the files that belong
> to the cahe of the group. This list in nil on Windows, so I had no problem, but on
> GNU/Linux, it raises an error, because I have no cache directory.

In that case, I probably should clarify how I fixed the problem: now
Emacs will signal an error on Windows as well.

I think there's a test missing in Gnus, but I'll defer to Gnus expert
to judge that.

bug#19701: 24.3; directory-files platform-dependent behaviour when directory doesn't exist

[Vida Gábor]

On Wed, Jan 28 2015, Eli Zaretskii wrote:

> I think there's a test missing in Gnus, but I'll defer to Gnus expert
> to judge that.

Thank you, I'll include this bug as a reference to the Gnus bug when I'll report it,
but I'm unsure as it should be seen frequently when someone doesn't use cache. So
it's highly possible that I misunderstand or misuse something.