empty-directory predicate, native implementation

empty-directory predicate, native implementation

[Arthur Miller]

[-- Attachment #1: Type: text/plain, Size: 1704 bytes --]

It is easy to check for an empty dir in elisp; we can just list files
and check if there is a list or not:

  (null (directory-files directory-name nil nodots t)))

where nodots is just regex to omit dot files (from dired+).

But then this is quite inneficient. We are listing all files in each
dir since directory-files will return entire content of directory. Also
we are matching every filename to a regex just to eliminate first two.
Alternative would be to take length and see if it is > 2; but then we
would iterate whole list twice. So I can't see anything avialable in
dired/elisp and I think a predicate implemented in low-level is better solution.
We are really interested just to see if there is some file; so we can
just open dir, and read first few entries, if there is more then 2 files
(. and .. on *nix) we can just abort and return true.

I have tested an idea with getdents (Linux syscall) and I can see
difference. Attached is a patch for dired.c and a test file to play with
some benchmark.

In somewhat synthetic test where I just looped a "lisp" and "native"
predicate over a several hundred directories, I can see quite drammatic
difference in performance. On a directory with something about ~800
subdirs, native prediate takes ~0.002s while lisp predicate goes in ~0.01s.

I have made also a small test to mark empty dirs in dired, and there I
see some difference. On same directory, I get consistently
~2.4s for lisp version and ~2s for native version.

This isn't any kind of drammatic difference for most use; file I/O
is dominated by disk access anyway, but i still don't like to spend cpu
on unnecessary evaluations, so I wonder if we could get native predicate
in elisp? 


[-- Attachment #2: dired-mark-empty.el --]
[-- Type: text/plain, Size: 2175 bytes --]

(require 'dired)

(defvar nodots "^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*")

(defun dired-go-to-first ()
    (interactive)
    (goto-char (point-min))
    (dired-next-line 1)
    (skip-chars-forward " \n\t"))

(defun dired-go-to-last ()
    (interactive)
    (goto-char (point-max))
    (dired-next-line -1)
    (skip-chars-forward " \n\t"))

(defun dired-is-empty-p (directory-name)
  (null (directory-files directory-name nil nodots t)))

(defun directory-number-files (directory-name &optional omit-filter)
  (length (directory-files directory-name nil omit-filter t)))

(defun dired-mark-empty-dirs ()
  (interactive)
  (when (equal major-mode 'dired-mode)
    (let ((curr-dir))
      (save-excursion
        (dired-go-to-first)
        
        (while (not (eobp))
          (setq curr-dir (dired-file-name-at-point))
          (cond ((or (null curr-dir)
                     (string= curr-dir ".")
                     (string= curr-dir ".."))
                     ;; do nothing here
                 )
                ((file-directory-p curr-dir)
                 (when (dired-is-empty-p curr-dir)
                   (dired-mark 1)
                   (dired-previous-line 1))))
          (dired-next-line 1))))))

(defun dired-mark-empty-dirs-native ()
  (interactive)
  (when (equal major-mode 'dired-mode)
    (let ((curr-dir))
      (save-excursion
        (dired-go-to-first)
        
        (while (not (eobp))
          (setq curr-dir (dired-file-name-at-point))
          (cond ((or (null curr-dir)
                     (string= curr-dir ".")
                     (string= curr-dir ".."))
                     ;; do nothing here
                 )
                ((file-directory-p curr-dir)
                 (when (directory-empty-p curr-dir)
                   (dired-mark 1)
                   (dired-previous-line 1))))
          (dired-next-line 1))))))

;;(benchmark-run-compiled 10 (directory-empty-p "/some/directory/here"))
;;(benchmark-run-compiled 10 (dired-is-empty-p "/some/directory/here"))

;; must be in some dired buffer for this
;;(benchmark-run-compiled 10 (dired-mark-empty-dirs))
;;(benchmark-run-compiled 10 (dired-mark-empty-dirs-native))

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: dired.patch --]
[-- Type: text/x-patch, Size: 2749 bytes --]

--- src/dired.c	2020-10-13 04:08:36.028838472 +0200
+++ ../dired.c	2020-10-13 04:07:48.374572510 +0200
@@ -21,6 +21,7 @@
 #include <config.h>
 
 #include <sys/stat.h>
+#include <sys/syscall.h>
 
 #ifdef HAVE_PWD_H
 #include <pwd.h>
@@ -39,6 +40,7 @@
 #include "systime.h"
 #include "buffer.h"
 #include "coding.h"
+#include "blockinput.h"
 
 #ifdef MSDOS
 #include "msdos.h"	/* for fstatat */
@@ -929,7 +931,7 @@
   struct stat s;
 
   /* An array to hold the mode string generated by filemodestring,
-     including its terminating space and null byte.  */
+     including its terminating space and NUL byte.  */
   char modes[sizeof "-rwxr-xr-x "];
 
   char *uname = NULL, *gname = NULL;
@@ -1078,6 +1080,50 @@
   return groups;
 }
 
+typedef struct dirent* pdirent;
+DEFUN ("directory-empty-p", Fdirectory_empty_p,
+       Sdirectory_empty_p, 1, 1, 0,
+       doc: /* Returns t if directory DIRNAME does not contain any
+       user files (special files . and .. are excluded
+       automatically), nil otherwise. */)
+(Lisp_Object dirname)
+{
+        #define BSIZE 1024
+        char buf[BSIZE];
+        const char* name;
+        int fd, n = 0, p = 0, c = 0;
+        pdirent d;
+
+        if(!STRINGP(dirname))
+                error("Directory name not a string object.");
+        
+        dirname = Fexpand_file_name(dirname, Qnil);
+        name = SSDATA(dirname);
+
+        fd = open (name, O_RDONLY | O_DIRECTORY);
+
+        if( fd == -1 )
+                error("Can't open directory.");
+
+        //block_input();
+        /* 32-bit version of getdents should be good enough;
+           we are just looking at first 3 files*/
+        n = syscall(SYS_getdents,fd,buf, BSIZE);
+        if(n == -1)
+                error("Can't read directory data.");
+
+        while(p < n && c < 3) {
+                d = (pdirent) (buf + p);
+                p += d->d_reclen;
+                c++;
+        }
+        //unblock_input();
+        
+        close(fd);
+        return (c > 2) ? Qnil : Qt;
+}
+
+
 void
 syms_of_dired (void)
 {
@@ -1089,7 +1135,8 @@
   DEFSYM (Qfile_attributes_lessp, "file-attributes-lessp");
   DEFSYM (Qdefault_directory, "default-directory");
   DEFSYM (Qdecomposed_characters, "decomposed-characters");
-
+  DEFSYM (Qdirectory_empty_p, "directory-empty-p")
+          
   defsubr (&Sdirectory_files);
   defsubr (&Sdirectory_files_and_attributes);
   defsubr (&Sfile_name_completion);
@@ -1098,6 +1145,7 @@
   defsubr (&Sfile_attributes_lessp);
   defsubr (&Ssystem_users);
   defsubr (&Ssystem_groups);
+  defsubr (&Sdirectory_empty_p);
 
   DEFVAR_LISP ("completion-ignored-extensions", Vcompletion_ignored_extensions,
 	       doc: /* Completion ignores file names ending in any string in this list.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi Arthur,

> This isn't any kind of drammatic difference for most use; file I/O
> is dominated by disk access anyway, but i still don't like to spend cpu
> on unnecessary evaluations, so I wonder if we could get native predicate
> in elisp?

I don't know whether we will add this function, but in case of: you
shall call a file name handler first. See for example Fdirectory_files.

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Arthur Miller]

[-- Attachment #1: Type: text/plain, Size: 948 bytes --]

Michael Albinus <michael.albinus@gmx.de> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
> Hi Arthur,
>
>> This isn't any kind of drammatic difference for most use; file I/O
>> is dominated by disk access anyway, but i still don't like to spend cpu
>> on unnecessary evaluations, so I wonder if we could get native predicate
>> in elisp?
>
> I don't know whether we will add this function, but in case of: you
> shall call a file name handler first. See for example Fdirectory_files.
>
> Best regards, Michael.
Thanks for looking at this.

I have seen in the source code calls to Ffind_file_handler, and have
been looking at the function in fileio.c, but I didn't figured out what
it really does, so I didn't use it. I tested without and it worked fine
:-). Anyway, I have added it now, but I am not sure if I using it
correctly, since I am not sure what it does.

By the way, what are "special constructs" referred to in the comment?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: dired.c patch --]
[-- Type: text/x-patch, Size: 3082 bytes --]

--- src/dired.c	2020-10-13 13:36:27.069967986 +0200
+++ ../dired.c	2020-10-13 13:35:47.695916558 +0200
@@ -21,6 +21,7 @@
 #include <config.h>
 
 #include <sys/stat.h>
+#include <sys/syscall.h>
 
 #ifdef HAVE_PWD_H
 #include <pwd.h>
@@ -39,6 +40,7 @@
 #include "systime.h"
 #include "buffer.h"
 #include "coding.h"
+#include "blockinput.h"
 
 #ifdef MSDOS
 #include "msdos.h"	/* for fstatat */
@@ -929,7 +931,7 @@
   struct stat s;
 
   /* An array to hold the mode string generated by filemodestring,
-     including its terminating space and null byte.  */
+     including its terminating space and NUL byte.  */
   char modes[sizeof "-rwxr-xr-x "];
 
   char *uname = NULL, *gname = NULL;
@@ -1078,6 +1080,58 @@
   return groups;
 }
 
+typedef struct dirent* pdirent;
+DEFUN ("directory-empty-p", Fdirectory_empty_p,
+       Sdirectory_empty_p, 1, 1, 0,
+       doc: /* Returns t if directory DIRNAME does not contain any
+       user files (special files . and .. are excluded
+       automatically), nil otherwise. */)
+(Lisp_Object dirname)
+{
+        #define BSIZE 1024
+        char buf[BSIZE];
+        const char* name;
+        int fd, n = 0, p = 0, c = 0;
+        pdirent d;
+        Lisp_Object handler;
+
+        if(!STRINGP(dirname))
+                error("Directory name not a string object.");
+        
+        dirname = Fexpand_file_name(dirname, Qnil);
+
+        /* If the file name has special constructs in it,
+           call the corresponding file name handler.  */
+        handler = Ffind_file_name_handler (dirname, Qdirectory_empty_p);
+        if (!NILP (handler))
+                return call2 (handler, Qdirectory_empty_p, dirname);
+        
+        name = SSDATA(dirname);
+
+        fd = open (name, O_RDONLY | O_DIRECTORY);
+
+        if( fd == -1 )
+                error("Can't open directory.");
+
+        //block_input();
+        /* 32-bit version of getdents should be good enough;
+           we are just looking at first 3 files*/
+        n = syscall(SYS_getdents,fd,buf, BSIZE);
+        if(n == -1)
+                error("Can't read directory data.");
+
+        while(p < n && c < 3) {
+                d = (pdirent) (buf + p);
+                p += d->d_reclen;
+                c++;
+        }
+        //unblock_input();
+        
+        close(fd);
+        return (c > 2) ? Qnil : Qt;
+}
+
+
 void
 syms_of_dired (void)
 {
@@ -1089,7 +1143,8 @@
   DEFSYM (Qfile_attributes_lessp, "file-attributes-lessp");
   DEFSYM (Qdefault_directory, "default-directory");
   DEFSYM (Qdecomposed_characters, "decomposed-characters");
-
+  DEFSYM (Qdirectory_empty_p, "directory-empty-p")
+          
   defsubr (&Sdirectory_files);
   defsubr (&Sdirectory_files_and_attributes);
   defsubr (&Sfile_name_completion);
@@ -1098,6 +1153,7 @@
   defsubr (&Sfile_attributes_lessp);
   defsubr (&Ssystem_users);
   defsubr (&Ssystem_groups);
+  defsubr (&Sdirectory_empty_p);
 
   DEFVAR_LISP ("completion-ignored-extensions", Vcompletion_ignored_extensions,
 	       doc: /* Completion ignores file names ending in any string in this list.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi Arthur,

> I have seen in the source code calls to Ffind_file_handler, and have
> been looking at the function in fileio.c, but I didn't figured out what
> it really does, so I didn't use it. I tested without and it worked fine
> :-). Anyway, I have added it now, but I am not sure if I using it
> correctly, since I am not sure what it does.

Looks good.

Simplified spoken, this code makes Tramp work. If you call for example
(directory-empty-p "/sudo::/") , find-file-name-handler (the ELisp
equivalent to Ffind_file_name_handler) checks, whether Tramp feels
responsible for this directory name, and it calls Tramp's function in
case.

You know, that your function cannot handle "/sudo::/".

> By the way, what are "special constructs" referred to in the comment?

Tramp file name syntax here. See (info "(elisp) Magic File Names")

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Eli Zaretskii]

> From: Arthur Miller <arthur.miller@live.com>
> Date: Tue, 13 Oct 2020 04:22:36 +0200
> 
> It is easy to check for an empty dir in elisp; we can just list files
> and check if there is a list or not:
> 
>   (null (directory-files directory-name nil nodots t)))
> 
> where nodots is just regex to omit dot files (from dired+).
> 
> But then this is quite inneficient. We are listing all files in each
> dir since directory-files will return entire content of directory. Also
> we are matching every filename to a regex just to eliminate first two.
> Alternative would be to take length and see if it is > 2; but then we
> would iterate whole list twice. So I can't see anything avialable in
> dired/elisp and I think a predicate implemented in low-level is better solution.
> We are really interested just to see if there is some file; so we can
> just open dir, and read first few entries, if there is more then 2 files
> (. and .. on *nix) we can just abort and return true.
> 
> I have tested an idea with getdents (Linux syscall) and I can see
> difference. Attached is a patch for dired.c and a test file to play with
> some benchmark.

If all we want is to stop reading a directory after N entries, why not
simply extend directory-files to accept one more argument: the maximum
number of file entries to read?  That should be easy to implement, and
will not require us to repeat all the code that is already there in
directory-files (and which you missed).  For example, file names need
to be encoded before they are passed to libc functions (or any
external APIs that expect file names).  As a bonus, we will be able to
return the file names we read, not just ignore them.  And the code
will be much more portable; if someone wants a more efficient
Linux-only version, that could be added as an additional feature
(assuming the speed difference justifies that).

WDYT?

Re: empty-directory predicate, native implementation

[Arthur Miller]

Michael Albinus <michael.albinus@gmx.de> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
> Hi Arthur,
>
>> I have seen in the source code calls to Ffind_file_handler, and have
>> been looking at the function in fileio.c, but I didn't figured out what
>> it really does, so I didn't use it. I tested without and it worked fine
>> :-). Anyway, I have added it now, but I am not sure if I using it
>> correctly, since I am not sure what it does.
>
> Looks good.
>
> Simplified spoken, this code makes Tramp work. If you call for example
> (directory-empty-p "/sudo::/") , find-file-name-handler (the ELisp
> equivalent to Ffind_file_name_handler) checks, whether Tramp feels
> responsible for this directory name, and it calls Tramp's function in
> case.
>
> You know, that your function cannot handle "/sudo::/".
:-) Actually I forgott that part completely.

>> By the way, what are "special constructs" referred to in the comment?
>
> Tramp file name syntax here. See (info "(elisp) Magic File Names")
>
Thank you very much for kind explanation; I have read the info now, and
I understand it now. Not sure that I know how to implement it though,
but it does not matter; Eli's suggestion seems even better; I think.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi Arthur,

>> Tramp file name syntax here. See (info "(elisp) Magic File Names")
>>
> Thank you very much for kind explanation; I have read the info now, and
> I understand it now. Not sure that I know how to implement it though,
> but it does not matter; Eli's suggestion seems even better; I think.

Agreed, we shall follow Eli's suggestion. But if we would take your
proposal, the Tramp implementation would happen in tramp.el.

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Arthur Miller]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Arthur Miller <arthur.miller@live.com>
>> Date: Tue, 13 Oct 2020 04:22:36 +0200
>> 
>> It is easy to check for an empty dir in elisp; we can just list files
>> and check if there is a list or not:
>> 
>>   (null (directory-files directory-name nil nodots t)))
>> 
>> where nodots is just regex to omit dot files (from dired+).
>> 
>> But then this is quite inneficient. We are listing all files in each
>> dir since directory-files will return entire content of directory. Also
>> we are matching every filename to a regex just to eliminate first two.
>> Alternative would be to take length and see if it is > 2; but then we
>> would iterate whole list twice. So I can't see anything avialable in
>> dired/elisp and I think a predicate implemented in low-level is better solution.
>> We are really interested just to see if there is some file; so we can
>> just open dir, and read first few entries, if there is more then 2 files
>> (. and .. on *nix) we can just abort and return true.
>> 
>> I have tested an idea with getdents (Linux syscall) and I can see
>> difference. Attached is a patch for dired.c and a test file to play with
>> some benchmark.
>
> If all we want is to stop reading a directory after N entries, why not
> simply extend directory-files to accept one more argument: the maximum
> number of file entries to read?  That should be easy to implement, and
> will not require us to repeat all the code that is already there in
> directory-files (and which you missed).  For example, file names need
> to be encoded before they are passed to libc functions (or any
> external APIs that expect file names).  As a bonus, we will be able to
> return the file names we read, not just ignore them.  And the code
> will be much more portable; if someone wants a more efficient
> Linux-only version, that could be added as an additional feature
> (assuming the speed difference justifies that).
Oh; and you say that now when I have just implemented a Windows
version :-).

> WDYT?
I think that sounds like a very good idea; or rather a splended one!
Much better then a predicate. If directory-files is asked just for 1
file, with regex to ignore, it would read max 3 files at most, and on
some filysystems maybe only one file.

> directory-files (and which you missed).  For example, file names need
> to be encoded before they are passed to libc functions (or any
> external APIs that expect file names).
Actually I didn't; I first tested with ENCODE_FILE but I didn't know
what it does, and it appeared like it is working without, so I ditched
it. I totally forgott about tramp & co, as Michael points out :-).

I didn't count on it to be correct; I am not so very familiar with
Emascs source, there are so many details to take care of, I was rather
just presenting an idea.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

> As a bonus, we will be able to return the file names we read, not just
> ignore them.

I agree with your proposal. However, I don't see why it is a bonus to
know which file names have read already, because they are taken in
random order. I fail to see how does it help, whether a file name is
returned under the first N matches, or not.

> WDYT?

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Eli Zaretskii]

> From: Arthur Miller <arthur.miller@live.com>
> Cc: emacs-devel@gnu.org
> Date: Tue, 13 Oct 2020 20:43:41 +0200
> 
> > directory-files (and which you missed).  For example, file names need
> > to be encoded before they are passed to libc functions (or any
> > external APIs that expect file names).
> Actually I didn't; I first tested with ENCODE_FILE but I didn't know
> what it does, and it appeared like it is working without, so I ditched
> it.

It worked without ENCODE_FILE by sheer luck: you must have tried that
on a system where file names are encoded in UTF-8, and so most file
names can be used without any conversion (but ENCODE_FILE already
knows that).

Re: empty-directory predicate, native implementation

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Arthur Miller <arthur.miller@live.com>,  emacs-devel@gnu.org
> Date: Tue, 13 Oct 2020 20:44:18 +0200
> 
> > As a bonus, we will be able to return the file names we read, not just
> > ignore them.
> 
> I agree with your proposal. However, I don't see why it is a bonus to
> know which file names have read already, because they are taken in
> random order.

The order depends on the filesystem.  For example, MS-Windows
filesystems always report in alphabetic order.

> I fail to see how does it help, whether a file name is returned
> under the first N matches, or not.

Well, it is easy to disregard the names if they aren't needed.

Re: empty-directory predicate, native implementation

[Arthur Miller]

[-- Attachment #1: Type: text/plain, Size: 1706 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Arthur Miller <arthur.miller@live.com>
>> Cc: emacs-devel@gnu.org
>> Date: Tue, 13 Oct 2020 20:43:41 +0200
>> 
>> > directory-files (and which you missed).  For example, file names need
>> > to be encoded before they are passed to libc functions (or any
>> > external APIs that expect file names).
>> Actually I didn't; I first tested with ENCODE_FILE but I didn't know
>> what it does, and it appeared like it is working without, so I ditched
>> it.
>
> It worked without ENCODE_FILE by sheer luck: you must have tried that
> on a system where file names are encoded in UTF-8, and so most file
> names can be used without any conversion (but ENCODE_FILE already
> knows that).
I see :-). I spent quite a lot time earlier today to get it to work on
windows, mostly for the conversion between utf8 and unicode; tried first
to manually convert between utf/wide, and after several tries discovered
some functions in w32.c I could reuse: filename_to_utf16 and 
filename_to_ansi, which worked for me (minus tramp of course).

Can I ask two questions, just for my learning; you seem to handle
ascii/unicode "manually"; in w32.c I have seen quite a few places where
switch is done on if(w32_unicode_filenames); is it possible to somehow
pass to configure script if Emacs should be compiled with wide or asci
version of win api? 

Second question: block_input; is there some general rule(s) what calls you
enclose with block/unblock_input? I guess OS will anyway block the
process after call to SYS_getdents so I didn't enclosed it; but I am not
sure if that would be needed at all or not.

I have attached my patch with win code as illustration to those
questions.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: dired.c --]
[-- Type: text/x-patch, Size: 5635 bytes --]

--- src/dired.c	2020-10-13 20:09:48.235962328 +0200
+++ ../dired.c	2020-10-13 20:09:10.865527164 +0200
@@ -22,6 +22,12 @@
 
 #include <sys/stat.h>
 
+#if __linux__
+#include <sys/syscall.h>
+#elif _WIN32
+#include <windows.h>
+#endif
+
 #ifdef HAVE_PWD_H
 #include <pwd.h>
 #endif
@@ -39,6 +45,7 @@
 #include "systime.h"
 #include "buffer.h"
 #include "coding.h"
+#include "blockinput.h"
 
 #ifdef MSDOS
 #include "msdos.h"	/* for fstatat */
@@ -929,7 +936,7 @@
   struct stat s;
 
   /* An array to hold the mode string generated by filemodestring,
-     including its terminating space and null byte.  */
+     including its terminating space and NUL byte.  */
   char modes[sizeof "-rwxr-xr-x "];
 
   char *uname = NULL, *gname = NULL;
@@ -1078,6 +1085,119 @@
   return groups;
 }
 
+DEFUN ("directory-empty-p", Fdirectory_empty_p,
+       Sdirectory_empty_p, 1, 1, 0,
+       doc: /* Returns t if directory DIRNAME does not contain any
+       user files (special files . and .. are excluded
+       automatically), nil otherwise. */)
+(Lisp_Object dirname)
+{
+        const char* name;
+        Lisp_Object handler;
+
+        if(!STRINGP(dirname))
+                error("Directory name not a string object.");
+        
+        dirname = Fexpand_file_name(dirname, Qnil);
+
+        /* If the file name has special constructs in it,
+           call the corresponding file name handler.  */
+        handler = Ffind_file_name_handler (dirname, Qdirectory_empty_p);
+        if (!NILP (handler))
+                return call2 (handler, Qdirectory_empty_p, dirname);
+        
+        name = SSDATA(dirname);
+        
+#ifdef __linux__
+        {
+                typedef struct dirent* pdirent;
+#define BSIZE 1024
+                pdirent d;
+                char buf[BSIZE];
+                int fd, n = 0, p = 0, c = 0;
+                
+                fd = open (name, O_RDONLY | O_DIRECTORY);
+
+                if( fd == -1 )
+                        error("Can't open directory.");
+
+                //block_input();
+                /* 32-bit version of getdents should be good enough;
+                   we are just looking at first 3 files*/
+                n = syscall(SYS_getdents,fd,buf, BSIZE);
+                if(n == -1)
+                        error("Can't read directory data.");
+
+                while(p < n && c < 3) {
+                        d = (pdirent) (buf + p);
+                        p += d->d_reclen;
+                        c++;
+                }
+                //unblock_input();
+        
+                close(fd);
+                return (c > 2) ? Qnil : Qt;
+        }
+#elif _WIN32
+        {
+                int len, c = 0;
+                HANDLE h = INVALID_HANDLE_VALUE;
+
+                if(w32_unicode_filenames){
+                        WIN32_FIND_DATAW ffd;
+                        wchar_t wname[MAX_PATH];
+                        filename_to_utf16(name, wname);
+                        len = wcslen(wname);
+
+                        if (!len)
+                                error(TEXT("Can't read directory name."));
+
+                        if(len > MAX_PATH - 3)
+                                error(TEXT("Directory path too long."));
+
+                        wcscat(wname, L"\\*");
+                
+                        h = FindFirstFileW(wname, &ffd);
+                        if (h == INVALID_HANDLE_VALUE)
+                                error(TEXT("Can't open directory."));
+
+                        while (FindNextFileW(h, &ffd) && c < 1){
+                                if(wcscmp(ffd.cFileName, L".") &&
+                                   wcscmp(ffd.cFileName, L".."))
+                                        c++;
+                        }
+                }else {
+                        WIN32_FIND_DATAA ffd;
+                        char aname[MAX_UTF8_PATH];
+                        filename_to_ansi(name, aname);
+                        len = strlen(aname);
+
+                        if (!len)
+                                error(TEXT("Can't read directory name."));
+
+                        if(len > MAX_PATH - 3)
+                                error(TEXT("Directory path too long."));
+
+                        strcat(aname, "\\*");
+                        
+                        h = FindFirstFileA(aname, &ffd);
+                        if (h == INVALID_HANDLE_VALUE)
+                                error(TEXT("Can't open directory."));
+                        
+                        while (FindNextFileA(h, &ffd) && c < 1){
+                                if(strcmp(ffd.cFileName, ".") &&
+                                   strcmp(ffd.cFileName, ".."))
+                                        c++;
+                        }
+                }
+                
+                FindClose(h);
+                return (c > 0) ? Qnil : Qt;
+        }
+#endif
+}
+
+
 void
 syms_of_dired (void)
 {
@@ -1089,7 +1209,8 @@
   DEFSYM (Qfile_attributes_lessp, "file-attributes-lessp");
   DEFSYM (Qdefault_directory, "default-directory");
   DEFSYM (Qdecomposed_characters, "decomposed-characters");
-
+  DEFSYM (Qdirectory_empty_p, "directory-empty-p")
+          
   defsubr (&Sdirectory_files);
   defsubr (&Sdirectory_files_and_attributes);
   defsubr (&Sfile_name_completion);
@@ -1098,6 +1219,7 @@
   defsubr (&Sfile_attributes_lessp);
   defsubr (&Ssystem_users);
   defsubr (&Ssystem_groups);
+  defsubr (&Sdirectory_empty_p);
 
   DEFVAR_LISP ("completion-ignored-extensions", Vcompletion_ignored_extensions,
 	       doc: /* Completion ignores file names ending in any string in this list.

Re: empty-directory predicate, native implementation

[Arthur Miller]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Michael Albinus <michael.albinus@gmx.de>
>> Cc: Arthur Miller <arthur.miller@live.com>,  emacs-devel@gnu.org
>> Date: Tue, 13 Oct 2020 20:44:18 +0200
>> 
>> > As a bonus, we will be able to return the file names we read, not just
>> > ignore them.
>> 
>> I agree with your proposal. However, I don't see why it is a bonus to
>> know which file names have read already, because they are taken in
>> random order.
>
> The order depends on the filesystem.  For example, MS-Windows
> filesystems always report in alphabetic order.
>
Not that it is important, just as a note:

I looked up today msdn while I was playing with this, they mention that
order is not guarantueed at all; FAT systems will returned file in order
they are written on disk, while on NTFS & CDFS they are usually returned
in alphabetical order. I don't know why they say "usually" :-), but they
do remark that any application should sort results if sorted order is
important.

https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-findnextfilea

>> I fail to see how does it help, whether a file name is returned
>> under the first N matches, or not.
>
> Well, it is easy to disregard the names if they aren't needed.
I am not sure of use-case, but it can be a bonus to get N number of
files instead of all. If Elisp was used for scripting in some very large
filesystems maybe? I don't know.

Re: empty-directory predicate, native implementation

[Arthur Miller]

Michael Albinus <michael.albinus@gmx.de> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
> Hi Arthur,
>
>>> Tramp file name syntax here. See (info "(elisp) Magic File Names")
>>>
>> Thank you very much for kind explanation; I have read the info now, and
>> I understand it now. Not sure that I know how to implement it though,
>> but it does not matter; Eli's suggestion seems even better; I think.
>
> Agreed, we shall follow Eli's suggestion. But if we would take your
> proposal, the Tramp implementation would happen in tramp.el.
>
> Best regards, Michael.
Thanks for the pointer. I was looking at tramp.el, but I it is ~6k loc
:-). I am not sure I understand what I would need to do to implement
for this; would like to learn though. Can you maybe point me to some
function which I can take a look that would be similar to what I tried
here use-case.

Anyway, Eli's idea is much simpler; I just implemented and tested and it
seems to work nice. I have a problem with helm-find-files but I'll post
in the answer to Eli together with the patch.

Re: empty-directory predicate, native implementation

[Arthur Miller]

[-- Attachment #1: Type: text/plain, Size: 1833 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Michael Albinus <michael.albinus@gmx.de>
>> Cc: Arthur Miller <arthur.miller@live.com>,  emacs-devel@gnu.org
>> Date: Tue, 13 Oct 2020 20:44:18 +0200
>> 
>> > As a bonus, we will be able to return the file names we read, not just
>> > ignore them.
>> 
>> I agree with your proposal. However, I don't see why it is a bonus to
>> know which file names have read already, because they are taken in
>> random order.
>
> The order depends on the filesystem.  For example, MS-Windows
> filesystems always report in alphabetic order.
>
>> I fail to see how does it help, whether a file name is returned
>> under the first N matches, or not.
>
> Well, it is easy to disregard the names if they aren't needed.

Your idea was definitely much simpler to implement, rather trivial :-).
I just tested for myself, and it seems to work well.

The only thing I am not sure about is how to deal with 0 count; I
mean if user explicitly ask for 0 count; the case when count is not
NILP. I have chosen not to deal with it all, I check for zer set
explicit limit for count to be [1,COUNT]. I am not sure if that is best
thing to do; maybe just to return Qnil? I am probably doing something
else wrong, advice apprecaiated. 

I had some problems until I realized i used wrong number converter; I
used fixnum, seems like I need fixnat. I am still not sure if I am
correct but it seems to work now. 

I tested this compared to my old "native" predicate, on GNU/Linux only
and just to run naked predicate on a big number of dirs, and it seems to
be just very slight difference; ~0.002 vs ~0.004 definitely not dramatic
as entire decimal place.

I needed to "touch" 2 other files due to chagned signature of
directory_files: lisp.h and sysdep.c.

In attached .el file are two small tests (I run them with M-:).


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: dired.c.patch --]
[-- Type: text/x-patch, Size: 5123 bytes --]

--- src/dired.c	2020-10-14 03:35:16.781786426 +0200
+++ ../dired2.c	2020-10-14 03:35:05.271568190 +0200
@@ -20,6 +20,7 @@
 
 #include <config.h>
 
+#include <stddef.h>
 #include <sys/stat.h>
 
 #ifdef HAVE_PWD_H
@@ -165,7 +166,7 @@
 Lisp_Object
 directory_files_internal (Lisp_Object directory, Lisp_Object full,
 			  Lisp_Object match, Lisp_Object nosort, bool attrs,
-			  Lisp_Object id_format)
+			  Lisp_Object id_format, Lisp_Object count_to_return)
 {
   if (!NILP (match))
     CHECK_STRING (match);
@@ -223,7 +224,14 @@
 #ifdef WINDOWSNT
   case_table = BVAR (&buffer_defaults, case_canon_table);
 #endif
-
+  
+  ptrdiff_t ind = 0, last = 0;
+  if(!NILP(count_to_return) && FIXNATP(count_to_return))
+      last = XFIXNAT(count_to_return);
+  
+  if(!last)
+    last = 1;
+  
   /* Read directory entries and accumulate them into LIST.  */
   Lisp_Object list = Qnil;
   for (struct dirent *dp; (dp = read_dirent (d, directory)); )
@@ -267,6 +275,13 @@
       else
 	finalname = name;
 
+      if(!NILP(count_to_return) && FIXNATP(count_to_return))
+        {
+          if(ind == last)
+            break;
+          ind ++;
+        }
+      
       list = Fcons (attrs ? Fcons (finalname, fileattrs) : finalname, list);
     }
 
@@ -282,13 +297,13 @@
   if (NILP (nosort))
     list = Fsort (Fnreverse (list),
 		  attrs ? Qfile_attributes_lessp : Qstring_lessp);
-
+  
   (void) directory_volatile;
   return list;
 }
 
 
-DEFUN ("directory-files", Fdirectory_files, Sdirectory_files, 1, 4, 0,
+DEFUN ("directory-files", Fdirectory_files, Sdirectory_files, 1, 5, 0,
        doc: /* Return a list of names of files in DIRECTORY.
 There are three optional arguments:
 If FULL is non-nil, return absolute file names.  Otherwise return names
@@ -296,9 +311,14 @@
 If MATCH is non-nil, mention only file names that match the regexp MATCH.
 If NOSORT is non-nil, the list is not sorted--its order is unpredictable.
  Otherwise, the list returned is sorted with `string-lessp'.
- NOSORT is useful if you plan to sort the result yourself.  */)
+ NOSORT is useful if you plan to sort the result yourself. 
+If COUNT is  non-nil, the function will return max (COUNT,length)
+ files, where length means number of files in directory. Order
+ in which files are returned is not guaranteed and is file system and
+ OS dependent. COUNT has to be an integral number in interval
+ [1,COUNT]. */) 
   (Lisp_Object directory, Lisp_Object full, Lisp_Object match,
-   Lisp_Object nosort)
+   Lisp_Object nosort, Lisp_Object count)
 {
   directory = Fexpand_file_name (directory, Qnil);
 
@@ -306,14 +326,15 @@
      call the corresponding file name handler.  */
   Lisp_Object handler = Ffind_file_name_handler (directory, Qdirectory_files);
   if (!NILP (handler))
-    return call5 (handler, Qdirectory_files, directory,
-                  full, match, nosort);
+    return call6 (handler, Qdirectory_files, directory,
+                  full, match, nosort, count);
 
-  return directory_files_internal (directory, full, match, nosort, false, Qnil);
+  return directory_files_internal (directory, full, match, nosort,
+                                   false, Qnil, count);
 }
 
 DEFUN ("directory-files-and-attributes", Fdirectory_files_and_attributes,
-       Sdirectory_files_and_attributes, 1, 5, 0,
+       Sdirectory_files_and_attributes, 1, 6, 0,
        doc: /* Return a list of names of files and their attributes in DIRECTORY.
 Value is a list of the form:
 
@@ -331,9 +352,14 @@
 ID-FORMAT specifies the preferred format of attributes uid and gid, see
 `file-attributes' for further documentation.
 On MS-Windows, performance depends on `w32-get-true-file-attributes',
-which see.  */)
-  (Lisp_Object directory, Lisp_Object full, Lisp_Object match,
-   Lisp_Object nosort, Lisp_Object id_format)
+which see. 
+If COUNT is  non-nil, the function will return max (COUNT,length)
+ files, where length means number of files in directory. Order
+ in which files are returned is not guaranteed and is file system and
+ OS dependent. COUNT has to be an integral number in interval
+ [1,COUNT]. */)
+(Lisp_Object directory, Lisp_Object full, Lisp_Object match,
+   Lisp_Object nosort, Lisp_Object id_format, Lisp_Object count)
 {
   directory = Fexpand_file_name (directory, Qnil);
 
@@ -342,11 +368,11 @@
   Lisp_Object handler
     = Ffind_file_name_handler (directory, Qdirectory_files_and_attributes);
   if (!NILP (handler))
-    return call6 (handler, Qdirectory_files_and_attributes,
-                  directory, full, match, nosort, id_format);
+    return call7 (handler, Qdirectory_files_and_attributes,
+                  directory, full, match, nosort, id_format, count);
 
   return directory_files_internal (directory, full, match, nosort,
-				   true, id_format);
+				   true, id_format, count);
 }
 
 \f
@@ -929,7 +955,7 @@
   struct stat s;
 
   /* An array to hold the mode string generated by filemodestring,
-     including its terminating space and null byte.  */
+     including its terminating space and NUL byte.  */
   char modes[sizeof "-rwxr-xr-x "];
 
   char *uname = NULL, *gname = NULL;

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: sysdep.c.patch --]
[-- Type: text/x-patch, Size: 534 bytes --]

--- src/sysdep.c	2020-10-13 23:22:37.521367461 +0200
+++ ../sysdep.c	2020-10-13 22:53:04.272682143 +0200
@@ -2892,7 +2892,8 @@
      process.  */
   procdir = build_string ("/proc");
   match = build_string ("[0-9]+");
-  proclist = directory_files_internal (procdir, Qnil, match, Qt, false, Qnil);
+  proclist = directory_files_internal (procdir, Qnil, match, Qt,
+                                       false, Qnil, Qnil);
 
   /* `proclist' gives process IDs as strings.  Destructively convert
      each string into a number.  */

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: lisp.h.patch --]
[-- Type: text/x-patch, Size: 510 bytes --]

--- src/lisp.h	2020-10-13 23:20:23.555386699 +0200
+++ ../lisp.h	2020-10-13 22:47:04.192203245 +0200
@@ -4633,7 +4633,7 @@
 extern void syms_of_dired (void);
 extern Lisp_Object directory_files_internal (Lisp_Object, Lisp_Object,
                                              Lisp_Object, Lisp_Object,
-                                             bool, Lisp_Object);
+                                             bool, Lisp_Object, Lisp_Object);
 
 /* Defined in term.c.  */
 extern int *char_ins_del_vector;

[-- Attachment #5: dired-mark-empty.el --]
[-- Type: text/plain, Size: 2395 bytes --]

(require 'dired)

(defvar nodots "^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*")

(defun dired-go-to-first ()
    (interactive)
    (goto-char (point-min))
    (dired-next-line 1)
    (skip-chars-forward " \n\t"))

(defun dired-is-empty-p (directory-name)
  (null (directory-files directory-name nil nodots t 1)))

(defun directory-number-files (directory-name &optional omit-filter)
  (length (directory-files directory-name nil omit-filter t)))

(defun dired-mark-empty-dirs ()
  (interactive)
  (when (equal major-mode 'dired-mode)
    (let ((curr-dir))
      (save-excursion
        (dired-go-to-first)
        
        (while (not (eobp))
          (setq curr-dir (dired-file-name-at-point))
          (cond ((or (null curr-dir)
                     (string= curr-dir ".")
                     (string= curr-dir ".."))
                     ;; do nothing here
                 )
                ((file-directory-p curr-dir)
                 (when (dired-is-empty-p curr-dir)
                   (dired-mark 1)
                   (dired-previous-line 1))))
          (dired-next-line 1))))))

(defun test-1 (dir-name count)
  "Ask for COUNT number of files from directory DIR-NAME."
  (let ((max (length (directory-files dir-name nil nil t)))
        (res (length (directory-files dir-name nil nil t count))))
    (cond ((>= max count)
           (if (= res count)
               (message "passed: asked: %d get: %d number files in dir: %d" count
                        res max)
             (message "not-passed: asked: %s get: %s number files in dir: %s" count
                      res max)))
          ((< max count)
           (if (< res count)
              (message "passed: asked: %d get: %d number files in dir: %d" count
                       res max)
            (message "not-passed: asked: %s get: %s number files in dir: %s" count
                     res max)))
          (t
           (message "Invalid case!")))))

(defun test-2 (dir-name count)
  "Ask for random number of files in interval [1,COUNT] files from every subdir
of directory DIR-NAME."
  (let ((dirs (directory-files dir-name t nodots t)))
    (dolist (curr-dir dirs)
      (when (file-directory-p curr-dir)
        (test-1 curr-dir (+ 1 (random count)))))))

;;(benchmark-run-compiled 10 (dired-is-empty-p "/some/path"))

;; must be in some dired buffer for this
;;(benchmark-run-compiled 10 (dired-mark-empty-dirs))

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi Arthur,

> Thanks for the pointer. I was looking at tramp.el, but I it is ~6k loc
> :-). I am not sure I understand what I would need to do to implement
> for this; would like to learn though. Can you maybe point me to some
> function which I can take a look that would be similar to what I tried
> here use-case.

Follow directory-files. tramp-handle-directory-files is the Tramp
implementation of that function.

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi Arthur,

> The only thing I am not sure about is how to deal with 0 count; I
> mean if user explicitly ask for 0 count; the case when count is not
> NILP.

>I have chosen not to deal with it all, I check for zer set
> explicit limit for count to be [1,COUNT]. I am not sure if that is best
> thing to do; maybe just to return Qnil? I am probably doing something
> else wrong, advice apprecaiated.

Returning Qnil seems to be OK, when count is 0.

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Arthur Miller]

Michael Albinus <michael.albinus@gmx.de> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
> Hi Arthur,
>
>> Thanks for the pointer. I was looking at tramp.el, but I it is ~6k loc
>> :-). I am not sure I understand what I would need to do to implement
>> for this; would like to learn though. Can you maybe point me to some
>> function which I can take a look that would be similar to what I tried
>> here use-case.
>
> Follow directory-files. tramp-handle-directory-files is the Tramp
> implementation of that function.
>
> Best regards, Michael.
Of course; I am so stupid sometimes :-) I was too fast, wasn't thinking, thanks!

Re: empty-directory predicate, native implementation

[Arthur Miller]

Michael Albinus <michael.albinus@gmx.de> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
> Hi Arthur,
>
>> The only thing I am not sure about is how to deal with 0 count; I
>> mean if user explicitly ask for 0 count; the case when count is not
>> NILP.
>
>>I have chosen not to deal with it all, I check for zer set
>> explicit limit for count to be [1,COUNT]. I am not sure if that is best
>> thing to do; maybe just to return Qnil? I am probably doing something
>> else wrong, advice apprecaiated.
>
> Returning Qnil seems to be OK, when count is 0.
>
> Best regards, Michael.
What about return the length, number of the files in that case?

In order to get number of the files currently we would get list of files
and get length of the list afterwards; if we asked to get 0 files, it
could return the length; I thought of it afterwards.

Re: empty-directory predicate, native implementation

[Eli Zaretskii]

> From: Arthur Miller <arthur.miller@live.com>
> Cc: emacs-devel@gnu.org
> Date: Tue, 13 Oct 2020 21:59:47 +0200
> 
> Can I ask two questions, just for my learning; you seem to handle
> ascii/unicode "manually"; in w32.c I have seen quite a few places where
> switch is done on if(w32_unicode_filenames); is it possible to somehow
> pass to configure script if Emacs should be compiled with wide or asci
> version of win api? 

The idea is not to decide this at configure and compile time, but at
runtime.  That way, the same Emacs binary can run both on systems that
support "wide" (a.k.a. "Unicode") APIs and on systems that don't.
Think about users who download the "official" binaries from the GNU
FTP site.

Besides, the compile-time "switch" to Unicode makes all TCHAR
variables be wchar_t under the hood, which means one cannot use
strlen, strcat, etc.  So this "feature" is a mixed blessing,
especially when the internal encoding is UTF-8.

> Second question: block_input; is there some general rule(s) what calls you
> enclose with block/unblock_input? I guess OS will anyway block the
> process after call to SYS_getdents so I didn't enclosed it; but I am not
> sure if that would be needed at all or not.

See the commentary in src/blockinput.h, I hope it answers the
question.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi Arthur,

>>>I have chosen not to deal with it all, I check for zer set
>>> explicit limit for count to be [1,COUNT]. I am not sure if that is best
>>> thing to do; maybe just to return Qnil? I am probably doing something
>>> else wrong, advice apprecaiated.
>>
>> Returning Qnil seems to be OK, when count is 0.
>>
> What about return the length, number of the files in that case?

No, this would be another signature of the response (number instead of
list of strings).

> In order to get number of the files currently we would get list of files
> and get length of the list afterwards; if we asked to get 0 files, it
> could return the length; I thought of it afterwards.

I cannot imagine that this information is needed somewhere. Why do you
need to know, how many files are in a directory?

Berst regards, Michael.

Re: empty-directory predicate, native implementation

[Arthur Miller]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Arthur Miller <arthur.miller@live.com>
>> Cc: emacs-devel@gnu.org
>> Date: Tue, 13 Oct 2020 21:59:47 +0200
>> 
>> Can I ask two questions, just for my learning; you seem to handle
>> ascii/unicode "manually"; in w32.c I have seen quite a few places where
>> switch is done on if(w32_unicode_filenames); is it possible to somehow
>> pass to configure script if Emacs should be compiled with wide or asci
>> version of win api? 
>
> The idea is not to decide this at configure and compile time, but at
> runtime.  That way, the same Emacs binary can run both on systems that
> support "wide" (a.k.a. "Unicode") APIs and on systems that don't.
> Think about users who download the "official" binaries from the GNU
> FTP site.
Ok, I understand the reason now.

> Besides, the compile-time "switch" to Unicode makes all TCHAR
> variables be wchar_t under the hood, which means one cannot use
> strlen, strcat, etc.  So this "feature" is a mixed blessing,
> especially when the internal encoding is UTF-8.
Yes; that is what they do in VS (they have option on general tab to use
unicode/multibyte/not-defined == ascii), so I was thinking about
something like that with configure, but now I understand why you don't
have it.

>> Second question: block_input; is there some general rule(s) what calls you
>> enclose with block/unblock_input? I guess OS will anyway block the
>> process after call to SYS_getdents so I didn't enclosed it; but I am not
>> sure if that would be needed at all or not.
>
> See the commentary in src/blockinput.h, I hope it answers the
> question.
Yes, will do. Thanks.

Re: empty-directory predicate, native implementation

[Arthur Miller]

[-- Attachment #1: Type: text/plain, Size: 1511 bytes --]

Arthur Miller <arthur.miller@live.com> writes:

> Michael Albinus <michael.albinus@gmx.de> writes:
>
>> Arthur Miller <arthur.miller@live.com> writes:
>>
>> Hi Arthur,
>>
>>> The only thing I am not sure about is how to deal with 0 count; I
>>> mean if user explicitly ask for 0 count; the case when count is not
>>> NILP.
>>
>>>I have chosen not to deal with it all, I check for zer set
>>> explicit limit for count to be [1,COUNT]. I am not sure if that is best
>>> thing to do; maybe just to return Qnil? I am probably doing something
>>> else wrong, advice apprecaiated.
>>
>> Returning Qnil seems to be OK, when count is 0.
>>
>> Best regards, Michael.
> What about return the length, number of the files in that case?
>
> In order to get number of the files currently we would get list of files
> and get length of the list afterwards; if we asked to get 0 files, it
> could return the length; I thought of it afterwards.

Ok; just a very-very fast hack; I am in very rush today, so this was my
coffee pause :-)

I have tested with those previous tests, seems to work, but I don't have
time to test on windows. I am also not sure if I can circut that early,
please see just as an idea.

If cunt 0 is asked, then directory-files will return number of files in
directory. If also "match" is provided it *should* give only number of
files that match the given regex and sort will have no effect (no
sorting). I don't have time to test with match or to write docs; but can
do probably tomorrow or on friday.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: dired.c.patch --]
[-- Type: text/x-patch, Size: 5572 bytes --]

--- /home/arthur/dired.c	2020-10-13 13:06:07.517648492 +0200
+++ src/dired.c	2020-10-14 16:18:03.750214680 +0200
@@ -20,6 +20,7 @@
 
 #include <config.h>
 
+#include <stddef.h>
 #include <sys/stat.h>
 
 #ifdef HAVE_PWD_H
@@ -165,7 +166,7 @@
 Lisp_Object
 directory_files_internal (Lisp_Object directory, Lisp_Object full,
 			  Lisp_Object match, Lisp_Object nosort, bool attrs,
-			  Lisp_Object id_format)
+			  Lisp_Object id_format, Lisp_Object count_to_return)
 {
   if (!NILP (match))
     CHECK_STRING (match);
@@ -224,6 +225,18 @@
   case_table = BVAR (&buffer_defaults, case_canon_table);
 #endif
 
+  bool only_length = false;
+  ptrdiff_t ind = 0, last = 0;
+  if(!NILP(count_to_return) && FIXNATP(count_to_return))
+    {
+      last = XFIXNAT(count_to_return);
+  
+      if(!last){
+        last = 1;
+        only_length = true;
+      }
+    }
+
   /* Read directory entries and accumulate them into LIST.  */
   Lisp_Object list = Qnil;
   for (struct dirent *dp; (dp = read_dirent (d, directory)); )
@@ -267,7 +280,22 @@
       else
 	finalname = name;
 
-      list = Fcons (attrs ? Fcons (finalname, fileattrs) : finalname, list);
+      if (only_length)
+        {
+          ind++;
+        }
+      else
+        {
+          if (!NILP(count_to_return) && FIXNATP(count_to_return))
+            {
+              if (ind == last)
+                break;
+              ind ++;
+            }
+      
+          list = Fcons (attrs ? Fcons (finalname, fileattrs) :
+                        finalname, list);
+        }
     }
 
   closedir (d);
@@ -279,16 +307,19 @@
   /* Discard the unwind protect.  */
   specpdl_ptr = specpdl + count;
 
+  if (only_length)
+    return make_fixnum(ind);
+
   if (NILP (nosort))
     list = Fsort (Fnreverse (list),
 		  attrs ? Qfile_attributes_lessp : Qstring_lessp);
-
+  
   (void) directory_volatile;
   return list;
 }
 
 
-DEFUN ("directory-files", Fdirectory_files, Sdirectory_files, 1, 4, 0,
+DEFUN ("directory-files", Fdirectory_files, Sdirectory_files, 1, 5, 0,
        doc: /* Return a list of names of files in DIRECTORY.
 There are three optional arguments:
 If FULL is non-nil, return absolute file names.  Otherwise return names
@@ -296,9 +327,14 @@
 If MATCH is non-nil, mention only file names that match the regexp MATCH.
 If NOSORT is non-nil, the list is not sorted--its order is unpredictable.
  Otherwise, the list returned is sorted with `string-lessp'.
- NOSORT is useful if you plan to sort the result yourself.  */)
+ NOSORT is useful if you plan to sort the result yourself. 
+If COUNT is  non-nil, the function will return max (COUNT,length)
+ files, where length means number of files in directory. Order
+ in which files are returned is not guaranteed and is file system and
+ OS dependent. COUNT has to be an integral number in interval
+ [1,COUNT]. */) 
   (Lisp_Object directory, Lisp_Object full, Lisp_Object match,
-   Lisp_Object nosort)
+   Lisp_Object nosort, Lisp_Object count)
 {
   directory = Fexpand_file_name (directory, Qnil);
 
@@ -306,14 +342,15 @@
      call the corresponding file name handler.  */
   Lisp_Object handler = Ffind_file_name_handler (directory, Qdirectory_files);
   if (!NILP (handler))
-    return call5 (handler, Qdirectory_files, directory,
-                  full, match, nosort);
+    return call6 (handler, Qdirectory_files, directory,
+                  full, match, nosort, count);
 
-  return directory_files_internal (directory, full, match, nosort, false, Qnil);
+  return directory_files_internal (directory, full, match, nosort,
+                                   false, Qnil, count);
 }
 
 DEFUN ("directory-files-and-attributes", Fdirectory_files_and_attributes,
-       Sdirectory_files_and_attributes, 1, 5, 0,
+       Sdirectory_files_and_attributes, 1, 6, 0,
        doc: /* Return a list of names of files and their attributes in DIRECTORY.
 Value is a list of the form:
 
@@ -331,9 +368,14 @@
 ID-FORMAT specifies the preferred format of attributes uid and gid, see
 `file-attributes' for further documentation.
 On MS-Windows, performance depends on `w32-get-true-file-attributes',
-which see.  */)
-  (Lisp_Object directory, Lisp_Object full, Lisp_Object match,
-   Lisp_Object nosort, Lisp_Object id_format)
+which see. 
+If COUNT is  non-nil, the function will return max (COUNT,length)
+ files, where length means number of files in directory. Order
+ in which files are returned is not guaranteed and is file system and
+ OS dependent. COUNT has to be an integral number in interval
+ [1,COUNT]. */)
+(Lisp_Object directory, Lisp_Object full, Lisp_Object match,
+   Lisp_Object nosort, Lisp_Object id_format, Lisp_Object count)
 {
   directory = Fexpand_file_name (directory, Qnil);
 
@@ -342,11 +384,11 @@
   Lisp_Object handler
     = Ffind_file_name_handler (directory, Qdirectory_files_and_attributes);
   if (!NILP (handler))
-    return call6 (handler, Qdirectory_files_and_attributes,
-                  directory, full, match, nosort, id_format);
+    return call7 (handler, Qdirectory_files_and_attributes,
+                  directory, full, match, nosort, id_format, count);
 
   return directory_files_internal (directory, full, match, nosort,
-				   true, id_format);
+				   true, id_format, count);
 }
 
 \f
@@ -929,7 +971,7 @@
   struct stat s;
 
   /* An array to hold the mode string generated by filemodestring,
-     including its terminating space and null byte.  */
+     including its terminating space and NUL byte.  */
   char modes[sizeof "-rwxr-xr-x "];
 
   char *uname = NULL, *gname = NULL;

Re: empty-directory predicate, native implementation

[Arthur Miller]

Michael Albinus <michael.albinus@gmx.de> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
> Hi Arthur,
>
>>>>I have chosen not to deal with it all, I check for zer set
>>>> explicit limit for count to be [1,COUNT]. I am not sure if that is best
>>>> thing to do; maybe just to return Qnil? I am probably doing something
>>>> else wrong, advice apprecaiated.
>>>
>>> Returning Qnil seems to be OK, when count is 0.
>>>
>> What about return the length, number of the files in that case?
>
> No, this would be another signature of the response (number instead of
> list of strings).
>
>> In order to get number of the files currently we would get list of files
>> and get length of the list afterwards; if we asked to get 0 files, it
>> could return the length; I thought of it afterwards.
>
> I cannot imagine that this information is needed somewhere. Why do you
> need to know, how many files are in a directory?
>
> Berst regards, Michael.
No idea :-).

It was just to test if empty; but now empty is better served with asking
just for one file and testing for null; and I was also thinking what to
do with 0 count if asked for after I sent patch yesterday; it seemed
like a bit more useful then just return nil early.

I can send tomorrow one that just returns nil and exits.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi Arthur,

> It was just to test if empty; but now empty is better served with asking
> just for one file and testing for null; and I was also thinking what to
> do with 0 count if asked for after I sent patch yesterday; it seemed
> like a bit more useful then just return nil early.
>
> I can send tomorrow one that just returns nil and exits.

Let's wait what Eli prefers.

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> Date: Wed, 14 Oct 2020 17:53:38 +0200
> 
> > It was just to test if empty; but now empty is better served with asking
> > just for one file and testing for null; and I was also thinking what to
> > do with 0 count if asked for after I sent patch yesterday; it seemed
> > like a bit more useful then just return nil early.
> >
> > I can send tomorrow one that just returns nil and exits.
> 
> Let's wait what Eli prefers.

I think 0 doesn't make much sense, so we might as well not support
that at all.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> I think 0 doesn't make much sense, so we might as well not support
> that at all.

In general, I agree. Calling directory-files with COUNT being 0,
literally, doesn't make sense. But COUNT could be the result of some
arithmetics, and instead urging the caller to test it, we could simply
return nil. Which is correct, reading nil as the empty list.

And it doesn't cost us anything.

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: arthur.miller@live.com,  emacs-devel@gnu.org
> Date: Wed, 14 Oct 2020 18:21:21 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I think 0 doesn't make much sense, so we might as well not support
> > that at all.
> 
> In general, I agree. Calling directory-files with COUNT being 0,
> literally, doesn't make sense. But COUNT could be the result of some
> arithmetics, and instead urging the caller to test it, we could simply
> return nil. Which is correct, reading nil as the empty list.
> 
> And it doesn't cost us anything.

I'm okay with interpreting "don't support" as "return nil".

Re: empty-directory predicate, native implementation

[Arthur Miller]

[-- Attachment #1: Type: text/plain, Size: 1222 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Michael Albinus <michael.albinus@gmx.de>
>> Cc: arthur.miller@live.com,  emacs-devel@gnu.org
>> Date: Wed, 14 Oct 2020 18:21:21 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > I think 0 doesn't make much sense, so we might as well not support
>> > that at all.
>> 
>> In general, I agree. Calling directory-files with COUNT being 0,
>> literally, doesn't make sense. But COUNT could be the result of some
>> arithmetics, and instead urging the caller to test it, we could simply
>> return nil. Which is correct, reading nil as the empty list.
>> 
>> And it doesn't cost us anything.
>
> I'm okay with interpreting "don't support" as "return nil".

Attached is patch which return nil early.

I was thinking a bit when it would be useful to return length; the only
thing I come up is to maybe allocate some structure, a vector or
something based on how many files there are without really carying about
files. In case there are many such objects, there would be lots of list
the lisp system does not care about, and GC will have to work a lot
unnecessary, but i think it is probably very rare case if not very
artifical, so nil is just fine for the most use-cases.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: dired.c.patch --]
[-- Type: text/x-patch, Size: 5369 bytes --]

--- src/dired.c	2020-10-15 07:43:02.633804624 +0200
+++ ../dired4.c	2020-10-15 07:42:01.358574645 +0200
@@ -17,7 +17,6 @@
 You should have received a copy of the GNU General Public License
 along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.  */
 
-
 #include <config.h>
 
 #include <sys/stat.h>
@@ -165,8 +164,19 @@
 Lisp_Object
 directory_files_internal (Lisp_Object directory, Lisp_Object full,
 			  Lisp_Object match, Lisp_Object nosort, bool attrs,
-			  Lisp_Object id_format)
+			  Lisp_Object id_format, Lisp_Object return_count)
 {
+  ptrdiff_t ind = 0, last = 0;
+
+  /* check count first for early exit */
+  if (FIXNATP(return_count))
+    {
+      last = XFIXNAT(return_count);
+  
+      if (!last)
+        return Qnil;
+    }
+
   if (!NILP (match))
     CHECK_STRING (match);
 
@@ -267,6 +277,13 @@
       else
 	finalname = name;
 
+      if (FIXNATP(return_count))
+        {
+          if (ind == last)
+            break;
+          ind ++;
+        }
+
       list = Fcons (attrs ? Fcons (finalname, fileattrs) : finalname, list);
     }
 
@@ -287,8 +304,7 @@
   return list;
 }
 
-
-DEFUN ("directory-files", Fdirectory_files, Sdirectory_files, 1, 4, 0,
+DEFUN ("directory-files", Fdirectory_files, Sdirectory_files, 1, 5, 0,
        doc: /* Return a list of names of files in DIRECTORY.
 There are three optional arguments:
 If FULL is non-nil, return absolute file names.  Otherwise return names
@@ -296,9 +312,14 @@
 If MATCH is non-nil, mention only file names that match the regexp MATCH.
 If NOSORT is non-nil, the list is not sorted--its order is unpredictable.
  Otherwise, the list returned is sorted with `string-lessp'.
- NOSORT is useful if you plan to sort the result yourself.  */)
+ NOSORT is useful if you plan to sort the result yourself.
+If COUNT is  non-nil, the function will return max of COUNT and length
+ files, where length is number of files in the directory. Order
+ in which files are returned is not guaranteed and is file system and
+ OS dependent. COUNT has to be an integral number in interval
+ [1,COUNT]. If 0 files are requested the function will return nil. */)
   (Lisp_Object directory, Lisp_Object full, Lisp_Object match,
-   Lisp_Object nosort)
+   Lisp_Object nosort, Lisp_Object count)
 {
   directory = Fexpand_file_name (directory, Qnil);
 
@@ -306,34 +327,40 @@
      call the corresponding file name handler.  */
   Lisp_Object handler = Ffind_file_name_handler (directory, Qdirectory_files);
   if (!NILP (handler))
-    return call5 (handler, Qdirectory_files, directory,
-                  full, match, nosort);
+    return call6 (handler, Qdirectory_files, directory,
+                  full, match, nosort, count);
 
-  return directory_files_internal (directory, full, match, nosort, false, Qnil);
+  return directory_files_internal (directory, full, match, nosort,
+                                   false, Qnil, count);
 }
 
 DEFUN ("directory-files-and-attributes", Fdirectory_files_and_attributes,
-       Sdirectory_files_and_attributes, 1, 5, 0,
-       doc: /* Return a list of names of files and their attributes in DIRECTORY.
+       Sdirectory_files_and_attributes, 1, 6, 0, doc
+       : /* Return a list of names of files and their attributes in DIRECTORY.
 Value is a list of the form:
 
-  ((FILE1 . FILE1-ATTRS) (FILE2 . FILE2-ATTRS) ...)
+((FILE1 . FILE1-ATTRS) (FILE2 . FILE2-ATTRS) ...)
 
 where each FILEn-ATTRS is the attributes of FILEn as returned
 by `file-attributes'.
 
 This function accepts four optional arguments:
 If FULL is non-nil, return absolute file names.  Otherwise return names
- that are relative to the specified directory.
+that are relative to the specified directory.
 If MATCH is non-nil, mention only file names that match the regexp MATCH.
 If NOSORT is non-nil, the list is not sorted--its order is unpredictable.
- NOSORT is useful if you plan to sort the result yourself.
+NOSORT is useful if you plan to sort the result yourself.
 ID-FORMAT specifies the preferred format of attributes uid and gid, see
 `file-attributes' for further documentation.
 On MS-Windows, performance depends on `w32-get-true-file-attributes',
-which see.  */)
-  (Lisp_Object directory, Lisp_Object full, Lisp_Object match,
-   Lisp_Object nosort, Lisp_Object id_format)
+which see.
+If COUNT is  non-nil, the function will return max of COUNT and length
+ files, where length is number of files in the directory. Order
+ in which files are returned is not guaranteed and is file system and
+ OS dependent. COUNT has to be an integral number in interval
+ [1,COUNT]. If 0 files are requested the function will return nil. */)
+(Lisp_Object directory, Lisp_Object full, Lisp_Object match,
+ Lisp_Object nosort, Lisp_Object id_format, Lisp_Object count)
 {
   directory = Fexpand_file_name (directory, Qnil);
 
@@ -342,11 +369,11 @@
   Lisp_Object handler
     = Ffind_file_name_handler (directory, Qdirectory_files_and_attributes);
   if (!NILP (handler))
-    return call6 (handler, Qdirectory_files_and_attributes,
-                  directory, full, match, nosort, id_format);
+    return call7 (handler, Qdirectory_files_and_attributes,
+                  directory, full, match, nosort, id_format, count);
 
   return directory_files_internal (directory, full, match, nosort,
-				   true, id_format);
+				   true, id_format, count);
 }
 
 \f

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi Arthur,

> Attached is patch which return nil early.

Thanks for this. I haven't seen your name on the FSF file; I guess your
legal papers are on the way?

In the meantime, we could prepare the other changes. These are

- Adapt the description of directory-files and
  directory-files-and-attributes in doc/lispref/files.texi.

- Add an entry to etc/NEWS.

- Adapt the file name handlers. These are ange-ftp-directory-files,
  ange-ftp-directory-files-and-attributes, tramp-handle-directory-files,
  tramp-handle-directory-files-and-attributes,
  tramp-adb-handle-directory-files-and-attributes,
  tramp-crypt-handle-directory-files,
  tramp-rclone-handle-directory-files,
  tramp-sh-handle-directory-files-and-attributes,
  tramp-smb-handle-directory-files.

- Adapt resp extend ERT tests. These are tramp-test16-directory-files,
  tramp-test19-directory-files-and-attributes,
  tramp-archive-test16-directory-files and
  tramp-archive-test19-directory-files-and-attributes.

- Optionally (but much appreciated), write new ERT tests for
  directory-files and directory-files-and-attributes.

And here are some minor comments about your patch:

> +      if (FIXNATP(return_count))
> +        {
> +          if (ind == last)
> +            break;
> +          ind ++;
> +        }
> +

Wouldn't it be more efficient to test

      if (!last && ind == last)
	break;
      ind ++;

> +DEFUN ("directory-files", Fdirectory_files, Sdirectory_files, 1, 5, 0,
>         doc: /* Return a list of names of files in DIRECTORY.
>  There are three optional arguments:
>  If FULL is non-nil, return absolute file names.  Otherwise return names
> @@ -296,9 +312,14 @@
>  If MATCH is non-nil, mention only file names that match the regexp MATCH.
>  If NOSORT is non-nil, the list is not sorted--its order is unpredictable.
>   Otherwise, the list returned is sorted with `string-lessp'.
> - NOSORT is useful if you plan to sort the result yourself.  */)
> + NOSORT is useful if you plan to sort the result yourself.
> +If COUNT is  non-nil, the function will return max of COUNT and length
> + files, where length is number of files in the directory. Order
> + in which files are returned is not guaranteed and is file system and
> + OS dependent. COUNT has to be an integral number in interval
> + [1,COUNT]. If 0 files are requested the function will return nil. */)

I know what you mean, but I had to read it twice in order to
understand. Why not simply

If COUNT is a natural number, the function will return up to COUNT files.

>  DEFUN ("directory-files-and-attributes", Fdirectory_files_and_attributes,
> -       Sdirectory_files_and_attributes, 1, 5, 0,
> -       doc: /* Return a list of names of files and their attributes in DIRECTORY.
> +       Sdirectory_files_and_attributes, 1, 6, 0, doc
> +       : /* Return a list of names of files and their attributes in DIRECTORY.
>  Value is a list of the form:
>
> -  ((FILE1 . FILE1-ATTRS) (FILE2 . FILE2-ATTRS) ...)
> +((FILE1 . FILE1-ATTRS) (FILE2 . FILE2-ATTRS) ...)

Why do you remove the indentation? It is intentional, I believe.

>  This function accepts four optional arguments:
>  If FULL is non-nil, return absolute file names.  Otherwise return names
> - that are relative to the specified directory.
> +that are relative to the specified directory.
>  If MATCH is non-nil, mention only file names that match the regexp MATCH.
>  If NOSORT is non-nil, the list is not sorted--its order is unpredictable.
> - NOSORT is useful if you plan to sort the result yourself.
> +NOSORT is useful if you plan to sort the result yourself.

Same here.

> +If COUNT is  non-nil, the function will return max of COUNT and length
> + files, where length is number of files in the directory. Order
> + in which files are returned is not guaranteed and is file system and
> + OS dependent. COUNT has to be an integral number in interval
> + [1,COUNT]. If 0 files are requested the function will return nil. */)

See above.

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Arthur Miller]

Michael Albinus <michael.albinus@gmx.de> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
> Hi Arthur,
>
>> Attached is patch which return nil early.
>
> Thanks for this. I haven't seen your name on the FSF file; I guess your
> legal papers are on the way?
They are not, but they can be if someone send me link. I've been looking
at this page:

https://www.gnu.org/prep/maintain/html_node/Copyright-Papers.html#Copyright-Papers

but I am not really sure where do I send request.

> In the meantime, we could prepare the other changes. These are
>
> - Adapt the description of directory-files and
>   directory-files-and-attributes in doc/lispref/files.texi.
>
> - Add an entry to etc/NEWS.
>
> - Adapt the file name handlers. These are ange-ftp-directory-files,
>   ange-ftp-directory-files-and-attributes, tramp-handle-directory-files,
>   tramp-handle-directory-files-and-attributes,
>   tramp-adb-handle-directory-files-and-attributes,
>   tramp-crypt-handle-directory-files,
>   tramp-rclone-handle-directory-files,
>   tramp-sh-handle-directory-files-and-attributes,
>   tramp-smb-handle-directory-files.

Aha, so it is how it works; didn't have time to look into tramp yet.
Ok, I'll do; I'll sent in patch(es) when I am done. Probably not today
though but during the weekend of beginning of the next week.

So many changes just for one signature :-).
> - Adapt resp extend ERT tests. These are tramp-test16-directory-files,
>   tramp-test19-directory-files-and-attributes,
>   tramp-archive-test16-directory-files and
>   tramp-archive-test19-directory-files-and-attributes.
>
> - Optionally (but much appreciated), write new ERT tests for
>   directory-files and directory-files-and-attributes.

Will have to look at ERT, I suppose it is not too complicated,
opportunity to learn it.

> And here are some minor comments about your patch:
>
>> +      if (FIXNATP(return_count))
>> +        {
>> +          if (ind == last)
>> +            break;
>> +          ind ++;
>> +        }
>> +
>
> Wouldn't it be more efficient to test
>
>       if (!last && ind == last)
> 	break;
>       ind ++;
It will :-)

I mainly try to be clear "what I intent" for some future reader; since
previous code uses lisp.h macros, I used same. I thought that one extra
'if' is not that much of a penalty, I'll gladly shortcircut with !last.

>> +DEFUN ("directory-files", Fdirectory_files, Sdirectory_files, 1, 5, 0,
>>         doc: /* Return a list of names of files in DIRECTORY.
>>  There are three optional arguments:
>>  If FULL is non-nil, return absolute file names.  Otherwise return names
>> @@ -296,9 +312,14 @@
>>  If MATCH is non-nil, mention only file names that match the regexp MATCH.
>>  If NOSORT is non-nil, the list is not sorted--its order is unpredictable.
>>   Otherwise, the list returned is sorted with `string-lessp'.
>> - NOSORT is useful if you plan to sort the result yourself.  */)
>> + NOSORT is useful if you plan to sort the result yourself.
>> +If COUNT is  non-nil, the function will return max of COUNT and length
>> + files, where length is number of files in the directory. Order
>> + in which files are returned is not guaranteed and is file system and
>> + OS dependent. COUNT has to be an integral number in interval
>> + [1,COUNT]. If 0 files are requested the function will return nil. */)
>
> I know what you mean, but I had to read it twice in order to
> understand. Why not simply
>
> If COUNT is a natural number, the function will return up to COUNT files.
I was thinking myself that the last argument took as much doc-space as
comment for the rest of entire function :-). I tried to be as
no-doubt-clear as well as not-so-mathematical, but I definitely think
your version is much nicer. We go with that.

>>  DEFUN ("directory-files-and-attributes", Fdirectory_files_and_attributes,
>> -       Sdirectory_files_and_attributes, 1, 5, 0,
>> -       doc: /* Return a list of names of files and their attributes in DIRECTORY.
>> +       Sdirectory_files_and_attributes, 1, 6, 0, doc
>> +       : /* Return a list of names of files and their attributes in DIRECTORY.
>>  Value is a list of the form:
>>
>> -  ((FILE1 . FILE1-ATTRS) (FILE2 . FILE2-ATTRS) ...)
>> +((FILE1 . FILE1-ATTRS) (FILE2 . FILE2-ATTRS) ...)
>
> Why do you remove the indentation? It is intentional, I believe.
I don't. It is my Emacs. I have by defualt "linux" as identation
style; and changed to "gnu" and even setq default, but Emacs arsed
about identing "linux" style anyway; so I "manually" idented everything
I sent in that changed, and I have just missed those since I didn't
edited those at all.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi Arthur,

>> Thanks for this. I haven't seen your name on the FSF file; I guess your
>> legal papers are on the way?
> They are not, but they can be if someone send me link. I've been looking
> at this page:
>
> https://www.gnu.org/prep/maintain/html_node/Copyright-Papers.html#Copyright-Papers
>
> but I am not really sure where do I send request.

Sorry, I should have asked. I'll send you the template off-list.

>> - Adapt the file name handlers. These are ange-ftp-directory-files,
>>   ange-ftp-directory-files-and-attributes, tramp-handle-directory-files,
>>   tramp-handle-directory-files-and-attributes,
>>   tramp-adb-handle-directory-files-and-attributes,
>>   tramp-crypt-handle-directory-files,
>>   tramp-rclone-handle-directory-files,
>>   tramp-sh-handle-directory-files-and-attributes,
>>   tramp-smb-handle-directory-files.
>
> Aha, so it is how it works; didn't have time to look into tramp yet.
> Ok, I'll do; I'll sent in patch(es) when I am done. Probably not today
> though but during the weekend of beginning of the next week.

There's no rush. Doing the legal work will take a week or so.

> So many changes just for one signature :-).

In order to enjoy you: Tramp is backward compatible back to Emacs
25. This must also be taken into account :-)

But don't worry, I'll make the needed changes after your commit.

>> - Adapt resp extend ERT tests. These are tramp-test16-directory-files,
>>   tramp-test19-directory-files-and-attributes,
>>   tramp-archive-test16-directory-files and
>>   tramp-archive-test19-directory-files-and-attributes.
>>
>> - Optionally (but much appreciated), write new ERT tests for
>>   directory-files and directory-files-and-attributes.
>
> Will have to look at ERT, I suppose it is not too complicated,
> opportunity to learn it.

A look at the ert manual might help. (info "(ert)")

> I mainly try to be clear "what I intent" for some future reader; since
> previous code uses lisp.h macros, I used same. I thought that one extra
> 'if' is not that much of a penalty, I'll gladly shortcircut with !last.

In case you feel the need, you could also provide a comment.

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Arthur Miller]

Michael Albinus <michael.albinus@gmx.de> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
> Hi Arthur,
>
>>> Thanks for this. I haven't seen your name on the FSF file; I guess your
>>> legal papers are on the way?
>> They are not, but they can be if someone send me link. I've been looking
>> at this page:
>>
>> https://www.gnu.org/prep/maintain/html_node/Copyright-Papers.html#Copyright-Papers
>>
>> but I am not really sure where do I send request.
>
> Sorry, I should have asked. I'll send you the template off-list.
>
>>> - Adapt the file name handlers. These are ange-ftp-directory-files,
>>>   ange-ftp-directory-files-and-attributes, tramp-handle-directory-files,
>>>   tramp-handle-directory-files-and-attributes,
>>>   tramp-adb-handle-directory-files-and-attributes,
>>>   tramp-crypt-handle-directory-files,
>>>   tramp-rclone-handle-directory-files,
>>>   tramp-sh-handle-directory-files-and-attributes,
>>>   tramp-smb-handle-directory-files.
>>
>> Aha, so it is how it works; didn't have time to look into tramp yet.
>> Ok, I'll do; I'll sent in patch(es) when I am done. Probably not today
>> though but during the weekend of beginning of the next week.
>
> There's no rush. Doing the legal work will take a week or so.
>
>> So many changes just for one signature :-).
>
> In order to enjoy you: Tramp is backward compatible back to Emacs
> 25. This must also be taken into account :-)
:-) You definitely understand my humour!

> But don't worry, I'll make the needed changes after your commit.
>
>>> - Adapt resp extend ERT tests. These are tramp-test16-directory-files,
>>>   tramp-test19-directory-files-and-attributes,
>>>   tramp-archive-test16-directory-files and
>>>   tramp-archive-test19-directory-files-and-attributes.
>>>
>>> - Optionally (but much appreciated), write new ERT tests for
>>>   directory-files and directory-files-and-attributes.
>>
>> Will have to look at ERT, I suppose it is not too complicated,
>> opportunity to learn it.
>
> A look at the ert manual might help. (info "(ert)")
>
>> I mainly try to be clear "what I intent" for some future reader; since
>> previous code uses lisp.h macros, I used same. I thought that one extra
>> 'if' is not that much of a penalty, I'll gladly shortcircut with !last.
>
> In case you feel the need, you could also provide a comment.
>
> Best regards, Michael.

Ok, I have sent a request for papers; and will deep dive into tramp soon.

Thank you a lot for guiding me, it is indispensible.

Best regards
/a

Re: empty-directory predicate, native implementation

[Stefan Monnier]

> They are not, but they can be if someone send me link. I've been looking
> at this page:

Here it is.  Please fill the form below and then send it as instructed
to the FSF so they can send you the relevant paperwork to sign.
Thank you for your contribution,


        Stefan


Please email the following information to assign@gnu.org, and we
will send you the assignment form for your past and future changes.

Please use your full legal name (in ASCII characters) as the subject
line of the message.
----------------------------------------------------------------------
REQUEST: SEND FORM FOR PAST AND FUTURE CHANGES

[What is the name of the program or package you're contributing to?]
Emacs

[Did you copy any files or text written by someone else in these changes?
Even if that material is free software, we need to know about it.]


[Do you have an employer who might have a basis to claim to own
your changes?  Do you attend a school which might make such a claim?]


[For the copyright registration, what country are you a citizen of?]


[What year were you born?]


[Please write your email address here.]


[Please write your postal address here.]





[Which files have you changed so far, and which new files have you written
so far?]

Re: empty-directory predicate, native implementation

[Arthur Miller]

Michael Albinus <michael.albinus@gmx.de> writes:

>>> - Adapt the file name handlers. These are ange-ftp-directory-files,
>>>   ange-ftp-directory-files-and-attributes, tramp-handle-directory-files,
>>>   tramp-handle-directory-files-and-attributes,
>>>   tramp-adb-handle-directory-files-and-attributes,
>>>   tramp-crypt-handle-directory-files,
>>>   tramp-rclone-handle-directory-files,
>>>   tramp-sh-handle-directory-files-and-attributes,
>>>   tramp-smb-handle-directory-files.

Sorry to bother, just one question: I am looking at ange-ftp, and see no
'nosort' and 'id-format' arguments passed in ange-ftp-directory-files:

(defun ange-ftp-directory-files (directory &optional full match &rest v19-args)

Are those (and count) taken care of as "rest" together with &rest
v19-args or should I specify them as it is done in ange-ftp-directory-and-attributes:

(defun ange-ftp-directory-files-and-attributes
  (directory &optional full match nosort id-format)


Would this work:

(defun ange-ftp-directory-files (directory &optional full match
                                           nosort id-format count
					   &rest v19-args)
  (setq directory (expand-file-name directory))
  (if (ange-ftp-ftp-name directory)
      (progn
	(ange-ftp-barf-if-not-directory directory)
	(let ((tail (ange-ftp-hash-table-keys
		     (ange-ftp-get-files directory)))
	      files f)
	  (setq directory (file-name-as-directory directory))
	  (while tail
	    (setq f (car tail)
		  tail (cdr tail))
	    (if (or (not match) (string-match-p match f))
		(setq files
		      (cons (if full (concat directory f) f) files))))
	  (nreverse files)))
    (apply 'ange-ftp-real-directory-files directory full match nosort
                                           id-format count v19-args))) 

(defun ange-ftp-directory-files-and-attributes
  (directory &optional full match nosort id-format count)
  (setq directory (expand-file-name directory))
  (if (ange-ftp-ftp-name directory)
      (mapcar
       (lambda (file)
	 (cons file (file-attributes (expand-file-name file directory))))
       (ange-ftp-directory-files directory full match nosort))
    (ange-ftp-real-directory-files-and-attributes
     directory full match nosort id-format count)))

Can I remove either &rest v19-version, or nosort through count?

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi Arthur,

> Sorry to bother, just one question: I am looking at ange-ftp, and see no
> 'nosort' and 'id-format' arguments passed in ange-ftp-directory-files:
>
> (defun ange-ftp-directory-files (directory &optional full match &rest v19-args)
>
> Are those (and count) taken care of as "rest" together with &rest
> v19-args or should I specify them as it is done in ange-ftp-directory-and-attributes:
>
> (defun ange-ftp-directory-files-and-attributes
>   (directory &optional full match nosort id-format)

ange-ftp.el hasn't been touched for years, so it would be great if we
could synchronize the signatures. v19-args aren't needed any longer.

> Can I remove either &rest v19-version, or nosort through count?

The former. And yes, nosort, id-format and count shall be implemented.

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Arthur Miller]

Michael Albinus <michael.albinus@gmx.de> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
> Hi Arthur,
>
>> Sorry to bother, just one question: I am looking at ange-ftp, and see no
>> 'nosort' and 'id-format' arguments passed in ange-ftp-directory-files:
>>
>> (defun ange-ftp-directory-files (directory &optional full match &rest v19-args)
>>
>> Are those (and count) taken care of as "rest" together with &rest
>> v19-args or should I specify them as it is done in ange-ftp-directory-and-attributes:
>>
>> (defun ange-ftp-directory-files-and-attributes
>>   (directory &optional full match nosort id-format)
>
> ange-ftp.el hasn't been touched for years, so it would be great if we
> could synchronize the signatures. v19-args aren't needed any longer.
>
>> Can I remove either &rest v19-version, or nosort through count?
>
> The former. And yes, nosort, id-format and count shall be implemented.
>
> Best regards, Michael.
Yes;

Thanks for the help! 

Re: empty-directory predicate, native implementation

[Arthur Miller]

Michael Albinus <michael.albinus@gmx.de> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
> Hi Arthur,
>
>>> Thanks for this. I haven't seen your name on the FSF file; I guess your
>>> legal papers are on the way?
>> They are not, but they can be if someone send me link. I've been looking
>> at this page:
>>
>> https://www.gnu.org/prep/maintain/html_node/Copyright-Papers.html#Copyright-Papers
>>
>> but I am not really sure where do I send request.
>
> Sorry, I should have asked. I'll send you the template off-list.
>
>>> - Adapt the file name handlers. These are ange-ftp-directory-files,
>>>   ange-ftp-directory-files-and-attributes, tramp-handle-directory-files,
>>>   tramp-handle-directory-files-and-attributes,
>>>   tramp-adb-handle-directory-files-and-attributes,
>>>   tramp-crypt-handle-directory-files,
>>>   tramp-rclone-handle-directory-files,
>>>   tramp-sh-handle-directory-files-and-attributes,
>>>   tramp-smb-handle-directory-files.
>>
>> Aha, so it is how it works; didn't have time to look into tramp yet.
>> Ok, I'll do; I'll sent in patch(es) when I am done. Probably not today
>> though but during the weekend of beginning of the next week.
>
> There's no rush. Doing the legal work will take a week or so.
>
>> So many changes just for one signature :-).
>
> In order to enjoy you: Tramp is backward compatible back to Emacs
> 25. This must also be taken into account :-)
>
> But don't worry, I'll make the needed changes after your commit.
>
>>> - Adapt resp extend ERT tests. These are tramp-test16-directory-files,
>>>   tramp-test19-directory-files-and-attributes,
>>>   tramp-archive-test16-directory-files and
>>>   tramp-archive-test19-directory-files-and-attributes.
>>>
>>> - Optionally (but much appreciated), write new ERT tests for
>>>   directory-files and directory-files-and-attributes.
>>
>> Will have to look at ERT, I suppose it is not too complicated,
>> opportunity to learn it.
>
> A look at the ert manual might help. (info "(ert)")
>
>> I mainly try to be clear "what I intent" for some future reader; since
>> previous code uses lisp.h macros, I used same. I thought that one extra
>> 'if' is not that much of a penalty, I'll gladly shortcircut with !last.
>
> In case you feel the need, you could also provide a comment.
>
> Best regards, Michael.
I have patched handlers; which version should I make patches against:
current master or latest stable? How would you like patches; each in own
file?

I will look at ert tests too.

Re: empty-directory predicate, native implementation

[Arthur Miller]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> They are not, but they can be if someone send me link. I've been looking
>> at this page:
>
> Here it is.  Please fill the form below and then send it as instructed
> to the FSF so they can send you the relevant paperwork to sign.
> Thank you for your contribution,
>
>
>         Stefan
>
>
> Please email the following information to assign@gnu.org, and we
> will send you the assignment form for your past and future changes.
>
> Please use your full legal name (in ASCII characters) as the subject
> line of the message.
> ----------------------------------------------------------------------
> REQUEST: SEND FORM FOR PAST AND FUTURE CHANGES
>
> [What is the name of the program or package you're contributing to?]
> Emacs
>
> [Did you copy any files or text written by someone else in these changes?
> Even if that material is free software, we need to know about it.]
>
>
> [Do you have an employer who might have a basis to claim to own
> your changes?  Do you attend a school which might make such a claim?]
>
>
> [For the copyright registration, what country are you a citizen of?]
>
>
> [What year were you born?]
>
>
> [Please write your email address here.]
>
>
> [Please write your postal address here.]
>
>
>
>
>
> [Which files have you changed so far, and which new files have you written
> so far?]
Michael sent me one, so I already sent it; thx anyway.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi Arthur,

> I have patched handlers; which version should I make patches against:
> current master or latest stable? How would you like patches; each in own
> file?

The patches shall be towards master. And I don't care, whether they are
just one file, or several ones :-)

> I will look at ert tests too.

Thanks. Best regards, Michael.

Re: empty-directory predicate, native implementation

[Arthur Miller]

[-- Attachment #1: Type: text/plain, Size: 458 bytes --]

Michael Albinus <michael.albinus@gmx.de> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
> Hi Arthur,
>
>> I have patched handlers; which version should I make patches against:
>> current master or latest stable? How would you like patches; each in own
>> file?
>
> The patches shall be towards master. And I don't care, whether they are
> just one file, or several ones :-)
>
>> I will look at ert tests too.
>
> Thanks. Best regards, Michael.


[-- Attachment #2: directory-files.patch --]
[-- Type: text/x-patch, Size: 22847 bytes --]

diff --git a/doc/lispref/files.texi b/doc/lispref/files.texi
index 3b8b4fb3a9..2f15176e79 100644
--- a/doc/lispref/files.texi
+++ b/doc/lispref/files.texi
@@ -2917,7 +2917,7 @@ Contents of Directories
 the latter case, it can optionally display information about each file,
 depending on the options passed to the @code{ls} command.
 
-@defun directory-files directory &optional full-name match-regexp nosort
+@defun directory-files directory &optional full-name match-regexp nosort count
 This function returns a list of the names of the files in the directory
 @var{directory}.  By default, the list is in alphabetical order.
 
@@ -2937,6 +2937,13 @@ Contents of Directories
 are processed in.  If the order of processing is visible to the user,
 then the user will probably be happier if you do sort the names.
 
+If @var{count} is non-@code{nil}, the function will return first
+@var{count} number of files, or all files, whichever occurs
+first. @var{count} has to be a natural number (> 0). You can use this
+function to short circuit evaluation in case you are just interested to
+find if a directory is empty or not (request one file and tell it to
+ignore dot-files).
+
 @example
 @group
 (directory-files "~lewis")
@@ -2946,6 +2953,14 @@ Contents of Directories
 @end group
 @end example
 
+@example
+@group
+  (null (directory-files directory-name nil
+  "^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*" t 1))
+     @result{} nil
+@end group
+@end example
+
 An error is signaled if @var{directory} is not the name of a directory
 that can be read.
 @end defun
@@ -2996,7 +3011,7 @@ Contents of Directories
 non-@code{nil} if that directory is the one it is looking for.
 @end defun
 
-@defun directory-files-and-attributes directory &optional full-name match-regexp nosort id-format
+@defun directory-files-and-attributes directory &optional full-name match-regexp nosort id-format count
 This is similar to @code{directory-files} in deciding which files
 to report on and how to report their names.  However, instead
 of returning a list of file names, it returns for each file a
diff --git a/etc/NEWS b/etc/NEWS
index 1838b6b38a..25c54d3dfe 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1550,6 +1550,13 @@ ledit.el, lmenu.el, lucid.el and old-whitespace.el.
 \f
 * Lisp Changes in Emacs 28.1
 
++++
+** 'diirectory-files' function can now take an additional count parameter
+This option makes directory-files return COUNT first files in
+directory. If match is given, the function vill return first COUNT files
+that match the expression. The option is useful for checking if
+directory is empty since it will check at most 3 files when COUNT = 1.
+
 +++
 ** 'truncate-string-ellipsis' now uses '…' by default.
 Modes that use 'truncate-string-to-width' with non-nil, non-string
diff --git a/lisp/net/ange-ftp.el b/lisp/net/ange-ftp.el
index 0cb8d7cb83..335a07914c 100644
--- a/lisp/net/ange-ftp.el
+++ b/lisp/net/ange-ftp.el
@@ -3427,8 +3427,8 @@ ange-ftp-file-accessible-directory-p
   (and (file-directory-p name)
        (file-readable-p name)))
 
-(defun ange-ftp-directory-files (directory &optional full match
-					   &rest v19-args)
+(defun ange-ftp-directory-files (directory &optional full match nosort
+					   count)
   (setq directory (expand-file-name directory))
   (if (ange-ftp-ftp-name directory)
       (progn
@@ -3444,18 +3444,19 @@ ange-ftp-directory-files
 		(setq files
 		      (cons (if full (concat directory f) f) files))))
 	  (nreverse files)))
-    (apply 'ange-ftp-real-directory-files directory full match v19-args)))
+    (apply 'ange-ftp-real-directory-files directory full match nosort count)))
 
 (defun ange-ftp-directory-files-and-attributes
-  (directory &optional full match nosort id-format)
+  (directory &optional full match nosort attrs id-format count)
   (setq directory (expand-file-name directory))
   (if (ange-ftp-ftp-name directory)
       (mapcar
        (lambda (file)
 	 (cons file (file-attributes (expand-file-name file directory))))
-       (ange-ftp-directory-files directory full match nosort))
+       (ange-ftp-directory-files directory full match nosort attrs
+                                 id_format count))
     (ange-ftp-real-directory-files-and-attributes
-     directory full match nosort id-format)))
+     directory full match nosort attrs id-format count)))
 
 (defun ange-ftp-file-attributes (file &optional id-format)
   (setq file (expand-file-name file))
diff --git a/lisp/net/tramp-adb.el b/lisp/net/tramp-adb.el
index 49ecaa58ee..26ec24a075 100644
--- a/lisp/net/tramp-adb.el
+++ b/lisp/net/tramp-adb.el
@@ -301,7 +301,7 @@ tramp-do-parse-file-attributes-with-ls
       file-properties)))
 
 (defun tramp-adb-handle-directory-files-and-attributes
-  (directory &optional full match nosort id-format)
+  (directory &optional full match nosort id-format count)
   "Like `directory-files-and-attributes' for Tramp files."
   (unless (file-exists-p directory)
     (tramp-error
@@ -312,7 +312,7 @@ tramp-adb-handle-directory-files-and-attributes
       (copy-tree
        (with-tramp-file-property
 	   v localname (format "directory-files-and-attributes-%s-%s-%s-%s"
-			       full match id-format nosort)
+			       full match id-format nosort count)
 	 (with-current-buffer (tramp-get-buffer v)
 	   (when (tramp-adb-send-command-and-check
 		  v (format "%s -a -l %s"
diff --git a/lisp/net/tramp-crypt.el b/lisp/net/tramp-crypt.el
index 3e96daa7b1..bda3735db4 100644
--- a/lisp/net/tramp-crypt.el
+++ b/lisp/net/tramp-crypt.el
@@ -667,7 +667,10 @@ tramp-crypt-handle-delete-file
     (let (tramp-crypt-enabled)
       (delete-file (tramp-crypt-encrypt-file-name filename) trash))))
 
-(defun tramp-crypt-handle-directory-files (directory &optional full match nosort)
+;; This function does not seem to pass match and nosort into
+;; directory-files at all; is that intentional or bug?
+(defun tramp-crypt-handle-directory-files (directory &optional full
+                                                     match nosort count)
   "Like `directory-files' for Tramp files."
   (unless (file-exists-p directory)
     (tramp-error
diff --git a/lisp/net/tramp-rclone.el b/lisp/net/tramp-rclone.el
index 3701bfc22c..787eead807 100644
--- a/lisp/net/tramp-rclone.el
+++ b/lisp/net/tramp-rclone.el
@@ -300,8 +300,10 @@ tramp-rclone-handle-delete-file
     (tramp-flush-file-properties v localname)
     (tramp-rclone-flush-directory-cache v)))
 
+;; This function did not pass nosort arguemnt into directory-files
+;; not sure if intentional or bug
 (defun tramp-rclone-handle-directory-files
-    (directory &optional full match nosort)
+    (directory &optional full match nosort count)
   "Like `directory-files' for Tramp files."
   (unless (file-exists-p directory)
     (tramp-error
@@ -312,7 +314,8 @@ tramp-rclone-handle-directory-files
     (with-parsed-tramp-file-name directory nil
       (let ((result
 	     (directory-files
-	      (tramp-rclone-local-file-name directory) full match)))
+	      (tramp-rclone-local-file-name directory) full match
+              nosort count)))
 	;; Massage the result.
 	(when full
 	  (let ((local (concat "^" (regexp-quote (tramp-rclone-mount-point v))))
diff --git a/lisp/net/tramp-sh.el b/lisp/net/tramp-sh.el
index 15eab0a4de..7a969979ac 100644
--- a/lisp/net/tramp-sh.el
+++ b/lisp/net/tramp-sh.el
@@ -1701,9 +1701,9 @@ tramp-sh-handle-file-ownership-preserved-p
 		    (tramp-get-remote-gid v 'integer)))))))))
 
 ;; Directory listings.
-
+;; what about perl & sta -> need to fix list to count?
 (defun tramp-sh-handle-directory-files-and-attributes
-  (directory &optional full match nosort id-format)
+  (directory &optional full match nosort id-format count)
   "Like `directory-files-and-attributes' for Tramp files."
   (unless id-format (setq id-format 'integer))
   (unless (file-exists-p directory)
@@ -1743,7 +1743,7 @@ tramp-sh-handle-directory-files-and-attributes
 	    (sort result (lambda (x y) (string< (car x) (car y)))))
 	  ;; The scripts could fail, for example with huge file size.
 	  (tramp-handle-directory-files-and-attributes
-	   directory full match nosort id-format)))))
+	   directory full match nosort id-format count)))))
 
 (defun tramp-do-directory-files-and-attributes-with-perl
   (vec localname &optional id-format)
diff --git a/lisp/net/tramp-smb.el b/lisp/net/tramp-smb.el
index 1b6af2a2e3..62135f514d 100644
--- a/lisp/net/tramp-smb.el
+++ b/lisp/net/tramp-smb.el
@@ -690,29 +690,36 @@ tramp-smb-handle-delete-file
 	   v 'file-error "%s `%s'" (match-string 0) filename))))))
 
 (defun tramp-smb-handle-directory-files
-  (directory &optional full match nosort)
+  (directory &optional full match nosort count)
   "Like `directory-files' for Tramp files."
   (unless (file-exists-p directory)
     (tramp-error
      (tramp-dissect-file-name directory) tramp-file-missing
      "No such file or directory" directory))
-  (let ((result (mapcar #'directory-file-name
-			(file-name-all-completions "" directory))))
-    ;; Discriminate with regexp.
-    (when match
-      (setq result
-	    (delete nil
-		    (mapcar (lambda (x) (when (string-match-p match x) x))
-			    result))))
-    ;; Prepend directory.
-    (when full
-      (setq result
-	    (mapcar
-	     (lambda (x) (format "%s/%s" (directory-file-name directory) x))
-	     result)))
-    ;; Sort them if necessary.
-    (unless nosort (setq result (sort result #'string-lessp)))
-    result))
+  (let ((result nil)
+        (numres 0))
+    (when (or (not count) (> count 0))
+        (setq result (mapcar #'directory-file-name
+			     (file-name-all-completions "" directory)))
+      ;; Discriminate with regexp.
+      (when match
+        (setq result
+	      (delete nil
+		      (mapcar (lambda (x) (when (string-match-p match x) x))
+			      result))))
+
+      ;; return [0,count) number of results
+      (setq result (cl-subseq result 0 count))
+
+      ;; Prepend directory.
+      (when full
+        (setq result
+	      (mapcar
+	       (lambda (x) (format "%s/%s" (directory-file-name directory) x))
+	       result)))
+      ;; Sort them if necessary.
+      (unless nosort (setq result (sort result #'string-lessp)))
+      result)))
 
 (defun tramp-smb-handle-expand-file-name (name &optional dir)
   "Like `expand-file-name' for Tramp files."
diff --git a/lisp/net/tramp.el b/lisp/net/tramp.el
index 6d44ad23ad..a99af70196 100644
--- a/lisp/net/tramp.el
+++ b/lisp/net/tramp.el
@@ -3114,7 +3114,8 @@ tramp-handle-directory-file-name
     (setq directory (substring directory 0 -1)))
   directory)
 
-(defun tramp-handle-directory-files (directory &optional full match nosort)
+(defun tramp-handle-directory-files (directory &optional full match
+                                               nosort count)
   "Like `directory-files' for Tramp files."
   (unless (file-exists-p directory)
     (tramp-error
@@ -3133,13 +3134,13 @@ tramp-handle-directory-files
       (if nosort result (sort result #'string<)))))
 
 (defun tramp-handle-directory-files-and-attributes
-  (directory &optional full match nosort id-format)
+  (directory &optional full match nosort id-format count)
   "Like `directory-files-and-attributes' for Tramp files."
   (mapcar
    (lambda (x)
      (cons x (file-attributes
 	      (if full x (expand-file-name x directory)) id-format)))
-   (directory-files directory full match nosort)))
+   (directory-files directory full match nosort count)))
 
 (defun tramp-handle-dired-uncache (dir)
   "Like `dired-uncache' for Tramp files."
diff --git a/src/dired.c b/src/dired.c
index 1584b6acf0..1fc8dd27fa 100644
--- a/src/dired.c
+++ b/src/dired.c
@@ -17,7 +17,6 @@
 You should have received a copy of the GNU General Public License
 along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.  */
 
-
 #include <config.h>
 
 #include <sys/stat.h>
@@ -165,8 +164,26 @@ read_dirent (DIR *dir, Lisp_Object dirname)
 Lisp_Object
 directory_files_internal (Lisp_Object directory, Lisp_Object full,
 			  Lisp_Object match, Lisp_Object nosort, bool attrs,
-			  Lisp_Object id_format)
+			  Lisp_Object id_format, Lisp_Object return_count)
 {
+  ptrdiff_t ind = 0, last = 0;
+
+  /* check count first for early exit */
+  if (FIXNUMP(return_count))
+    {
+      last = XFIXNUM (return_count);
+      if (last <= 0)
+        return Qnil;
+    }
+
+  if (FIXNATP(return_count))
+    {
+      last = XFIXNAT(return_count);
+
+      if (!last)
+        return Qnil;
+    }
+
   if (!NILP (match))
     CHECK_STRING (match);
 
@@ -267,6 +284,10 @@ directory_files_internal (Lisp_Object directory, Lisp_Object full,
       else
 	finalname = name;
 
+      if (last && ind == last)
+          break;
+      ind ++;
+
       list = Fcons (attrs ? Fcons (finalname, fileattrs) : finalname, list);
     }
 
@@ -287,8 +308,7 @@ directory_files_internal (Lisp_Object directory, Lisp_Object full,
   return list;
 }
 
-
-DEFUN ("directory-files", Fdirectory_files, Sdirectory_files, 1, 4, 0,
+DEFUN ("directory-files", Fdirectory_files, Sdirectory_files, 1, 5, 0,
        doc: /* Return a list of names of files in DIRECTORY.
 There are three optional arguments:
 If FULL is non-nil, return absolute file names.  Otherwise return names
@@ -296,9 +316,12 @@ DEFUN ("directory-files", Fdirectory_files, Sdirectory_files, 1, 4, 0,
 If MATCH is non-nil, mention only file names that match the regexp MATCH.
 If NOSORT is non-nil, the list is not sorted--its order is unpredictable.
  Otherwise, the list returned is sorted with `string-lessp'.
- NOSORT is useful if you plan to sort the result yourself.  */)
+ NOSORT is useful if you plan to sort the result yourself.
+If COUNT is  non-nil, the function will return max of COUNT and length
+ files, where length is number of files in the directory. COUNT has to
+ be a natural number > 0. */)
   (Lisp_Object directory, Lisp_Object full, Lisp_Object match,
-   Lisp_Object nosort)
+   Lisp_Object nosort, Lisp_Object count)
 {
   directory = Fexpand_file_name (directory, Qnil);
 
@@ -306,34 +329,38 @@ DEFUN ("directory-files", Fdirectory_files, Sdirectory_files, 1, 4, 0,
      call the corresponding file name handler.  */
   Lisp_Object handler = Ffind_file_name_handler (directory, Qdirectory_files);
   if (!NILP (handler))
-    return call5 (handler, Qdirectory_files, directory,
-                  full, match, nosort);
+    return call6 (handler, Qdirectory_files, directory,
+                  full, match, nosort, count);
 
-  return directory_files_internal (directory, full, match, nosort, false, Qnil);
+  return directory_files_internal (directory, full, match, nosort,
+                                   false, Qnil, count);
 }
 
 DEFUN ("directory-files-and-attributes", Fdirectory_files_and_attributes,
-       Sdirectory_files_and_attributes, 1, 5, 0,
-       doc: /* Return a list of names of files and their attributes in DIRECTORY.
+       Sdirectory_files_and_attributes, 1, 6, 0, doc
+       : /* Return a list of names of files and their attributes in DIRECTORY.
 Value is a list of the form:
 
-  ((FILE1 . FILE1-ATTRS) (FILE2 . FILE2-ATTRS) ...)
+((FILE1 . FILE1-ATTRS) (FILE2 . FILE2-ATTRS) ...)
 
 where each FILEn-ATTRS is the attributes of FILEn as returned
 by `file-attributes'.
 
 This function accepts four optional arguments:
 If FULL is non-nil, return absolute file names.  Otherwise return names
- that are relative to the specified directory.
+that are relative to the specified directory.
 If MATCH is non-nil, mention only file names that match the regexp MATCH.
 If NOSORT is non-nil, the list is not sorted--its order is unpredictable.
- NOSORT is useful if you plan to sort the result yourself.
+NOSORT is useful if you plan to sort the result yourself.
 ID-FORMAT specifies the preferred format of attributes uid and gid, see
 `file-attributes' for further documentation.
 On MS-Windows, performance depends on `w32-get-true-file-attributes',
-which see.  */)
-  (Lisp_Object directory, Lisp_Object full, Lisp_Object match,
-   Lisp_Object nosort, Lisp_Object id_format)
+which see.
+If COUNT is  non-nil, the function will return max of COUNT and length
+ files, where length is number of files in the directory. COUNT has to
+ be a natural number > 0. */)
+(Lisp_Object directory, Lisp_Object full, Lisp_Object match,
+ Lisp_Object nosort, Lisp_Object id_format, Lisp_Object count)
 {
   directory = Fexpand_file_name (directory, Qnil);
 
@@ -342,11 +369,11 @@ DEFUN ("directory-files-and-attributes", Fdirectory_files_and_attributes,
   Lisp_Object handler
     = Ffind_file_name_handler (directory, Qdirectory_files_and_attributes);
   if (!NILP (handler))
-    return call6 (handler, Qdirectory_files_and_attributes,
-                  directory, full, match, nosort, id_format);
+    return call7 (handler, Qdirectory_files_and_attributes,
+                  directory, full, match, nosort, id_format, count);
 
   return directory_files_internal (directory, full, match, nosort,
-				   true, id_format);
+				   true, id_format, count);
 }
 
 \f
diff --git a/src/lisp.h b/src/lisp.h
index 45353fbef3..a3cfb5044d 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -4612,7 +4612,7 @@ maybe_disable_address_randomization (int argc, char **argv)
 extern void syms_of_dired (void);
 extern Lisp_Object directory_files_internal (Lisp_Object, Lisp_Object,
                                              Lisp_Object, Lisp_Object,
-                                             bool, Lisp_Object);
+                                             bool, Lisp_Object, Lisp_Object);
 
 /* Defined in term.c.  */
 extern int *char_ins_del_vector;
diff --git a/src/sysdep.c b/src/sysdep.c
index f6c0ddee01..9111f1863e 100644
--- a/src/sysdep.c
+++ b/src/sysdep.c
@@ -2892,7 +2892,7 @@ list_system_processes (void)
      process.  */
   procdir = build_string ("/proc");
   match = build_string ("[0-9]+");
-  proclist = directory_files_internal (procdir, Qnil, match, Qt, false, Qnil);
+  proclist = directory_files_internal (procdir, Qnil, match, Qt, false, Qnil, Qnil);
 
   /* `proclist' gives process IDs as strings.  Destructively convert
      each string into a number.  */
diff --git a/test/src/dired-tests.el b/test/src/dired-tests.el
new file mode 100644
index 0000000000..3b739e59cc
--- /dev/null
+++ b/test/src/dired-tests.el
@@ -0,0 +1,92 @@
+;;; dired-tests.el --- Tests for directory-files in dired.c  -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2015-2020 Free Software Foundation, Inc.
+
+;; Author: Arthur Miller <arthur.miller@live.com>
+;; Keywords:
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;;
+
+;;; Code:
+(require 'ert)
+
+(ert-deftest directory-files-tests ()
+  (let ((name (expand-file-name "directory-files-test"
+                                (temporary-file-directory)))
+        ;; nodots expression from dired+ by Drew A.
+        (nodots "^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*"))
+
+    (message name)
+    (when (file-directory-p name)
+      (delete-directory name t))
+    (make-directory name)
+    (when (file-directory-p name)
+      (should (= 2 (length (directory-files name))))
+      (should-not (directory-files name nil nodots t 1))
+      (dolist (file '(a b c d))
+        (make-empty-file (expand-file-name (symbol-name file) name)))
+      (should (= 6 (length (directory-files name))))
+      (should (equal "abcd" (string-join (directory-files name nil
+                                                          nodots) "")))
+      (should (= 2 (length (directory-files name nil "[bc]"))))
+      (should (= 3 (length (directory-files name nil nodots nil 3))))
+      (dolist (file '(5 4 3 2 1))
+        (make-empty-file (expand-file-name (number-to-string file) name)))
+      (should (= 0 (length (directory-files name nil "[0-9]" t -1))))
+      (should (= 5 (length (directory-files name nil "[0-9]" t))))
+      (should (= 5 (length (directory-files name nil "[0-9]" t 50))))
+  )))
+
+(ert-deftest directory-files-and-attributes-tests ()
+  (let ((name (expand-file-name "directory-files-test"
+                                (temporary-file-directory)))
+        ;; nodots expression from dired+ by Drew A.
+        (nodots "^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*"))
+
+    (message name)
+    (when (file-directory-p name)
+      (delete-directory name t))
+    (make-directory name)
+    (when (file-directory-p name)
+      (should (= 2 (length (directory-files name))))
+      (should-not (directory-files-and-attributes name t nodots t 1))
+      (dolist (file '(a b c d))
+        (make-directory (expand-file-name (symbol-name file) name)))
+      (should (= 6 (length (directory-files-and-attributes name))))
+      (dolist (dir (directory-files-and-attributes name t nodots))
+        (should (file-directory-p (car dir)))
+        (should-not (file-regular-p (car dir))))
+      (should (= 2 (length
+                    (directory-files-and-attributes name nil "[bc]"))))
+      (should (= 3 (length
+                    (directory-files-and-attributes name nil nodots
+                                                    nil nil 3))))
+      (dolist (file '(5 4 3 2 1))
+        (make-empty-file (expand-file-name (number-to-string file) name)))
+      (should (= 0 (length
+                    (directory-files-and-attributes name nil
+                                                    "[0-9]" t 1 -1))))
+      (should (= 5 (length
+                    (directory-files-and-attributes name nil "[0-9]" t))))
+      (should (= 5 (length
+                    (directory-files-and-attributes name nil
+                                                    "[0-9]" t nil 50))))
+      )))
+
+(provide 'dired-tests)
+;;; dired-tests.el ends here

[-- Attachment #3: Type: text/plain, Size: 877 bytes --]


See how this works.I have done patch between two branches.

Please check that handlers stuff. I am not sure how to test it, and am
not sure I get those correct; especially tramp-crypt.

I am using a regular expression from Dired+ by Drew in two places. I
have mention it the comment in ert tests, but don't know how to mention
it in the example in manual. Maybe remove example, or maybe it can stay
without creds?

I also discovered that I wasn't covered with FIXNUM all the way; thought
it was unsigned int so -1 would be converted into ffffffff, which
would just yeild all results. Seems like fixnum is not what I thought so I
have added test for <=0 which return nil.

Ert test are mostly foxused on new argument. I haven't found something
related to dired in test/src so I have created dired-tests.el. If it
shoudl be in some other place I can rename it.

Best regards
/a

RE: empty-directory predicate, native implementation

[Drew Adams]

Sorry I'm coming to this late.  It's a long thread,
and I've only checked your patch, not the messages.

I don't understand why we would do this.  Is the
real aim to be able to have an empty-directory
predicate, i.e., to be able to see if there are
no files in a directory?

If so, then why not do just that, witout
bothering (changing the signature of)
`directory-files'?

Is there really some use case for getting only
the first N files in a directory?  I can
understand wanting to test whether a dir is
empty.  I can't see why someone would want the
first N files (however "first" might be defined).

The doc string itself seems to suggest that the
(only?) use case is to test for emptiness:

 You can use this function to short circuit evaluation
 in case you are just interested to find if a directory
 is empty or not (request one file and tell it to
 ignore dot-files).

That need is clear enough.  But what else is
COUNT useful for?  How about having the doc
string also tell us what we can use a COUNT
value > 1 for?  That's the case that really
needs pointing out, not the empty-dir-p case.

And what is the order that defines which file
is first, second, etc.?  Without knowing that,
what use is getting the "first" N?

If this has already been raised, discussed,
and dismissed, fine - apologies for the noise.
If not, what's the raison d'etre for adding
optional parameter COUNT?

Why not just add a predicate `directory-empty-p'?

Re: empty-directory predicate, native implementation

[Arthur Miller]

[-- Attachment #1: Type: text/plain, Size: 3228 bytes --]

Drew Adams <drew.adams@oracle.com> writes:

> Sorry I'm coming to this late.  It's a long thread,
> and I've only checked your patch, not the messages.
>
> I don't understand why we would do this.  Is the
> real aim to be able to have an empty-directory
> predicate, i.e., to be able to see if there are
> no files in a directory?
Yes.
> If so, then why not do just that, witout
> bothering (changing the signature of)
> `directory-files'?
Because it is not possible to that efficiently with current elisp; how
would you do that directly? OS does not have such call, one have to
iterate through directories. At least I am not aware of such call. If
you know, please tell me, I would be happy to use it, at least for myself.

Fastest I am aware of is to get a buffered list with SYS_getdents and then
check for first non-dot file. That changes complexity from linear in
number of files in directory to constant (3 files at max). It works only
on Linux kernel, and I don't know if there is nt version of such call,
or what there is on macs.

I suggested such predicate, but to not re-iterate entire discussion, pls
check my first mail and followed 2 or 3 I think.

> Is there really some use case for getting only
> the first N files in a directory?  I can
> understand wanting to test whether a dir is
> empty.  I can't see why someone would want the
> first N files (however "first" might be defined).
Probably not; it is just a result of having count parameter. 

> The doc string itself seems to suggest that the
> (only?) use case is to test for emptiness:
>
>  You can use this function to short circuit evaluation
>  in case you are just interested to find if a directory
>  is empty or not (request one file and tell it to
>  ignore dot-files).
It does not say only, but it suggests it can be usd in that way. But
yes, that was the driver behind.

Check the attachment; maybe something you would like to have in Dired+?

I don't know if it could become a part of Dired itself, I believe it is
probably too niche and subjective.

> That need is clear enough.  But what else is
> COUNT useful for?  How about having the doc
> string also tell us what we can use a COUNT
> value > 1 for?  That's the case that really
> needs pointing out, not the empty-dir-p case.
>
> And what is the order that defines which file
> is first, second, etc.?  Without knowing that,
> what use is getting the "first" N?
OS.

There is no hard guarantee by OS:s that you will get files in
particular order. I had that sentence in docs, but it was a bit too
verse. I am not very literal of me, if you can phrase it better and more
concise, please go ahead, I would appreciate. Check all docs if you have
time and feel for it, I always need help with the language.

> If this has already been raised, discussed,
> and dismissed, fine - apologies for the noise.
> If not, what's the raison d'etre for adding
> optional parameter COUNT?
I hope you don't mind me snitching your regex for nodots. I was trying
to get one fast myself, and gave up after half hour, would never thought
it was so picky to get that one right.

> Why not just add a predicate `directory-empty-p'?
:-) Check my first mail. I implemented one with that name.

Best regards
/a


[-- Attachment #2: dired-mark-empty-dirs.el --]
[-- Type: text/plain, Size: 961 bytes --]

(require 'dired)

(defvar nodots "^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*")

(defun dired-go-to-first ()
    (interactive)
    (goto-char (point-min))
    (dired-next-line 1)
    (skip-chars-forward " \n\t"))

(defun directory-is-empty-p (directory-name)
  (null (directory-files directory-name nil nodots t 1)))

(defun dired-mark-empty-dirs ()
  (interactive)
  (when (equal major-mode 'dired-mode)
    (let ((curr-dir))
      (save-excursion
        (dired-go-to-first)
        
        (while (not (eobp))
          (setq curr-dir (dired-file-name-at-point))
          (cond ((or (null curr-dir)
                     (string= curr-dir ".")
                     (string= curr-dir ".."))
                     ;; do nothing here
                 )
                ((file-directory-p curr-dir)
                 (when (directory-is-empty-p curr-dir)
                   (dired-mark 1)
                   (dired-previous-line 1))))
          (dired-next-line 1))))))

Re: empty-directory predicate, native implementation

[Arthur Miller]

Drew Adams <drew.adams@oracle.com> writes:

> Sorry I'm coming to this late.  It's a long thread,
> and I've only checked your patch, not the messages.
>
> I don't understand why we would do this.  Is the
> real aim to be able to have an empty-directory
> predicate, i.e., to be able to see if there are
> no files in a directory?
I would just like to add to the previous mail; that this optimisation
really does not matter on very first call.

Since read_files calls readdir, which reads in 32k batch (I think),
system will still fetch quite many inodes. The first call is probably
one that matters most for most application, or in most cases, in Emacs
for Dired's purposes, so there is not much savings to directly to SYS_getdents
directly so I agree with Eli that we don't need to go to Linux specific calls.

The most effect we get on subsequent calls comes probably from
Lisp itsel, by cutting of unnecessary lists traversals and list elements
creation as well as comparing entire directory contents with regex to
eliminate dots with such text. At least what I think currently, maybe I
am wrong about, I am not really expert in systems programming, so I am
not super on all the details.

RE: empty-directory predicate, native implementation

[Drew Adams]

> > Sorry I'm coming to this late.  It's a long thread,
> > and I've only checked your patch, not the messages.
> >
> > I don't understand why we would do this.  Is the
> > real aim to be able to have an empty-directory
> > predicate, i.e., to be able to see if there are
> > no files in a directory?
>
> Yes.
>
> > If so, then why not do just that, without
> > bothering (changing the signature of)
> > `directory-files'?
>
> Because it is not possible to that efficiently with current elisp; how
> would you do that directly? OS does not have such call, one have to
> iterate through directories. At least I am not aware of such call. If
> you know, please tell me, I would be happy to use it, at least for myself.
> 
> Fastest I am aware of is to get a buffered list with SYS_getdents and then
> check for first non-dot file. That changes complexity from linear in
> number of files in directory to constant (3 files at max). It works only
> on Linux kernel, and I don't know if there is nt version of such call,
> or what there is on macs.
> 
> I suggested such predicate, but to not re-iterate entire discussion, pls
> check my first mail and followed 2 or 3 I think.

I took a quick look at the first few messages etc.

I see that Eli suggested not implementing an
empty-directory predicate directly but instead
adding a COUNT parameter to `directory-files' (Lisp).

That's just what I questioned.  If the aim is an
empty-directory pred then why not code it in C, like
we do `directory-files'?  And instead of duplicating
code, if that would be the case, extend the C code
underlying `directory-files' to be usable also for
such a predicate, stopping as soon as it finds one
file.

IOW, what you've done, but not expose the COUNT
parameter to Lisp.  If its real purpose is just to
check for emptiness, then from Lisp I'd think we'd
have a separate predicate for that, and we wouldn't
change the signature of `directory-files'.

UNLESS there really IS some use for a COUNT value > 1.

That was my question.  If there's some use case, in
Lisp, for COUNT, i.e., for returning the "first" N
files, then fine.  But if not, I don't see why we
would add COUNT to `directory-files' (Lisp), as
opposed to just getting the emptiness check from C
(perhaps by adding a COUNT there, but not changing
the Lisp interface for `directory-files').

Not a big deal.  Just a question; just something
that occurred to me when I saw this.  I think a
Lisp user is likely to ask herself the same thing:
apart from passing 1 for COUNT, why would I ever
use COUNT?

And if it were agreed that there's no use case for
COUNT > 1, but for some reason a separate predicate,
`directory-empty-p' isn't wanted, then change COUNT
to a Boolean parameter that just says you want an
emptiness check.

If COUNT > 1 has no utility then, if you have to add
a parameter for Lisp, its name, doc, and behavior
should at least be more indicative of its purpose, no?

Don't get me wrong.  I'm generally in favor of Lisp
Boolean functions and such that return useful non-nil
truth values.  I _really_ like that.  But here I don't
see the usefulness of the non-nil return value, which
means the thing will confuse and mislead sometimes.

> > Is there really some use case for getting only
> > the first N files in a directory?  I can
> > understand wanting to test whether a dir is
> > empty.  I can't see why someone would want the
> > first N files (however "first" might be defined).
>
> Probably not; it is just a result of having count parameter.

Should be the other way round, no?  We should have
a COUNT parameter only if it's useful.  If we're
adding it just for implementation purposes, then why
not just do that under the covers in the C code?

Anyway, I don't mean to belabor this.  I've asked
my question and clarified why I asked.

RE: empty-directory predicate, native implementation

[Drew Adams]

> > Sorry I'm coming to this late.  It's a long thread,
> > and I've only checked your patch, not the messages.
> >
> > I don't understand why we would do this.  Is the
> > real aim to be able to have an empty-directory
> > predicate, i.e., to be able to see if there are
> > no files in a directory?
>
> I would just like to add to the previous mail; that this optimisation
> really does not matter on very first call...

Believe me, I'm not trying to second-guess any
implementation.  I haven't (and won't) look at the
C code, and I'm not very qualified to do so anyway.

I was guessing that COUNT = 1 would be quicker for an
emptiness check than COUNT > 1 (e.g. COUNT = 80000).

But my argument wasn't about implementation or
efficiency.  Even if, for some reason, C had to "get"
(info about) all of the files in the directory, just
to determine whether the dir is empty, my argument
would be the same wrt Lisp: have a separate predicate,
or have a Boolean new parameter, instead of having a
COUNT new parameter.

Re: empty-directory predicate, native implementation

[Arthur Miller]

Drew Adams <drew.adams@oracle.com> writes:

>> > Sorry I'm coming to this late.  It's a long thread,
>> > and I've only checked your patch, not the messages.
>> >
>> > I don't understand why we would do this.  Is the
>> > real aim to be able to have an empty-directory
>> > predicate, i.e., to be able to see if there are
>> > no files in a directory?
>>
>> I would just like to add to the previous mail; that this optimisation
>> really does not matter on very first call...
>
> Believe me, I'm not trying to second-guess any
> implementation.  I haven't (and won't) look at the
> C code, and I'm not very qualified to do so anyway.
>
> I was guessing that COUNT = 1 would be quicker for an
> emptiness check than COUNT > 1 (e.g. COUNT = 80000).
>
> But my argument wasn't about implementation or
> efficiency.  Even if, for some reason, C had to "get"
> (info about) all of the files in the directory, just
> to determine whether the dir is empty, my argument
> would be the same wrt Lisp: have a separate predicate,
> or have a Boolean new parameter, instead of having a
> COUNT new parameter.
I am not sure I follow; but it is a building block to build that
predicate. I agree with you that it is much nice to have such predicate
built-in than if anyone is building their own if it is just one liner.

I have no problems to build that predicate on directory_files_internal
either, it is trivial; and it would remove need to change the function
signature and count confusion, but than instead of changing one
parameter in a signature it would be a new predicate in Lisp
implementation. Wold be nice exercise to me to learn how to write
a tramp handler :-), but I am not sure it is desired.

Don't know, what Michael and Eli have to say about?

Re: empty-directory predicate, native implementation

[Arthur Miller]

Drew Adams <drew.adams@oracle.com> writes:

>> > Sorry I'm coming to this late.  It's a long thread,
>> > and I've only checked your patch, not the messages.
>> >
>> > I don't understand why we would do this.  Is the
>> > real aim to be able to have an empty-directory
>> > predicate, i.e., to be able to see if there are
>> > no files in a directory?
>>
>> Yes.
>>
>> > If so, then why not do just that, without
>> > bothering (changing the signature of)
>> > `directory-files'?
>>
>> Because it is not possible to that efficiently with current elisp; how
>> would you do that directly? OS does not have such call, one have to
>> iterate through directories. At least I am not aware of such call. If
>> you know, please tell me, I would be happy to use it, at least for myself.
>> 
>> Fastest I am aware of is to get a buffered list with SYS_getdents and then
>> check for first non-dot file. That changes complexity from linear in
>> number of files in directory to constant (3 files at max). It works only
>> on Linux kernel, and I don't know if there is nt version of such call,
>> or what there is on macs.
>> 
>> I suggested such predicate, but to not re-iterate entire discussion, pls
>> check my first mail and followed 2 or 3 I think.
>
> I took a quick look at the first few messages etc.
>
> I see that Eli suggested not implementing an
> empty-directory predicate directly but instead
> adding a COUNT parameter to `directory-files' (Lisp).
>
> That's just what I questioned.  If the aim is an
> empty-directory pred then why not code it in C, like
> we do `directory-files'?  And instead of duplicating
> code, if that would be the case, extend the C code
> underlying `directory-files' to be usable also for
> such a predicate, stopping as soon as it finds one
> file.
>
> IOW, what you've done, but not expose the COUNT
> parameter to Lisp.  If its real purpose is just to
> check for emptiness, then from Lisp I'd think we'd
> have a separate predicate for that, and we wouldn't
> change the signature of `directory-files'.
>
> UNLESS there really IS some use for a COUNT value > 1.
>
> That was my question.  If there's some use case, in
> Lisp, for COUNT, i.e., for returning the "first" N
> files, then fine.  But if not, I don't see why we
> would add COUNT to `directory-files' (Lisp), as
> opposed to just getting the emptiness check from C
> (perhaps by adding a COUNT there, but not changing
> the Lisp interface for `directory-files').
>
> Not a big deal.  Just a question; just something
> that occurred to me when I saw this.  I think a
> Lisp user is likely to ask herself the same thing:
> apart from passing 1 for COUNT, why would I ever
> use COUNT?
>
> And if it were agreed that there's no use case for
> COUNT > 1, but for some reason a separate predicate,
> `directory-empty-p' isn't wanted, then change COUNT
> to a Boolean parameter that just says you want an
> emptiness check.
>
> If COUNT > 1 has no utility then, if you have to add
> a parameter for Lisp, its name, doc, and behavior
> should at least be more indicative of its purpose, no?
>
> Don't get me wrong.  I'm generally in favor of Lisp
> Boolean functions and such that return useful non-nil
> truth values.  I _really_ like that.  But here I don't
> see the usefulness of the non-nil return value, which
> means the thing will confuse and mislead sometimes.
>
>> > Is there really some use case for getting only
>> > the first N files in a directory?  I can
>> > understand wanting to test whether a dir is
>> > empty.  I can't see why someone would want the
>> > first N files (however "first" might be defined).
>>
>> Probably not; it is just a result of having count parameter.
>
> Should be the other way round, no?  We should have
> a COUNT parameter only if it's useful.  If we're
> adding it just for implementation purposes, then why
> not just do that under the covers in the C code?
>
> Anyway, I don't mean to belabor this.  I've asked
> my question and clarified why I asked.

Yeah, sure; I completely buy your reasoning. When I think of, one user
case for COUNT is maybe if a user have some kind of limited structure to
fill and just wish to taken N number of files. I don't know if that is
very common in Elisp, it's not directly a system programming lang, so I
think it is an artifical example.

Re: empty-directory predicate, native implementation

[Eli Zaretskii]

> Date: Sat, 17 Oct 2020 13:03:35 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> 
> I don't understand why we would do this.  Is the
> real aim to be able to have an empty-directory
> predicate, i.e., to be able to see if there are
> no files in a directory?

It could be useful to know that a directory has at least N files in
it, and the complexity of stopping a loop after N iterations is the
same as stopping after 1 iteration.

> If so, then why not do just that, witout
> bothering (changing the signature of)
> `directory-files'?

The signature got an additional optional argument, which is backward
compatible.  And it is now trivial to implement the predicate for an
empty directory based on the modified directory-files.

> And what is the order that defines which file
> is first, second, etc.?  Without knowing that,
> what use is getting the "first" N?

As discussed: on some filesystems the order is known in advance.  Not
sure how important that is.

RE: empty-directory predicate, native implementation

[Drew Adams]

> > I don't understand why we would do this.
> 
> It could be useful to know that a directory has at least N files in it

OK, if you say so.  That was my question, though:
why/when would that be useful?  No doubt I lack
imagination.

It's not important - to me, anyway.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi Arthur,

> Please check that handlers stuff. I am not sure how to test it, and am
> not sure I get those correct; especially tramp-crypt.

See comments below.

> I am using a regular expression from Dired+ by Drew in two places. I
> have mention it the comment in ert tests, but don't know how to mention
> it in the example in manual. Maybe remove example, or maybe it can stay
> without creds?

You don't need that regexp from Drew. We have the constant
directory-files-no-dot-files-regexp for that purpose.

> I also discovered that I wasn't covered with FIXNUM all the way; thought
> it was unsigned int so -1 would be converted into ffffffff, which
> would just yeild all results. Seems like fixnum is not what I thought so I
> have added test for <=0 which return nil.

What's wrong using FIXNATP and XFIXNAT everywhere?

> Ert test are mostly foxused on new argument. I haven't found something
> related to dired in test/src so I have created dired-tests.el. If it
> shoudl be in some other place I can rename it.

test/src/dired-tests.el is perfect. Maybe you add a comment, that just
the tests for COUNT are added.

> diff --git a/doc/lispref/files.texi b/doc/lispref/files.texi
> index 3b8b4fb3a9..2f15176e79 100644
> --- a/doc/lispref/files.texi
> +++ b/doc/lispref/files.texi
>
> +If @var{count} is non-@code{nil}, the function will return first
> +@var{count} number of files, or all files, whichever occurs
> +first. @var{count} has to be a natural number (> 0). You can use this
> +function to short circuit evaluation in case you are just interested to
> +find if a directory is empty or not (request one file and tell it to
> +ignore dot-files).

In Emacs documentation, we use two spaces after an end-of-sentence
period. Furthermore, pls say "file name" instead of "file"; this is what
we return.

> +@example
> +@group
> +  (null (directory-files directory-name nil
> +  "^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*" t 1))

Please format properly. It shall be

(null
 (directory-files directory-name
   nil directory-files-no-dot-files-regexp t 1))

> diff --git a/etc/NEWS b/etc/NEWS
> index 1838b6b38a..25c54d3dfe 100644
> --- a/etc/NEWS
> +++ b/etc/NEWS
> +** 'diirectory-files' function can now take an additional count parameter

Pls fix the spelling error. Finish the header line with a period. Use
capital letters COUNT.

> +This option makes directory-files return COUNT first files in

It is not an option, but rather an optional parameter. Quote
directory-files. Use two spaces. Use "file names". Fix "vill".

> The option is useful for checking if
> +directory is empty since it will check at most 3 files when COUNT = 1.

As it stands it is not understandable, without mentioning the respective
regexp. Better, you keep this sentence out.

> diff --git a/lisp/net/ange-ftp.el b/lisp/net/ange-ftp.el
> index 0cb8d7cb83..335a07914c 100644
> --- a/lisp/net/ange-ftp.el
> +++ b/lisp/net/ange-ftp.el
> +(defun ange-ftp-directory-files (directory &optional full match nosort
> +					   count)

Could be in  the same line.

> @@ -3444,18 +3444,19 @@ ange-ftp-directory-files
>  		(setq files
>  		      (cons (if full (concat directory f) f) files))))
>  	  (nreverse files)))
> -    (apply 'ange-ftp-real-directory-files directory full match v19-args)))
> +    (apply 'ange-ftp-real-directory-files directory full match nosort count)))

That is the fallback. But I don't see an implementation of
COUNT. (Should be in the "while tail" loop).

>  (defun ange-ftp-directory-files-and-attributes
> -  (directory &optional full match nosort id-format)
> +  (directory &optional full match nosort attrs id-format count)

What it attrs good for?

> -       (ange-ftp-directory-files directory full match nosort))
> +       (ange-ftp-directory-files directory full match nosort attrs
> +                                 id_format count))

Remove attrs. Use id-format.

>      (ange-ftp-real-directory-files-and-attributes
> -     directory full match nosort id-format)))
> +     directory full match nosort attrs id-format count)))

Remove attrs.

> --- a/lisp/net/tramp-adb.el
> +++ b/lisp/net/tramp-adb.el
> @@ -312,7 +312,7 @@ tramp-adb-handle-directory-files-and-attributes
>        (copy-tree
>         (with-tramp-file-property
>  	   v localname (format "directory-files-and-attributes-%s-%s-%s-%s"
> -			       full match id-format nosort)
> +			       full match id-format nosort count)

If you want to add count to the property name, you must adapt the format
string.

I miss the implementation of count.

> diff --git a/lisp/net/tramp-crypt.el b/lisp/net/tramp-crypt.el
> index 3e96daa7b1..bda3735db4 100644
> --- a/lisp/net/tramp-crypt.el
> +++ b/lisp/net/tramp-crypt.el
> +;; This function does not seem to pass match and nosort into
> +;; directory-files at all; is that intentional or bug?

ATM: yes. Leave it as it is, we could change later on. Add FIXME in the comment.

> diff --git a/lisp/net/tramp-rclone.el b/lisp/net/tramp-rclone.el
> index 3701bfc22c..787eead807 100644
> --- a/lisp/net/tramp-rclone.el
> +++ b/lisp/net/tramp-rclone.el
> +;; This function did not pass nosort arguemnt into directory-files
> +;; not sure if intentional or bug

You can remove the comment, because you did it right.

> diff --git a/lisp/net/tramp-sh.el b/lisp/net/tramp-sh.el
> index 15eab0a4de..7a969979ac 100644
> +;; what about perl & sta -> need to fix list to count?

That's correct. Mark the comment as FIXME.

> diff --git a/lisp/net/tramp-smb.el b/lisp/net/tramp-smb.el
> index 1b6af2a2e3..62135f514d 100644
> --- a/lisp/net/tramp-smb.el
> +++ b/lisp/net/tramp-smb.el
> +  (let ((result nil)
> +        (numres 0))

numres is not used. The rest I need to test later; I'm currently just
dry reading.

> diff --git a/lisp/net/tramp.el b/lisp/net/tramp.el
> index 6d44ad23ad..a99af70196 100644
> --- a/lisp/net/tramp.el
> +++ b/lisp/net/tramp.el
> +(defun tramp-handle-directory-files (directory &optional full match
> +                                               nosort count)

This fits into one line (80 chars). I'm missiong an implementation for count.

> diff --git a/src/dired.c b/src/dired.c
> index 1584b6acf0..1fc8dd27fa 100644
> --- a/src/dired.c
> +++ b/src/dired.c
> @@ -17,7 +17,6 @@
> +  if (FIXNUMP(return_count))
> +    {
> +      last = XFIXNUM (return_count);
> +      if (last <= 0)
> +        return Qnil;
> +    }

This is superfluous. Remove it.

> +If COUNT is  non-nil, the function will return max of COUNT and length
> + files, where length is number of files in the directory. COUNT has to
> + be a natural number > 0. */)

So you haven't changed the docstring?

>  DEFUN ("directory-files-and-attributes", Fdirectory_files_and_attributes,
> -       Sdirectory_files_and_attributes, 1, 5, 0,
> -       doc: /* Return a list of names of files and their attributes in DIRECTORY.
> +       Sdirectory_files_and_attributes, 1, 6, 0, doc
> +       : /* Return a list of names of files and their attributes in DIRECTORY.
>  Value is a list of the form:
>
> -  ((FILE1 . FILE1-ATTRS) (FILE2 . FILE2-ATTRS) ...)
> +((FILE1 . FILE1-ATTRS) (FILE2 . FILE2-ATTRS) ...)

...

Still the indentation doesn't fit with the original.

> diff --git a/test/src/dired-tests.el b/test/src/dired-tests.el
> new file mode 100644
> index 0000000000..3b739e59cc
> --- /dev/null
> +++ b/test/src/dired-tests.el
> +;; Copyright (C) 2015-2020 Free Software Foundation, Inc.

This shall be just 2020.

> +        ;; nodots expression from dired+ by Drew A.
> +        (nodots "^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*"))

Not needed. Use directory-files-no-dot-files-regexp.

> +    (message name)

That's OK for debugging purposes. Finally, the tests shall be silent.

> +    (make-directory name)

Tests shall not kepp garbage. Please delete the directory at the end of
the test, preferred as unwindorm of unwind-protect.

> +(ert-deftest directory-files-and-attributes-tests ()

Same comments apply.

> Best regards

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi,

>> But my argument wasn't about implementation or
>> efficiency.  Even if, for some reason, C had to "get"
>> (info about) all of the files in the directory, just
>> to determine whether the dir is empty, my argument
>> would be the same wrt Lisp: have a separate predicate,
>> or have a Boolean new parameter, instead of having a
>> COUNT new parameter.
>
> I am not sure I follow; but it is a building block to build that
> predicate. I agree with you that it is much nice to have such predicate
> built-in than if anyone is building their own if it is just one liner.
>
> I have no problems to build that predicate on directory_files_internal
> either, it is trivial; and it would remove need to change the function
> signature and count confusion, but than instead of changing one
> parameter in a signature it would be a new predicate in Lisp
> implementation. Wold be nice exercise to me to learn how to write
> a tramp handler :-), but I am not sure it is desired.
>
> Don't know, what Michael and Eli have to say about?

I'm in line with Eli here. I have no glaring example that we need COUNT
greater than one. Just a vague idea, that it could help to implement a
more performant file name completion, especially in the remote
case. That is, if you have a very large directory, you let complete file
names on-the-fly (while typing in the minibuffer) with just the first 50
hits from directory-files, which could be retrieved much faster than,
say, more than 2000 file names (this is what I have in /usr/bin). The
complete file names for completion are retrieved, when the user hits TAB.

Just a vague idea; I haven't checked how to implement.

So I propose we still use the COUNT optional argument. This isn't worse
than an argument NOT-EMPTY, which would let directory-files return
immediately after the first hit has found.

Best regards, Michael.

RE: empty-directory predicate, native implementation

[Drew Adams]

> > I am using a regular expression from Dired+ by Drew in two places. I
> > have mention it the comment in ert tests, but don't know how to mention
> > it in the example in manual. Maybe remove example, or maybe it can stay
> > without creds?
> 
> You don't need that regexp from Drew. We have the constant
> directory-files-no-dot-files-regexp for that purpose.

FWIW -

`directory-files-no-dot-files-regexp' was added to Emacs 23.
Its value then was the same as that of `diredp-re-no-dot':
"^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*".  The value was
changed in Emacs 27, to "[^.]\\|\\.\\.\\.".

For my purposes (Dired) I want the former, not the latter,
so `diredp-re-no-dot' remains the former.  The two behave
quite differently.  

See https://lists.gnu.org/archive/html/emacs-devel/2020-04/msg00764.html

Re: empty-directory predicate, native implementation

[Michael Albinus]

Drew Adams <drew.adams@oracle.com> writes:

Hi Drew,

> `directory-files-no-dot-files-regexp' was added to Emacs 23.
> Its value then was the same as that of `diredp-re-no-dot':
> "^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*".  The value was
> changed in Emacs 27, to "[^.]\\|\\.\\.\\.".
>
> For my purposes (Dired) I want the former, not the latter,
> so `diredp-re-no-dot' remains the former.  The two behave
> quite differently.
>
> See https://lists.gnu.org/archive/html/emacs-devel/2020-04/msg00764.html

FTR, I have also problems to understand how the current value of
directory-files-no-dot-files-regexp works. But I fail to find a case
where it is wrong.

(string-match directory-files-no-dot-files-regexp ".") => nil
(string-match directory-files-no-dot-files-regexp "..") => nil
(string-match directory-files-no-dot-files-regexp ".a") => 1
(string-match directory-files-no-dot-files-regexp "..a") => 2

Could you pls give me an example which shows the problem with that
constant? In case there is, I'll lobby for your request in the given
message :-)

Thanks, and best regards, Michael.

Re: empty-directory predicate, native implementation

[Stefan Monnier]

> For my purposes (Dired) I want the former, not the latter,

Why?

> The two behave quite differently.

Can you give an example?

> See https://lists.gnu.org/archive/html/emacs-devel/2020-04/msg00764.html

I don't see any answer to those two questions in that message either.


        Stefan

RE: empty-directory predicate, native implementation

[Drew Adams]

> > `directory-files-no-dot-files-regexp' was added to Emacs 23.
> > Its value then was the same as that of `diredp-re-no-dot':
> > "^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*".  The value was
> > changed in Emacs 27, to "[^.]\\|\\.\\.\\.".
> >
> > For my purposes (Dired) I want the former, not the latter,
> > so `diredp-re-no-dot' remains the former.  The two behave
> > quite differently.
> >
> > See https://urldefense.com/v3/__https://lists.gnu.org/archive/html/emacs-
> devel/2020-
> 04/msg00764.html__;!!GqivPVa7Brio!PWSpyl3EDfFC2LKEXIP7mdNKcGl6HzDDLwE2SFOWdxS
> ZaQ3phv5AVvVuUb-CN6kG$
> 
> FTR, I have also problems to understand how the current value of
> directory-files-no-dot-files-regexp works. But I fail to find a case
> where it is wrong.
> 
> (string-match directory-files-no-dot-files-regexp ".") => nil
> (string-match directory-files-no-dot-files-regexp "..") => nil
> (string-match directory-files-no-dot-files-regexp ".a") => 1
> (string-match directory-files-no-dot-files-regexp "..a") => 2
> 
> Could you pls give me an example which shows the problem with that
> constant? In case there is, I'll lobby for your request in the given
> message :-)

Dunno.  And perhaps I misspoke in saying they behave quite
differently.  They _can_ behave quite differently, depending
on how they're used.

And frankly I think that the only current Dired+ uses of the
regexp don't depend on the difference, as they all just pass
it to `directory-files' as the MATCH arg.  And in that case
the new regexp is just as usable.

In general, the difference between the two is this, AFAICT:
the old one (which is the one still used by Dired+) matches
the complete file name (the nondirectory part), whereas the
new one matches only enough of it to distinguish the . and
.. cases.

IOW, what's different, AFAICS, is the match data: the match.

So if you use the regexp only with `string-match-p' (which
doesn't care about the match data), or if you use it only
with `directory-files', then there's no real difference in
the effect.  But if you use it for some context where the
matched parts are important, that is, where the match-data
matters, then there's a big difference.

Perhaps in the past I used the regexp also for purposes of
grabbing the matched part(s).  I don't recall.

I didn't complain about Emacs changing the value of the
variable - no lobbying is needed.  What I said was that
"it's not clear to me" why people were claiming that the
new regexp is "more correct" than the old one.  (No one
ever responded to that, explaining in what way the old
one was somehow incorrect.)

Paul's mail responding to my mail in that emacs-devel
thread says, BTW:

  As Drew's comments make evident, the doc string is
  unclear. It should be something like 'Regexp that
  matches part of a nonempty string if the string is
  ^^^^^^^^^^^^
  neither "." nor "..".'

But I couldn't find where in that thread I said that.

Anyway, I've said it now.  The old regexp matches all
chars in the nondir part of the file name.  The new
regexp doesn't.  The match data for the old regexp
gives you the matched name.  But no, that's not
needed for `directory-file-names'.
___

[BTW, neither manual nor doc string for `directory-files'
says what MATCH is matched against, other than "file names".
But apparently it's matched only against the nondirectory
part of file names, even if FULL is non-nil.]

RE: empty-directory predicate, native implementation

[Drew Adams]

> > For my purposes (Dired) I want the former, not the latter,
> 
> Why?
> 
> > The two behave quite differently.
> 
> Can you give an example?

See my reply to Michael.  What they match (match-data)
is different.

But for the Dired use I make of it, it turns out that I
don't care: it in fact makes no difference which I use.

(My use of it is as old as the original `dired-re-no-dot'.)

Re: empty-directory predicate, native implementation

[Stefan Monnier]

> ever responded to that, explaining in what way the old
> one was somehow incorrect.)

One case that's mishandled by the old regexp is the case where
the file name includes an LF char (not matched by `.`).


        Stefan

Re: empty-directory predicate, native implementation

[Arthur Miller]

Michael Albinus <michael.albinus@gmx.de> writes:

> What's wrong using FIXNATP and XFIXNAT everywhere?
If we return nil for 0 count, then it make sense to return nil for
negative count too?

XFIXNAT will turn it negative num into some big int.
I thought first it is fine, but after second thought I don't think it is
best thing to do.

if (FIXNUMP(return_count))
    {
      last = XFIXNUM(return_count);

      if (last < 1)
        return Qnil;
    }

This works better.

Negative bignums end up also as big unsigneds which results with all
file namess returned, but I don't know if it matters. Correct me if I am wrong.

>> --- a/lisp/net/tramp-adb.el
>> +++ b/lisp/net/tramp-adb.el
>> @@ -312,7 +312,7 @@ tramp-adb-handle-directory-files-and-attributes
>>        (copy-tree
>>         (with-tramp-file-property
>>  	   v localname (format "directory-files-and-attributes-%s-%s-%s-%s"
>> -			       full match id-format nosort)
>> +			       full match id-format nosort count)
I did that first, but then decided to skip count in format and forgot
to delete the count from the argument list. I didn't know how format was
used so I didn't want to mess with it; but if it is only for the name,
then it is maybe useful?

>> diff --git a/lisp/net/tramp.el b/lisp/net/tramp.el
>> index 6d44ad23ad..a99af70196 100644
>> --- a/lisp/net/tramp.el
>> +++ b/lisp/net/tramp.el
>> +(defun tramp-handle-directory-files (directory &optional full match
>> +                                               nosort count)
>
> This fits into one line (80 chars).
It is 81 chars so Emacs breaks the line. I can call it num, saves 2 chars.

I have fixed identation and docs; as well as other stuff you pointed
out. I can send patch when I get answers to questions above.

Best regards; sorry for giving you so much work!

/a

Re: empty-directory predicate, native implementation

[Arthur Miller]

Drew Adams <drew.adams@oracle.com> writes:

> FWIW -
>
> `directory-files-no-dot-files-regexp' was added to Emacs 23.
> Its value then was the same as that of `diredp-re-no-dot':
> "^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*".  The value was
> changed in Emacs 27, to "[^.]\\|\\.\\.\\.".
>
> For my purposes (Dired) I want the former, not the latter,
> so `diredp-re-no-dot' remains the former.  The two behave
> quite differently.  
Do you mind if I ask what is the difference? I have tried now both and I
am seemingly getting same results from both. Is there some special case
where `directory-files-no-dot-files-regexp' does not work correct?

Re: empty-directory predicate, native implementation

[Arthur Miller]

Drew Adams <drew.adams@oracle.com> writes:

>> > `directory-files-no-dot-files-regexp' was added to Emacs 23.
>> > Its value then was the same as that of `diredp-re-no-dot':
>> > "^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*".  The value was
>> > changed in Emacs 27, to "[^.]\\|\\.\\.\\.".
>> >
>> > For my purposes (Dired) I want the former, not the latter,
>> > so `diredp-re-no-dot' remains the former.  The two behave
>> > quite differently.
>> >
>> > See https://urldefense.com/v3/__https://lists.gnu.org/archive/html/emacs-
>> devel/2020-
>> 04/msg00764.html__;!!GqivPVa7Brio!PWSpyl3EDfFC2LKEXIP7mdNKcGl6HzDDLwE2SFOWdxS
>> ZaQ3phv5AVvVuUb-CN6kG$
>> 
>> FTR, I have also problems to understand how the current value of
>> directory-files-no-dot-files-regexp works. But I fail to find a case
>> where it is wrong.
>> 
>> (string-match directory-files-no-dot-files-regexp ".") => nil
>> (string-match directory-files-no-dot-files-regexp "..") => nil
>> (string-match directory-files-no-dot-files-regexp ".a") => 1
>> (string-match directory-files-no-dot-files-regexp "..a") => 2
>> 
>> Could you pls give me an example which shows the problem with that
>> constant? In case there is, I'll lobby for your request in the given
>> message :-)
>
> Dunno.  And perhaps I misspoke in saying they behave quite
> differently.  They _can_ behave quite differently, depending
> on how they're used.
>
> And frankly I think that the only current Dired+ uses of the
> regexp don't depend on the difference, as they all just pass
> it to `directory-files' as the MATCH arg.  And in that case
> the new regexp is just as usable.
>
> In general, the difference between the two is this, AFAICT:
> the old one (which is the one still used by Dired+) matches
> the complete file name (the nondirectory part), whereas the
> new one matches only enough of it to distinguish the . and
> .. cases.
>
> IOW, what's different, AFAICS, is the match data: the match.
>
> So if you use the regexp only with `string-match-p' (which
> doesn't care about the match data), or if you use it only
> with `directory-files', then there's no real difference in
> the effect.  But if you use it for some context where the
> matched parts are important, that is, where the match-data
> matters, then there's a big difference.
>
> Perhaps in the past I used the regexp also for purposes of
> grabbing the matched part(s).  I don't recall.
>
> I didn't complain about Emacs changing the value of the
> variable - no lobbying is needed.  What I said was that
> "it's not clear to me" why people were claiming that the
> new regexp is "more correct" than the old one.  (No one
> ever responded to that, explaining in what way the old
> one was somehow incorrect.)
>
> Paul's mail responding to my mail in that emacs-devel
> thread says, BTW:
>
>   As Drew's comments make evident, the doc string is
>   unclear. It should be something like 'Regexp that
>   matches part of a nonempty string if the string is
>   ^^^^^^^^^^^^
>   neither "." nor "..".'
>
> But I couldn't find where in that thread I said that.
>
> Anyway, I've said it now.  The old regexp matches all
> chars in the nondir part of the file name.  The new
> regexp doesn't.  The match data for the old regexp
> gives you the matched name.  But no, that's not
> needed for `directory-file-names'.
> ___
>
> [BTW, neither manual nor doc string for `directory-files'
> says what MATCH is matched against, other than "file names".
> But apparently it's matched only against the nondirectory
> part of file names, even if FULL is non-nil.]

Thanks, that was really informative! Thank you for writing and
explaining all that. I guess for purpose of tests and manual where I
have used your regex, we can live with built-in one.

Please ignore my other mail; I sent it before I red the rest of resonses.

I am obviously to new here, so before I suggest something again :-), can
I ask if you have already considered having a predicate as a filter
function instead of regex? Is it considered as too much work to
implement? 

RE: empty-directory predicate, native implementation

[Drew Adams]

> I guess for purpose of tests and manual where I
> have used your regex, we can live with built-in one.

Yes, most likely.

> Please ignore my other mail; I sent it before I red the rest of resonses.

No problem.

> I am obviously to new here, so before I suggest something again :-), can
> I ask if you have already considered having a predicate as a filter
> function instead of regex? Is it considered as too much work to
> implement?

Is that a question for me?  If so, a predicate where,
in what context?

As I mentioned, the current uses of the regexp, in
Dired+ as in vanilla Dired, are just to pass to
`directory-files' as the MATCH argument.

There are also Dired+ and Dired functions that accept
a predicate as argument.  `diredp-get-files'
`diredp-get-subdirs', and `diredp-files-within', for
example.

What is it that you'd like to do/suggest?  Or are you
asking something about the predicate of this thread?

Re: empty-directory predicate, native implementation

[Arthur Miller]

Drew Adams <drew.adams@oracle.com> writes:

>> I guess for purpose of tests and manual where I
>> have used your regex, we can live with built-in one.
>
> Yes, most likely.
>
>> Please ignore my other mail; I sent it before I red the rest of resonses.
>
> No problem.
>
>> I am obviously to new here, so before I suggest something again :-), can
>> I ask if you have already considered having a predicate as a filter
>> function instead of regex? Is it considered as too much work to
>> implement?
>
> Is that a question for me?
In general; for you and anyone who has been developing this
functionality and have time and will to answer.

>  If so, a predicate where, in what context?
> There are also Dired+ and Dired functions that accept
> a predicate as argument.  `diredp-get-files'
> `diredp-get-subdirs', and `diredp-files-within', for
> example.
>
> What is it that you'd like to do/suggest?
I don't know for sure; so I guess I'll play with it when I have time and
sent in suggestion if I make something. :-). Sorry for the noise.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Drew Adams <drew.adams@oracle.com> writes:

Hi Drew,

> IOW, what's different, AFAICS, is the match data: the match.
>
> So if you use the regexp only with `string-match-p' (which
> doesn't care about the match data), or if you use it only
> with `directory-files', then there's no real difference in
> the effect.  But if you use it for some context where the
> matched parts are important, that is, where the match-data
> matters, then there's a big difference.

Yes. But to be fair, the docstring of directory-files-no-dot-files-regexp
didn't promise to return any kind of match data. Using it was just using
an undocumented side effect.

> I didn't complain about Emacs changing the value of the
> variable - no lobbying is needed.  What I said was that
> "it's not clear to me" why people were claiming that the
> new regexp is "more correct" than the old one.  (No one
> ever responded to that, explaining in what way the old
> one was somehow incorrect.)

I believe this statement was rather for the different instances of
regexps over the code, all of them claiming to match just "." and
"..". And some of them might have been wrong.

> [BTW, neither manual nor doc string for `directory-files'
> says what MATCH is matched against, other than "file names".
> But apparently it's matched only against the nondirectory
> part of file names, even if FULL is non-nil.]

I've fixed the docstrings of directory-files-no-dot-files-regexp,
directory-files and directory-files-and-attributes.

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

Hi Stefan,

> One case that's mishandled by the old regexp is the case where
> the file name includes an LF char (not matched by `.`).

Well, I regard this rather as a regression. Do we want to handle file
names containing a line break?

>         Stefan

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi Arthur,

>> What's wrong using FIXNATP and XFIXNAT everywhere?
>
> If we return nil for 0 count, then it make sense to return nil for
> negative count too?

*Any* non-nil COUNT, which is not a natural number, is handled as it
were nil. That's what my proposal for the docstring tries to say ("If
COUNT is a natural number ...").

> XFIXNAT will turn it negative num into some big int.
> I thought first it is fine, but after second thought I don't think it is
> best thing to do.
>
> if (FIXNUMP(return_count))
>     {
>       last = XFIXNUM(return_count);
>
>       if (last < 1)
>         return Qnil;
>     }
>
> This works better.

But if you use FIXNATP instead of FIXNUMP, there's no problem at all.

>>> --- a/lisp/net/tramp-adb.el
>>> +++ b/lisp/net/tramp-adb.el
>>> @@ -312,7 +312,7 @@ tramp-adb-handle-directory-files-and-attributes
>>>        (copy-tree
>>>         (with-tramp-file-property
>>>  	   v localname (format "directory-files-and-attributes-%s-%s-%s-%s"
>>> -			       full match id-format nosort)
>>> +			       full match id-format nosort count)
> I did that first, but then decided to skip count in format and forgot
> to delete the count from the argument list. I didn't know how format was
> used so I didn't want to mess with it; but if it is only for the name,
> then it is maybe useful?

Well, the name is used for caches. If we want to use the cache also for
a non-nil COUNT, we must use it also for the cache entry.

>>> diff --git a/lisp/net/tramp.el b/lisp/net/tramp.el
>>> index 6d44ad23ad..a99af70196 100644
>>> --- a/lisp/net/tramp.el
>>> +++ b/lisp/net/tramp.el
>>> +(defun tramp-handle-directory-files (directory &optional full match
>>> +                                               nosort count)
>>
>> This fits into one line (80 chars).
> It is 81 chars so Emacs breaks the line. I can call it num, saves 2 chars.

It's not important. But in my Emacs, the closing parenthesis is at
position 80, IIRC :-)

> Best regards; sorry for giving you so much work!

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Arthur Miller]

Michael Albinus <michael.albinus@gmx.de> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
> Hi Arthur,
>
>>> What's wrong using FIXNATP and XFIXNAT everywhere?
>>
>> If we return nil for 0 count, then it make sense to return nil for
>> negative count too?
>
> *Any* non-nil COUNT, which is not a natural number, is handled as it
> were nil. That's what my proposal for the docstring tries to say ("If
> COUNT is a natural number ...").
Yes, that is how I interpreted it; just wanted to confirm.

>> XFIXNAT will turn it negative num into some big int.
>> I thought first it is fine, but after second thought I don't think it is
>> best thing to do.
>>
>> if (FIXNUMP(return_count))
>>     {
>>       last = XFIXNUM(return_count);
>>
>>       if (last < 1)
>>         return Qnil;
>>     }
>>
>> This works better.
>
> But if you use FIXNATP instead of FIXNUMP, there's no problem at all.
I tryed, but it discards negative numbers - from lisp.h:

INLINE bool
FIXNATP (Lisp_Object x)
{
  return FIXNUMP (x) && 0 <= XFIXNUM (x);
}

This discards negative numbers, which then leaves last = 0, which
results later in all files returned. Either I have to add another test
or use FIXNUM. Or do I missinterpret the code?

With fixnum negative numbers are catched unless they are bignumbers.

FIXNATP uses under the hood FIXNUMP, so check for FIXNUMP is cheaper too.

>>>> --- a/lisp/net/tramp-adb.el
>>>> +++ b/lisp/net/tramp-adb.el
>>>> @@ -312,7 +312,7 @@ tramp-adb-handle-directory-files-and-attributes
>>>>        (copy-tree
>>>>         (with-tramp-file-property
>>>>  	   v localname (format "directory-files-and-attributes-%s-%s-%s-%s"
>>>> -			       full match id-format nosort)
>>>> +			       full match id-format nosort count)
>> I did that first, but then decided to skip count in format and forgot
>> to delete the count from the argument list. I didn't know how format was
>> used so I didn't want to mess with it; but if it is only for the name,
>> then it is maybe useful?
>
> Well, the name is used for caches. If we want to use the cache also for
> a non-nil COUNT, we must use it also for the cache entry.
Yes, I will add the count to format.

>>>> diff --git a/lisp/net/tramp.el b/lisp/net/tramp.el
>>>> index 6d44ad23ad..a99af70196 100644
>>>> --- a/lisp/net/tramp.el
>>>> +++ b/lisp/net/tramp.el
>>>> +(defun tramp-handle-directory-files (directory &optional full match
>>>> +                                               nosort count)
>>>
>>> This fits into one line (80 chars).
>> It is 81 chars so Emacs breaks the line. I can call it num, saves 2 chars.
>
> It's not important. But in my Emacs, the closing parenthesis is at
> position 80, IIRC :-)
Ha :-) Mine was 81. Emacs ways are strange sometimes. Anyway, I changed
to 'num' - 2 chars off! :D

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi Arthur,

>> *Any* non-nil COUNT, which is not a natural number, is handled as it
>> were nil. That's what my proposal for the docstring tries to say ("If
>> COUNT is a natural number ...").
>
> Yes, that is how I interpreted it; just wanted to confirm.

See my other message. Likely we're served better, if we raise an error
for a non-nil COUNT not being a natural number. "Principle of least
surprise" :-)

Best regards, Michael.

RE: empty-directory predicate, native implementation

[Drew Adams]

> > IOW, what's different, AFAICS, is the match data: the match.
> >
> > So if you use the regexp only with `string-match-p' (which
> > doesn't care about the match data), or if you use it only
> > with `directory-files', then there's no real difference in
> > the effect.  But if you use it for some context where the
> > matched parts are important, that is, where the match-data
> > matters, then there's a big difference.
> 
> Yes. But to be fair, the docstring of directory-files-no-dot-files-regexp
> didn't promise to return any kind of match data. Using it was just using
> an undocumented side effect.

Sure.  It's just a regexp.  Its value says what it's good for.

> > I didn't complain about Emacs changing the value of the
> > variable - no lobbying is needed.  What I said was that
> > "it's not clear to me" why people were claiming that the
> > new regexp is "more correct" than the old one.  (No one
> > ever responded to that, explaining in what way the old
> > one was somehow incorrect.)
> 
> I believe this statement was rather for the different instances of
> regexps over the code, all of them claiming to match just "." and
> "..". And some of them might have been wrong.

My reading of that statement was that the new regexp
was somehow more correct than the old one, at least
for the occurrences in the code.  I asked how/where
it was more correct.

But in this current thread Stefan has mentioned that
it is more correct in that it matches also file names
containing newline chars.  So far, that's the only
indication I've seen of how the new is more correct
than the old.

(Aside from whether/how it might be more correct,
the new is no doubt quicker.)

> > [BTW, neither manual nor doc string for `directory-files'
> > says what MATCH is matched against, other than "file names".
> > But apparently it's matched only against the nondirectory
> > part of file names, even if FULL is non-nil.]
> 
> I've fixed the docstrings of directory-files-no-dot-files-regexp,
> directory-files and directory-files-and-attributes.

Cool.  Thanks for that.

Re: empty-directory predicate, native implementation

[Michael Albinus]

Arthur Miller <arthur.miller@live.com> writes:

Hi Arthur,

> See if patch works; I have done pull this morning, so I hope it is all
> up-to-date.

I have pushed your patch to master. While testing, I have pushed another
patch (in my name) for some oddities I have seen, most of them are
rather aesthetic. The major changes are moving directory-empty-p form
dired.el to files.el, and moving all the new tests to
lisp/dired-test.el. And fixing some test errors seen in Tramp. See the
diff in git.

> Best regards; and thanks for the patience and help!

Best regards, Michael.

Re: empty-directory predicate, native implementation

[Arthur Miller]

Michael Albinus <michael.albinus@gmx.de> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
> Hi Arthur,
>
>> See if patch works; I have done pull this morning, so I hope it is all
>> up-to-date.
>
> I have pushed your patch to master. While testing, I have pushed another
> patch (in my name) for some oddities I have seen, most of them are
> rather aesthetic. The major changes are moving directory-empty-p form
> dired.el to files.el, and moving all the new tests to
> lisp/dired-test.el. And fixing some test errors seen in Tramp. See the
> diff in git.

Cool to see it done; thanks. I didn't really know wherte to stash that
directory-empty-p; good you found better place than dired.el.

best regards too
/a