Human-readable file sorting

Human-readable file sorting

[Lars Ingebrigtsen]

Most file browsers these days sort files that have mixed
alphabetic/numerical content in a "logical" way for humans, and not
strictly according to their character values.  I wrote a bit about it,
but forgot to start the discussion here.  :-)

http://lars.ingebrigtsen.no/2014/10/13/campaign-for-humane-sorting/

To summarise:

When you have a directory with files like

file8.png
file9.png
file10.png
file11.png

humans recognise that these have increasing numerical content, and that
they probably should be sorted this way.  However, Emacs will sort them
this way:

file10.png
file11.png
file8.png
file9.png

Most modern file browsers will sort these files the human way, and I
think that Emacs should, too.

This applies to dired, but also to functions like `(directory-files
foo)'.  Possibly via new parameters or something, but I kinda think
that's not necessary.

Of course, we could have endless discussions about "what's a consecutive
sequence of numbers?"  Like file1,004.45.png?  Is that one number or
three?  But I don't think we need to.  :-) It's three.  We get 99% of
the use cases right by just being simple and stupid, I think.  (See the
longer discussion on the blog.)

So what do all y'all think?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Human-readable file sorting

[John Wiegley]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:

> So what do all y'all think?

It looks like GNU ls support "human sorting" using -v, so you can always use
C-u s to add this flag, or change your default set of flags to use it.

Whether it needs to be the dired sorting default... I don't see a strong need
for that, given that it can be customized.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Human-readable file sorting

[Lars Ingebrigtsen]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> So what do all y'all think?
>
> It looks like GNU ls support "human sorting" using -v, so you can always use
> C-u s to add this flag, or change your default set of flags to use it.
>
> Whether it needs to be the dired sorting default... I don't see a strong need
> for that, given that it can be customized.

Well, it's not just dired.  For instance, if you say `C-x f file<TAB>',
you get a help buffer listing all the files that match.  This should
also be sorted in a human readable fashion.  When you push `n' in an
image mode buffer, it takes you to the next file, and that should also
be the next, as humans see it.

This stuff is all over Emacs, and it's either because the functions rely
on the sorting from `directory-files', or because they call
`string-lessp' explicitly.

The practical way to implement this is to provide a `file-string-lessp'
function that can be configured to sort the way the user prefers,
defaulting to human readable sorting, and then we adjust
`directory-files' and other file-ey commands to call that instead of
`string-lessp'.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Human-readable file sorting

[John Wiegley]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:

> The practical way to implement this is to provide a `file-string-lessp'
> function that can be configured to sort the way the user prefers, defaulting
> to human readable sorting, and then we adjust `directory-files' and other
> file-ey commands to call that instead of `string-lessp'.

If directory-files were made extensible, so a sorting method could be
provided... that sounds like a nice extension to me. So, having a
customization variable defaulting to `string-lessp', with an option to switch
to `string-human-lessp'.

Whether the default should human-lessp is another matter, and something I'd
instinctively want to defer until feedback. But having the extensibility
does sound nice.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Human-readable file sorting

[Lars Ingebrigtsen]

John Wiegley <jwiegley@gmail.com> writes:

> If directory-files were made extensible, so a sorting method could be
> provided... that sounds like a nice extension to me. So, having a
> customization variable defaulting to `string-lessp', with an option to switch
> to `string-human-lessp'.

It currently calls `string-lessp' unconditionally, as do all the other
functions that operate on files.  This is a very typical example:

(defun image-mode--images-in-directory (file)
  (let* ((dir (file-name-directory buffer-file-name))
         (files (directory-files dir nil
                                 (image-file-name-regexp) t)))
    ;; Add the current file to the list of images if necessary, in
    ;; case it does not match `image-file-name-regexp'.
    (unless (member file files)
      (push file files))
    (sort files 'string-lessp)))

So I think the easiest way to get all this consistent throughout Emacs
isn't to allow `directory-files' to be customisable, but to instead
introduce a new function `file-string-lessp' (or a better name if you
have it), which would basically look like:

(defun file-string-lessp (s1 s2)
  (pcase file-sorting-method
    (,unicode
     (string-lessp s1 s2))
    (,human
     (human-string-lessp s1 s2)))
    ..)

(Hey!  Did I get the pcase syntax right?  Bonus points!)

Then it's "only a matter" of changing these functions to call
`file-string-lessp', and that would include `directory-files'.

(Perhaps it should be in the C layer for speed?  Does it matter when
sorting files names?  That's probably not the bottleneck...)

> Whether the default should human-lessp is another matter, and something I'd
> instinctively want to defer until feedback.

Sure.

> But having the extensibility does sound nice.

If the proposed method sounds fine, I can start implementing...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Human-readable file sorting

[Marcin Borkowski]

On 2016-02-20, at 07:00, Lars Ingebrigtsen <larsi@gnus.org> wrote:

> John Wiegley <jwiegley@gmail.com> writes:
>
>> If directory-files were made extensible, so a sorting method could be
>> provided... that sounds like a nice extension to me. So, having a
>> customization variable defaulting to `string-lessp', with an option to switch
>> to `string-human-lessp'.
>
> It currently calls `string-lessp' unconditionally, as do all the other
> functions that operate on files.  This is a very typical example:
>
> (defun image-mode--images-in-directory (file)
>   (let* ((dir (file-name-directory buffer-file-name))
>          (files (directory-files dir nil
>                                  (image-file-name-regexp) t)))
>     ;; Add the current file to the list of images if necessary, in
>     ;; case it does not match `image-file-name-regexp'.
>     (unless (member file files)
>       (push file files))
>     (sort files 'string-lessp)))
>
> So I think the easiest way to get all this consistent throughout Emacs
> isn't to allow `directory-files' to be customisable, but to instead
> introduce a new function `file-string-lessp' (or a better name if you
> have it), which would basically look like:
>
> (defun file-string-lessp (s1 s2)
>   (pcase file-sorting-method
>     (,unicode
>      (string-lessp s1 s2))
>     (,human
>      (human-string-lessp s1 s2)))
>     ..)

Wouldn't it be cleaner (and more flexible) to introduce a variable
`file-string-lessp-function'?  Then the user could also account for
1,000.23 and similar cases by themselves when needed.  One could then
even put a special value in .dir-locals.el for e.g. music files (which
sometimes follow strange naming conventions).

BTW, I like this idea.  FWIW, exactly because of this issue I use
trailing zeros when naming files, and do not expect file sorting to be
too smart, but many people don't do this, and renaming files I get from
them is a pain.

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Human-readable file sorting

[Lars Ingebrigtsen]

Marcin Borkowski <mbork@mbork.pl> writes:

> Wouldn't it be cleaner (and more flexible) to introduce a variable
> `file-string-lessp-function'?  Then the user could also account for
> 1,000.23 and similar cases by themselves when needed.  One could then
> even put a special value in .dir-locals.el for e.g. music files (which
> sometimes follow strange naming conventions).

We could allow `file-sorting-method' to be a function in addition to the
special symbols.  I kinda like having predefined symbols for
user-oriented variables, though.  Less chance of users getting
confused.  :-)

> BTW, I like this idea.  FWIW, exactly because of this issue I use
> trailing zeros when naming files, and do not expect file sorting to be
> too smart, but many people don't do this, and renaming files I get from
> them is a pain.

I had a collection of stuff that I had padded with four zeros, and then
I needed another digit.  *sigh*

Computers are really good at doing stuff like sorting, so we should let
them do that, so that humans don't have to do so much work.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Human-readable file sorting

[Marcin Borkowski]

On 2016-02-20, at 08:15, Lars Ingebrigtsen <larsi@gnus.org> wrote:

> Marcin Borkowski <mbork@mbork.pl> writes:
>
>> Wouldn't it be cleaner (and more flexible) to introduce a variable
>> `file-string-lessp-function'?  Then the user could also account for
>> 1,000.23 and similar cases by themselves when needed.  One could then
>> even put a special value in .dir-locals.el for e.g. music files (which
>> sometimes follow strange naming conventions).
>
> We could allow `file-sorting-method' to be a function in addition to the
> special symbols.  I kinda like having predefined symbols for
> user-oriented variables, though.  Less chance of users getting
> confused.  :-)

Good point.

>> BTW, I like this idea.  FWIW, exactly because of this issue I use
>> trailing zeros when naming files, and do not expect file sorting to be
>> too smart, but many people don't do this, and renaming files I get from
>> them is a pain.
>
> I had a collection of stuff that I had padded with four zeros, and then
> I needed another digit.  *sigh*

Ha, once I had to decide on a policy for naming files.  I carefully
estimated how much digits I'd need (three), and then settled for four
just to be safe.  (I've been using that system for some 8 years now, and
still have to reach the 0500 mark, so I don't expect an overflow within
my life-span.)

> Computers are really good at doing stuff like sorting, so we should let
> them do that, so that humans don't have to do so much work.  :-)

And this is a very good point!  I'd be the first to test this feature.

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Human-readable file sorting

[Nikolaus Rath]

On Feb 20 2016, Lars Ingebrigtsen <larsi@gnus.org> wrote:
>> BTW, I like this idea.  FWIW, exactly because of this issue I use
>> trailing zeros when naming files, and do not expect file sorting to be
>> too smart, but many people don't do this, and renaming files I get from
>> them is a pain.
>
> I had a collection of stuff that I had padded with four zeros, and then
> I needed another digit.  *sigh*

$ mmv 'prefix*.jpg' 'prefix0#1.jpg'

That wasn't too painful, was it? :-).


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Human-readable file sorting

[Marcin Borkowski]

On 2016-02-22, at 19:09, Nikolaus Rath <Nikolaus@rath.org> wrote:

> On Feb 20 2016, Lars Ingebrigtsen <larsi@gnus.org> wrote:
>>> BTW, I like this idea.  FWIW, exactly because of this issue I use
>>> trailing zeros when naming files, and do not expect file sorting to be
>>> too smart, but many people don't do this, and renaming files I get from
>>> them is a pain.
>>
>> I had a collection of stuff that I had padded with four zeros, and then
>> I needed another digit.  *sigh*
>
> $ mmv 'prefix*.jpg' 'prefix0#1.jpg'
>
> That wasn't too painful, was it? :-).

What about thumbnails?  Metadata kept in seperate files by image viewing
programs?

That wasn't too painful, but it was too simple. (Still, thanks for
teaching me that mmv command - nice!)

> Best,
> -Nikolaus

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Human-readable file sorting

[Yuri Khan]

On Tue, Feb 23, 2016 at 3:31 AM, Marcin Borkowski <mbork@mbork.pl> wrote:

>> $ mmv 'prefix*.jpg' 'prefix0#1.jpg'
>
> What about thumbnails?  Metadata kept in seperate files by image viewing
> programs?

Those are caches and should be invalidated by the rename.


> That wasn't too painful, but it was too simple. (Still, thanks for
> teaching me that mmv command - nice!)

Perl comes with a handy script rename(1) which performs multi-file
renames by applying a Perl regexp substitution, optionally filtering
by wildcard:

$ rename 's/^prefix(.*?)\.jpg$/prefix0$1.jpg/'


The util-linux package contains another utility, also named rename(1).
Its man page explicitly gives this use case as an example:

       Given the files foo1, ..., foo9, foo10, ..., foo278, the commands

              rename foo foo0 foo?
              rename foo foo0 foo??

       will turn them into foo001, ..., foo009, foo010, ..., foo278.

Re: Human-readable file sorting

[Andreas Schwab]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> Perl comes with a handy script rename(1) which performs multi-file
> renames by applying a Perl regexp substitution, optionally filtering
> by wildcard:

Emacs comes with wdired which applies its editing capabilites to
directories.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: Human-readable file sorting

[Marcin Borkowski]

On 2016-02-23, at 09:32, Andreas Schwab <schwab@suse.de> wrote:

> Yuri Khan <yuri.v.khan@gmail.com> writes:
>
>> Perl comes with a handy script rename(1) which performs multi-file
>> renames by applying a Perl regexp substitution, optionally filtering
>> by wildcard:
>
> Emacs comes with wdired which applies its editing capabilites to
> directories.

And in fact I use Wdired for precisely that.  Wdired+keyboard macros is
such a powerful combination that it makes a nuke seem a child's toy.

> Andreas.

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Human-readable file sorting

[John Wiegley]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:

> (defun file-string-lessp (s1 s2)
>   (pcase file-sorting-method
>     (,unicode
>      (string-lessp s1 s2))
>     (,human
>      (human-string-lessp s1 s2)))
>     ..)

> (Hey!  Did I get the pcase syntax right?  Bonus points!)

Exception that you picked an instance better suited to `cl-case'. :)

>> But having the extensibility does sound nice.

> If the proposed method sounds fine, I can start implementing...

It sounds reasonable to me. So for changing a single invocation of
`directory-files', would I rely on `file-sorting-method' being a dynamic
variable?

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Human-readable file sorting

[Lars Ingebrigtsen]

John Wiegley <jwiegley@gmail.com> writes:

> It sounds reasonable to me. So for changing a single invocation of
> `directory-files', would I rely on `file-sorting-method' being a dynamic
> variable?

Yes, I think so.  But I don't think that's something that somebody would
do very often...  If you want a particular sorting method for one
specific call (as opposed to in the normal work flow), you'd most likely
say something like `(sort (directory-files dir nil nil t) 'my-special-sort)'
or something.  I think.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Human-readable file sorting

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sat, 20 Feb 2016 17:00:01 +1100
> 
> So I think the easiest way to get all this consistent throughout Emacs
> isn't to allow `directory-files' to be customisable, but to instead
> introduce a new function `file-string-lessp' (or a better name if you
> have it)

logical-string-lessp, alphanumeric-string-lessp, human-string-lessp.

> (defun file-string-lessp (s1 s2)
>   (pcase file-sorting-method
>     (,unicode
>      (string-lessp s1 s2))
>     (,human
>      (human-string-lessp s1 s2)))
>     ..)
> 
> (Hey!  Did I get the pcase syntax right?  Bonus points!)

Actually, pcase is not needed here, as you are comparing literal
symbols.  cond should be do the job.

> (Perhaps it should be in the C layer for speed?  Does it matter when
> sorting files names?  That's probably not the bottleneck...)

Measure it before deciding.

Btw, MS-Windows filesystems sort files internally in the order you
want.  E.g., here's what I get for a directory I set up for testing
this:

  (directory-files "~/data/tsort")
  => ("." ".." "file1.png" "file10.png" "file11.png" "file2.png" "file3.png" "file31.png" "file4.png" "file5.png" "file6.png" "file8.png" "file9.png")

Look, ma: no sorting required!

> If the proposed method sounds fine, I can start implementing...

It's certainly a good feature, thanks.

Re: Human-readable file sorting

[martin rudalics]

 > Btw, MS-Windows filesystems sort files internally in the order you
 > want.  E.g., here's what I get for a directory I set up for testing
 > this:
 >
 >    (directory-files "~/data/tsort")
 >    => ("." ".." "file1.png" "file10.png" "file11.png" "file2.png" "file3.png" "file31.png" "file4.png" "file5.png" "file6.png" "file8.png" "file9.png")

I suppose the OP wants "file2.png" listed before "file10.png".

martin

Re: Human-readable file sorting

[Eli Zaretskii]

> Date: Sat, 20 Feb 2016 11:24:43 +0100
> From: martin rudalics <rudalics@gmx.at>
> Cc: emacs-devel@gnu.org
> 
>  > Btw, MS-Windows filesystems sort files internally in the order you
>  > want.  E.g., here's what I get for a directory I set up for testing
>  > this:
>  >
>  >    (directory-files "~/data/tsort")
>  >    => ("." ".." "file1.png" "file10.png" "file11.png" "file2.png" "file3.png" "file31.png" "file4.png" "file5.png" "file6.png" "file8.png" "file9.png")
> 
> I suppose the OP wants "file2.png" listed before "file10.png".

The Explorer already does that, using a special system API, and we
could use it in Emacs as well.

Re: Human-readable file sorting

[Michael Heerdegen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> (defun file-string-lessp (s1 s2)
>   (pcase file-sorting-method
>     (,unicode
>      (string-lessp s1 s2))
>     (,human
>      (human-string-lessp s1 s2)))
>     ..)
>
> (Hey!  Did I get the pcase syntax right?  Bonus points!)

These patterns are invalid.  I think the patterns you want are 'unicode
and 'human.


Michael.

RE: Human-readable file sorting

[Drew Adams]

> > (defun file-string-lessp (s1 s2)
> >   (pcase file-sorting-method
> >     (,unicode
> >      (string-lessp s1 s2))
> >     (,human
> >      (human-string-lessp s1 s2)))
> >     ..)
> >
> > (Hey!  Did I get the pcase syntax right?  Bonus points!)
> 
> These patterns are invalid.  I think the patterns you want are
> 'unicode and 'human.

`eql' tests of the value of a symbol are exactly what Common Lisp
`case' is for (`cl-case' in Elisp).

 (cl-case file-sorting-method
   (unicode    (string-lessp s1 s2))
   (human      (human-string-lessp s1 s2))
   (otherwise  (my-default-string-less-p s1 s2)))

Use `pcase' when you need to do something fancier - that makes the
fancy need clear to a human reader.  If Lars's "..)" requires fancy
stuff then `pcase' can make sense here.  Otherwise, it is overkill.

----

FWIW:

That should be `human-string-less-p', not `human-string-lessp'.

`string-lessp' (and `smie-rule-bolp') are (AFAICT) the only exceptions
(found in the Emacs manuals, at least) to the rule stated in (elisp)
`Coding Conventions' that a predicate name that uses multiple words
should end in `-p' (not just `p'):

  If the purpose of a function is to tell you whether a certain
  condition is true or false, give the function a name that ends in
  'p' (which stands for "predicate").  If the name is one word, add
  just 'p'; if the name is multiple words, add '-p'.  Examples are
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  'framep' and 'frame-live-p'.

Both of the names `string-lessp' and `smie-rule-bolp' could claim to
reuse well known names `lessp' and `bolp', but that claim is lame.
`time-less-p' is correct; `string-lessp' not so much.
 
`string-lessp' was perhaps named before the naming rule was adopted.
`smie-rule-bolp' presumably has no such excuse.  (And `string-lessp'
at least has an alias: `string<'.)

dired and ls (was: Human-readable file sorting)

[Lars Ingebrigtsen]

This all reminds me of something I forgot to prattle on about in that
dired bug report: ls-lisp.el.

dired (on Unixey systems) uses "ls" to get the contents of directories,
and then parses the results.  And generates a buffer.  Not all systems
have "ls", so in that case it uses ls-lisp.el, which just calls
directory-files and then file-attributes in a loop.

I think this is rather suboptimal for many reasons:

1) dired look unnecessarily different on different systems

2) the customisability is limited -- both the look of the buffers and
the sorting is limited by what the native "ls" returns

3) Reasons!  So many reasons!

I've asked "but why" before, but the only response I can remember is "ls
is faster".  Which may be true!  (I may be misremembering this being the
only answer.)  Because `directory-files' uses opendir/readdir first to
get the file names, and then we have to stat each and every file just to
find out whether they're files or directories.

If we instead had `directory-files-with-attributes' in C, we could just
do what "ls" does, which is...  opening the directory, reading it, and
then statting all the files.  (I just straced "ls -l".)  So it should be
just as fast?  I think?  And if you have your dired set up not to show
ownership, date or file permissions, you don't even have to stat
anything on Linux/BSD -- the d_type says whether something is a file or
a directory directly.

Hm...  and there are a lot of functions in Emacs that only wants to know
the file name and whether it's a directory or not...  I mean, basically
every file prompting function only needs that info...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: dired and ls

[John Wiegley]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:

> If we instead had `directory-files-with-attributes' in C, we could just do
> what "ls" does, which is... opening the directory, reading it, and then
> statting all the files. (I just straced "ls -l".) So it should be just as
> fast? I think? And if you have your dired set up not to show ownership, date
> or file permissions, you don't even have to stat anything on Linux/BSD --
> the d_type says whether something is a file or a directory directly.

The /bin/ls process is asynchronous, so a really long "ls -R" listing doesn't
need to block Emacs, does it?

> Hm... and there are a lot of functions in Emacs that only wants to know the
> file name and whether it's a directory or not... I mean, basically every
> file prompting function only needs that info...

Are you also suggesting another improvement (Rubs hands expectantly)?  Or does
this follow from the above, in which case I missed the implication...

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: dired and ls

[Lars Ingebrigtsen]

John Wiegley <jwiegley@gmail.com> writes:

> The /bin/ls process is asynchronous, so a really long "ls -R" listing doesn't
> need to block Emacs, does it?

That's true.  With "ls" the opportunity for async behaviour is greater.
I don't think dired currently is async, but it could be...  

>> Hm... and there are a lot of functions in Emacs that only wants to know the
>> file name and whether it's a directory or not... I mean, basically every
>> file prompting function only needs that info...
>
> Are you also suggesting another improvement (Rubs hands expectantly)?

:-)

> Or does this follow from the above, in which case I missed the
> implication...

I was thinking out loud.  :-)  I'm looking at file_name_completion now,
and the basic loop is:

  for (struct dirent *dp; (dp = read_dirent (d, dirname)); )
    {

[...]

      if (file_name_completion_stat (fd, dp, &st) < 0)
        continue;

      directoryp = S_ISDIR (st.st_mode) != 0;

and that seems to be the only data used from the stat call.  So a
`(directory-files-with-attributes DIR ... FILE-OR-DIR-ONLY-P)' function
seems like it would speed things up a lot if you hit TAB while looking
for a file in a large directory (on Linux and BSD).

The function could have two modes -- one where it only returned the file
name + dirp, and one where it returns full attributes.  The former case
would be just a readdir loop, so it would be very fast, while the second
would need to stat every file, so it would... not.

`dired' would typically need the latter, while file completion would
need the former.

Modulo that I'm reading the code right.  The completion code is terra
incognito to me.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: dired and ls

[John Wiegley]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:

> That's true. With "ls" the opportunity for async behaviour is greater. I
> don't think dired currently is async, but it could be...

See dired-async.el for some attempts.  For example, you can mark multiple
large files for copy, and when you execute the copy it colors them all yellow
(for "in progress"), and removes the coloring as each one is completed.  It
still need a decent interface for canceling operations in progress, though.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: dired and ls

[Thierry Volpiatto]

John Wiegley <jwiegley@gmail.com> writes:

> See dired-async.el for some attempts.  For example, you can mark multiple
> large files for copy, and when you execute the copy it colors them all yellow
> (for "in progress"), and removes the coloring as each one is
> completed.

No, it behaves differently since long time now, you have a message in
mode-line notifying async(s) processes running with the number of jobs,
when a job finish the number of job is updated in mode-line when all
jobs finished you have a message in mode-line.

> It still need a decent interface for canceling operations in progress,
> though.

M-x dired-async-kill-process

-- 
Thierry

Re: dired and ls

[John Wiegley]

>>>>> Thierry Volpiatto <thierry.volpiatto@gmail.com> writes:

> No, it behaves differently since long time now, you have a message in
> mode-line notifying async(s) processes running with the number of jobs, when
> a job finish the number of job is updated in mode-line when all jobs
> finished you have a message in mode-line.
>
> M-x dired-async-kill-process

Hah, the joys of having an active maintainer.  Thanks for all your work,
Thierry. :)

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: dired and ls (was: Human-readable file sorting)

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sat, 20 Feb 2016 17:16:09 +1100
> 
> This all reminds me of something I forgot to prattle on about in that
> dired bug report: ls-lisp.el.
> 
> dired (on Unixey systems) uses "ls" to get the contents of directories,
> and then parses the results.  And generates a buffer.  Not all systems
> have "ls", so in that case it uses ls-lisp.el, which just calls
> directory-files and then file-attributes in a loop.
> 
> I think this is rather suboptimal for many reasons:
> 
> 1) dired look unnecessarily different on different systems
> 
> 2) the customisability is limited -- both the look of the buffers and
> the sorting is limited by what the native "ls" returns
> 
> 3) Reasons!  So many reasons!
> 
> I've asked "but why" before, but the only response I can remember is "ls
> is faster".

There's another reason: ls-lisp implements only a subset of options
supported by 'ls' the program.

> Which may be true!  (I may be misremembering this being the
> only answer.)  Because `directory-files' uses opendir/readdir first to
> get the file names, and then we have to stat each and every file just to
> find out whether they're files or directories.
> 
> If we instead had `directory-files-with-attributes' in C, we could just
> do what "ls" does, which is...  opening the directory, reading it, and
> then statting all the files.

ls-lisp already uses directory-files-with-attributes.

> So it should be just as fast?  I think?

You should measure it, instead of trying to reason about that.  IME,
reasoning about relative timings of programs is never a good idea,
because measurements always produce surprising results.

One reason why ls-lisp might be slower is that it performs quite a bit
of processing after reading the directory, and that processing is
faster in C than in Lisp.  But it could be faster in insignificant
ways.

Re: dired and ls

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> One reason why ls-lisp might be slower is that it performs quite a bit of
> processing after reading the directory, and that processing is faster in C
> than in Lisp. But it could be faster in insignificant ways.

There's also em-ls (eshell's implementation of ls in Lisp). I wonder how it
compares to ls-lisp on this performance point.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: dired and ls

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Cc: Lars Ingebrigtsen <larsi@gnus.org>,  emacs-devel@gnu.org
> Date: Sat, 20 Feb 2016 11:21:37 -0800
> 
> >>>>> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > One reason why ls-lisp might be slower is that it performs quite a bit of
> > processing after reading the directory, and that processing is faster in C
> > than in Lisp. But it could be faster in insignificant ways.
> 
> There's also em-ls (eshell's implementation of ls in Lisp). I wonder how it
> compares to ls-lisp on this performance point.

em-ls is about 6 times slower than ls-lisp.

Re: dired and ls

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> em-ls is about 6 times slower than ls-lisp.

Thanks for checking, Eli.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: dired and ls

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> If we instead had `directory-files-with-attributes' in C, we could just
>> do what "ls" does, which is...  opening the directory, reading it, and
>> then statting all the files.
>
> ls-lisp already uses directory-files-with-attributes.

Aha.  I didn't know about `directory-files-and-attributes'.  And it,
indeed, just reads the directory and then stats all the files...

>> So it should be just as fast?  I think?
>
> You should measure it, instead of trying to reason about that.  IME,
> reasoning about relative timings of programs is never a good idea,
> because measurements always produce surprising results.

Sure.  But it helps talking about stuff before starting to do the real
work...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Human-readable file sorting

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sat, 20 Feb 2016 13:26:51 +1100
> 
> The practical way to implement this is to provide a `file-string-lessp'
> function that can be configured to sort the way the user prefers,
> defaulting to human readable sorting, and then we adjust
> `directory-files' and other file-ey commands to call that instead of
> `string-lessp'.

Please don't call that function `file-string-lessp', or anything that
has "file" in it.  This sorting order has nothing to do with files.
Names I can suggest include "logical", "intuitive", and "numerical".

Btw, ls-lisp by default doesn't use string-lessp, it uses
string-collate-lessp.

Re: Human-readable file sorting

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Please don't call that function `file-string-lessp', or anything that
> has "file" in it.  This sorting order has nothing to do with files.

Well, this is the function that's supposed to sort file names in the
order requested by the user, so that it has "file" in the name makes
sense to me.  `file-name-lessp' is probably better, though.

> Names I can suggest include "logical", "intuitive", and "numerical".

Those are names for sorting predicates that behave in a specific manner,
and not something that directory-files (etc) should call directly.

> Btw, ls-lisp by default doesn't use string-lessp, it uses
> string-collate-lessp.

Hm, interesting...

(sort '("11" "12" "1 1" "1 2" "1.1" "1.2") 'string-collate-lessp)
  => ("11" "1 1" "1.1" "12" "1 2" "1.2")

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Human-readable file sorting

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Sat, 20 Feb 2016 22:32:02 +1100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Please don't call that function `file-string-lessp', or anything that
> > has "file" in it.  This sorting order has nothing to do with files.
> 
> Well, this is the function that's supposed to sort file names in the
> order requested by the user

I think it's supposed to sort strings while treating numeric parts of
them specially.  Its usage domain is wider than just file names.

> > Btw, ls-lisp by default doesn't use string-lessp, it uses
> > string-collate-lessp.
> 
> Hm, interesting...
> 
> (sort '("11" "12" "1 1" "1 2" "1.1" "1.2") 'string-collate-lessp)
>   => ("11" "1 1" "1.1" "12" "1 2" "1.2")

You will see that 'ls' in a UTF-8 locale does the same.

Re: Human-readable file sorting

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> > Please don't call that function `file-string-lessp', or anything that
>> > has "file" in it.  This sorting order has nothing to do with files.
>> 
>> Well, this is the function that's supposed to sort file names in the
>> order requested by the user
>
> I think it's supposed to sort strings while treating numeric parts of
> them specially.  Its usage domain is wider than just file names.

No no.  This is the function that all Emacs things that want to sort
file names should use.  

The definition of that function will be just

(if some-user-setting
    (string-lessp ...)
  (logical-string-lessp ...))

>> Hm, interesting...
>> 
>> (sort '("11" "12" "1 1" "1 2" "1.1" "1.2") 'string-collate-lessp)
>>   => ("11" "1 1" "1.1" "12" "1 2" "1.2")
>
> You will see that 'ls' in a UTF-8 locale does the same.

Huh, I never noticed:

larsi@mouse:/tmp/zot$ echo $LANG
en_US.UTF-8
larsi@mouse:/tmp/zot$ ls 1*
10  11  1.1  12

But:

larsi@mouse:/tmp/zot$ ls foo*
foo12.txt   foo1.txt  foo2.txt

Sorting is hard!

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Human-readable file sorting

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Sat, 20 Feb 2016 22:45:31 +1100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> > Please don't call that function `file-string-lessp', or anything that
> >> > has "file" in it.  This sorting order has nothing to do with files.
> >> 
> >> Well, this is the function that's supposed to sort file names in the
> >> order requested by the user
> >
> > I think it's supposed to sort strings while treating numeric parts of
> > them specially.  Its usage domain is wider than just file names.
> 
> No no.  This is the function that all Emacs things that want to sort
> file names should use.  

It's okay to sort file names using a comparison function that can be
used with strings that are not file names.  But if you call it
"file-SOMETHING", then it will look weird if someone uses it to sort
strings that are not file names.

Re: Human-readable file sorting

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> It's okay to sort file names using a comparison function that can be
> used with strings that are not file names.  But if you call it
> "file-SOMETHING", then it will look weird if someone uses it to sort
> strings that are not file names.

That's true.  But we were talking just about sorting file names, and
allowing the users to control how file name should be sorted.  And
having that customisation apply to non-file functions, too, would be
rather odd.

But if we're instead talking about allowing the user to customise how
any and all strings should be sorted, then a function with "file" in its
name would be inappropriate.

But are we?  :-)  I don't know...  perhaps we are?  Perhaps what we want
to offer the users is "every time Emacs sorts strings, then it should be
done *this* way?"  I have a feeling that there's plenty of opportunity
for breakage.  Which there is for file sorting, too, but perhaps even
more so...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Human-readable file sorting

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > That's true.  But we were talking just about sorting file names, and
  > allowing the users to control how file name should be sorted.  And
  > having that customisation apply to non-file functions, too, would be
  > rather odd.

I think we should have a new comparison function that compares
two strings in this "natural" way.  Perhaps 'string-natural-lessp'.

Then we should have a global variable that specifies which function
to use to compare file names.  Its value could be 'string-lessp' or
'string-natural-lessp' or something else.  Its name could be
'file-name-comparison-function'.

Then we want some user interface to make it easy to choose one of
those two functions as the value of 'file-name-comparison-function'.
-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Human-readable file sorting

[Lars Ingebrigtsen]

Richard Stallman <rms@gnu.org> writes:

> Then we should have a global variable that specifies which function
> to use to compare file names.  Its value could be 'string-lessp' or
> 'string-natural-lessp' or something else.  Its name could be
> 'file-name-comparison-function'.
>
> Then we want some user interface to make it easy to choose one of
> those two functions as the value of 'file-name-comparison-function'.

Yes, I agree.  But Eli pointed out (I think :-)) that this setting may
actually be something that will (or should) affect more than file name
sorting.

I think that may be a good idea, but I'm not sure.  This is a problem
that mostly rears its head when dealing with file names, which very
commonly use embedded numbers.  I can't (off the top of my head)
remember other situations where you get a list of strings that may have
embedded numbers, and you want to have them sorted and displayed?

If somebody had examples where that happens, I think we should go for a
non-file-specific setting.  Otherwise, we should do this setting for
file names only.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Human-readable file sorting

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Then we want some user interface to make it easy to choose one of
  > > those two functions as the value of 'file-name-comparison-function'.

  > Yes, I agree.  But Eli pointed out (I think :-)) that this setting may
  > actually be something that will (or should) affect more than file name
  > sorting.

I don't think we should do that.  I think rather that we should
have separate variables to select a sort method for other things.

I doubt that users will want this "natural" sort method
for anything other than file names.  But if they ask for that,
we can easily give them variables to set to get it.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Human-readable file sorting

[Pierpaolo Bernardi]

On Mon, Feb 22, 2016 at 7:01 PM, Richard Stallman <rms@gnu.org> wrote:

> I doubt that users will want this "natural" sort method
> for anything other than file names.  But if they ask for that,
> we can easily give them variables to set to get it.

In software I write, I use a natural sort for *any* sorted data that
is meant to be presented to a human. I see no reason to do otherwise.
Users are happy.

Re: Human-readable file sorting

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: eliz@gnu.org, emacs-devel@gnu.org
> Date: Mon, 22 Feb 2016 13:01:25 -0500
> 
>   > > Then we want some user interface to make it easy to choose one of
>   > > those two functions as the value of 'file-name-comparison-function'.
> 
>   > Yes, I agree.  But Eli pointed out (I think :-)) that this setting may
>   > actually be something that will (or should) affect more than file name
>   > sorting.
> 
> I don't think we should do that.  I think rather that we should
> have separate variables to select a sort method for other things.

Why do we need a variable?  The relevant functions accept an argument
that is the comparison function, so the callers could simply pass the
function they wanted.

Re: Human-readable file sorting

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > I don't think we should do that.  I think rather that we should
  > > have separate variables to select a sort method for other things.

  > Why do we need a variable?  The relevant functions accept an argument
  > that is the comparison function, so the callers could simply pass the
  > function they wanted.

I think we are talking about two different questions.
You seem to be talking about low-level primitives.  I am talking
about how users customize the behavior of commands.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Human-readable file sorting

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: larsi@gnus.org, emacs-devel@gnu.org
> Date: Tue, 23 Feb 2016 12:43:22 -0500
> 
>   > > I don't think we should do that.  I think rather that we should
>   > > have separate variables to select a sort method for other things.
> 
>   > Why do we need a variable?  The relevant functions accept an argument
>   > that is the comparison function, so the callers could simply pass the
>   > function they wanted.
> 
> I think we are talking about two different questions.
> You seem to be talking about low-level primitives.  I am talking
> about how users customize the behavior of commands.

Fair enough, but which commands were you talking about, specifically?

(Lars did talk about invoking functions from Lisp programs, which is
why I talked about that as well.  But I don't recall commands for
sorting files where this issue would be applicable; the only one that
comes to mind -- Dired -- delegates sorting to 'ls'.)

Re: Human-readable file sorting

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> (Lars did talk about invoking functions from Lisp programs, which is
> why I talked about that as well.  But I don't recall commands for
> sorting files where this issue would be applicable; the only one that
> comes to mind -- Dired -- delegates sorting to 'ls'.)

I proposed making `directory-files' use this user-controlled sorting
function when sorting files.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Human-readable file sorting

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I proposed making `directory-files' use this user-controlled sorting
  > function when sorting files.

Maybe we should do that.  Or we could give it a way to specify the
sort function with an argument (as I think someone else suggested),
and the next higher level up would be able to decide which sort
function to specify.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Human-readable file sorting

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > I think we are talking about two different questions.
  > > You seem to be talking about low-level primitives.  I am talking
  > > about how users customize the behavior of commands.

  > Fair enough, but which commands were you talking about, specifically?

Any command that displays a list of file names.

For instance, listing file name completions.

In principle, list-directory and dired's subroutines ought to do this.
If in fact they outsource the sorting to ls, then maybe this issue
does not really exist -- maybe there are no real Emacs functions
that need to worry about what order for sorting file names.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Human-readable file sorting

[Alexis]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> The definition of that function will be just
>
> (if some-user-setting 
>     (string-lessp ...) 
>   (logical-string-lessp ...))
>
>>> Hm, interesting...
>>> 
>>> (sort '("11" "12" "1 1" "1 2" "1.1" "1.2") 
>>> 'string-collate-lessp) 
>>>   => ("11" "1 1" "1.1" "12" "1 2" "1.2")
>>
>> You will see that 'ls' in a UTF-8 locale does the same.
>
> Huh, I never noticed:
>
> larsi@mouse:/tmp/zot$ echo $LANG en_US.UTF-8 
> larsi@mouse:/tmp/zot$ ls 1* 10  11  1.1  12
>
> But:
>
> larsi@mouse:/tmp/zot$ ls foo* foo12.txt   foo1.txt  foo2.txt
>
> Sorting is hard!

Indeed. :-)

But, having read all the other posts in this discussion, i'm still 
not clear whether `logical-string-lessp` would use 
`string-collate-lessp` "under the hood" (or at least follow the 
value of LC_COLLATE or its closest equivalent)? i ask due to 
things like this:

https://github.com/flexibeast/org-vcard/issues/18#issuecomment-92006062


Alexis.

Re: Human-readable file sorting

[Eli Zaretskii]

> From: Alexis <flexibeast@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> Date: Sun, 21 Feb 2016 01:23:19 +1100
> 
> But, having read all the other posts in this discussion, i'm still 
> not clear whether `logical-string-lessp` would use 
> `string-collate-lessp` "under the hood" (or at least follow the 
> value of LC_COLLATE or its closest equivalent)?

That's up to the person who will implement that, but IMO it's
unlikely: these two use different comparison routines that are not
easy to reconcile into one.

> i ask due to things like this:
> 
> https://github.com/flexibeast/org-vcard/issues/18#issuecomment-92006062

They don't understand how supporting multiple languages makes anything
locale-specific go against the design.

Re: Human-readable file sorting

[Lars Ingebrigtsen]

Alexis <flexibeast@gmail.com> writes:

> But, having read all the other posts in this discussion, i'm still not
> clear whether `logical-string-lessp` would use `string-collate-lessp`
> "under the hood" (or at least follow the value of LC_COLLATE or its
> closest equivalent)? i ask due to things like this:
>
> https://github.com/flexibeast/org-vcard/issues/18#issuecomment-92006062

I didn't know about `string-collate-lessp'.  Looking at the code, it
seems to end up calling wcscoll (gotta love C library function names).
Which sorts according to the LC_COLLATE locale setting...

Hm...  I guess the numerical string sorting function could use that to
sort the textual segments.  Or perhaps there should be one with and one
without?  That is, the built-in sorting arsenal would be `unicode',
`collate', `numerical', and `numerical-and-collate'?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Human-readable file sorting

[Alexis]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I didn't know about `string-collate-lessp'.  Looking at the 
> code, it seems to end up calling wcscoll (gotta love C library 
> function names).  Which sorts according to the LC_COLLATE locale 
> setting...
>
> Hm...  I guess the numerical string sorting function could use 
> that to sort the textual segments.  Or perhaps there should be 
> one with and one without?  That is, the built-in sorting arsenal 
> would be `unicode', `collate', `numerical', and 
> `numerical-and-collate'?

Off the top of my head, that sounds like a good option to me (and 
i assume by 'unicode' you mean "sort by Unicode codepoint"?); but 
perhaps there are a number of possible issues with this approach 
that i'm not aware of ....


Alexis.

Re: Human-readable file sorting

[Lars Ingebrigtsen]

Alexis <flexibeast@gmail.com> writes:

> Off the top of my head, that sounds like a good option to me (and i
> assume by 'unicode' you mean "sort by Unicode codepoint"?);

Yup.  I guess this is what the Emacs documentation currently calls
"lexicographical" sorting, although I'm not sure that's very
informative, either...

> but perhaps there are a number of possible issues with this approach
> that i'm not aware of ....

Perhaps...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Human-readable file sorting

[Yuri Khan]

On Sun, Feb 21, 2016 at 3:30 PM, Alexis <flexibeast@gmail.com> wrote:

> Off the top of my head, that sounds like a good option to me (and i assume
> by 'unicode' you mean "sort by Unicode codepoint"?); but perhaps there are a
> number of possible issues with this approach that i'm not aware of ....

Sorting lexicographically by codepoint has the issue that it is not
consistent with composition equivalence. E.g. U+00E0 “Latin small
letter a with acute” is equivalent to U+0061 U+0301 (“Latin small
letter a” followed by “Combining acute accent”), but codepoint sorting
will sort them far away.

Re: Human-readable file sorting

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Mon, 22 Feb 2016 01:27:23 +0600
> Cc: Lars Ingebrigtsen <larsi@gnus.org>, Eli Zaretskii <eliz@gnu.org>, Emacs developers <emacs-devel@gnu.org>
> 
> Sorting lexicographically by codepoint has the issue that it is not
> consistent with composition equivalence. E.g. U+00E0 “Latin small
> letter a with acute” is equivalent to U+0061 U+0301 (“Latin small
> letter a” followed by “Combining acute accent”), but codepoint sorting
> will sort them far away.

If you need this, use string-collate-lessp in a UTF-8 locale.

Re: Human-readable file sorting

[Pierpaolo Bernardi]

On Sat, Feb 20, 2016 at 9:53 AM, Eli Zaretskii <eliz@gnu.org> wrote:

> Please don't call that function `file-string-lessp', or anything that
> has "file" in it.  This sorting order has nothing to do with files.
> Names I can suggest include "logical", "intuitive", and "numerical".

The established name is "natural sort".  There's no reason to invent a new one?

https://en.wikipedia.org/wiki/Natural_sort_order

Re: Human-readable file sorting

[Eli Zaretskii]

> Date: Sat, 20 Feb 2016 16:35:42 +0100
> From: Pierpaolo Bernardi <olopierpa@gmail.com>
> Cc: Lars Ingebrigtsen <larsi@gnus.org>, Emacs developers <emacs-devel@gnu.org>
> 
> On Sat, Feb 20, 2016 at 9:53 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > Please don't call that function `file-string-lessp', or anything that
> > has "file" in it.  This sorting order has nothing to do with files.
> > Names I can suggest include "logical", "intuitive", and "numerical".
> 
> The established name is "natural sort".  There's no reason to invent a new one?
> 
> https://en.wikipedia.org/wiki/Natural_sort_order

I didn't invent anything.  The names I suggested are established as
well, in various environments.  Of course, "natural" is also fine.

My point was that "file sorting order" is not something we should use
for this feature, as doing so would unnecessarily narrow its intended
application domain.

Re: Human-readable file sorting

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > My point was that "file sorting order" is not something we should use
  > for this feature, as doing so would unnecessarily narrow its intended
  > application domain.

I agree.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

RE: Human-readable file sorting

[Herring, Davis]

> The established name is "natural sort".  There's no reason to invent a new one?
> 
> https://en.wikipedia.org/wiki/Natural_sort_order

gnulib and ls call the idea "version sort": (coreutils)Details about version sort.  (The name "natural sort" seems to suggest a universality that it doesn't have.)

This is not quite the same as the general "treat digit sequences as integers" associated with the name "Natural sort order", and is also slightly different from Emacs' own `version<'.  I fear the confusion from all these almost-equivalent algorithms...

Davis

Re: Human-readable file sorting

[Paul Eggert]

Herring, Davis wrote:
> I fear the confusion from all these almost-equivalent algorithms...

For what it's worth, Gnulib filevercmp is intended to be a replacement for glibc 
strverscmp, and to implement Debian-style version string comparison. It should 
be easy to add this to Emacs if there's a need.

Re: Human-readable file sorting

[Lars Ingebrigtsen]

Pierpaolo Bernardi <olopierpa@gmail.com> writes:

> The established name is "natural sort".  There's no reason to invent a
> new one?

Calling your favourite something "natural" is a great PR trick, because
then you imply that everything that isn't your favourite something is
"unnatural".  I don't think Emacs needs to get on that bandwagon.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Human-readable file sorting

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > The established name is "natural sort".  There's no reason to invent a new one?

  > https://en.wikipedia.org/wiki/Natural_sort_order

We could call it 'string-natural-lessp'.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Human-readable file sorting

[Stefan Monnier]

> The practical way to implement this is to provide a `file-string-lessp'
> function that can be configured to sort the way the user prefers,
> defaulting to human readable sorting, and then we adjust
> `directory-files' and other file-ey commands to call that instead of
> `string-lessp'.

FWIW, mpc.el does use "human sorting" (see mpc-compare-strings).


        Stefan

Re: Human-readable file sorting

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Most modern file browsers will sort these files the human way, and I
  > think that Emacs should, too.

It would be a useful option, but it should be easy to switch to the
low-level sorting order.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.