bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Óscar Fuentes]

The mentioned docstring ends with


     Do _not_ use this function to compare file names for equality, only
     for sorting them.


which makes no sense to me. First of all, what's so special about file
names to mention them here? And then, how one is supposed to use an
equality predicate for sorting? (We have string-collate-lessp for that)



In GNU Emacs 25.1.50.1 (x86_64-pc-linux-gnu, X toolkit)
 of 2016-06-29 built on qcore
Repository revision: 652b638b0f80fda2abc316f3d1b0f005c7d28e1a
Windowing system distributor 'The X.Org Foundation', version 11.0.11803000
System Description:	Ubuntu 16.04 LTS

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Eli Zaretskii]

> From: oscarfv@telefonica.net (Óscar Fuentes)
> Date: Wed, 06 Jul 2016 00:05:57 +0200
> 
> The mentioned docstring ends with
> 
>      Do _not_ use this function to compare file names for equality, only
>      for sorting them.
> 
> which makes no sense to me. First of all, what's so special about file
> names to mention them here?

As with any such notes in doc strings, we warn about popular mistakes
that people make frequently.  Trying to compare file names using this
function is such a mistake.  Any string that is actually a kind of
handle to a system object, for which the system itself uses binary
comparison, will do as an example of such a mistake, but doing that
with file names is a common enough example to warrant mentioning.

> And then, how one is supposed to use an
> equality predicate for sorting? (We have string-collate-lessp for that)

A sorting algorithm can have its own ideas what to do when
string-collate-lessp returns zero.  In any case, the important part
here is not to use this function for file-name comparison.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> As with any such notes in doc strings, we warn about popular mistakes
> that people make frequently.  Trying to compare file names using this
> function is such a mistake.  Any string that is actually a kind of
> handle to a system object, for which the system itself uses binary
> comparison, will do as an example of such a mistake, but doing that
> with file names is a common enough example to warrant mentioning.

I also have trouble to understand.  How can I compare file names modulo
collate-equality, if not with this function?

> > And then, how one is supposed to use an
> > equality predicate for sorting? (We have string-collate-lessp for that)
>
> A sorting algorithm can have its own ideas what to do when
> string-collate-lessp returns zero.

But we are speaking about the docstring of `string-collate-equal-p'.  It
is not clear here how it (as an equivalence predicate) could be useful
for sorting at all.


Michael.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: oscarfv@telefonica.net (Óscar Fuentes),
>   23902@debbugs.gnu.org
> Date: Wed, 06 Jul 2016 17:08:25 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > As with any such notes in doc strings, we warn about popular mistakes
> > that people make frequently.  Trying to compare file names using this
> > function is such a mistake.  Any string that is actually a kind of
> > handle to a system object, for which the system itself uses binary
> > comparison, will do as an example of such a mistake, but doing that
> > with file names is a common enough example to warrant mentioning.
> 
> I also have trouble to understand.  How can I compare file names modulo
> collate-equality, if not with this function?

With string=, of course.  (Or, better yet, with file-equal-p, but
that's a different story.)

> But we are speaking about the docstring of `string-collate-equal-p'.  It
> is not clear here how it (as an equivalence predicate) could be useful
> for sorting at all.

If it is not useful, then the issue against which the note warns
doesn't exist, does it?

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> I also have trouble to understand.  How can I compare file names modulo
>> collate-equality, if not with this function?
>
> With string=, of course.  (Or, better yet, with file-equal-p, but
> that's a different story.)

I understand why comparing file names with string-collate-equalp is a
bad idea (it can return t for cases where the names are different from
the POV of the file system) but then explaining this in full and
pointing to file-equal-p doesn't take too much space (about the same as
the current note, I'll say).

(using string= for comparing file names is a bad idea for the same
reason idea when the FS is case-insensitive, BTW)

>> But we are speaking about the docstring of `string-collate-equal-p'.  It
>> is not clear here how it (as an equivalence predicate) could be useful
>> for sorting at all.
>
> If it is not useful, then the issue against which the note warns
> doesn't exist, does it?

Putting notes about non-existing issues is surely confusing, isn't it?

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Glenn Morris]

FWIW, I also think the doc would be better without this addition.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> > I also have trouble to understand.  How can I compare file names modulo
> > collate-equality, if not with this function?
>
> With string=, of course.  (Or, better yet, with file-equal-p, but
> that's a different story.)

Does that compare mod collate-equality?

> > But we are speaking about the docstring of `string-collate-equal-p'.
> > It is not clear here how it (as an equivalence predicate) could be
> > useful for sorting at all.
>
> If it is not useful, then the issue against which the note warns
> doesn't exist, does it?

Useful for equality testing, or for comparing (and sorting)?  You speak
about the latter in the doc.  I don't understand why the doc of
`string-collate-equal-p' mentions sorting at all.  Is it useful for
that?  (not a rhetorical question but for my understanding of the
current text)


Thanks,

Michael.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Eli Zaretskii]

> From: oscarfv@telefonica.net (Óscar Fuentes)
> Cc: Michael Heerdegen <michael_heerdegen@web.de>,  23902@debbugs.gnu.org
> Date: Wed, 06 Jul 2016 17:46:39 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> I also have trouble to understand.  How can I compare file names modulo
> >> collate-equality, if not with this function?
> >
> > With string=, of course.  (Or, better yet, with file-equal-p, but
> > that's a different story.)
> 
> I understand why comparing file names with string-collate-equalp is a
> bad idea (it can return t for cases where the names are different from
> the POV of the file system) but then explaining this in full and
> pointing to file-equal-p doesn't take too much space (about the same as
> the current note, I'll say).

If you think adding that will make the issue more clear, go ahead and
propose or push a patch.

> (using string= for comparing file names is a bad idea for the same
> reason idea when the FS is case-insensitive, BTW)

Which is why I mentioned file-equal-p.

> Putting notes about non-existing issues is surely confusing, isn't it?

Indeed.  But this issue does exist, and is real.  See, for example:

  http://archives.miloush.net/michkap/archive/2005/10/17/481600.html

(CompareString is what we use on MS-Windows to implement
string-collate-equalp and string-collate-lessp.)

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: oscarfv@telefonica.net (Óscar Fuentes),
>   23902@debbugs.gnu.org
> Date: Wed, 06 Jul 2016 11:48:03 -0400
> 
> 
> FWIW, I also think the doc would be better without this addition.

I obviously disagree.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: oscarfv@telefonica.net,  23902@debbugs.gnu.org
> Date: Wed, 06 Jul 2016 17:55:16 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > > I also have trouble to understand.  How can I compare file names modulo
> > > collate-equality, if not with this function?
> >
> > With string=, of course.  (Or, better yet, with file-equal-p, but
> > that's a different story.)
> 
> Does that compare mod collate-equality?

Sorry, I cannot parse this.

> > > But we are speaking about the docstring of `string-collate-equal-p'.
> > > It is not clear here how it (as an equivalence predicate) could be
> > > useful for sorting at all.
> >
> > If it is not useful, then the issue against which the note warns
> > doesn't exist, does it?
> 
> Useful for equality testing, or for comparing (and sorting)?

Could be for either.  It depends on what the programmer wants to
achieve; we are talking some hypothetical application here.

> I don't understand why the doc of `string-collate-equal-p' mentions
> sorting at all.  Is it useful for that?

<Shrug> It could be used to gather similar "equivalent" file names
together, for example.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> > > With string=, of course.  (Or, better yet, with file-equal-p, but
> > > that's a different story.)
> > 
> > Does that compare mod collate-equality?
>
> Sorry, I cannot parse this.

I can, but I think my question didn't make sense.

> > I don't understand why the doc of `string-collate-equal-p' mentions
> > sorting at all.  Is it useful for that?
>
> <Shrug> It could be used to gather similar "equivalent" file names
> together, for example.

Like this? (doesn't work)

  (sort (list "b" "c" "a" "f" "b" "a" "c") #'string-collate-equalp)

Would you agree to remove "only for sorting them" at the end of the docstring?


Michael.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: oscarfv@telefonica.net,  23902@debbugs.gnu.org
> Date: Wed, 06 Jul 2016 18:44:35 +0200
> 
> Would you agree to remove "only for sorting them" at the end of the docstring?

Yes, if that's the confusing part, feel free to remove.  The important
part is the one that precedes it.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> I understand why comparing file names with string-collate-equalp is a
>> bad idea (it can return t for cases where the names are different from
>> the POV of the file system) but then explaining this in full and
>> pointing to file-equal-p doesn't take too much space (about the same as
>> the current note, I'll say).
>
> If you think adding that will make the issue more clear, go ahead and
> propose or push a patch.

What about replacing the paragraph with this one:

  Note that this function is not adequate for testing if two strings
  name the same file. Use `file-equal-p' for that purpose.

?

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Eli Zaretskii]

> From: oscarfv@telefonica.net (Óscar Fuentes)
> Cc: michael_heerdegen@web.de,  23902@debbugs.gnu.org
> Date: Wed, 06 Jul 2016 19:44:50 +0200
> 
> What about replacing the paragraph with this one:
> 
>   Note that this function is not adequate for testing if two strings
>   name the same file. Use `file-equal-p' for that purpose.
> 
> ?

I'd rather just drop the reference to sorting, and leave everything
else intact.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> Yes, if that's the confusing part, feel free to remove.  The important
> part is the one that precedes it.

Ok, then I'll do this tomorrow.  On which branch, emacs-25 still ok for
this?

Michael.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: oscarfv@telefonica.net,  23902@debbugs.gnu.org
> Date: Thu, 07 Jul 2016 00:49:15 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Yes, if that's the confusing part, feel free to remove.  The important
> > part is the one that precedes it.
> 
> Ok, then I'll do this tomorrow.  On which branch, emacs-25 still ok for
> this?

Yes, documentation changes are okay on emacs-25.

Thanks.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> Yes, documentation changes are okay on emacs-25.

Oh, I see now this function is defined in C.  I'm not sure what I
exactly have to do (the docstring is in a comment?).

Maybe you can make the change this time, Eli?


Thanks,

Michael.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: oscarfv@telefonica.net,  23902@debbugs.gnu.org
> Date: Thu, 07 Jul 2016 23:52:30 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Yes, documentation changes are okay on emacs-25.
> 
> Oh, I see now this function is defined in C.  I'm not sure what I
> exactly have to do (the docstring is in a comment?).

Yes, doc strings in C are comments.  They are collected by
make-docfile and written to etc/DOC at build time.

> Maybe you can make the change this time, Eli?

I did, but you shouldn't be scared like that of making such trivial
changes in the C sources.  There are lots of examples in the *.c files
that you could use to see the simple formatting we use.  The results
are easy to check: just build Emacs and type "C-h f".  And of course,
you can ask questions if you feel uncertain of something.

Thanks.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> I did, but you shouldn't be scared like that of making such trivial
> changes in the C sources.  There are lots of examples in the *.c files
> that you could use to see the simple formatting we use.

Sure, I was just working on several other things, and I thought it would
be better when it's just done.

> [...]  And of course, you can ask questions if you feel uncertain of
> something.

Thanks - Are the spaces before the closing comment delimiter (delimiting
the docstring) significant?  When not, why are they used?


Michael.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: oscarfv@telefonica.net,  23902-done@debbugs.gnu.org
> Date: Fri, 08 Jul 2016 22:59:25 +0200
> 
> Are the spaces before the closing comment delimiter (delimiting
> the docstring) significant?

You mean, before the closing "*/" ?  No, these are not significant.

> When not, why are they used?

To keep C comments in their usual GNU C style, I suppose.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Are the spaces before the closing comment delimiter (delimiting
  > > the docstring) significant?

  > You mean, before the closing "*/" ?  No, these are not significant.

  > > When not, why are they used?

  > To keep C comments in their usual GNU C style, I suppose.

We often put two spaces there so that M-e will see it as the
end of a sentence.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Glenn Morris]

Eli Zaretskii wrote:

> Indeed.  But this issue does exist, and is real.  See, for example:
>
>   http://archives.miloush.net/michkap/archive/2005/10/17/481600.html
>
> (CompareString is what we use on MS-Windows to implement
> string-collate-equalp and string-collate-lessp.)

But no-one using Emacs will reason in this way.
"I want to compare if two files are the same.
I don't know about file-equal-p, so I will just do a string comparison
of the filenames.
I know that the MS Windows function to compare strings is CompareString.
Therefore I will see which Emacs function uses that.
Ah, it is string-collate-equalp.
Therefore I will use string-collate-equalp to compare two file names.
I won't use the more obvious string-equal, nor will I read the
documentation of string-collate-equalp, which makes it obvious that this
is wrong."


And suppose I have two strings, and want to know if they are equal,
respecting my locale's convention about characters that are not
literally identical, but have the same meaning. I should use
string-collate-equalp for this. This is true whether the strings
represent the names of elephants in a zoo, or files on a disk.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: oscarfv@telefonica.net (Óscar Fuentes),
>   michael_heerdegen@web.de,  23902@debbugs.gnu.org
> Date: Sat, 09 Jul 2016 13:06:57 -0400
> 
> But no-one using Emacs will reason in this way.
> "I want to compare if two files are the same.
> I don't know about file-equal-p, so I will just do a string comparison
> of the filenames.
> I know that the MS Windows function to compare strings is CompareString.
> Therefore I will see which Emacs function uses that.
> Ah, it is string-collate-equalp.
> Therefore I will use string-collate-equalp to compare two file names.
> I won't use the more obvious string-equal, nor will I read the
> documentation of string-collate-equalp, which makes it obvious that this
> is wrong."

No, no one will reason this way.

But they can reason this way instead:

"I want to compare two file names.
Collation is a way to compare strings, for example the man page for
strcoll says the function returns zero if one string is equal to the
other.
And Emacs just learned how to use collation to compare strings, it
has this great new function string-collate-equalp.
Therefore, let's use string-collate-equalp for comparing two file
names."

> And suppose I have two strings, and want to know if they are equal,
> respecting my locale's convention about characters that are not
> literally identical, but have the same meaning. I should use
> string-collate-equalp for this. This is true whether the strings
> represent the names of elephants in a zoo, or files on a disk.

And that is exactly the fallacy that the note warns against.  Because
filesystems don't compare as equal characters that have the same
meaning, they compare bytes in a byte stream that is the file name in
its raw byte form, as recorded on disk.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> But they can reason this way instead:
>
> "I want to compare two file names.
> Collation is a way to compare strings, for example the man page for
> strcoll says the function returns zero if one string is equal to the
> other.
> And Emacs just learned how to use collation to compare strings, it
> has this great new function string-collate-equalp.
> Therefore, let's use string-collate-equalp for comparing two file
> names."

This implies that the user knows about string comparisons with collation
but he is a complete ignoramus about file systems. A bit unrealistic.

Anyway, saying "don't use this for comparing file names" is cryptic.
Mentioning file-equal-p would be helpful if you insist on mentioning
file names on the docstring of string-collate-equalp.

>> And suppose I have two strings, and want to know if they are equal,
>> respecting my locale's convention about characters that are not
>> literally identical, but have the same meaning. I should use
>> string-collate-equalp for this. This is true whether the strings
>> represent the names of elephants in a zoo, or files on a disk.
>
> And that is exactly the fallacy that the note warns against.  Because
> filesystems don't compare as equal characters that have the same
> meaning, they compare bytes in a byte stream that is the file name in
> its raw byte form, as recorded on disk.

I think that Glenn is saying that you can compare file names for other
purposes than knowing if they name the same file.

Maybe the user wishes to find a file with a given name, modulo
collation. Search for "tu.txt" and find "tú.txt".

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Michael Heerdegen]

Richard Stallman <rms@gnu.org> writes:

> We often put two spaces there so that M-e will see it as the end of a
> sentence.

I see, thanks.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Eli Zaretskii]

> From: oscarfv@telefonica.net (Óscar Fuentes)
> Cc: Glenn Morris <rgm@gnu.org>,  michael_heerdegen@web.de,  23902@debbugs.gnu.org
> Date: Sat, 09 Jul 2016 23:36:37 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > But they can reason this way instead:
> >
> > "I want to compare two file names.
> > Collation is a way to compare strings, for example the man page for
> > strcoll says the function returns zero if one string is equal to the
> > other.
> > And Emacs just learned how to use collation to compare strings, it
> > has this great new function string-collate-equalp.
> > Therefore, let's use string-collate-equalp for comparing two file
> > names."
> 
> This implies that the user knows about string comparisons with collation
> but he is a complete ignoramus about file systems. A bit unrealistic.

I'm not sure.  People might know a lot about strings, but not how the
filesystem stores file names.

> >> And suppose I have two strings, and want to know if they are equal,
> >> respecting my locale's convention about characters that are not
> >> literally identical, but have the same meaning. I should use
> >> string-collate-equalp for this. This is true whether the strings
> >> represent the names of elephants in a zoo, or files on a disk.
> >
> > And that is exactly the fallacy that the note warns against.  Because
> > filesystems don't compare as equal characters that have the same
> > meaning, they compare bytes in a byte stream that is the file name in
> > its raw byte form, as recorded on disk.
> 
> I think that Glenn is saying that you can compare file names for other
> purposes than knowing if they name the same file.

Then it's not "equality", it's "equivalence".

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> > I think that Glenn is saying that you can compare file names for other
> > purposes than knowing if they name the same file.
>
> Then it's not "equality", it's "equivalence".

Equivalence implies equality of equivalence classes.  These terms depend
on the viewpoint.  Does `file-equal-p' test whether the files named by
the strings are equal, or whether these strings name the same file and
are thus equivalent (Maybe it should be named
`file-names-equivalent-p')?

Michael.

bug#23902: 25.1.50; Strange warning on string-collate-equalp's docstring

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: oscarfv@telefonica.net (Óscar Fuentes),
>   23902@debbugs.gnu.org
> Date: Sun, 10 Jul 2016 05:06:49 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > > I think that Glenn is saying that you can compare file names for other
> > > purposes than knowing if they name the same file.
> >
> > Then it's not "equality", it's "equivalence".
> 
> Equivalence implies equality of equivalence classes.

I think we shouldn't split hair on this matter.  File-name equality is
something that doesn't need explanation, and we don't want to muddy
the waters by looking how to confuse matters even more.

> Does `file-equal-p' test whether the files named by the strings are
> equal, or whether these strings name the same file and are thus
> equivalent

IIUC, it does neither (e.g., because it supports links and other
stuff).

Now, can we please move on to more important things?