bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Michael Heerdegen]

Hello,

Please see #2536 for a similar issue.

I'm working with Gnu/Linux.  If I hit ! in dired on a file starting with
"-", the hyphen is not quoted so that the specified program will
interpret the file name as an option.  I guess this is nearly never
what the user wants.


In GNU Emacs 24.0.92.1 (i486-pc-linux-gnu, GTK+ Version 3.2.3)
 of 2012-01-05 on zelenka, modified by Debian
 (emacs-snapshot package, version 1:20120105-1)
Windowing system distributor `The X.Org Foundation', version 11.0.11102902
configured using `configure  '--build' 'i486-linux-gnu' '--host' 'i486-linux-gnu' '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib' '--localstatedir=/var' '--infodir=/usr/share/info' '--mandir=/usr/share/man' '--with-pop=yes' '--enable-locallisppath=/etc/emacs-snapshot:/etc/emacs:/usr/local/share/emacs/24.0.92/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.0.92/site-lisp:/usr/share/emacs/site-lisp' '--without-compress-info' '--with-crt-dir=/usr/lib/i386-linux-gnu/' '--with-x=yes' '--with-x-toolkit=gtk3' '--with-imagemagick=yes' 'build_alias=i486-linux-gnu' 'host_alias=i486-linux-gnu' 'CFLAGS=-DDEBIAN -DSITELOAD_PURESIZE_EXTRA=5000 -g -O2''

Important settings:
  value of $LC_ALL: de_DE.utf8
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: C
  value of $LANG: de_DE.utf8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Stefan Monnier]

> I'm working with Gnu/Linux.  If I hit ! in dired on a file starting with
> "-", the hyphen is not quoted so that the specified program will
> interpret the file name as an option.  I guess this is nearly never
> what the user wants.

Indeed.  Quoting the file name by doing something like:

    foobar '-toto' titi

won't help (`foobar' won't see the quoting anyway because the shell
will strip it away before passing the result to `foobar').
But we could do something like:

   foobar ./-toto titi

which would not suffer from this problem.


        Stefan

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Glenn Morris]

Stefan Monnier wrote:

> But we could do something like:
>
>    foobar ./-toto titi

Or stick a "--" after the command (is that a universal convention?). if
the user did not supply one. This is also what anyone could use to work
around the problem in the meantime.

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Mon, 09 Jan 2012 12:27:30 -0500
> Cc: michael_heerdegen@web.de, 10458@debbugs.gnu.org
> 
> Stefan Monnier wrote:
> 
> > But we could do something like:
> >
> >    foobar ./-toto titi
> 
> Or stick a "--" after the command (is that a universal convention?).

AFAIK, -- is supported only by GNU getopt (and maybe compatibles).  I
think there's no guarantee that an arbitrary program will support it.

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Stefan Monnier]

>> But we could do something like:
>> foobar ./-toto titi
> Or stick a "--" after the command (is that a universal convention?)

not only I don't think it's a universal convention, but I can't think of
any easy reliable way to do that.  E.g. compare

    gcc -o foo -toto.c
vs
    gcc -c foo.c -toto.c


-- Stefan

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Richard Stallman]

    AFAIK, -- is supported only by GNU getopt (and maybe compatibles).  I
    think there's no guarantee that an arbitrary program will support it.

If you encounter a program that doesn't use GNU getopt
or equivalent, please suggest adopting that feature.
It would be convenient for users if all programs supported it.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Jan D.]

Stefan Monnier skrev 2012-01-09 23:40:
>>> But we could do something like:
>>> foobar ./-toto titi
>> Or stick a "--" after the command (is that a universal convention?)
>
> not only I don't think it's a universal convention, but I can't think of
> any easy reliable way to do that.  E.g. compare
>
>      gcc -o foo -toto.c
> vs
>      gcc -c foo.c -toto.c
>

It is universal in that it is in POSIX 1003.1.

	Jan D.

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Glenn Morris]

Stefan Monnier wrote:

> not only I don't think it's a universal convention, but I can't think of
> any easy reliable way to do that.  E.g. compare
>
>     gcc -o foo -toto.c
> vs
>     gcc -c foo.c -toto.c

Then perhaps it is academic, but I see no ambiguity, since dired keeps
command and file(s) to operate on separate:

  ! gcc -o foo RET
  ! gcc -c RET

It's not like we have to try and decode an entire arbitrary command
string.

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Stefan Monnier]

> Then perhaps it is academic, but I see no ambiguity, since dired keeps
> command and file(s) to operate on separate:

>   ! gcc -o foo RET
>   ! gcc -c RET

How 'bout "! diff -u oldfoo RET"?


        Stefan

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Glenn Morris]

Stefan Monnier wrote:

> How 'bout "! diff -u oldfoo RET"?

<shakes fist>

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Glenn Morris]

> Stefan Monnier wrote:
>
>> How 'bout "! diff -u oldfoo RET"?

Actually, GNU diff accepts this just fine:

  diff -u oldfoo -- -newfoo

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Stefan Monnier]

>>> How 'bout "! diff -u oldfoo RET"?
> Actually, GNU diff accepts this just fine:
>   diff -u oldfoo -- -newfoo

Interesting.  So we need yet another hack in case the file is named
"--".


        Stefan

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Jan Djärv]

10 jan 2012 kl. 21:03 skrev Stefan Monnier:

>>>> How 'bout "! diff -u oldfoo RET"?
>> Actually, GNU diff accepts this just fine:
>>  diff -u oldfoo -- -newfoo
> 
> Interesting.  So we need yet another hack in case the file is named
> "--".

diff -u oldfoo -- --

should work.

	Jan D.

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Lars Ingebrigtsen]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> Indeed.  Quoting the file name by doing something like:
>
>     foobar '-toto' titi
>
> won't help (`foobar' won't see the quoting anyway because the shell
> will strip it away before passing the result to `foobar').
> But we could do something like:
>
>    foobar ./-toto titi
>
> which would not suffer from this problem.

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

The discussion then turned to whether "--" was a possibility, and it
isn't.

But adding ./ to file names that start with - should be safe, I think?
So I've now done that in Emacs 29.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sun, 20 Feb 2022 14:50:36 +0100
> Cc: michael_heerdegen@web.de, 10458@debbugs.gnu.org
> 
> But adding ./ to file names that start with - should be safe, I think?

It changes semantics.  The list of file names passed to a command
doesn't necessarily mean "open each file and process it", it could
mean something else, like "make a list of files".  So I don't think we
should do that.

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> It changes semantics.  The list of file names passed to a command
> doesn't necessarily mean "open each file and process it", it could
> mean something else, like "make a list of files".  So I don't think we
> should do that.

Commands like "echo" will behave differently, but I tried to imagine
other functions where this would make a difference, and I failed.  Do
you have any examples?

The old behaviour would misbehave for the vast majority of commands, so
I think it's an improvement.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: monnier@IRO.UMontreal.CA,  michael_heerdegen@web.de,  10458@debbugs.gnu.org
> Date: Sun, 20 Feb 2022 15:10:05 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > It changes semantics.  The list of file names passed to a command
> > doesn't necessarily mean "open each file and process it", it could
> > mean something else, like "make a list of files".  So I don't think we
> > should do that.
> 
> Commands like "echo" will behave differently, but I tried to imagine
> other functions where this would make a difference, and I failed.  Do
> you have any examples?

Here:

   find /foo/bar -name %f

I think substituting "./bar" for %f has different semantics that
"bar", doesn't it?

Basically, any scenario where "foo" doesn't mean 'the file "foo" in
the directory where the command is invoked'.

> The old behaviour would misbehave for the vast majority of commands, so
> I think it's an improvement.

In some cases, yes.  I'm worried about those where it changes the
meaning, and the fact that we do this silently.

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Here:
>
>    find /foo/bar -name %f
>
> I think substituting "./bar" for %f has different semantics that
> "bar", doesn't it?

Hm, right.  I guess that's a possible command somebody could use (even
though it doesn't work well in general (if you've ticked several files)).

But note that it doesn't replace "bar" with "./bar" -- it only does this
if the file name starts with "-".

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: monnier@IRO.UMontreal.CA,  michael_heerdegen@web.de,  10458@debbugs.gnu.org
> Date: Sun, 20 Feb 2022 15:31:56 +0100
> 
> But note that it doesn't replace "bar" with "./bar" -- it only does this
> if the file name starts with "-".

Sure.  Which is why the problems will be rare and subtle.

Btw, the "usual" way of telling a program "all arguments from here on
are not options" is to use "--".

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> But adding ./ to file names that start with - should be safe, I think?
> So I've now done that in Emacs 29.

The problem with that is that it's hackish and ad-hoc.  It will
sometimes do the right thing, but other times it' not what's expected
and users will find it odd.

An alternative is to pass absolute file names.  And I think it would
make sense in that case to let the user choose between relative names
and absolute names.


        Stefan

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Btw, the "usual" way of telling a program "all arguments from here on
> are not options" is to use "--".

It's not supported by all commands, though.  (And it would fail even
more in the "find" example.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> The problem with that is that it's hackish and ad-hoc.  It will
> sometimes do the right thing, but other times it' not what's expected
> and users will find it odd.

Statistically speaking, it'll work on more cases than `!' used to,
though.  I'm pretty sure.  And I think it's fine for a command like `!'
to try to DWIM.

> An alternative is to pass absolute file names.  And I think it would
> make sense in that case to let the user choose between relative names
> and absolute names.

Giving absolute names would break more stuff than the DWIM tweak,
though, so we can't default to that.  And if we can't default to having
`!' work for files like this, there isn't much point.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Michael Heerdegen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Giving absolute names would break more stuff than the DWIM tweak,
> though, so we can't default to that.

What kinds of stuff would typically break if we did that?

Michael.

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Lars Ingebrigtsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

>> Giving absolute names would break more stuff than the DWIM tweak,
>> though, so we can't default to that.
>
> What kinds of stuff would typically break if we did that?

See Eli's example.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> The problem with that is that it's hackish and ad-hoc.  It will
>> sometimes do the right thing, but other times it' not what's expected
>> and users will find it odd.
>
> Statistically speaking, it'll work on more cases than `!' used to,
> though.  I'm pretty sure.  And I think it's fine for a command like `!'
> to try to DWIM.

I was thinking of making it less-DWIMish and leave it in control of
the user.  IOW, rather than try to magically "do the right thing" for
files starting with `-`, allow the users to request explicitly absolute
file names (maybe with a prefix command or something like that) when
*they* see it's needed.

It won't fix the user's mistakes, but it will give a simpler semantics
on which powerusers can more easily rely.


        Stefan

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I was thinking of making it less-DWIMish and leave it in control of
> the user.  IOW, rather than try to magically "do the right thing" for
> files starting with `-`, allow the users to request explicitly absolute
> file names (maybe with a prefix command or something like that) when
> *they* see it's needed.

The prefix is already taken on the `!' command...

> It won't fix the user's mistakes, but it will give a simpler semantics
> on which powerusers can more easily rely.

Using -file-names as they are is a usability problem, and may be a
security problem (the file name may have been created by someone with
the idea in mind that interpreted as a command line switch it'll lead to
nefarious results), so I don't think `!' should go back to the old
behaviour by default.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#10458: 24.0.92; ! in dired on a file starting with a hyphen

[Drew Adams]

> > But adding ./ to file names that start with - should be safe, I
> > think?  So I've now done that in Emacs 29.
> 
> The problem with that is that it's hackish and ad-hoc.  It will
> sometimes do the right thing, but other times it' not what's expected
> and users will find it odd.

FWIW, I think the cure is worse than the problem.

And as Michael indicated, a similar problem was
reported in bug #2536.

No simple solution has been found.  The simplistic
one you've (apparently) adopted is, I think, a bad
idea.  Better to just document this (and similarly
for bug #2536) as a limitation of `!' (and `&' and
`M-!').

That at least gives straightforward, if limited,
behavior - easy to understand (but needs to be
documented).  Users can find other ways (e.g.
outside Dired) to take care of such (relatively
corner) cases.

That's unfortunate, but not as unfortunate as the
change you've (apparently) made.

Just one opinion.  Document the limitations, and
leave these two bugs open, hoping that Someone(TM)
at some point will dream up a real, comprehensive
solution.