bug#17950: 24.4.50; REGRESSION: `read-file-name' from a menu (mouse) treats "~/" as installation dir

bug#17950: 24.4.50; REGRESSION: `read-file-name' from a menu (mouse) treats "~/" as installation dir

[Drew Adams]

emacs -Q

(defun foo (file)
  (interactive (list (read-file-name "Foo: " "~/")))
  (message "File is `%s'" file))

(define-key menu-bar-file-menu [foo] '(menu-item "FOOOO" foo))

(expand-file-name "~/") ; correctly returns my home directory, as in
previous Emacs versions.

And `M-x foo' correctly uses `~/' as the directory.

But invoking the same command `foo' from the menu (item `FOOOO') uses
the Emacs installation directory instead.


In GNU Emacs 24.4.50.1 (i686-pc-mingw32)
 of 2014-06-28 on ODIEONE
Bzr revision: 117431 rgm@gnu.org-20140628015517-eku6hj8mpgcvfnso
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure --prefix=/c/Devel/emacs/snapshot/trunk
 --enable-checking=yes,glyphs 'CFLAGS=-O0 -g3'
 LDFLAGS=-Lc:/Devel/emacs/lib 'CPPFLAGS=-DGC_MCHECK=1
 -Ic:/Devel/emacs/include''

bug#17950: 24.4.50; REGRESSION: `read-file-name' from a menu (mouse) treats "~/" as installation dir

[Eli Zaretskii]

> Date: Sat, 5 Jul 2014 14:29:24 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> 
> emacs -Q
> 
> (defun foo (file)
>   (interactive (list (read-file-name "Foo: " "~/")))
>   (message "File is `%s'" file))
> 
> (define-key menu-bar-file-menu [foo] '(menu-item "FOOOO" foo))
> 
> (expand-file-name "~/") ; correctly returns my home directory, as in
> previous Emacs versions.
> 
> And `M-x foo' correctly uses `~/' as the directory.
> 
> But invoking the same command `foo' from the menu (item `FOOOO') uses
> the Emacs installation directory instead.

I cannot reproduce this here with today's trunk.  When I select the
foo item from the menu, the file selection dialog that pops up shows
me my home directory, as I'd expect.

So something else is at work here.  Perhaps the important detail you
left out is how exactly did you invoke "emacs -Q".

bug#17950: 24.4.50; REGRESSION: `read-file-name' from a menu (mouse) treats "~/" as installation dir

[Drew Adams]

> I cannot reproduce this here with today's trunk.  When I select the
> foo item from the menu, the file selection dialog that pops up shows
> me my home directory, as I'd expect.
> 
> So something else is at work here.  Perhaps the important detail you
> left out is how exactly did you invoke "emacs -Q".

I invoked emacs -Q from a Windows shortcut with `Target':
Z:\path\to\runemacs.exe -Q --debug-init
and with `Start in' set to its default: z:\path\to\Emacs\bin

I then used *scratch* to evaluate the recipe code.
I have env var HOME defined, also. And Emacs recognizes "~/" correctly
(apart from the bug).

When I check `default-directory' it is "Z:\\path\\to\\bin/" (dunno
why there is a forward slash at the end but the others are backslashes).
If I change `Start in' then `default-directory' changes to reflect it,
correctly.  But the dir used by the menu item is still the (same) wrong
one.

It works fine in Emacs 24.3 and prior.  Broken in the reported build.
It works fine also as recently as this build:
GNU Emacs 24.4.50.1 (i686-pc-mingw32) of 2014-06-17 on ODIEONE

However, it seems that the directory used for the file selection box
is not related to `Start in'.  It seems to be the something like a
dir used in a different or a previous Emacs session (?).  Not sure
about that, but it definitely comes up with a directory that is
unrelated to either my HOME or the directory in `Start in'.

The directory I see it coming up with now (which is different from
what I saw when I reported the bug yesterday) is a directory that
I normally use a lot, and which contains much of my Lisp code.
But there is nothing in this emacs -Q recipe that points to anything
to do with that directory or my code.  All I can guess is that it is
coming from another Emacs session somehow, or from a cache file
somewhere.  But looking around (e.g., in HOME, .emacs.d, etc.) I see
nothing resembling a cache file that it could be picking the dir up
from.

I added `M-x debug-on-entry read-file-name' to the recipe, and went
through the debugger after invoking from menu FOOOO.  E.g.:

* x-file-dialog("Foo: " "~/" nil nil nil)
* read-file-name-default("Foo: " "~/" nil nil nil nil)
* read-file-name("Foo: " "~/")

When I hit `d' on `x-file-dialog' the file-selection box is popped
up with the incorrect directory in field `Look in:'.  `x-file-dialog'
is defined in C, and I have no C sources.  But it seems that would
be the place to look for the cause of the problem.  Perhaps its
code has changed recently.  HTH.

Maybe you can play with it a bit more, to try to see what's going on.

In any case, what is not happening, which has always happened before,
is that the file-selection directory is HOME (since "~/" is used in
the recipe).

bug#17950: 24.4.50; REGRESSION: `read-file-name' from a menu (mouse) treats "~/" as installation dir

[Eli Zaretskii]

> Date: Sun, 6 Jul 2014 08:10:35 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 17950@debbugs.gnu.org
> 
> I invoked emacs -Q from a Windows shortcut with `Target':
> Z:\path\to\runemacs.exe -Q --debug-init
> and with `Start in' set to its default: z:\path\to\Emacs\bin
> 
> I then used *scratch* to evaluate the recipe code.
> I have env var HOME defined, also. And Emacs recognizes "~/" correctly
> (apart from the bug).
> 
> When I check `default-directory' it is "Z:\\path\\to\\bin/" (dunno
> why there is a forward slash at the end but the others are backslashes).
> If I change `Start in' then `default-directory' changes to reflect it,
> correctly.  But the dir used by the menu item is still the (same) wrong
> one.

Repeating this, everything I see is the same as you, except that the
file selection dialog still shows my home directory.

> However, it seems that the directory used for the file selection box
> is not related to `Start in'.  It seems to be the something like a
> dir used in a different or a previous Emacs session (?).  Not sure
> about that, but it definitely comes up with a directory that is
> unrelated to either my HOME or the directory in `Start in'.

Could be a Windows 7 thing (I'm testing on XP here).  I think it
remembers the last directory you were in, or something.  I'll try on
Windows 7 when I can.

bug#17950: 24.4.50; REGRESSION: `read-file-name' from a menu (mouse) treats "~/" as installation dir

[Drew Adams]

> Could be a Windows 7 thing (I'm testing on XP here).  I think it
> remembers the last directory you were in, or something.  I'll try on
> Windows 7 when I can.

OK, yes, I am on Windows 7.  But you might also want to see whether
the code for `x-file-dialog' changed recently.  There are only 8 days
separating a build that worked from this build where this is broken.

bug#17950: 24.4.50; REGRESSION: `read-file-name' from a menu (mouse) treats "~/" as installation dir

[Eli Zaretskii]

> Date: Sun, 6 Jul 2014 09:05:26 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 17950@debbugs.gnu.org
> 
> But you might also want to see whether the code for `x-file-dialog'
> changed recently.

I already did: it was last changed in Nov 2013.

> There are only 8 days separating a build that worked from this build
> where this is broken.

It would help to know the bzr revisions of these builds.

bug#17950: 24.4.50; REGRESSION: `read-file-name' from a menu (mouse) treats "~/" as installation dir

[Eli Zaretskii]

> Date: Sun, 6 Jul 2014 08:10:35 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 17950@debbugs.gnu.org
> 
> I added `M-x debug-on-entry read-file-name' to the recipe, and went
> through the debugger after invoking from menu FOOOO.  E.g.:
> 
> * x-file-dialog("Foo: " "~/" nil nil nil)
> * read-file-name-default("Foo: " "~/" nil nil nil nil)
> * read-file-name("Foo: " "~/")
> 
> When I hit `d' on `x-file-dialog' the file-selection box is popped
> up with the incorrect directory in field `Look in:'.

So you are saying that x-file-dialog is called with the correct
directory "~/", and yet the file selection dialog still shows another
directory, is that right?

bug#17950: 24.4.50; REGRESSION: `read-file-name' from a menu (mouse) treats "~/" as installation dir

[Drew Adams]

> > * x-file-dialog("Foo: " "~/" nil nil nil)
> > * read-file-name-default("Foo: " "~/" nil nil nil nil)
> > * read-file-name("Foo: " "~/")
> >
> > When I hit `d' on `x-file-dialog' the file-selection box is popped
> > up with the incorrect directory in field `Look in:'.
> 
> So you are saying that x-file-dialog is called with the correct
> directory "~/", and yet the file selection dialog still shows another
> directory, is that right?

Apparently so.  As soon as I hit `d', at the point shown above, the
file-selection box popped up, with the wrong directory.

bug#17950: 24.4.50; REGRESSION: `read-file-name' from a menu (mouse) treats "~/" as installation dir

[Drew Adams]

> > There are only 8 days separating a build that worked from this build
> > where this is broken.
> 
> It would help to know the bzr revisions of these builds.

Does this help at all?

In GNU Emacs 24.4.50.1 (i686-pc-mingw32)
 of 2014-06-17 on ODIEONE
Repository revision: 117359 monnier@iro.umontreal.ca-20140617193358-2t1nl1te9gc2mqrx
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure --prefix=/c/Devel/emacs/snapshot/trunk
 --enable-checking=yes,glyphs 'CFLAGS=-O0 -g3'
 LDFLAGS=-Lc:/Devel/emacs/lib 'CPPFLAGS=-DGC_MCHECK=1
 -Ic:/Devel/emacs/include''

In GNU Emacs 24.4.50.1 (i686-pc-mingw32)
 of 2014-06-28 on ODIEONE
Repository revision: 117431 rgm@gnu.org-20140628015517-eku6hj8mpgcvfnso
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure --prefix=/c/Devel/emacs/snapshot/trunk
 --enable-checking=yes,glyphs 'CFLAGS=-O0 -g3'
 LDFLAGS=-Lc:/Devel/emacs/lib 'CPPFLAGS=-DGC_MCHECK=1
 -Ic:/Devel/emacs/include''

bug#17950: 24.4.50; REGRESSION: `read-file-name' from a menu (mouse) treats "~/" as installation dir

[Eli Zaretskii]

> Date: Sun, 6 Jul 2014 16:00:21 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 17950@debbugs.gnu.org
> 
> > It would help to know the bzr revisions of these builds.
> 
> Does this help at all?
> 
> In GNU Emacs 24.4.50.1 (i686-pc-mingw32)
>  of 2014-06-17 on ODIEONE
> Repository revision: 117359 monnier@iro.umontreal.ca-20140617193358-2t1nl1te9gc2mqrx
> Windowing system distributor `Microsoft Corp.', version 6.1.7601
> Configured using:
>  `configure --prefix=/c/Devel/emacs/snapshot/trunk
>  --enable-checking=yes,glyphs 'CFLAGS=-O0 -g3'
>  LDFLAGS=-Lc:/Devel/emacs/lib 'CPPFLAGS=-DGC_MCHECK=1
>  -Ic:/Devel/emacs/include''
> 
> In GNU Emacs 24.4.50.1 (i686-pc-mingw32)
>  of 2014-06-28 on ODIEONE
> Repository revision: 117431 rgm@gnu.org-20140628015517-eku6hj8mpgcvfnso
> Windowing system distributor `Microsoft Corp.', version 6.1.7601
> Configured using:
>  `configure --prefix=/c/Devel/emacs/snapshot/trunk
>  --enable-checking=yes,glyphs 'CFLAGS=-O0 -g3'
>  LDFLAGS=-Lc:/Devel/emacs/lib 'CPPFLAGS=-DGC_MCHECK=1
>  -Ic:/Devel/emacs/include''

Yes, thanks.

bug#17950: 24.4.50; REGRESSION: `read-file-name' from a menu (mouse) treats "~/" as installation dir

[Eli Zaretskii]

> Date: Sun, 06 Jul 2014 18:56:31 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 17950@debbugs.gnu.org
> 
> > However, it seems that the directory used for the file selection box
> > is not related to `Start in'.  It seems to be the something like a
> > dir used in a different or a previous Emacs session (?).  Not sure
> > about that, but it definitely comes up with a directory that is
> > unrelated to either my HOME or the directory in `Start in'.
> 
> Could be a Windows 7 thing (I'm testing on XP here).  I think it
> remembers the last directory you were in, or something.  I'll try on
> Windows 7 when I can.

OK, I do see this on Windows 7.  But it's not due to something Emacs
does or started to do lately.  This is due to a deliberate change in
behavior of the file selection dialogs introduced in Windows 7.  It is
explicitly documented in the pertinent parameter we pass to the API
that pops up the dialog:

  lpstrInitialDir

      Type: LPCTSTR

      The initial directory. The algorithm for selecting the initial
      directory varies on different platforms.

      Windows 7:

	  If lpstrInitialDir has the same value as was passed the
	  first time the application used an Open or Save As dialog
	  box, the path most recently selected by the user is used as
	  the initial directory.

	  Otherwise, if lpstrFile contains a path, that path is the
	  initial directory.

	  Otherwise, if lpstrInitialDir is not NULL, it specifies the
	  initial directory.  If lpstrInitialDir is NULL and the
	  current directory contains any files of the specified filter
	  types, the initial directory is the current directory.

	  Otherwise, the initial directory is the personal files
	  directory of the current user.

	  Otherwise, the initial directory is the Desktop folder.

      Windows 2000/XP/Vista:

	  If lpstrFile contains a path, that path is the initial
	  directory.

	  Otherwise, lpstrInitialDir specifies the initial directory.

	  Otherwise, if the application has used an Open or Save As
	  dialog box in the past, the path most recently used is
	  selected as the initial directory. However, if an
	  application is not run for a long time, its saved selected
	  path is discarded.  If lpstrInitialDir is NULL and the
	  current directory contains any files of the specified filter
	  types, the initial directory is the current directory.

	  Otherwise, the initial directory is the personal files
	  directory of the current user.

	  Otherwise, the initial directory is the Desktop folder.

IOW, whenever you call x-file-dialog with the same 2nd argument as the
last time, you will be presented with the directory where you selected
a file at that prior call.

So I'm quite sure your previous binary (and all the older ones)
behaves exactly like your current binary does.  All you need to
trigger this "feature" is to navigate away from your home directory
using the file selection dialog, and actually select a file in another
directory, then invoke x-file-dialog again with the same "~/" argument
as the first call -- you will see that the file selection dialog
displays that other directory.

Given that this is standard behavior of the file selection dialog on
Windows 7 and later, the question is, should we try to work around it
(assuming there is a workaround, which is something I'm not yet sure)?

And if the workaround comes at a price, like initially having
something like "*.*" in the "File Name" field, which currently starts
empty, is that price acceptable, or would it be a nuisance?

bug#17950: 24.4.50; REGRESSION: `read-file-name' from a menu (mouse) treats "~/" as installation dir

[Drew Adams]

> > > However, it seems that the directory used for the file selection box
> > > is not related to `Start in'.  It seems to be the something like a
> > > dir used in a different or a previous Emacs session (?).  Not sure
> > > about that, but it definitely comes up with a directory that is
> > > unrelated to either my HOME or the directory in `Start in'.
> >
> > Could be a Windows 7 thing (I'm testing on XP here).  I think it
> > remembers the last directory you were in, or something.  I'll try on
> > Windows 7 when I can.
> 
> OK, I do see this on Windows 7.  But it's not due to something Emacs
> does or started to do lately.  This is due to a deliberate change in
> behavior of the file selection dialogs introduced in Windows 7.  It is
> explicitly documented in the pertinent parameter we pass to the API
> that pops up the dialog:
> 
>   lpstrInitialDir
> 
>       Type: LPCTSTR
> 
>       The initial directory. The algorithm for selecting the initial
>       directory varies on different platforms.
> 
>       Windows 7:
> 
> 	  If lpstrInitialDir has the same value as was passed the
> 	  first time the application used an Open or Save As dialog
> 	  box, the path most recently selected by the user is used as
> 	  the initial directory.
> 
> 	  Otherwise, if lpstrFile contains a path, that path is the
> 	  initial directory.
> 
> 	  Otherwise, if lpstrInitialDir is not NULL, it specifies the
> 	  initial directory.  If lpstrInitialDir is NULL and the
> 	  current directory contains any files of the specified filter
> 	  types, the initial directory is the current directory.
> 
> 	  Otherwise, the initial directory is the personal files
> 	  directory of the current user.
> 
> 	  Otherwise, the initial directory is the Desktop folder.
> 
>       Windows 2000/XP/Vista:
> 
> 	  If lpstrFile contains a path, that path is the initial
> 	  directory.
> 
> 	  Otherwise, lpstrInitialDir specifies the initial directory.
> 
> 	  Otherwise, if the application has used an Open or Save As
> 	  dialog box in the past, the path most recently used is
> 	  selected as the initial directory. However, if an
> 	  application is not run for a long time, its saved selected
> 	  path is discarded.  If lpstrInitialDir is NULL and the
> 	  current directory contains any files of the specified filter
> 	  types, the initial directory is the current directory.
> 
> 	  Otherwise, the initial directory is the personal files
> 	  directory of the current user.
> 
> 	  Otherwise, the initial directory is the Desktop folder.
> 
> IOW, whenever you call x-file-dialog with the same 2nd argument as the
> last time, you will be presented with the directory where you selected
> a file at that prior call.
> 
> So I'm quite sure your previous binary (and all the older ones)
> behaves exactly like your current binary does.  All you need to
> trigger this "feature" is to navigate away from your home directory
> using the file selection dialog, and actually select a file in another
> directory, then invoke x-file-dialog again with the same "~/" argument
> as the first call -- you will see that the file selection dialog
> displays that other directory.
> 
> Given that this is standard behavior of the file selection dialog on
> Windows 7 and later, the question is, should we try to work around it
> (assuming there is a workaround, which is something I'm not yet sure)?
> 
> And if the workaround comes at a price, like initially having
> something like "*.*" in the "File Name" field, which currently starts
> empty, is that price acceptable, or would it be a nuisance?

Got it.  Thanks for looking into this.  I don't have a particular
opinion about how Emacs Dev should handle this.  I do see that this
could lead to user errors or at least confusion.

I'm OK with whatever you decide is TRT to do about this (including if
it is nothing). Thx.

bug#17950: 24.4.50; REGRESSION: `read-file-name' from a menu (mouse) treats "~/" as installation dir

[Stefan Monnier]

> Given that this is standard behavior of the file selection dialog on
> Windows 7 and later, the question is, should we try to work around it
> (assuming there is a workaround, which is something I'm not yet sure)?

To the extent that using the system's dialog is meant to follow the
system's behavior, I think it makes sense to just "go with the flow".


        Stefan

bug#17950: 24.4.50; REGRESSION: `read-file-name' from a menu (mouse) treats "~/" as installation dir

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: drew.adams@oracle.com,  17950@debbugs.gnu.org
> Date: Mon, 07 Jul 2014 12:39:43 -0400
> 
> > Given that this is standard behavior of the file selection dialog on
> > Windows 7 and later, the question is, should we try to work around it
> > (assuming there is a workaround, which is something I'm not yet sure)?
> 
> To the extent that using the system's dialog is meant to follow the
> system's behavior, I think it makes sense to just "go with the flow".

No further comments, so I documented the changed behavior in emacs-24
branch, revision 117373, and I'm closing the bug.