bug#4128: 23.1; term/ns-win.el does "too much", assumes wrong run order

bug#4128: 23.1; term/ns-win.el does "too much", assumes wrong run order

[John Prevost]

Please write in English if possible, because the Emacs maintainers
usually do not have translators to read other languages for them.

Your bug report will be posted to the bug-gnu-emacs@gnu.org mailing list,
and to the gnu.emacs.bug news group.

Please describe exactly what actions triggered the bug
and the precise symptoms of the bug:

The term/ns-win.el terminal init file does a lot of questionable things,
and more importantly, it seems to assume the wrong run order.
Specifically, a lot of the items in the file assume that they're run
before the user's startup file is loaded, when in fact this file is run
after the user's startup file.  (First: "(emacs) Terminal Init" says
that the .emacs file can prevent loading of this by setting
term-file-prefix to nil.  Second: When run in --daemon mode, term init
files appear to be loaded once when the first terminal of that type is
created.)

Specifically problemmatic:

  1) ns-win.el contains a number of defvaralias declarations intended to
     make transition from the old "mac-X" variables to the new "ns-X"
     variables (e.g. mac-command-modifier -> ns-command-modifier)
     easier.  These defvaraliases run after the user's startup file,
     which means that they are not in effect when the user sets the
     old-style "mac-X" variables.

  2) ns-win.el contains a number of rather questionable keyboard
     bindings on the global-map.  Some of these are nextstep-specific
     events (ns-power-off, ns-open-file, etc.).  More upsetting is a
     wholesale slaughter of the super- modifier, with some 44
     keybindings set for "compatibility" bindings for that modifier.
     Extremely troubling is the binding for S-mouse-1 to
     'mouse-save-then-kill by default, which may be a standard nextstep
     behavior, but is definitely not a standard mac behavior.

     The real problem with these keybindings is that they are set when
     the term/ns-win.el term init file is loaded, which as I noted above
     is *after* the user's startup file.  That means that in order to
     replace any bindings in this set (for example, if the user has his
     own super- bindings, or if he wants to replace S-mouse-1 with
     something a bit less strange, or even if he wants to replace the
     default behavior of dropping a file inserting its contents in the
     current buffer with the older behavior of dropping a file visiting
     that file), then the user *must* use term-setup-hook to run the
     appropriate commands after the term is loaded.

     For reference:

         $ gzip -dc ns-win.el.gz | grep key | grep global | wc -l
         83
         $ gzip -dc w32-win.el.gz | grep key | grep global | wc -l
         3
         $ gzip -dc x-win.el.gz | grep key | grep global | wc -l
         1

     And specifically:

          $ gzip -dc w32-win.el.gz | grep key | grep global
          (global-set-key [drag-n-drop] 'w32-drag-n-drop)
          (global-set-key [C-drag-n-drop] 'w32-drag-n-drop-other-frame)
          (global-set-key [language-change] 'ignore)

          $ gzip -dc x-win.el.gz | grep key | grep global
          ;;     (global-set-key [f10] 'ignore))

          whereas the ns-win.el bindings are all rather substantive.

  3) Another example of something the term/ns-win.el file really
     shouldn't be mucking with is:

          ;; Don't show the frame name; that's redundant with Nextstep.
          (setq-default mode-line-frame-identification '("  "))

     This does nothing to the display of windowed frames, but makes tty
     frames *fail* to display the frame number once the ns-win.el
     terminal init has been loaded.


A number of these features really need to be moved elsewhere, although I
can't say exactly where.  Some of them need to be turned off by default
(for example, the 44 super-based nextstep compatibility keybindings) or
at the very least configurable by use of a variable to select whether
they're desired.



If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
If you would like to further debug the crash, please read the file
/Users/prevost1/Documents/src/emacs-23.1/nextstep/Emacs.app/Contents/Resources/etc/DEBUG for instructions.


In GNU Emacs 23.1.1 (i386-apple-darwin9.7.0, NS apple-appkit-949.46)
 of 2009-08-07 on TELPERION.WV.CC.CMU.EDU
Windowing system distributor `Apple', version 10.3.949
configured using `configure  '--prefix=/opt/emacs-23.1' '--with-ns''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: nil
  default-enable-multibyte-characters: t

Major mode: Fundamental

Minor modes in effect:
  diff-auto-refine-mode: t
  tooltip-mode: t
  tool-bar-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  global-auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<return> y <down> <down> <down> <down> C-x b <return> 
C-c C-c n C-x C-g <help-echo> <down-mouse-1> <mouse-movement> 
<mouse-1> C-c C-c y <help-echo> <help-echo> <help-echo> 
<help-echo> <down-mouse-1> <mouse-1> <down-mouse-1> 
<mouse-1> s-x M-x r e p o r <tab> <return>

Recent messages:
Checking new news...done
Auto-saving...
Sending...
Already sent message via mail; resend? (y or n) 
message-send: No methods specified to send by
Auto-saving...
Sending...
Already sent message via mail; resend? (y or n) 
Sending via mail...
Sending...done
kill-region: The mark is not set now, so there is no region

bug#4128: 23.1; term/ns-win.el does "too much", assumes wrong run order

[Jason Rumney]

John Prevost wrote:
> The term/ns-win.el terminal init file does a lot of questionable things,
> and more importantly, it seems to assume the wrong run order.
> Specifically, a lot of the items in the file assume that they're run
> before the user's startup file is loaded, when in fact this file is run
> after the user's startup file.
The term/*-win.el files are preloaded in Emacs 23 on the platforms they 
are appropriate for.
So they are certainly loaded before the user's init file.

> Specifically problemmatic:
>
>   1) ns-win.el contains a number of defvaralias declarations intended to
>      make transition from the old "mac-X" variables to the new "ns-X"
>      variables (e.g. mac-command-modifier -> ns-command-modifier)
>      easier.  These defvaraliases run after the user's startup file,
>      which means that they are not in effect when the user sets the
>      old-style "mac-X" variables.
>   

Is this something you have actually seen happening, or are you 
extrapolating based on your assumption above?

>   2) ns-win.el contains a number of rather questionable keyboard
>      bindings on the global-map.  Some of these are nextstep-specific
>      events (ns-power-off, ns-open-file, etc.).  More upsetting is a
>      wholesale slaughter of the super- modifier, with some 44
>      keybindings set for "compatibility" bindings for that modifier.
>      Extremely troubling is the binding for S-mouse-1 to
>      'mouse-save-then-kill by default, which may be a standard nextstep
>      behavior, but is definitely not a standard mac behavior.
>   

These sound problematic, as does 3 (not quoted). S-mouse-1 is bound to 
the buffer font menu on other platforms, and we don't generally bind 
keys with the super modifier, but if they are bound to functions that 
the user would expect on nextstep derived platforms, then that might be 
OK. However, binding anything to a power-off command within Emacs sounds 
like a bad idea.

bug#4128: 23.1; term/ns-win.el does "too much", assumes wrong run order

[John Prevost]

Jason Rumney <jasonr@gnu.org> writes:

> The term/*-win.el files are preloaded in Emacs 23 on the platforms they 
> are appropriate for.
> So they are certainly loaded before the user's init file.

Hmm.  This does seem to be correct--I was using a pre-release version at
one point while trying to get this going, and that may have confused
me, if things behaved differently.

The keybindings can be overridden, which I've just tested, although
there's still something weird going on (see below).

Is it only the term/*-win.el files that are so pre-loaded?  It might be
good to document the fact in the info documentation for
Terminal-specific Initialization ((emacs) Terminal Init) that these
files are special and therefore cannot be bypassed by -q or setting
term-file-prefix in your .emacs file.  (I must admit I was somewhat
confused when I was trying to figure out what was happening where and
when and modified the term/ns-win.el file... which had no effect.  Now
it is more clear.)

>>   1) ns-win.el contains a number of defvaralias declarations intended to
>>      make transition from the old "mac-X" variables to the new "ns-X"
>>      variables (e.g. mac-command-modifier -> ns-command-modifier)
>>      easier.  These defvaraliases run after the user's startup file,
>>      which means that they are not in effect when the user sets the
>>      old-style "mac-X" variables.
>>   
>
> Is this something you have actually seen happening, or are you 
> extrapolating based on your assumption above?

My apologies.  This is not exactly what is happening, as I've determined
after I went back to reproduce the problem.  Please let me know if I
should make a fresh bug report for this problem:

The problem it only appears to happen if emacs is started in --daemon
mode.  So my suppositions about what exactly is happening are clearly
wrong.  It does not matter if mac-command-modifier or
ns-command-modifier is used, the problem happens either way.

Here's some test code:

(setq mac-command-modifier 'meta)

(message (symbol-name mac-command-modifier))

(defun after-term-test ()
  (message (symbol-name mac-command-modifier))
  (message "Terminal setup done"))

(add-hook 'term-setup-hook 'after-term-test)

If the above is the .emacs file, and the NS version of emacs is started
normally, it results in the following in *Messages* after normal
startup:

meta
For information about GNU Emacs and the GNU system, type C-h C-a.
meta
Terminal setup done

If I then do C-h k (command)-x, I get:

M-x runs the command execute-extended-command, which is an interactive

If I run in daemon mode, I get the same messages as above output to
stdout, but when I then use emacsclient to create a window and do C-h k
(command)-x, I get:

s-x runs the command kill-region, which is an interactive compiled

If I then set mac-command-modifier to meta in that session, command is
interpreted as meta, and it continues to be interpreted that way after I
have closed the emacsclient session and started a new one.

If after starting the daemon I run emacsclient, I can see that the
setting is in place:

$ emacsclient -e 'ns-command-modifier'
meta

And then if I immediately check again after creating my first windowed
session:

$ emacsclient -e 'ns-command-modifier'
super

If I instead create a TTY terminal, the value of ns-command-modifier
does not change.  It is only the first time I create a windowed terminal
that it changes.

So the defvaraliases are just fine, but something else is resetting the
value of ns-command-modifier (and friends) in --daemon mode only, after
the startup file has been run, when the first nextstep windowed terminal
is created.


> These sound problematic, as does 3 (not quoted). S-mouse-1 is bound to 
> the buffer font menu on other platforms, and we don't generally bind 
> keys with the super modifier, but if they are bound to functions that 
> the user would expect on nextstep derived platforms, then that might be 
> OK. However, binding anything to a power-off command within Emacs sounds 
> like a bad idea.

Well, the "power off" command in this case is a window event saying "Hi,
I am politely shutting down now", which is bound to
save-buffers-kill-emacs.  This is an example of something pretty
sensible.

I'm really not sure I think the large collection of "compatibility"
keybindings being activated by default is really a good idea, but now
that I've determined what's actually mucking with things (only the weird
behavior of ns-command-modifier as described above), it's a bit easier
to stomach.

That really leaves the only super troubling binding as:

;;; Allow shift-clicks to work similarly to under Nextstep
(define-key global-map [S-mouse-1] 'mouse-save-then-kill)
(global-unset-key [S-down-mouse-1])

which provides a very surprising behavior that is unlike any modern
computer that runs something "nextstep derived"

and the less troubling but rather irking:

(define-key global-map [home] 'beginning-of-buffer)
(define-key global-map [end] 'end-of-buffer)
(define-key global-map [kp-home] 'beginning-of-buffer)
(define-key global-map [kp-end] 'end-of-buffer)

which does change the behavior of the keys to a behavior common on a
popular modern nextstep-derived system, but with the addendum that it's
just as common for individual applications to treat those keys in the
fashion emacs treats them on other platforms.


Anyway, sorry for wasting your time with my mis-interpretation of what
was going on in the keybindings.  I'll try to verify future reports in
more depth before making them.


Please do let me know if I should submit all or part of this message as
separate bug reports.

John.

bug#4128: 23.1; term/ns-win.el does "too much", assumes wrong run order

[Dan Nicolaescu]

John Prevost <prevost1@cert.org> writes:

  > Jason Rumney <jasonr@gnu.org> writes:
  > 
  > > The term/*-win.el files are preloaded in Emacs 23 on the platforms they 
  > > are appropriate for.
  > > So they are certainly loaded before the user's init file.
  > 
  > Hmm.  This does seem to be correct--I was using a pre-release version at
  > one point while trying to get this going, and that may have confused
  > me, if things behaved differently.
  > 
  > The keybindings can be overridden, which I've just tested, although
  > there's still something weird going on (see below).
  > 
  > Is it only the term/*-win.el files that are so pre-loaded?  It might be
  > good to document the fact in the info documentation for
  > Terminal-specific Initialization ((emacs) Terminal Init) that these
  > files are special and therefore cannot be bypassed by -q or setting
  > term-file-prefix in your .emacs file.  (I must admit I was somewhat
  > confused when I was trying to figure out what was happening where and
  > when and modified the term/ns-win.el file... which had no effect.  Now
  > it is more clear.)
  > 
  > >>   1) ns-win.el contains a number of defvaralias declarations intended to
  > >>      make transition from the old "mac-X" variables to the new "ns-X"
  > >>      variables (e.g. mac-command-modifier -> ns-command-modifier)
  > >>      easier.  These defvaraliases run after the user's startup file,
  > >>      which means that they are not in effect when the user sets the
  > >>      old-style "mac-X" variables.
  > >>   
  > >
  > > Is this something you have actually seen happening, or are you 
  > > extrapolating based on your assumption above?
  > 
  > My apologies.  This is not exactly what is happening, as I've determined
  > after I went back to reproduce the problem.  Please let me know if I
  > should make a fresh bug report for this problem:
  > 
  > The problem it only appears to happen if emacs is started in --daemon
  > mode.  So my suppositions about what exactly is happening are clearly
  > wrong.  It does not matter if mac-command-modifier or
  > ns-command-modifier is used, the problem happens either way.
  > 
  > Here's some test code:
  > 
  > (setq mac-command-modifier 'meta)
  > 
  > (message (symbol-name mac-command-modifier))
  > 
  > (defun after-term-test ()
  >   (message (symbol-name mac-command-modifier))
  >   (message "Terminal setup done"))
  > 
  > (add-hook 'term-setup-hook 'after-term-test)
  > 
  > If the above is the .emacs file, and the NS version of emacs is started
  > normally, it results in the following in *Messages* after normal
  > startup:
  > 
  > meta
  > For information about GNU Emacs and the GNU system, type C-h C-a.
  > meta
  > Terminal setup done
  > 
  > If I then do C-h k (command)-x, I get:
  > 
  > M-x runs the command execute-extended-command, which is an interactive
  > 
  > If I run in daemon mode, I get the same messages as above output to
  > stdout, but when I then use emacsclient to create a window and do C-h k
  > (command)-x, I get:
  > 
  > s-x runs the command kill-region, which is an interactive compiled
  > 
  > If I then set mac-command-modifier to meta in that session, command is
  > interpreted as meta, and it continues to be interpreted that way after I
  > have closed the emacsclient session and started a new one.
  > 
  > If after starting the daemon I run emacsclient, I can see that the
  > setting is in place:
  > 
  > $ emacsclient -e 'ns-command-modifier'
  > meta
  > 
  > And then if I immediately check again after creating my first windowed
  > session:
  > 
  > $ emacsclient -e 'ns-command-modifier'
  > super
  > 
  > If I instead create a TTY terminal, the value of ns-command-modifier
  > does not change.  It is only the first time I create a windowed terminal
  > that it changes.
  > 
  > So the defvaraliases are just fine, but something else is resetting the
  > value of ns-command-modifier (and friends) in --daemon mode only, after
  > the startup file has been run, when the first nextstep windowed terminal
  > is created.


nsterm.m:ns_set_default_prefs


  > Please do let me know if I should submit all or part of this message as
  > separate bug reports.

IMHO that would be a good idea.
Even better, submit it together with patches that fix the problems that
you found, that would make it much more likely to get things going.
There are a lot of bugs filed for the NS port and very few people work
on fixing them...

bug#4128: 23.1; term/ns-win.el does "too much", assumes wrong run order

[Eli Zaretskii]

> From: John Prevost <prevost1@cert.org>
> Date: Wed, 12 Aug 2009 12:35:56 -0400
> Cc: "4128@emacsbugs.donarmstrong.com" <4128@emacsbugs.donarmstrong.com>
> Reply-To: John Prevost <prevost1@cert.org>, 4128@emacsbugs.donarmstrong.com
> 
> Is it only the term/*-win.el files that are so pre-loaded?

No, there are many more.  See the file lisp/loadup.el for the full
list of Lisp packages that are preloaded.

> It might be good to document the fact in the info documentation for
> Terminal-specific Initialization ((emacs) Terminal Init) that these
> files are special and therefore cannot be bypassed by -q or setting
> term-file-prefix in your .emacs file.

It would be a maintenance nightmare, IMO, to document each file that
is preloaded, because these things change over time.  And I don't
think it should matter, anyway.  If there are specific problems with
the fact that they are preloaded, they might be bugs or misfeatures
that need to be fixed, rather than relying on documentation of what is
preloaded and letting users figure out what that means.  (I suspect
that most users don't understand the subtleties of this, nor do I
think they need to.)

bug#4128: 23.1; term/ns-win.el does "too much", assumes wrong run order

[Adrian Robert]

Hello,

Thanks for this report.


> (define-key global-map [home] 'beginning-of-buffer)
> (define-key global-map [end] 'end-of-buffer)
> (define-key global-map [kp-home] 'beginning-of-buffer)
> (define-key global-map [kp-end] 'end-of-buffer)
>
> which does change the behavior of the keys to a behavior common on a
> popular modern nextstep-derived system, but with the addendum that  
> it's
> just as common for individual applications to treat those keys in the
> fashion emacs treats them on other platforms.

Can you be more specific?  Are you talking about Home/End = beginning/ 
end of line?  Which other  applications on a "popular modern nextstep- 
derived system" are doing this?  I haven't found any, whereas  
browsers, Terminal, iWork at least go to beginning/end of document.   
But perhaps we should make this change anyway to accomodate those  
coming from a Windows background.



> ;;; Allow shift-clicks to work similarly to under Nextstep
> (define-key global-map [S-mouse-1] 'mouse-save-then-kill)
> (global-unset-key [S-down-mouse-1])
>
> which provides a very surprising behavior that is unlike any modern
> computer that runs something "nextstep derived"

While the name sounds odd, the primary behavior is to create/extend  
the selection, which is common with other apps.  This IS different  
from putting up a font menu on other platforms, but this is a tough  
call since the font panel is accessible via the tools menu and Cmd-t  
already, and the shift-extend-selection behavior is one of the  
oldest / most basic / most common gestures in non-X11 environments.


Regarding ns-power-off, there is some confusion about these bindings;  
they are strictly internally used for passing information between the  
C and lisp levels and don't relate to the power button on some  
keyboards, or to events passed by the OS itself.


The daemon situation IS problematic.  At least the aliases can be  
worked around by using the ns- equivalents.  You can put code  
in .emacs conditional on windowing-system = 'ns or 'mac (or emacs- 
major-version 22/23) to use under multiple emacsen.

bug#4128: 23.1; term/ns-win.el does "too much", assumes wrong run order

[John Prevost]

Adrian Robert <adrian.b.robert@gmail.com> writes:

> I haven't found any, whereas browsers, Terminal, iWork at least go to
> beginning/end of document.  But perhaps we should make this change
> anyway to accomodate those coming from a Windows background.

You are correct--I must have tested against a Microsoft app I was
running at the time, which is of course a questionable source for "how
should things work on a Mac?"  Everything else I've tried treats
home/end consistently as beginning/end-of-file.  The only question,
then, would be "is it better to be consistent with other applications on
this OS, or to to be consistent with Emacs on other OSs?"

>> ;;; Allow shift-clicks to work similarly to under Nextstep
>> (define-key global-map [S-mouse-1] 'mouse-save-then-kill)
>> (global-unset-key [S-down-mouse-1])
>>
>> which provides a very surprising behavior that is unlike any modern
>> computer that runs something "nextstep derived"
>
> While the name sounds odd, the primary behavior is to create/extend  
> the selection, which is common with other apps.  This IS different  
> from putting up a font menu on other platforms, but this is a tough  
> call since the font panel is accessible via the tools menu and Cmd-t  
> already, and the shift-extend-selection behavior is one of the  
> oldest / most basic / most common gestures in non-X11 environments.

Uh.  That's not the behavior I was talking about.  The behavior I'm
concerned about is what happens when you shift-double-click with the
above definition in force.  That is very much *not* normal for any
system I'm familiar with.

And again, the question here is: Is it better to be consistent with
other applications on the host OS, or to be consistent with Emacs on
other OSs?

At the very least, the choice of whether OS trumps Emacs tradition or
Emacs tradition trumps OS should be consistent from OS to OS.

Historically, the "normal" Emacs distribution on MacOS has always
preferred to behave like Emacs on other systems, and the "change
everything to make it more Mac-like and friendly" solution was Aquamacs
(which is why everybody I know avoids it.)

John.