bug#22348: 24.5; Mode hooks doesn't run as expected if the major mode of a new buffer is determined by the default major mode

bug#22348: 24.5; Mode hooks doesn't run as expected if the major mode of a new buffer is determined by the default major mode

[Rolf Ade]

Although reported from my all day working emacs version I see the same
behavior with emacs-25.1.50.2.

Recipe to reproduce the problem:

- Prepare a file with a name/suffix, that isn't matched by anything in
  auto-mode-alist (and interpreter-mode-alist and magic-mode-alist),
  with some content in it. From what I see in my tests, the exact name
  and content doesn't matter (as long as it fullfills the above
  condition). To give an exact example recipe I see it with a file named
  test.abc with the content of (exclude the ""):

"
abc abc abc"

  (Nothing in the default value of auto-mode-alist matches this file
  name test.abc.)

- Create a copy of that file from above with the name test.txt.

  Please note, that the suffix .txt is (beside other ones) handled /
  matched by the default value of auto-mode-alist and points to
  text-mode.

- Start emacs -Q

- In the *scratch* buffer evaluate this lisp code:

(setq-default major-mode 'text-mode)
(add-hook 'text-mode-hook 'end-of-buffer)

  Please note that I'm aware that end-of-buffer sets the mark. That's
  irrelevant for what I want to show. I used end-of-buffer just for
  simplicity of the recipe. If I use a lamba with (goto-char
  (point-max)) I see the same mis-behaviour (see below).

- Open the above prepared file test.abc (just a normal C-x C-f ...)

  Emacs presents me a new buffer, showing the content of the file. The
  buffer has major mode text-mode. (Since nothing out of auto-mode-alist
  et. al. matched, the above set default major mode applies. That's
  expected.) But the point is at the beginning of the buffer, not at the
  end. This is not expected.

- To contrast, now open (also just a normal C-x C-f ...) the second file
  from above, test.txt.

  Emacs presents me another new buffer, showing the content of this file
  (since this file is a copy of the first one, it's exactly the same
  content). The buffer is in major mode text-mode. (No surprise here,
  .txt is handled by the default value of auto-mode-alist.) But the
  point is in this case at the end of the buffer (as expected).

Expected behavior: In both buffers the point should be at the end of the
buffer. Or, to put it more general: the mode hooks of the major mode of
a new buffer should have the same effects for both files.

Futher notes:

- This isn't special to text-mode. I see the same mis-behaviour, if I do
  the recipe from above (appropriate applied) with other major modes.

- Initially, from older days, I had (setq default-major-mode 'text-mode)
  in my init.el, just to learn while analysing the problem, that this
  variable is obsolete since 23.2. The documentation of
  default-major-mode points to major-mode. The (mis-)behaviour is the
  same, regardless which of the both variables I use.

- In general, the hooks do run, it isn't that they just not run in the
  above described circumstances. So, e.g. one is able the enable
  auto-fill-mode by hook.

- One may argue this works as documented. The documentation of
  major-mode includes this (not as cristal clear as it could be, I'd
  comment) paragraph:

 "When a mode is used by default, `find-file' switches to it before it
  reads the contents into the buffer and before it finishes setting up
  the buffer. Thus, the mode and its hooks should not expect certain
  variables such as `buffer-read-only' and `buffer-file-coding-system'
  to be set up."

  This seems to suggest, that (in case the default major mode is used,
  because nothing from auto-mode-alist et. al. matches) the mode hooks
  run before the file content is available in the buffer. So it seems it
  is no surprise, that point movements within some hook doesn't work as
  expected, because there isn't some content, to work on.

  But beside the fact, that this paragraph doesn't seem to tell the
  exact truth about what in fact happens (see another to be written bug
  report - I'll use the title "Documentation of variable major-mode does
  not seem to tell the exactly true") I'd call this a mis-feature. It is
  surprising and confusing to open two different files and then have two
  new buffers, both in the same major mode, but some of the hooks of
  that major mode have different results/effects, for (at least at first
  look) no obvious reason.

  In fact, I stubled over this in the wild. A small configuration change
  that I expected to be done in at max minutes took me hours. Well, this
  is of course not the fault of the emacs developers but completely
  mine, to spend so much time on this. I note this just as a random data
  point, that the current behaviour is confusing und surprising at least
  for some users and therefor not providing the best emacs experience.

- As a partly workaround I've added some more suffixes to the
  auto-mode-alist of my configuration. It may seems possible, to specify
  the wished default major mode with a last entry to auto-mode-alist,
  that matches all and use the form (REGEXP FUNCTION NON-NIL). But this
  seems to be fragil (how to ensure, that entry stays the last one in
  auto-mode-alist?) and much more complicated then setting just
  major-mode (and using such a 'magic' auto-mode-alist entry isn't - at
  least, as far as I see - adviced anywhere).




In GNU Emacs 24.5.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.10)
 of 2015-04-11 on linux-qg7d
Windowing system distributor `The X.Org Foundation', version 11.0.11203000
System Description:	openSUSE 12.2 (x86_64)

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=local
  locale-coding-system: utf-8-unix

Major mode: Outline

Minor modes in effect:
  goto-address-mode: t
  TeX-PDF-mode: t
  diff-auto-refine-mode: t
  shell-dirtrack-mode: t
  desktop-save-mode: t
  show-paren-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t
  view-mode: t

Recent messages:
Quit
C-c C-p is undefined
Mark set
View mode: type C-h for help, h for commands, q to quit.
Mark saved where search started
Quit [2 times]
Mark saved where search started
Type C-x 1 to delete the help window.

Making completion list...

Load-path shadows:
/home/rolf/.emacs.d/elpa/icicles-20151231.1515/icicles hides ~/.emacs.d/site-lisp/icicles

Features:
(benchmark vc-git follow debbugs-org rainbow-mode holidays hol-loaddefs
icomplete fireplace org-capture web-beautify debbugs-browse descr-text
thai-util thai-word lao-util info-look make-mode gnus-draft
expand-region text-mode-expansions cc-mode-expansions
the-org-mode-expansions ruby-mode-expansions
python-el-fgallina-expansions nxml-mode-expansions latex-mode-expansions
web-mode-expansions hippie-exp ox-latex ox-icalendar ox-html ox-ascii
ox-publish ox hungry-delete ert ewoc timezone texinfo toolbar-x reporter
context plain-tex debbugs-gnu add-log debbugs soap-client superword
subword ruby-mode rx python json html-mode-expansions sgml-mode
er-basic-expansions expand-region-core expand-region-custom diff linum
mailalias smtpmail filecache arc-mode archive-mode goto-addr binhex
ps-mule ps-print ps-def lpr url-queue eww mm-url canlock ido calc-misc
calc-aent calc-yank finder rcirc image-file face-remap mb-depth
icicles-mode icicles-face icicles-cmd2 icicles-cmd1 imenu completion
icicles-mcmd image-dired icicles-fn icicles-var icicles-opt cus-theme
filesets icicles-mac cl-macs warnings yow etags cookie1 bookmark recentf
tree-widget cl gv autoload tar-mode lisp-mnt url-http url-gw url-cache
url-auth url url-proxy url-privacy url-expand url-methods url-history
url-cookie url-domsuf url-util url-handlers epg finder-inf time view
org-agenda calc-incom rect sh-script smie executable animate misc nnir
preview prv-emacs tex-buf font-latex latex tex-style tex dbus xml crm
tex-mode latexenc profiler pcase tramp-cmds lunar solar cal-dst
tramp-cache tramp-sh tramp tramp-compat tramp-loaddefs trampver calc-alg
calc-ext calc-menu calc calc-loaddefs calc-macs ffap url-parse url-vars
smerge-mode diff-mode compare-w cal-iso cal-move pp shadow emacsbug
sendmail shell debug shr-color color ibuf-ext ibuffer tabify org-table
grep compile dired-aux flyspell cus-edit ifold two-column iso-transl
eieio-opt speedbar sb-image ezimage dframe gnus-fun flow-fill mule-util
shr browse-url mm-archive sort smiley gnus-cite mail-extr gnus-async
gnus-bcklg qp gnus-ml gnus-topic nndraft nnmh gnutls nnfolder parse-time
netrc network-stream auth-source eieio byte-opt bytecomp byte-compile
cconv eieio-core starttls tls gnus-agent gnus-srvr gnus-score score-mode
nnvirtual gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime smime
password-cache dig mailcap nntp gnus-cache gnus-sum nnoo gnus-group
gnus-undo nnmail mail-source gnus-start gnus-spec gnus-int gnus-range
message rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse
rfc2231 rfc2047 rfc2045 ietf-drums mailabbrev gmm-utils mailheader
gnus-win gnus gnus-ems nnheader mail-utils mm-util mail-prsvr wid-edit
misearch multi-isearch dabbrev thingatpt help-mode org-clock org-colview
cl-extra org-element org-rmail org-mhe org-irc org-info org-gnus
gnus-util org-docview doc-view jka-compr image-mode dired org-bibtex
bibtex org-bbdb org-w3m org org-macro org-footnote org-pcomplete
pcomplete org-list org-faces org-entities org-version ob-emacs-lisp ob
ob-tangle ob-ref ob-lob ob-table ob-exp org-src ob-keys ob-comint
ob-core ob-eval org-compat org-macs org-loaddefs format-spec find-func
cal-menu calendar cal-loaddefs noutline outline easy-mmode tcl comint
ansi-color ring footnote nxml-uchnm rng-xsd xsd-regexp rng-cmpct
rng-nxml rng-valid nxml-mode nxml-outln nxml-rap nxml-glyph tex-site
info icicles-autoloads package epg-config windmove goto-chg key-chord
web-mode advice help-fns derived edmacro kmacro disp-table desktop
frameset rng-loc rng-uri rng-parse rng-match rng-dt rng-util rng-pttrn
nxml-parse nxml-ns nxml-enc xmltok nxml-util delsel cus-start cus-load
vc-fossil vc vc-dispatcher paren server klone-line ispell cc-langs
cl-loaddefs cl-lib cc-mode cc-fonts easymenu cc-guess cc-menus cc-cmds
cc-styles cc-align cc-engine cc-vars cc-defs time-date tooltip electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
dbusbind gfilenotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty emacs)

Memory information:
((conses 16 2847334 1094304)
 (symbols 48 87985 352)
 (miscs 40 6241 8511)
 (strings 32 373670 98708)
 (string-bytes 1 10550494)
 (vectors 16 109922)
 (vector-slots 8 2743525 278850)
 (floats 8 2285 1943)
 (intervals 56 451125 51727)
 (buffers 960 459)
 (heap 1024 234127 39534))

bug#22348: 24.5; Mode hooks doesn't run as expected if the major mode of a new buffer is determined by the default major mode

[Glenn Morris]

Gosh, that's rather a long bug report. :)

If you do:

./src/emacs -Q --eval "(progn (setq-default major-mode 'text-mode)
  (add-hook 'text-mode-hook (lambda () (message \"FOO\") (sit-for 3))))"
  /tmp/foo.bar 

you'll see it works fine. This suggest that the issue is specific to
point-motion (your example used end-of-buffer).

set-auto-mode calls set-buffer-major-mode if nothing else specifies a mode.
set-buffer-major-mode can operate on buffers other than the current one,
and uses save-excursion to preserve buffer. But set-auto-mode calls it
to operate on current-buffer. Thus save-excursion undoes your hook's
point changes. So that's the real issue here.

bug#22348: 24.5; Mode hooks doesn't run as expected if the major mode of a new buffer is determined by the default major mode

[Rolf Ade]

Am 01/11/2016 11:27 PM, Glenn Morris wrote:
> 
> Gosh, that's rather a long bug report. :)

Sorry. Unfortunately I feel a bit uncomfortable to express myself in
English; I'm afraid I can't make my point clear enough.

But you're right. Being longish doesn't help either.

> If you do:
> 
> ./src/emacs -Q --eval "(progn (setq-default major-mode 'text-mode)
>   (add-hook 'text-mode-hook (lambda () (message \"FOO\") (sit-for 3))))"
>   /tmp/foo.bar 
> 
> you'll see it works fine. This suggest that the issue is specific to
> point-motion (your example used end-of-buffer).

Yes, I know, I even mentioned that in the bug report, somewhere within
the endless words ...

I probably should have written better "Some Hooks doesn't run ...".
But that would have been an even longer title ...

> set-auto-mode calls set-buffer-major-mode if nothing else specifies a mode.
> set-buffer-major-mode can operate on buffers other than the current one,
> and uses save-excursion to preserve buffer. But set-auto-mode calls it
> to operate on current-buffer. Thus save-excursion undoes your hook's
> point changes. So that's the real issue here.

There's an explanation for why it behaves as it does, I'm sure. My
main point is it shouldn't behave this way. I think, your "that's the
real issue here" does this say, too, right?

bug#22348: 24.5; Mode hooks doesn't run as expected if the major mode of a new buffer is determined by the default major mode

[Glenn Morris]

I'm saying the solution is probably as simple as:

--- a/src/buffer.c
+++ b/src/buffer.c
@@ -1999,7 +1999,7 @@ the current buffer's major mode.  */)
   /* To select a nonfundamental mode,
      select the buffer temporarily and then call the mode function.  */
 
-  record_unwind_protect (save_excursion_restore, save_excursion_save ());
+  record_unwind_current_buffer ();
 
   Fset_buffer (buffer);
   call0 (function);

bug#22348: 24.5; Mode hooks doesn't run as expected if the major mode of a new buffer is determined by the default major mode

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Mon, 11 Jan 2016 18:16:30 -0500
> Cc: 22348@debbugs.gnu.org
> 
> 
> I'm saying the solution is probably as simple as:
> 
> --- a/src/buffer.c
> +++ b/src/buffer.c
> @@ -1999,7 +1999,7 @@ the current buffer's major mode.  */)
>    /* To select a nonfundamental mode,
>       select the buffer temporarily and then call the mode function.  */
>  
> -  record_unwind_protect (save_excursion_restore, save_excursion_save ());
> +  record_unwind_current_buffer ();
>  
>    Fset_buffer (buffer);
>    call0 (function);

I think we should do this on master.

bug#22348: 24.5; Mode hooks doesn't run as expected if the major mode of a new buffer is determined by the default major mode

[Glenn Morris]

Version: 25.2

>> -  record_unwind_protect (save_excursion_restore, save_excursion_save ());
>> +  record_unwind_current_buffer ();

> I think we should do this on master.

OK.

bug#22348: 24.5; Mode hooks doesn't run as expected if the major mode of a new buffer is determined by the default major mode

[Michael Heerdegen]

Glenn Morris <rgm@gnu.org> writes:

> I'm saying the solution is probably as simple as:
>
> --- a/src/buffer.c
> +++ b/src/buffer.c
> @@ -1999,7 +1999,7 @@ the current buffer's major mode.  */)
>    /* To select a nonfundamental mode,
>       select the buffer temporarily and then call the mode function.  */
>  
> -  record_unwind_protect (save_excursion_restore, save_excursion_save ());
> +  record_unwind_current_buffer ();
>  
>    Fset_buffer (buffer);
>    call0 (function);

I can confirm that this fixes the issue.


Michael.