bug#35177: 27.0.50; Binding deactivate-mark to nil unexpectedly deactivates the mark

bug#35177: 27.0.50; Binding deactivate-mark to nil unexpectedly deactivates the mark

[Markus Triska]

Please start Emacs with "emacs -Q" and insert the following form in the
*scratch* buffer:

    (progn
      (let (deactivate-mark)
        (switch-to-buffer (get-buffer-create "t"))
        (erase-buffer)
        (insert "hello"))
      (let (deactivate-mark)
        (set-mark-command nil)
        (move-beginning-of-line nil)))

then place point at the end of the form and evaluate it with C-x C-e.

You will see a new buffer ("t"), where "hello" is inserted and nothing
is highlighted because the mark is inactive.

However, I expect the mark to be active and the region highlighted,
because the form binds deactivate-mark to nil. This is the way that is
recommended in the Elisp manual, see for Example 31.7 "The Mark":

     To write Lisp code that modifies the buffer without causing
     deactivation of the mark at the end of the command, bind
     ‘deactivate-mark’ to ‘nil’ around the code that does the
     modification.  For example:

          (let (deactivate-mark)
            (insert " "))


When I remove the second let-binding in the snippet, then the mark is in
fact active after the form is evaluated. For example, when I evaluate:

    (let (deactivate-mark)
      (switch-to-buffer (get-buffer-create "t"))
      (erase-buffer)
      (insert "hello")
      (set-mark-command nil)
      (move-beginning-of-line nil))

then "hello" is highlighted.

Hence, if I bind deactivate-mark to nil, then the mark is in some cases
unexpectedly deactivated.

As another test case, here is the first form bound to F4:

    (global-set-key [f4]
                    (lambda ()
                      (interactive)
                      (progn
                        (let (deactivate-mark)
                          (switch-to-buffer (get-buffer-create "t"))
                          (erase-buffer)
                          (insert "hello"))
                        (let (deactivate-mark)
                          (set-mark-command nil)
                          (move-beginning-of-line nil)))))

with this binding in place, when I press F4, I also expect "hello" to be
highlighted, since this form binds deactivate-mark to nil. However,
"hello" is unexpectedly not highlighted when I then press F4.

In contrast, when I evaluate the first form with C-M-x, then the region
is highlighted. Could you please make C-x C-e and F4 (with this binding
in place) give the same result as using C-M-x on the form?

Thank you and all the best!
Markus


In GNU Emacs 27.0.50 (build 1, x86_64-apple-darwin18.0.0, X toolkit, Xaw scroll bars)
 of 2018-11-15 built on mt-computer
Repository revision: b4eb908f858284a7962851fd99c94598f76afa6f
Windowing system distributor 'The X.Org Foundation', version 11.0.11804000
System Description:  Mac OS X 10.14.2

bug#35177: 27.0.50; Binding deactivate-mark to nil unexpectedly deactivates the mark

[Eli Zaretskii]

> From: Markus Triska <triska@metalevel.at>
> Date: Sun, 07 Apr 2019 02:11:02 +0200
> 
> Please start Emacs with "emacs -Q" and insert the following form in the
> *scratch* buffer:
> 
>     (progn
>       (let (deactivate-mark)
>         (switch-to-buffer (get-buffer-create "t"))
>         (erase-buffer)
>         (insert "hello"))
>       (let (deactivate-mark)
>         (set-mark-command nil)
>         (move-beginning-of-line nil)))
> 
> then place point at the end of the form and evaluate it with C-x C-e.
> 
> You will see a new buffer ("t"), where "hello" is inserted and nothing
> is highlighted because the mark is inactive.
> 
> However, I expect the mark to be active and the region highlighted,
> because the form binds deactivate-mark to nil. This is the way that is
> recommended in the Elisp manual, see for Example 31.7 "The Mark":
> 
>      To write Lisp code that modifies the buffer without causing
>      deactivation of the mark at the end of the command, bind
>      ‘deactivate-mark’ to ‘nil’ around the code that does the
>      modification.  For example:
> 
>           (let (deactivate-mark)
>             (insert " "))
> 

I think you need to put the entire progn inside the binding of
deactivate-mark.  IOW, the binding should be in effect when the
command terminates.

bug#35177: 27.0.50; Binding deactivate-mark to nil unexpectedly deactivates the mark

[Markus Triska]

Eli Zaretskii <eliz@gnu.org> writes:

> I think you need to put the entire progn inside the binding of
> deactivate-mark.  IOW, the binding should be in effect when the
> command terminates.

Please consider the following additional test cases: Both of the
following forms keep the mark activated, while neither of them has the
binding in effect when the commond terminates.

Form 1:

    (progn
      (let (deactivate-mark)
        (switch-to-buffer (get-buffer-create "t"))
        (erase-buffer)
        (insert "hello"))
      (set-mark-command nil)
      (move-beginning-of-line nil))

Form 2:

    (progn
      (let (deactivate-mark)
        (switch-to-buffer (get-buffer-create "t"))
        (erase-buffer)
        (insert "hello"))
      (let (deactivate-mark-1)
        (set-mark-command nil)
        (move-beginning-of-line nil)))

So, it seems very unexpected to me that an additional let-binding of
deactivate-mark (to nil) causes deactivation of the mark, whereas both
of these forms keep the mark activated (when evaluated with C-x C-e).

If anything, I expect an additional let-binding of deactivate-mark to
nil to retain the active mark "all the more". In other words, if:

    (progn
      (let (deactivate-mark)
        (switch-to-buffer (get-buffer-create "t"))
        (erase-buffer)
        (insert "hello"))
      (let (deactivate-mark-1)
        (set-mark-command nil)
        (move-beginning-of-line nil)))

keeps the mark activated, then I expect the following to do too:

    (progn
      (let (deactivate-mark)
        (switch-to-buffer (get-buffer-create "t"))
        (erase-buffer)
        (insert "hello"))
      (let (deactivate-mark)
        (set-mark-command nil)
        (move-beginning-of-line nil)))

Note that I only changed deactivate-mark-1 to deactivate-mark.

So, in other words, binding deactivate-mark to nil deactivates the mark,
which is the opposite of what I expect it to do.

bug#35177: 27.0.50; Binding deactivate-mark to nil unexpectedly deactivates the mark

[Eli Zaretskii]

> From: Markus Triska <triska@metalevel.at>
> Cc: 35177@debbugs.gnu.org
> Date: Mon, 08 Apr 2019 00:39:04 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I think you need to put the entire progn inside the binding of
> > deactivate-mark.  IOW, the binding should be in effect when the
> > command terminates.
> 
> Please consider the following additional test cases: Both of the
> following forms keep the mark activated, while neither of them has the
> binding in effect when the commond terminates.

None of your additional test cases do what I said I thought you should
do, though, do they?

> So, in other words, binding deactivate-mark to nil deactivates the mark,
> which is the opposite of what I expect it to do.

I don't think it's something in your code that deactivates the mark,
it's what Emacs does internally when a command finishes.

bug#35177: 27.0.50; Binding deactivate-mark to nil unexpectedly deactivates the mark

[Markus Triska]

Eli Zaretskii <eliz@gnu.org> writes:

> None of your additional test cases do what I said I thought you should
> do, though, do they?

In emacs -Q, please evaluate the following form with C-x C-e:

    (let (deactivate-mark)
      (switch-to-buffer (get-buffer-create "t"))
      (insert "hello")
      (set-mark-command nil)
      (move-beginning-of-line nil))

This does what you say. However, the mark is deactivated after the form
is executed, which is the opposite of what I expect from this snippet.

> I don't think it's something in your code that deactivates the mark,
> it's what Emacs does internally when a command finishes.

I noticed that the mark stays activate (as expected), when I add a
"redisplay" after switch-to-buffer:

    (let (deactivate-mark)
      (switch-to-buffer (get-buffer-create "t"))
      (redisplay)
      (insert "hello")
      (set-mark-command nil)
      (move-beginning-of-line nil))

After I evaluate this, the first form also consistently retains the
active mark when I evaluate it. It is expected that the mark stays
active, but it is unexpected that it does so after I evaluate the second
form, yet is deactivated if I only evaluate the first form.

Please also note that the documentation of deactivate-mark states
"around the code that does the modification". If other preconditions are
necessary to consistently retain the active mark, could you please
document them? Thank you.

bug#35177: 27.0.50; Binding deactivate-mark to nil unexpectedly deactivates the mark

[Eli Zaretskii]

> From: Markus Triska <triska@metalevel.at>
> Cc: 35177@debbugs.gnu.org
> Date: Mon, 08 Apr 2019 12:03:11 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > None of your additional test cases do what I said I thought you should
> > do, though, do they?
> 
> In emacs -Q, please evaluate the following form with C-x C-e:
> 
>     (let (deactivate-mark)
>       (switch-to-buffer (get-buffer-create "t"))
>       (insert "hello")
>       (set-mark-command nil)
>       (move-beginning-of-line nil))
> 
> This does what you say. However, the mark is deactivated after the form
> is executed, which is the opposite of what I expect from this snippet.

I cannot reproduce your result: the above does show an active mark
after the execution, both in Emacs 26.2 and Emacs 27.0.50.

> Please also note that the documentation of deactivate-mark states
> "around the code that does the modification". If other preconditions are
> necessary to consistently retain the active mark, could you please
> document them? Thank you.

You seem to think that I know something you don't, and just withhold
it.  But that's not so: I'm exactly like you, trying to make sense of
the behavior and the documentation.  I just arrived at slightly
different conclusions, and saw that by modifying your examples I can
get the behavior described by the docs, while I cannot reproduce some
of your results, that's all.

bug#35177: 27.0.50; Binding deactivate-mark to nil unexpectedly deactivates the mark

[Markus Triska]

Eli Zaretskii <eliz@gnu.org> writes:

> I cannot reproduce your result: the above does show an active mark
> after the execution, both in Emacs 26.2 and Emacs 27.0.50.

Here is a more complete recipe, could you please try it:

  1. download deactivate_mark.el with
     $ wget https://www.metalevel.at/ei/deactivate_mark.el

  2. start Emacs with
     $ emacs -Q deactivate_mark.el

  3. move point at the end of the form and evaluate it with C-x C-e.

The result is that "hello" is displayed in buffer "t", and the mark is
not active. The content of deactivate_mark.el is:

(let (deactivate-mark)
  (switch-to-buffer (get-buffer-create "t"))
  (insert "hello")
  (set-mark-command nil)
  (move-beginning-of-line nil))

It seems to play a role whether its buffer is modified or not, hence
this receipe starting from a fresh file and an unmodified buffer.

Can you reproduce this up to here? If so, please proceed as follows:

  4. Kill the "t" buffer with C-x C-k RET

  5. Add a space to the end of deactivate_mark.el by pressing SPACE.

  6. Evaluate the form again with C-x C-e.

After these steps, "hello" is highlighted and the mark is active. Also,
on all further iterations of (4) to (6), the region is again active.

Can you reproduce this? If possible, could you please make it so that
the mark is active also on the first run, i.e., after steps (1) to (3)?

Thank you and all the best,
Markus

bug#35177: 27.0.50; Binding deactivate-mark to nil unexpectedly deactivates the mark

[Eli Zaretskii]

> From: Markus Triska <triska@metalevel.at>
> Cc: 35177@debbugs.gnu.org
> Date: Mon, 08 Apr 2019 17:50:10 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I cannot reproduce your result: the above does show an active mark
> > after the execution, both in Emacs 26.2 and Emacs 27.0.50.
> 
> Here is a more complete recipe, could you please try it:
> 
>   1. download deactivate_mark.el with
>      $ wget https://www.metalevel.at/ei/deactivate_mark.el
> 
>   2. start Emacs with
>      $ emacs -Q deactivate_mark.el
> 
>   3. move point at the end of the form and evaluate it with C-x C-e.
> 
> The result is that "hello" is displayed in buffer "t", and the mark is
> not active. The content of deactivate_mark.el is:
> 
> (let (deactivate-mark)
>   (switch-to-buffer (get-buffer-create "t"))
>   (insert "hello")
>   (set-mark-command nil)
>   (move-beginning-of-line nil))
> 
> It seems to play a role whether its buffer is modified or not, hence
> this receipe starting from a fresh file and an unmodified buffer.
> 
> Can you reproduce this up to here? If so, please proceed as follows:
> 
>   4. Kill the "t" buffer with C-x C-k RET
> 
>   5. Add a space to the end of deactivate_mark.el by pressing SPACE.
> 
>   6. Evaluate the form again with C-x C-e.
> 
> After these steps, "hello" is highlighted and the mark is active. Also,
> on all further iterations of (4) to (6), the region is again active.
> 
> Can you reproduce this?

Yes.

> If possible, could you please make it so that the mark is active
> also on the first run, i.e., after steps (1) to (3)?

Not sure.

bug#35177: 27.0.50; Binding deactivate-mark to nil unexpectedly deactivates the mark

[Noam Postavsky]

Markus Triska <triska@metalevel.at> writes:

> (let (deactivate-mark)
>   (switch-to-buffer (get-buffer-create "t"))
>   (insert "hello")
>   (set-mark-command nil)
>   (move-beginning-of-line nil))
>
> It seems to play a role whether its buffer is modified or not, hence
> this receipe starting from a fresh file and an unmodified buffer.

I think the key detail is whether deactivate-mark has a buffer-local
value or not (modifying the buffer gives it one).  If it does have a
buffer-local value, then the let-binding affects just the buffer-local
one, otherwise, it binds the global value.  So C-x C-e on the following
always gives an active mark:

(progn
  (make-local-variable 'deactivate-mark)
  ;; (kill-local-variable 'deactivate-mark)
  (let (deactivate-mark)
    (switch-to-buffer (get-buffer-create "t"))
    (insert "hello")
    (set-mark-command nil)
    (move-beginning-of-line nil)))

And if you uncomment the kill-local-variable statement, then C-x C-e
will never give an active mark.

C-M-x and M-x eval-region still do give an active mark for all variants.
I'm not entirely sure why, but I'm guessing it's because they juggle
buffers in a slightly different way, such that the command loop ends up
consulting the value of deactivate-mark from somewhere else.

bug#35177: 27.0.50; Binding deactivate-mark to nil unexpectedly deactivates the mark

[Markus Triska]

Noam Postavsky <npostavs@gmail.com> writes:

> I think the key detail is whether deactivate-mark has a buffer-local
> value or not (modifying the buffer gives it one).  If it does have a
> buffer-local value, then the let-binding affects just the buffer-local
> one, otherwise, it binds the global value.  So C-x C-e on the following
> always gives an active mark:

Thank you very much for looking into this!

I have constructed an additional test case, which in a sense complements
the previous one in that the mark is now active on the first run, but
inactive in subsequent runs. To reproduce it, please do the following:

1. Download mark_test_run.el with:
   $ wget https://www.metalevel.at/ei/mark_test_run.el

2. Invoke Emacs with:
   $ emacs -Q mark_test_run.el

3. In that Emacs instance, please do:

   M-x evaluate-buffer RET M-x mark-test-run RET

   After this, "hello" is displayed and the mark is active.

4. Kill the current buffer (i.e., "b") with C-x k RET

5. Invoke mark-test-run again with M-x mark-test-run RET

   After this, "hello" is displayed and the mark is *inactive*.

On subsequent invocations of mark-test-run, the mark is likewise
inactive. However, I expect the mark to be active in all runs, due to
the structure of mark-test-run, which binds deactivate-mark to nil:

    (defun mark-test-run ()
      (interactive)
      (let (deactivate-mark)
	...))

Can you reproduce this?

This seems related to the current issue. However, if you consider this
unrelated, please let me know, and I will file this as a separate issue.

For completeness, I include the content of mark_test_run.el below.

All the best,
Markus


(defvar mark-test-forms
  '((new-buffer "b")
    (insert "hello")
    (set-mark-command)
    (move-beginning-of-line)))

(defun mark-test-run ()
  (interactive)
  (let (deactivate-mark)
    (mapc 'mark-test-interpret mark-test-forms)))


(defun mark-test-interpret (expr)
  (interactive)
  (let ((e (car expr)))
    (cond ((eq e 'insert)
           (insert (cadr expr)))
          ((eq e 'move-beginning-of-line)
           (move-beginning-of-line nil))
          ((eq e 'new-buffer)
           (let ((buf (get-buffer-create (cadr expr))))
             (with-current-buffer buf
               (erase-buffer))
             (switch-to-buffer buf)))
          ((eq e 'set-mark-command)
           (let ((inhibit-message t))
             (set-mark-command nil))))))

bug#35177: 27.0.50; Binding deactivate-mark to nil unexpectedly deactivates the mark

[Noam Postavsky]

Markus Triska <triska@metalevel.at> writes:

> 3. In that Emacs instance, please do:
>
>    M-x evaluate-buffer RET M-x mark-test-run RET
>
>    After this, "hello" is displayed and the mark is active.

> Can you reproduce this?

Sort of.  There is some interaction with the suggest-key-bindings
feature.  On the first run, the mark stays active until the message with
the suggestion goes away, and only then gets deactivated.

    You can run the command ‘mark-test-run’ with M-x ma-t RET

If set suggest-key-bindings to nil, or type in exactly M-x ma-t RET,
then the mark is deactivated the first time too.

> This seems related to the current issue. However, if you consider this
> unrelated, please let me know, and I will file this as a separate issue.

Yeah, I think it's the same issue.  If I include (setq
suggest-key-bindings nil) and (make-local-variable 'deactivate-mark)
then the mark stays active consistently.