bug#40375: 27.0.50; yank unexpectedly depends on progn

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Markus Triska]

Suppose yank.el comprises the following forms:

    (erase-buffer)
    (let* ((size 144)
           (data (with-temp-buffer
                   (insert (format "P1\n%s %s\n" size size))
                   (dotimes (_ size)
                     (dotimes (_ size)
                       (insert "1 "))
                     (insert "\n"))
                   (buffer-string))))
      (insert-image `(image :type pbm :data ,data :ascent center) "t"))
    (kill-new (buffer-string))
    (erase-buffer)
    (yank)

When I start Emacs with:

    $ emacs -Q yank.el --eval='(eval-buffer)'

then Emacs starts, and I see an empty buffer.

However, when I embed the entire contents of yank.el in (progn ...),
i.e., if yank.el consists of the following form:

    (progn
      (erase-buffer)
      (let* ((size 144)
             (data (with-temp-buffer
                     (insert (format "P1\n%s %s\n" size size))
                     (dotimes (_ size)
                       (dotimes (_ size)
                         (insert "1 "))
                       (insert "\n"))
                     (buffer-string))))
        (insert-image `(image :type pbm :data ,data :ascent center) "t"))
      (kill-new (buffer-string))
      (erase-buffer)
      (yank))

and I then start Emacs (again) with:

    $ emacs -Q yank.el --eval='(eval-buffer)'

Then I see a black box in the buffer.

Hence, what (yank) inserts seems to unexpectedly depend on whether progn
is used. Is there a way to get the same result also if progn is not used?

Thank you and all the best!
Markus



In GNU Emacs 27.0.50 (build 1, x86_64-apple-darwin18.0.0, X toolkit, Xaw scroll bars)
 of 2018-11-15 built on mts-mac
Repository revision: b4eb908f858284a7962851fd99c94598f76afa6f
Windowing system distributor 'The X.Org Foundation', version 11.0.11804000
System Description:  Mac OS X 10.14.2

Configured using:
 'configure --prefix=/opt/local --without-ns --without-dbus
 --without-gconf --without-libotf --without-m17n-flt --without-gpm
 --with-gnutls --with-xml2 --with-modules --infodir
 /opt/local/share/info/emacs --with-json --with-x-toolkit=lucid
 --without-xaw3d --without-imagemagick --with-xpm --with-jpeg
 --with-tiff --with-gif --with-png --with-lcms2 --without-rsvg
 --with-xft 'CFLAGS=-pipe -Os
 -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
 -arch x86_64' 'CPPFLAGS=-I/opt/local/include
 -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk'
 'LDFLAGS=-L/opt/local/lib -Wl,-headerpad_max_install_names -lfreetype
 -lfontconfig -Wl,-no_pie
 -Wl,-syslibroot,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
 -arch x86_64''

Configured features:
XPM JPEG TIFF GIF PNG NOTIFY KQUEUE ACL GNUTLS LIBXML2 FREETYPE XFT ZLIB
TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS JSON LCMS2 GMP

Important settings:
  value of $LC_CTYPE: UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Noam Postavsky]

Markus Triska <triska@metalevel.at> writes:


>     $ emacs -Q yank.el --eval='(eval-buffer)'

A simpler reproducer is

    (erase-buffer)
    (message "hello")

will print nothing, whereas

    (progn (erase-buffer)
           (message "hello"))

prints "hello".  The problem is the first case erases the code which
calls message before it is read in.

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Markus Triska]

Noam Postavsky <npostavs@gmail.com> writes:

> prints "hello".  The problem is the first case erases the code which
> calls message before it is read in.

OK, here is a better test case. Let yank.el consist of the single form:

   (let ((data (base64-decode-string
   "iVBORw0KGgoAAAANSUhEUgAAAA4AAAAbBAMAAAC+faPKAAAALVBMVEX///8AAACgoKDg4OBAQEDA
   wMAQEBCQkJAgICCAgIBwcHAwMDBgYGDQ0NDw8PALucPOAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA
   X0lEQVQIHWNgYGBQBmIQcIVQDKkQmlkUQrNDaQ4oPRFCswuCaatGQWkXFwcGy5mNMjNnTgBqcYSq
   o4Q2gpoTgoVeKMTAUAUUvygDcTKXYACTA5DPoCG95QGIZjC9wAAAJ0UR8BmY4ewAAAAASUVORK5C
   YII=")))
     (kill-new
      (propertize data 'display `(image :type png :data ,data :ascent center))))

When I start Emacs with:

    $ emacs -Q yank.el --eval='(eval-buffer)'

and then press C-y to yank, the image that was most recently killed via
kill-new is unexpectedly not inserted. However, when I invoke Emacs
with the form wrapped in (progn ...), followed by (yank), i.e. with:

    $ emacs -Q yank.el --eval='(progn (eval-buffer) (yank))'

then the image is indeed inserted.

Why is there this difference? Thank you a lot for looking into this!

All the best,
Markus

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Noam Postavsky]

Markus Triska <triska@metalevel.at> writes:
>
>    (let ((data (base64-decode-string
>    "iVBORw0KGgoAAAANSUhEUgAAAA4AAAAbBAMAAAC+faPKAAAALVBMVEX///8AAACgoKDg4OBAQEDA
>    wMAQEBCQkJAgICCAgIBwcHAwMDBgYGDQ0NDw8PALucPOAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA
>    X0lEQVQIHWNgYGBQBmIQcIVQDKkQmlkUQrNDaQ4oPRFCswuCaatGQWkXFwcGy5mNMjNnTgBqcYSq
>    o4Q2gpoTgoVeKMTAUAUUvygDcTKXYACTA5DPoCG95QGIZjC9wAAAJ0UR8BmY4ewAAAAASUVORK5C
>    YII=")))
>      (kill-new
>       (propertize data 'display `(image :type png :data ,data :ascent center))))
>
> When I start Emacs with:
>
>     $ emacs -Q yank.el --eval='(eval-buffer)'
>
> and then press C-y to yank, the image that was most recently killed via
> kill-new is unexpectedly not inserted.

I'm not able to reproduce this, the image (of a lower case t) yanks just
fine for me.

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Noam Postavsky]

tags 40375 unreproducible
close 40375
quit

Noam Postavsky <npostavs@gmail.com> writes:

> Markus Triska <triska@metalevel.at> writes:
>>
>>    (let ((data (base64-decode-string
>>    "iVBORw0KGgoAAAANSUhEUgAAAA4AAAAbBAMAAAC+faPKAAAALVBMVEX///8AAACgoKDg4OBAQEDA
>>    wMAQEBCQkJAgICCAgIBwcHAwMDBgYGDQ0NDw8PALucPOAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA
>>    X0lEQVQIHWNgYGBQBmIQcIVQDKkQmlkUQrNDaQ4oPRFCswuCaatGQWkXFwcGy5mNMjNnTgBqcYSq
>>    o4Q2gpoTgoVeKMTAUAUUvygDcTKXYACTA5DPoCG95QGIZjC9wAAAJ0UR8BmY4ewAAAAASUVORK5C
>>    YII=")))
>>      (kill-new
>>       (propertize data 'display `(image :type png :data ,data :ascent center))))
>>
>> When I start Emacs with:
>>
>>     $ emacs -Q yank.el --eval='(eval-buffer)'
>>
>> and then press C-y to yank, the image that was most recently killed via
>> kill-new is unexpectedly not inserted.
>
> I'm not able to reproduce this, the image (of a lower case t) yanks just
> fine for me.

No further response, so I'm closing the bug.

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Markus Triska]

Noam Postavsky <npostavs@gmail.com> writes:

> No further response, so I'm closing the bug.

I can still reproduce it. Is there anything else I can provide to help
with this issue?

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Noam Postavsky]

Markus Triska <triska@metalevel.at> writes:

> Noam Postavsky <npostavs@gmail.com> writes:
>
>> No further response, so I'm closing the bug.
>
> I can still reproduce it. Is there anything else I can provide to help
> with this issue?

Oh, hmm.  Could this be macOS specific?  Does this happen only on
27.0.50?  What about the latest 27.0.90 pretest?  26.3?

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Andreas Schwab]

On Apr 01 2020, Markus Triska wrote:

> When I start Emacs with:
>
>     $ emacs -Q yank.el --eval='(eval-buffer)'
>
> and then press C-y to yank, the image that was most recently killed via
> kill-new is unexpectedly not inserted. However, when I invoke Emacs
> with the form wrapped in (progn ...), followed by (yank), i.e. with:
>
>     $ emacs -Q yank.el --eval='(progn (eval-buffer) (yank))'
>
> then the image is indeed inserted.

What does your interprogram-paste-function return?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Markus Triska]

Andreas Schwab <schwab@linux-m68k.org> writes:

> What does your interprogram-paste-function return?

Its value is `gui-selection-value'.

I have now repeatedly tried it, and sometimes it indeed works as
expected (i.e., the image is inserted), but I can reliably trigger the
problem when I:

1. open Firefox and browse to the description of this bug, i.e.:
   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=403752
2. click on the URL bar, so that the text in the URL becomes selected
3. switch to a terminal application using only the keyboard, and then,
   from the terminal, perform the steps I outlined.

In this case, the text that is inserted is not the expected image that
was most recently killed in Emacs, but rather the URL from Firefox that
is selected in that application.

I can reproduce this with XQuartz 1.18.4 (xorg-server 1.18.4) on OSX.

Is it possible that yanking in Emacs does not work as expected when text
is selected in other applications on this platform?

Once this issue occurs, what is yanked is no longer predictable for me:
Sometimes it is text that is selected in other applications, sometimes
it is text that was copied to the clipboard from other applications, but
is no longer selected. It feels like there is no clear pattern.

What I would prefer in all these cases is that Emacs yank the text that
was most recently killed in this example, which is the sample image that
you can reproduce.

Thank you and all the best,
Markus

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Eli Zaretskii]

> From: Markus Triska <triska@metalevel.at>
> Date: Mon, 13 Apr 2020 20:01:13 +0200
> Cc: 40375@debbugs.gnu.org, Noam Postavsky <npostavs@gmail.com>
> 
> Once this issue occurs, what is yanked is no longer predictable for me:
> Sometimes it is text that is selected in other applications, sometimes
> it is text that was copied to the clipboard from other applications, but
> is no longer selected. It feels like there is no clear pattern.

Do you have some clipboard-managing software installed and active?  If
so, can you disable it?

> What I would prefer in all these cases is that Emacs yank the text that
> was most recently killed in this example, which is the sample image that
> you can reproduce.

When the system clipboard holds stuff from another application, C-y
will by default yank that first.  If some software on your system
rearranges the clipboard in some way that makes Emacs think the stuff
in the clipboard was copied there after you killed something in Emacs,
Emacs will behave as you describe.

IOW, I don't believe this is an Emacs problem.

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Markus Triska]

Eli Zaretskii <eliz@gnu.org> writes:

> Do you have some clipboard-managing software installed and active?  If
> so, can you disable it?

To the best of my knowledge, I have no such tool installed or active.

> When the system clipboard holds stuff from another application, C-y
> will by default yank that first.  If some software on your system
> rearranges the clipboard in some way that makes Emacs think the stuff
> in the clipboard was copied there after you killed something in Emacs,
> Emacs will behave as you describe.
>
> IOW, I don't believe this is an Emacs problem.

From the documentation of `yank', I got the impression that the most
recently killed text should reliably be inserted in the example I
posted, because the description of `yank' describes it as:

   Reinsert ("paste") the last stretch of killed text.
   More precisely, reinsert the most recent kill, which is the
   stretch of killed text most recently killed OR yanked.

However, from your description, it seems this documentation is not
accurate, because the clipboard may unexpcetedly also play a role.

Could you please consider updating the documentation if necessary?
Alternatively, is there a way to get `yank' to do what this states,
i.e., to always insert the most recently killed or yanked text,
disregarding the clipboard contents of the operating platform?

Thank you and all the best,
Markus

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Eli Zaretskii]

> From: Markus Triska <triska@metalevel.at>
> Cc: schwab@linux-m68k.org,  40375@debbugs.gnu.org,  npostavs@gmail.com
> Date: Mon, 13 Apr 2020 20:52:48 +0200
> 
> Alternatively, is there a way to get `yank' to do what this states,
> i.e., to always insert the most recently killed or yanked text,
> disregarding the clipboard contents of the operating platform?

Yes, set interprogram-paste-function to nil.

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Markus Triska]

Eli Zaretskii <eliz@gnu.org> writes:

> Yes, set interprogram-paste-function to nil.

Perfect, this makes the test case work as intended!

Please consider documenting this in the description of `yank'. Other
than that, I consider this issue resolved.

Thank you a lot!

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Noam Postavsky]

Markus Triska <triska@metalevel.at> writes:

> Please consider documenting this in the description of `yank'. Other
> than that, I consider this issue resolved.

Does this look okay?  (modulo a fill-paragraph which I've held off on
just to make the patch easier to read)

--- i/lisp/simple.el
+++ w/lisp/simple.el
@@ -5110,7 +5110,9 @@ yank-pop
 (defun yank (&optional arg)
   "Reinsert (\"paste\") the last stretch of killed text.
 More precisely, reinsert the most recent kill, which is the
-stretch of killed text most recently killed OR yanked.  Put point
+stretch of text most recently killed or yanked.  Or text from the
+system clipboard if that was modified more recently (see
+`interprogram-paste-function').  Put point
 at the end, and set mark at the beginning without activating it.
 With just \\[universal-argument] as argument, put point at beginning, and mark at end.
 With argument N, reinsert the Nth most recent kill.

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Drew Adams]

> Does this look okay?  (modulo a fill-paragraph which I've held off on
> just to make the patch easier to read)
> 
> --- i/lisp/simple.el
> +++ w/lisp/simple.el
> @@ -5110,7 +5110,9 @@ yank-pop
>  (defun yank (&optional arg)
>    "Reinsert (\"paste\") the last stretch of killed text.
>  More precisely, reinsert the most recent kill, which is the
> -stretch of killed text most recently killed OR yanked.  Put point
> +stretch of text most recently killed or yanked.  Or text from the
> +system clipboard if that was modified more recently (see
> +`interprogram-paste-function').  Put point
>  at the end, and set mark at the beginning without activating it.
>  With just \\[universal-argument] as argument, put point at beginning,
> and mark at end.
>  With argument N, reinsert the Nth most recent kill.

Sorry for butting in here; just happened to see this in passing.

Till now, it was OK to talk only about killing or yanking, as the source of the text to be yanked ("the most recent kill").

But there's also copying text to the kill ring, as if it were killed.  And putting text on the kill ring is analogous to putting it onto the system clipboard.  In both cases you can get it to that saved-text place by killing/cutting OR by copying.

So to be parallel in the doc maybe we should say "text from the kill-ring or from the system clipboard if..."

Users should have a good sense that the same kind of thing is involved: text gets put on some save location (kill ring or clipboard), and then it gets yanked from there.  With your text above, it seems like only killing and yanking is used for the kill ring, but killing or copying is used for the clipboard.

(If what I'm saying isn't clear to you, ignore it.  It's not a big deal.)

bug#40375: 27.0.50; yank unexpectedly depends on progn

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  40375@debbugs.gnu.org,
>   schwab@linux-m68k.org
> Date: Mon, 13 Apr 2020 18:16:26 -0400
> 
> Markus Triska <triska@metalevel.at> writes:
> 
> > Please consider documenting this in the description of `yank'. Other
> > than that, I consider this issue resolved.
> 
> Does this look okay?  (modulo a fill-paragraph which I've held off on
> just to make the patch easier to read)

Thanks, I fixed this in a slightly different manner (as just
mentioning interprogram-paste-function may not be accurate enough).