X selection access in xterm (OSC 52)

X selection access in xterm (OSC 52)

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 334 bytes --]

As described in http://invisible-island.net/xterm/ctlseqs/ctlseqs.html,
xterm supports escape sequences to read or change the X clipboard. This has
been implemented in HTerm, a JavaScript-based terminal emulator:
https://github.com/chromium/hterm/blob/master/etc/osc52.el
Would it make sense to integrate this into xterm.el directly?

[-- Attachment #2: Type: text/html, Size: 510 bytes --]

Re: X selection access in xterm (OSC 52)

[Eli Zaretskii]

> From: Philipp Stephani <p.stephani2@gmail.com>
> Date: Sun, 08 Feb 2015 18:18:23 +0000
> 
> As described in http://invisible-island.net/xterm/ctlseqs/ctlseqs.html, xterm
> supports escape sequences to read or change the X clipboard. This has been
> implemented in HTerm, a JavaScript-based terminal emulator:
> https://github.com/chromium/hterm/blob/master/etc/osc52.el
> Would it make sense to integrate this into xterm.el directly?

Isn't it already implemented on master?

Re: X selection access in xterm (OSC 52)

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

Eli Zaretskii <eliz@gnu.org> schrieb am Sun Feb 08 2015 at 19:43:21:

> > From: Philipp Stephani <p.stephani2@gmail.com>
> > Date: Sun, 08 Feb 2015 18:18:23 +0000
> >
> > As described in http://invisible-island.net/xterm/ctlseqs/ctlseqs.html,
> xterm
> > supports escape sequences to read or change the X clipboard. This has
> been
> > implemented in HTerm, a JavaScript-based terminal emulator:
> > https://github.com/chromium/hterm/blob/master/etc/osc52.el
> > Would it make sense to integrate this into xterm.el directly?
>
> Isn't it already implemented on master?
>

Which file? I can't find it in
http://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/term/xterm.el. (The
xterm-paste is something different.)

[-- Attachment #2: Type: text/html, Size: 1359 bytes --]

Re: X selection access in xterm (OSC 52)

[Stefan Monnier]

> Would it make sense to integrate this into xterm.el directly?

Yes.


        Stefan

Re: X selection access in xterm (OSC 52)

[Yuri Khan]

On Mon, Feb 9, 2015 at 12:18 AM, Philipp Stephani <p.stephani2@gmail.com> wrote:
> As described in http://invisible-island.net/xterm/ctlseqs/ctlseqs.html,
> xterm supports escape sequences to read or change the X clipboard. This has
> been implemented in HTerm, a JavaScript-based terminal emulator:
> https://github.com/chromium/hterm/blob/master/etc/osc52.el
> Would it make sense to integrate this into xterm.el directly?

It’s a nice feature but be aware that most DE-specific X-based
terminal emulators (GNOME, Xfce, Konsole, …) do not implement it and
still advertise themselves as TERM=xterm[-256color].

Re: X selection access in xterm (OSC 52)

[Stefan Monnier]

> It’s a nice feature but be aware that most DE-specific X-based
> terminal emulators (GNOME, Xfce, Konsole, …) do not implement it and
> still advertise themselves as TERM=xterm[-256color].

Right, the code needs to be able to handle the situation where the
underlying terminal emulator doesn't support the feature.

There are typically 2 situations:
- the terminal emulator just ignores the special codes we send, so we
  can happily delude ourselves into thinking they always work.
- we check the terminal version info before enabling/using the feature
  (see xterm--version-handler in lisp/xterm.el).

It seems the first option won't work here, since we may send large
chunks of text as clipboard contents, which are unlikely to be
silently ignored.


        Stefan

Re: X selection access in xterm (OSC 52)

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 1486 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> schrieb am Mon Feb 09 2015 at
5:23:09 AM:

> > It’s a nice feature but be aware that most DE-specific X-based
> > terminal emulators (GNOME, Xfce, Konsole, …) do not implement it and
> > still advertise themselves as TERM=xterm[-256color].
>
> Right, the code needs to be able to handle the situation where the
> underlying terminal emulator doesn't support the feature.
>
> There are typically 2 situations:
> - the terminal emulator just ignores the special codes we send, so we
>   can happily delude ourselves into thinking they always work.
> - we check the terminal version info before enabling/using the feature
>   (see xterm--version-handler in lisp/xterm.el).
>
> It seems the first option won't work here, since we may send large
> chunks of text as clipboard contents, which are unlikely to be
> silently ignored.
>
>
>

http://invisible-island.net/xterm/xterm.log.html#xterm_203 says "add
experimental option to allow applications to get or set the selection data
as a BASE64 string (adapted from patch by Joe Allen)." I assume this is the
OSC 52 functionality, so setting 203 as the minimum requirement sounds like
a reasonable choice. On my Ubuntu 14.04 box I found the following version
strings reported:

hterm: "0;256;0"
xterm: "41;297;0"
gnome-terminal: "1;3409;0"

gnome-terminal gets adjusted to 200, so I assume that the test for >= 203
should be OK and cover both xterm and hterm.

[-- Attachment #2: Type: text/html, Size: 1989 bytes --]

Re: X selection access in xterm (OSC 52)

[Stefan Monnier]

> gnome-terminal gets adjusted to 200, so I assume that the test for >= 203
> should be OK and cover both xterm and hterm.

Indeed, sounds good.  Can someone prepare a patch for that feature?


        Stefan

Re: X selection access in xterm (OSC 52)

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 299 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> schrieb am Mon Feb 09 2015 at
6:12:20 PM:

> > gnome-terminal gets adjusted to 200, so I assume that the test for >= 203
> > should be OK and cover both xterm and hterm.
>
> Indeed, sounds good.  Can someone prepare a patch for that feature?
>
>
I'm on it.

[-- Attachment #2: Type: text/html, Size: 584 bytes --]

Re: X selection access in xterm (OSC 52)

[Philipp Stephani]

[-- Attachment #1.1: Type: text/plain, Size: 422 bytes --]

Philipp Stephani <p.stephani2@gmail.com> schrieb am Di., 10. Feb. 2015 um
11:30 Uhr:

>
>
> Stefan Monnier <monnier@iro.umontreal.ca> schrieb am Mon Feb 09 2015 at
> 6:12:20 PM:
>
> > gnome-terminal gets adjusted to 200, so I assume that the test for >= 203
>> > should be OK and cover both xterm and hterm.
>>
>> Indeed, sounds good.  Can someone prepare a patch for that feature?
>>
>>
> I'm on it.
>

Attached a patch.

[-- Attachment #1.2: Type: text/html, Size: 1062 bytes --]

[-- Attachment #2: 0001-Add-support-for-OSC-52.txt --]
[-- Type: text/plain, Size: 6448 bytes --]

From 06019ced7d34dade2d40d97605f6b2c1962fe67a Mon Sep 17 00:00:00 2001
From: Philipp Stephani <phst@google.com>
Date: Fri, 27 Feb 2015 19:51:01 +0100
Subject: [PATCH] Add support for OSC 52.

This is an xterm escape sequence to save text to the X clipboard.  Taken from
https://github.com/chromium/hterm/blob/64b3819692526f41eb960f6c9444c2c58e658f79/etc/osc52.el.
---
 lisp/term/xterm.el | 95 +++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 91 insertions(+), 4 deletions(-)

diff --git a/lisp/term/xterm.el b/lisp/term/xterm.el
index 519f691..2cfc36d 100644
--- a/lisp/term/xterm.el
+++ b/lisp/term/xterm.el
@@ -37,7 +37,8 @@ If a list, assume that the listed features are supported, without checking.
 
 The relevant features are:
   modifyOtherKeys  -- if supported, more key bindings work (e.g., \"\\C-,\")
-  reportBackground -- if supported, Xterm reports its background color"
+  reportBackground -- if supported, Xterm reports its background color
+  interprogramCut  -- if supported, Xterm saves yanked text to the X selection"
   :version "24.1"
   :group 'xterm
   :type '(choice (const :tag "No" nil)
@@ -45,7 +46,24 @@ The relevant features are:
                  ;; NOTE: If you add entries here, make sure to update
                  ;; `terminal-init-xterm' as well.
                  (set (const :tag "modifyOtherKeys support" modifyOtherKeys)
-                      (const :tag "report background" reportBackground))))
+                      (const :tag "report background" reportBackground)
+                      (const :tag "X selection cut" interprogramCut))))
+
+(defcustom xterm-max-cut-length 100000
+  "Maximum number of bytes to cut into xterm using the OSC 52 sequence.
+
+The OSC 52 sequence requires a terminator byte.  Some terminals will ignore or
+mistreat a terminated sequence that is longer than a certain size, usually to
+protect users from runaway sequences.
+
+This variable allows you to tweak the maximum number of bytes that will be sent
+using the OSC 52 sequence.
+
+If you select a region larger than this size, it won't be copied to your system
+clipboard.  Since clipboard data is base 64 encoded, the actual number of
+string bytes that can be copied is 3/4 of this value."
+  :group 'xterm
+  :type 'integer)
 
 (defconst xterm-paste-ending-sequence "\e[201~"
   "Characters send by the terminal to end a bracketed paste.")
@@ -620,7 +638,13 @@ The relevant features are:
         ;; introduced) or higher, initialize the
         ;; modifyOtherKeys support.
         (when (>= version 216)
-          (terminal-init-xterm-modify-other-keys))))))
+          (terminal-init-xterm-modify-other-keys))
+        ;; In version 203 support for accessing the X selection was
+        ;; added.  Hterm reports itself as version 256 and supports it
+        ;; as well.  gnome-terminal doesn't and is excluded by this
+        ;; test.
+        (when (>= version 203)
+          (terminal-init-xterm-activate-interprogram-cut))))))
 
 (defun xterm--query (query handlers)
   "Send QUERY string to the terminal and watch for a response.
@@ -699,7 +723,10 @@ We run the first FUNCTION whose STRING matches the input events."
                     '(("\e]11;" .  xterm--report-background-handler))))
 
     (when (memq 'modifyOtherKeys xterm-extra-capabilities)
-      (terminal-init-xterm-modify-other-keys)))
+      (terminal-init-xterm-modify-other-keys))
+
+    (when (memq 'interprogramCut xterm-extra-capabilities)
+      (terminal-init-xterm-activate-interprogram-cut)))
 
   ;; Unconditionally enable bracketed paste mode: terminals that don't
   ;; support it just ignore the sequence.
@@ -719,6 +746,66 @@ We run the first FUNCTION whose STRING matches the input events."
   (push "\e[?2004l" (terminal-parameter nil 'tty-mode-reset-strings))
   (push "\e[?2004h" (terminal-parameter nil 'tty-mode-set-strings)))
 
+(defun terminal-init-xterm-activate-interprogram-cut ()
+  "Terminal initialization for interprogram cut."
+  (setq interprogram-cut-function
+        (if (string-prefix-p "screen" (getenv "TERM"))
+            #'xterm--screen-interprogram-cut
+          #'xterm--interprogram-cut)))
+ 
+(defun xterm--interprogram-cut (text)
+  "Copy TEXT to the X selection using the OSC 52 escape sequence.
+
+Set `interprogram-cut-function' to this when using a compatible terminal, and
+your system clipboard will be updated whenever you copy a region of text in
+Emacs.
+
+If the resulting OSC 52 sequence would be longer than
+`xterm-max-cut-length', then the TEXT is not sent to the system
+clipboard.
+
+This function sends a raw OSC 52 sequence and will work on a bare terminal
+emulators.  It does not work on screen or tmux terminals, since they don't
+natively support OSC 52; use `xterm--screen-interprogram-cut' for that."
+  (xterm--interprogram-cut-bytes
+   (base64-encode-string text :no-line-break)))
+
+(defun xterm--screen-interprogram-cut (text)
+  "Copy TEXT to the X selection using the OSC 52 escape sequence, for screen users.
+
+Set `interprogram-cut-function' to this when using the screen program, and your
+system clipboard will be updated whenever you copy a region of text in Emacs.
+
+If the resulting OSC 52 sequence would be longer than
+`xterm-max-cut-length', then the STRING is not sent to the system
+clipboard.
+
+This function wraps the OSC 52 in a Device Control String sequence.  This causes
+screen to pass the wrapped OSC 52 sequence along to the host terminal.  This
+function also chops long DCS sequences into multiple smaller ones to avoid
+hitting screen's max DCS length."
+  (xterm--interprogram-cut-bytes
+   (replace-regexp-in-string
+    "\n" "\e\\\eP" (base64-encode-string text) :fixedcase :literal)
+   "\eP" "\e\\"))
+
+(defun xterm--interprogram-cut-bytes (bytes &optional dcs-start dcs-end)
+  (let ((length (string-bytes bytes)))
+    (if (> length xterm-max-cut-length)
+        (progn
+          (warn "Selection too long to send to terminal: %d bytes" length)
+          (sit-for 2))
+      (send-string-to-terminal
+       (concat
+        dcs-start
+        "\e]52;"
+        (when select-enable-clipboard "c")
+        (when select-enable-primary "p")
+        ";"
+        bytes
+        "\a"
+        dcs-end)))))
+
 ;; Set up colors, for those versions of xterm that support it.
 (defvar xterm-standard-colors
   ;; The names in the comments taken from XTerm-col.ad in the xterm
-- 
2.2.0.rc0.207.ga3a616c

Re: X selection access in xterm (OSC 52)

[Stefan Monnier]

>>> Indeed, sounds good.  Can someone prepare a patch for that feature?
>> I'm on it.

Thanks.  I don't see your name in copyright.list.  Did you sign the
copyright paperwork yet?  If not, then please fill the form below and
email it as instructed so the FSF can send you the relevant paperwork
to sign.

> Taken from
> https://github.com/chromium/hterm/blob/64b3819692526f41eb960f6c9444c2c58e658f79/etc/osc52.el.

Can you figure out who are the authors of that file, so we can get their
copyright cleared?

> This is an xterm escape sequence to save text to the X clipboard.
[...]
> +  interprogramCut  -- if supported, Xterm saves yanked text to the X selection"

Is it the "clipboard" or the "selection"?

> +  (setq interprogram-cut-function
> +        (if (string-prefix-p "screen" (getenv "TERM"))
> +            #'xterm--screen-interprogram-cut
> +          #'xterm--interprogram-cut)))

Please don't change interprogram-cut-function, since this breaks down if
you use a single Emacs session with both GUI and text frames (and with
potentially various different terminal kinds).

Instead use gui-method-declare for gui-set-selection (new in Emacs-25).

Also, do I understand it correctly, that this only handles transfers in
one direction (i.e. pushing Emacs's selection to the X server, but not
fetching the X server's selection for use in Emacs)?


        Stefan


Please email the following information to assign@gnu.org, and we
will send you the assignment form for your past and future changes.

Please use your full legal name (in ASCII characters) as the subject
line of the message.
----------------------------------------------------------------------
REQUEST: SEND FORM FOR PAST AND FUTURE CHANGES

[What is the name of the program or package you're contributing to?]
Emacs

[Did you copy any files or text written by someone else in these changes?
Even if that material is free software, we need to know about it.]


[Do you have an employer who might have a basis to claim to own
your changes?  Do you attend a school which might make such a claim?]


[For the copyright registration, what country are you a citizen of?]


[What year were you born?]


[Please write your email address here.]


[Please write your postal address here.]





[Which files have you changed so far, and which new files have you written
so far?]

Re: X selection access in xterm (OSC 52)

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 2124 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> schrieb am Fr., 13. März 2015 um
23:08 Uhr:

> >>> Indeed, sounds good.  Can someone prepare a patch for that feature?
> >> I'm on it.
>
> Thanks.  I don't see your name in copyright.list.  Did you sign the
> copyright paperwork yet?  If not, then please fill the form below and
> email it as instructed so the FSF can send you the relevant paperwork
> to sign.
>

I work for Google, so this should be covered by the general agreement
between Google and the FSF.


>
> > Taken from
> > https://github.com/chromium/hterm/blob/64b3819692526f41eb960f6c9444c2
> c58e658f79/etc/osc52.el.
>
> Can you figure out who are the authors of that file, so we can get their
> copyright cleared?
>

According to the GitHub commit log, rginda is the only contributor. He's
also a Googler and approved of the patch.


>
> > This is an xterm escape sequence to save text to the X clipboard.
> [...]
> > +  interprogramCut  -- if supported, Xterm saves yanked text to the X
> selection"
>
> Is it the "clipboard" or the "selection"?
>

It should probably be "X Window selection" because both the primary
selection and the clipboard can be accessed.


>
> > +  (setq interprogram-cut-function
> > +        (if (string-prefix-p "screen" (getenv "TERM"))
> > +            #'xterm--screen-interprogram-cut
> > +          #'xterm--interprogram-cut)))
>
> Please don't change interprogram-cut-function, since this breaks down if
> you use a single Emacs session with both GUI and text frames (and with
> potentially various different terminal kinds).
>
> Instead use gui-method-declare for gui-set-selection (new in Emacs-25).
>

Thanks, I'll modify the patch accordingly tomorrow.


>
> Also, do I understand it correctly, that this only handles transfers in
> one direction (i.e. pushing Emacs's selection to the X server, but not
> fetching the X server's selection for use in Emacs)?
>
>
>
Yes. IIUC pasting has always been supported, either via the slow path of
character-by-character insertion, or the bracketed paste functionality.

[-- Attachment #2: Type: text/html, Size: 3321 bytes --]

Re: X selection access in xterm (OSC 52)

[Stefan Monnier]

> Yes.  IIUC pasting has always been supported, either via the slow path of
> character-by-character insertion, or the bracketed paste functionality.

Tho it's not quite the same: this relies on the terminal emulator's
access to the clipboard/primary so you don't get to use Emacs's own
commands to choose which you want to use (e.g. C-y in Emacs still won't
be able to fetch the contents of the clipboard).


        Stefan

Re: X selection access in xterm (OSC 52)

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 921 bytes --]

>
>
>
>>
>> > +  (setq interprogram-cut-function
>> > +        (if (string-prefix-p "screen" (getenv "TERM"))
>> > +            #'xterm--screen-interprogram-cut
>> > +          #'xterm--interprogram-cut)))
>>
>> Please don't change interprogram-cut-function, since this breaks down if
>> you use a single Emacs session with both GUI and text frames (and with
>> potentially various different terminal kinds).
>>
>> Instead use gui-method-declare for gui-set-selection (new in Emacs-25).
>>
>
> Thanks, I'll modify the patch accordingly tomorrow.
>
>

Sorry for the delay. I've tried implementing this, but I'm a bit at a loss
here: If I understand the GUI method code correctly, it only works for real
GUIs, not terminal emulators that happen to be GUI applications, right? At
least the window-system variable, which gui-method apparently uses, is
always nil in my tests with hterm. Should this be something like 'xterm?

[-- Attachment #2: Type: text/html, Size: 1536 bytes --]

Re: X selection access in xterm (OSC 52)

[Stefan Monnier]

> Sorry for the delay. I've tried implementing this, but I'm a bit at a loss
> here: If I understand the GUI method code correctly, it only works for real
> GUIs, not terminal emulators that happen to be GUI
> applications, right?

No, they work for text terminals as well.

> At least the window-system variable, which gui-method apparently uses,
> is always nil in my tests with hterm. Should this be something
> like 'xterm?

No, it will be nil.  Of course, it will apply to all terminals, so
you'll then have to check whether the current text terminal is an xterm.


        Stefan

Re: X selection access in xterm (OSC 52)

[Olaf Rogalsky]

Hi,

I just stumbeld upon this thread.

Not long ago I wrote a minor mode, which provides two-way
interoperability between the X11 selections and the kill ring of emacs
via OSC 52 control sequences.

The code is by no means polished, but it works for me.

If you are interested, you can have a look at

https://github.com/olaf-rogalsky/xterm-clip

Olaf

Re: X selection access in xterm (OSC 52)

[Stefan Monnier]

> If you are interested, you can have a look at
>
> https://github.com/olaf-rogalsky/xterm-clip

It looks more complete than what we have so far (e.g. it includes
support to fetch the clipboard/selection as well).

It also seems to use new functions such as gui-set-selection, but oddly
enough it still modifies things like interprogram-cut-function instead
of using gui-method-define.

Any hope your code and Philipp can be merged and then installed in
term/xterm.el?


        Stefan

Re: X selection access in xterm (OSC 52)

[Philipp Stephani]

[-- Attachment #1.1: Type: text/plain, Size: 814 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> schrieb am Di., 24. März 2015 um
22:55 Uhr:

> > Sorry for the delay. I've tried implementing this, but I'm a bit at a
> loss
> > here: If I understand the GUI method code correctly, it only works for
> real
> > GUIs, not terminal emulators that happen to be GUI
> > applications, right?
>
> No, they work for text terminals as well.
>
> > At least the window-system variable, which gui-method apparently uses,
> > is always nil in my tests with hterm. Should this be something
> > like 'xterm?
>
> No, it will be nil.  Of course, it will apply to all terminals, so
> you'll then have to check whether the current text terminal is an xterm.
>
>
OK, here's another patch. I've done some lightweight testing, it works in
hterm with or without screen.

[-- Attachment #1.2: Type: text/html, Size: 1135 bytes --]

[-- Attachment #2: 0001-Implement-OSC-52-functionality-for-setting-the-X-sel.patch --]
[-- Type: application/octet-stream, Size: 6395 bytes --]

From affef6455cdcbb879b5063859deec3641312b0fd Mon Sep 17 00:00:00 2001
From: Philipp Stephani <phst@google.com>
Date: Wed, 8 Apr 2015 18:53:47 +0200
Subject: [PATCH] Implement OSC-52 functionality for setting the X selection

---
 lisp/term/xterm.el | 93 +++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 89 insertions(+), 4 deletions(-)

diff --git a/lisp/term/xterm.el b/lisp/term/xterm.el
index 519f691..f4d1a84 100644
--- a/lisp/term/xterm.el
+++ b/lisp/term/xterm.el
@@ -37,7 +37,8 @@ If a list, assume that the listed features are supported, without checking.
 
 The relevant features are:
   modifyOtherKeys  -- if supported, more key bindings work (e.g., \"\\C-,\")
-  reportBackground -- if supported, Xterm reports its background color"
+  reportBackground -- if supported, Xterm reports its background color
+  setSelection     -- if supported, Xterm saves yanked text to the X selection"
   :version "24.1"
   :group 'xterm
   :type '(choice (const :tag "No" nil)
@@ -45,7 +46,24 @@ The relevant features are:
                  ;; NOTE: If you add entries here, make sure to update
                  ;; `terminal-init-xterm' as well.
                  (set (const :tag "modifyOtherKeys support" modifyOtherKeys)
-                      (const :tag "report background" reportBackground))))
+                      (const :tag "report background" reportBackground)
+                      (const :tag "set X selection" setSelection))))
+
+(defcustom xterm-max-cut-length 100000
+  "Maximum number of bytes to cut into xterm using the OSC 52 sequence.
+
+The OSC 52 sequence requires a terminator byte.  Some terminals will ignore or
+mistreat a terminated sequence that is longer than a certain size, usually to
+protect users from runaway sequences.
+
+This variable allows you to tweak the maximum number of bytes that will be sent
+using the OSC 52 sequence.
+
+If you select a region larger than this size, it won't be copied to your system
+clipboard.  Since clipboard data is base 64 encoded, the actual number of
+string bytes that can be copied is 3/4 of this value."
+  :group 'xterm
+  :type 'integer)
 
 (defconst xterm-paste-ending-sequence "\e[201~"
   "Characters send by the terminal to end a bracketed paste.")
@@ -620,7 +638,13 @@ The relevant features are:
         ;; introduced) or higher, initialize the
         ;; modifyOtherKeys support.
         (when (>= version 216)
-          (terminal-init-xterm-modify-other-keys))))))
+          (terminal-init-xterm-modify-other-keys))
+        ;; In version 203 support for accessing the X selection was
+        ;; added.  Hterm reports itself as version 256 and supports it
+        ;; as well.  gnome-terminal doesn't and is excluded by this
+        ;; test.
+        (when (>= version 203)
+          (terminal-init-xterm-activate-set-selection))))))
 
 (defun xterm--query (query handlers)
   "Send QUERY string to the terminal and watch for a response.
@@ -699,7 +723,10 @@ We run the first FUNCTION whose STRING matches the input events."
                     '(("\e]11;" .  xterm--report-background-handler))))
 
     (when (memq 'modifyOtherKeys xterm-extra-capabilities)
-      (terminal-init-xterm-modify-other-keys)))
+      (terminal-init-xterm-modify-other-keys))
+
+    (when (memq 'setSelection xterm-extra-capabilities)
+      (terminal-init-xterm-activate-set-selection)))
 
   ;; Unconditionally enable bracketed paste mode: terminals that don't
   ;; support it just ignore the sequence.
@@ -719,6 +746,64 @@ We run the first FUNCTION whose STRING matches the input events."
   (push "\e[?2004l" (terminal-parameter nil 'tty-mode-reset-strings))
   (push "\e[?2004h" (terminal-parameter nil 'tty-mode-set-strings)))
 
+(defun terminal-init-xterm-activate-set-selection ()
+  "Terminal initialization for `gui-set-selection'."
+  ;; All text terminals are represented by the nil GUI type.  We need
+  ;; to detect XTerm again in `xterm--set-selection' using the
+  ;; terminal parameters.
+  (gui-method-define gui-set-selection nil #'xterm--set-selection))
+
+(defun xterm--set-selection (type data)
+  "Copy DATA to the X selection using the OSC 52 escape sequence.
+
+TYPE specifies which selection to set; it must be either
+`PRIMARY' or `CLIPBOARD'.  DATA must be a string.
+
+This can be used as a `gui-set-selection' method for
+xterm-compatible terminal emulators.  Then your system clipboard
+will be updated whenever you copy a region of text in Emacs.
+
+If the resulting OSC 52 sequence would be longer than
+`xterm-max-cut-length', then the TEXT is not sent to the system
+clipboard.
+
+This function either sends a raw OSC 52 sequence or wraps the OSC
+52 in a Device Control String sequence.  This way, it will work
+on a bare terminal emulators as well as inside the screen
+program.  When inside the screen program, this function also
+chops long DCS sequences into multiple smaller ones to avoid
+hitting screen's max DCS length."
+  (let* ((init-function (terminal-parameter nil 'terminal-initted))
+         (xterm (eq init-function 'terminal-init-xterm))
+         (screen (eq init-function 'terminal-init-screen)))
+    ;; Only do something if the current terminal is actually an XTerm
+    ;; or screen.
+    (when (or xterm screen)
+      (let* ((bytes (encode-coding-string data 'utf-8-unix))
+             (base-64 (if screen
+                          (replace-regexp-in-string
+                           "\n" "\e\\\eP"
+                           (base64-encode-string bytes)
+                           :fixedcase :literal)
+                        (base64-encode-string bytes :no-line-break)))
+             (length (string-bytes base-64)))
+        (if (> length xterm-max-cut-length)
+            (progn
+              (warn "Selection too long to send to terminal: %d bytes" length)
+              (sit-for 2))
+          (send-string-to-terminal
+           (concat
+            (when screen "\eP")
+            "\e]52;"
+            (cond
+             ((eq type 'PRIMARY) "p")
+             ((eq type 'CLIPBOARD) "c")
+             (t (error "Invalid type %S" type)))
+            ";"
+            base-64
+            "\a"
+            (when screen "\e\\"))))))))
+
 ;; Set up colors, for those versions of xterm that support it.
 (defvar xterm-standard-colors
   ;; The names in the comments taken from XTerm-col.ad in the xterm
-- 
2.3.5

Re: X selection access in xterm (OSC 52)

[Stefan Monnier]

> OK, here's another patch.  I've done some lightweight testing, it works in
> hterm with or without screen.

Thanks, it looks sound.  Have you had a chance to look at Olaf's code?


        Stefan

Re: X selection access in xterm (OSC 52)

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 462 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> schrieb am Do., 9. Apr. 2015 um
04:10 Uhr:

> > OK, here's another patch.  I've done some lightweight testing, it works
> in
> > hterm with or without screen.
>
> Thanks, it looks sound.  Have you had a chance to look at Olaf's code?
>
>
>
Yes, I took a look and I'll work on integrating the paste functionality.
Since cut and paste are mostly independent of each other, maybe you could
already integrate the cut patch?

[-- Attachment #2: Type: text/html, Size: 768 bytes --]

Re: X selection access in xterm (OSC 52)

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 1005 bytes --]

I took a second look at HTerm, and found that its FAQ (
http://git.chromium.org/gitweb/?p=chromiumos/platform/assets.git;a=blob;f=chromeapps/nassh/doc/faq.txt)
states:

"Clipboard read is not implemented.  Reading is a security hole you
probably didn't want anyway."

Since I can't test the get-selection function and it's considered a
security hole, I'm afraid I can't implement it.
This only affects get-selection, not set-selection.

Philipp Stephani <p.stephani2@gmail.com> schrieb am Do., 9. Apr. 2015 um
17:48 Uhr:

> Stefan Monnier <monnier@iro.umontreal.ca> schrieb am Do., 9. Apr. 2015 um
> 04:10 Uhr:
>
>> > OK, here's another patch.  I've done some lightweight testing, it works
>> in
>> > hterm with or without screen.
>>
>> Thanks, it looks sound.  Have you had a chance to look at Olaf's code?
>>
>>
>>
> Yes, I took a look and I'll work on integrating the paste functionality.
> Since cut and paste are mostly independent of each other, maybe you could
> already integrate the cut patch?
>

[-- Attachment #2: Type: text/html, Size: 2023 bytes --]

Re: X selection access in xterm (OSC 52)

[Philipp Stephani]

[-- Attachment #1.1: Type: text/plain, Size: 737 bytes --]

Philipp Stephani <p.stephani2@gmail.com> schrieb am Do., 9. Apr. 2015 um
20:47 Uhr:

> I took a second look at HTerm, and found that its FAQ (
> http://git.chromium.org/gitweb/?p=chromiumos/platform/assets.git;a=blob;f=chromeapps/nassh/doc/faq.txt)
> states:
>
> "Clipboard read is not implemented.  Reading is a security hole you
> probably didn't want anyway."
>
> Since I can't test the get-selection function and it's considered a
> security hole, I'm afraid I can't implement it.
> This only affects get-selection, not set-selection.
>

FWIW, here's my WIP patch for gui-get-selection, in case anybody wants to
implement this. It differs from Olaf's code mainly in using a temp buffer
and the existing xterm--query helper function.

[-- Attachment #1.2: Type: text/html, Size: 1468 bytes --]

[-- Attachment #2: 0001-Implement-OSC-52-functionality-for-getting-the-X-sel.patch --]
[-- Type: application/octet-stream, Size: 4560 bytes --]

From f0af6dea559e000c90873f30c294ae6ce9a5922a Mon Sep 17 00:00:00 2001
From: Philipp Stephani <phst@google.com>
Date: Thu, 9 Apr 2015 22:05:19 +0200
Subject: [PATCH] Implement OSC-52 functionality for getting the X selection

---
 lisp/term/xterm.el | 41 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 40 insertions(+), 1 deletion(-)

diff --git a/lisp/term/xterm.el b/lisp/term/xterm.el
index f4d1a84..695091f 100644
--- a/lisp/term/xterm.el
+++ b/lisp/term/xterm.el
@@ -38,7 +38,8 @@ If a list, assume that the listed features are supported, without checking.
 The relevant features are:
   modifyOtherKeys  -- if supported, more key bindings work (e.g., \"\\C-,\")
   reportBackground -- if supported, Xterm reports its background color
-  setSelection     -- if supported, Xterm saves yanked text to the X selection"
+  getSelection     -- if supported, Xterm yanks text from the X selection
+  setSelection     -- if supported, Xterm saves killed text to the X selection"
   :version "24.1"
   :group 'xterm
   :type '(choice (const :tag "No" nil)
@@ -47,6 +48,7 @@ The relevant features are:
                  ;; `terminal-init-xterm' as well.
                  (set (const :tag "modifyOtherKeys support" modifyOtherKeys)
                       (const :tag "report background" reportBackground)
+                      (const :tag "get X selection" getSelection)
                       (const :tag "set X selection" setSelection))))
 
 (defcustom xterm-max-cut-length 100000
@@ -644,6 +646,7 @@ string bytes that can be copied is 3/4 of this value."
         ;; as well.  gnome-terminal doesn't and is excluded by this
         ;; test.
         (when (>= version 203)
+          (terminal-init-xterm-activate-get-selection)
           (terminal-init-xterm-activate-set-selection))))))
 
 (defun xterm--query (query handlers)
@@ -725,6 +728,8 @@ We run the first FUNCTION whose STRING matches the input events."
     (when (memq 'modifyOtherKeys xterm-extra-capabilities)
       (terminal-init-xterm-modify-other-keys))
 
+    (when (memq 'getSelection xterm-extra-capabilities)
+      (terminal-init-xterm-activate-get-selection))
     (when (memq 'setSelection xterm-extra-capabilities)
       (terminal-init-xterm-activate-set-selection)))
 
@@ -746,6 +751,13 @@ We run the first FUNCTION whose STRING matches the input events."
   (push "\e[?2004l" (terminal-parameter nil 'tty-mode-reset-strings))
   (push "\e[?2004h" (terminal-parameter nil 'tty-mode-set-strings)))
 
+(defun terminal-init-xterm-activate-get-selection ()
+  "Terminal initialization for `gui-get-selection'."
+  ;; All text terminals are represented by the nil GUI type.  We need
+  ;; to detect XTerm again in `xterm--get-selection' using the
+  ;; terminal parameters.
+  (gui-method-define gui-get-selection nil #'xterm--get-selection))
+
 (defun terminal-init-xterm-activate-set-selection ()
   "Terminal initialization for `gui-set-selection'."
   ;; All text terminals are represented by the nil GUI type.  We need
@@ -753,6 +765,33 @@ We run the first FUNCTION whose STRING matches the input events."
   ;; terminal parameters.
   (gui-method-define gui-set-selection nil #'xterm--set-selection))
 
+(defun xterm--get-selection (type data-type)
+  (unless (eq data-type 'STRING)
+    (error "Unsupported data type %S" data-type))
+  (let* ((init-function (terminal-parameter nil 'terminal-initted))
+         (xterm (eq init-function 'terminal-init-xterm))
+         (screen (eq init-function 'terminal-init-screen)))
+    ;; Only do something if the current terminal is actually an XTerm
+    ;; or screen.
+    (when (or xterm screen)
+      (let ((query (concat
+                    "\e]52;"
+                    (cond
+                     ((eq type 'PRIMARY) "p")
+                     ((eq type 'CLIPBOARD) "c")
+                     (t (error "Invalid type %S" type)))
+                    ";")))
+        (with-temp-buffer
+          (xterm--query
+           (concat (when screen "\eP") query "?\a" (when screen "\e\\"))
+           (list (cons query (lambda ()
+                               (while (let ((char (read-char)))
+                                        (unless (eq char ?\a)
+                                          (insert char)
+                                          t)))))))
+          (base64-decode-region (point-min) (point-max))
+          (buffer-substring-no-properties (point-min) (point-max)))))))
+
 (defun xterm--set-selection (type data)
   "Copy DATA to the X selection using the OSC 52 escape sequence.
 
-- 
2.3.5

Re: X selection access in xterm (OSC 52)

[Stefan Monnier]

> Yes, I took a look and I'll work on integrating the paste functionality.
> Since cut and paste are mostly independent of each other, maybe you could
> already integrate the cut patch?

I just installed it (after adding a ChangeLog, and although it still
lacks an etc/NEWS entry).
Thank you.

Adding support for the getSelection part shouldn't directly introduce
any security problem, although you can indeed argue that this feature is
a security risk in some circumstances (but the security risk is not due
to Emacs's support of that feature, but to the terminal's support of
that feature).


        Stefan

Re: X selection access in xterm (OSC 52)

[Stefan Monnier]

>> Yes, I took a look and I'll work on integrating the paste functionality.
>> Since cut and paste are mostly independent of each other, maybe you could
>> already integrate the cut patch?
> I just installed it (after adding a ChangeLog, and although it still
> lacks an etc/NEWS entry).

BTW, I can't seem to make this feature work for me.  I do:

   emacs -Q -nw
   M-x trace-function RET xterm--set-selection RET
   C-SPC M-f M-f M-f M-w
   <go to a previously running Emacs session in GUI mode>
   C-y

and instead of getting the three words from *scratch*, I get whatever
was already there before in the clipboard.  Yet, the trace buffer shows
that xterm--set-selection was called alright (and edebugging it also
indicates that it seems to be doing what it should).


        Stefan

Re: X selection access in xterm (OSC 52)

[Stefan Monnier]

Is that normal?  Do you guys see the same?  I'm using Debian's "xterm"
package version 312-2, for what it's worth.


        Stefan


>>>>> "Stefan" == Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>>> Yes, I took a look and I'll work on integrating the paste functionality.
>>> Since cut and paste are mostly independent of each other, maybe you could
>>> already integrate the cut patch?
>> I just installed it (after adding a ChangeLog, and although it still
>> lacks an etc/NEWS entry).

> BTW, I can't seem to make this feature work for me.  I do:

>    emacs -Q -nw
>    M-x trace-function RET xterm--set-selection RET
>    C-SPC M-f M-f M-f M-w
>    <go to a previously running Emacs session in GUI mode>
>    C-y

> and instead of getting the three words from *scratch*, I get whatever
> was already there before in the clipboard.  Yet, the trace buffer shows
> that xterm--set-selection was called alright (and edebugging it also
> indicates that it seems to be doing what it should).


>         Stefan

Re: X selection access in xterm (OSC 52)

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 1309 bytes --]

Maybe something needs to be enabled? The documentation says "These controls
may be disabled using the allowWindowOps resource." I'll try it today.

Stefan Monnier <monnier@iro.umontreal.ca> schrieb am Fr., 17. Apr. 2015 um
04:40 Uhr:

> Is that normal?  Do you guys see the same?  I'm using Debian's "xterm"
> package version 312-2, for what it's worth.
>
>
>         Stefan
>
>
> >>>>> "Stefan" == Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>
> >>> Yes, I took a look and I'll work on integrating the paste
> functionality.
> >>> Since cut and paste are mostly independent of each other, maybe you
> could
> >>> already integrate the cut patch?
> >> I just installed it (after adding a ChangeLog, and although it still
> >> lacks an etc/NEWS entry).
>
> > BTW, I can't seem to make this feature work for me.  I do:
>
> >    emacs -Q -nw
> >    M-x trace-function RET xterm--set-selection RET
> >    C-SPC M-f M-f M-f M-w
> >    <go to a previously running Emacs session in GUI mode>
> >    C-y
>
> > and instead of getting the three words from *scratch*, I get whatever
> > was already there before in the clipboard.  Yet, the trace buffer shows
> > that xterm--set-selection was called alright (and edebugging it also
> > indicates that it seems to be doing what it should).
>
>
> >         Stefan
>

[-- Attachment #2: Type: text/html, Size: 1873 bytes --]

Re: X selection access in xterm (OSC 52)

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 1606 bytes --]

If I understand https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=384593,
this functionality was disabled by default on Debian-based systems for
security reasons.

Philipp Stephani <p.stephani2@gmail.com> schrieb am Fr., 17. Apr. 2015 um
08:25 Uhr:

> Maybe something needs to be enabled? The documentation says "These
> controls may be disabled using the allowWindowOps resource." I'll try it
> today.
>
> Stefan Monnier <monnier@iro.umontreal.ca> schrieb am Fr., 17. Apr. 2015
> um 04:40 Uhr:
>
>> Is that normal?  Do you guys see the same?  I'm using Debian's "xterm"
>> package version 312-2, for what it's worth.
>>
>>
>>         Stefan
>>
>>
>> >>>>> "Stefan" == Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>>
>> >>> Yes, I took a look and I'll work on integrating the paste
>> functionality.
>> >>> Since cut and paste are mostly independent of each other, maybe you
>> could
>> >>> already integrate the cut patch?
>> >> I just installed it (after adding a ChangeLog, and although it still
>> >> lacks an etc/NEWS entry).
>>
>> > BTW, I can't seem to make this feature work for me.  I do:
>>
>> >    emacs -Q -nw
>> >    M-x trace-function RET xterm--set-selection RET
>> >    C-SPC M-f M-f M-f M-w
>> >    <go to a previously running Emacs session in GUI mode>
>> >    C-y
>>
>> > and instead of getting the three words from *scratch*, I get whatever
>> > was already there before in the clipboard.  Yet, the trace buffer shows
>> > that xterm--set-selection was called alright (and edebugging it also
>> > indicates that it seems to be doing what it should).
>>
>>
>> >         Stefan
>>
>

[-- Attachment #2: Type: text/html, Size: 2436 bytes --]

Re: X selection access in xterm (OSC 52)

[Stefan Monnier]

> If I understand https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=384593,
> this functionality was disabled by default on Debian-based systems for
> security reasons.

Ah, indeed I see in "man xterm" that allowWindowOps defaults to false
and that disallowedWindowOps includes both GetSelection and SetSelection.
If I try

   xterm -xrm '*.allowWindowOps: true'

Then things work.  Yay!

I don't see why SetSelection would be a serious security issue (tho
I guess if a program does the right SetSelection at the right time, you
could end up pasting dangerous commands into a shell).
For GetSelection, the problem can show up if you view "raw data" without
going though a pager, but if your terminal is busy running Emacs you're
safe ;-)

Hmm... these WindowOps really need to be fixed.  E.g. they could require
a secret key (à la xauth), so an attacker wouldn't be able to send the
right command.  But of course, that can't be fixed on Emacs's side.


        Stefan


> Philipp Stephani <p.stephani2@gmail.com> schrieb am Fr., 17. Apr. 2015 um
> 08:25 Uhr:

>> Maybe something needs to be enabled? The documentation says "These
>> controls may be disabled using the allowWindowOps resource." I'll try it
>> today.
>> 
>> Stefan Monnier <monnier@iro.umontreal.ca> schrieb am Fr., 17. Apr. 2015
>> um 04:40 Uhr:
>> 
>>> Is that normal?  Do you guys see the same?  I'm using Debian's "xterm"
>>> package version 312-2, for what it's worth.
>>> 
>>> 
>>> Stefan
>>> 
>>> 
>>> >>>>> "Stefan" == Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>>> 
>>> >>> Yes, I took a look and I'll work on integrating the paste
>>> functionality.
>>> >>> Since cut and paste are mostly independent of each other, maybe you
>>> could
>>> >>> already integrate the cut patch?
>>> >> I just installed it (after adding a ChangeLog, and although it still
>>> >> lacks an etc/NEWS entry).
>>> 
>>> > BTW, I can't seem to make this feature work for me.  I do:
>>> 
>>> >    emacs -Q -nw
>>> >    M-x trace-function RET xterm--set-selection RET
>>> >    C-SPC M-f M-f M-f M-w
>>> >    <go to a previously running Emacs session in GUI mode>
>>> >    C-y
>>> 
>>> > and instead of getting the three words from *scratch*, I get whatever
>>> > was already there before in the clipboard.  Yet, the trace buffer shows
>>> > that xterm--set-selection was called alright (and edebugging it also
>>> > indicates that it seems to be doing what it should).
>>> 
>>> 
>>> >         Stefan
>>> 
>> 

Re: X selection access in xterm (OSC 52)

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 1375 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> schrieb am Fr., 17. Apr. 2015 um
15:52 Uhr:

> > If I understand https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=384593
> ,
> > this functionality was disabled by default on Debian-based systems for
> > security reasons.
>
> Ah, indeed I see in "man xterm" that allowWindowOps defaults to false
> and that disallowedWindowOps includes both GetSelection and SetSelection.
> If I try
>
>    xterm -xrm '*.allowWindowOps: true'
>
> Then things work.  Yay!
>
> I don't see why SetSelection would be a serious security issue (tho
> I guess if a program does the right SetSelection at the right time, you
> could end up pasting dangerous commands into a shell).
> For GetSelection, the problem can show up if you view "raw data" without
> going though a pager, but if your terminal is busy running Emacs you're
> safe ;-)
>

I think the attack vector is: you can trust SSH to not destroy or leak data
on your machine, so you can SSH into arbitrary untrusted machines and run
arbitrary programs there. This trust is broken if the program can initiate
a read of the clipboard of the local machine (the clipboard could contain
confidential information). So I can see why terminal emulator authors would
want to disable/omit this function. Agreed that it wouldn't be an issue to
support it on Emacs's side. I'll try to get my patch working.

[-- Attachment #2: Type: text/html, Size: 1829 bytes --]

Re: X selection access in xterm (OSC 52)

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 1660 bytes --]

Philipp Stephani <p.stephani2@gmail.com> schrieb am Fr., 17. Apr. 2015 um
16:00 Uhr:

> Stefan Monnier <monnier@iro.umontreal.ca> schrieb am Fr., 17. Apr. 2015
> um 15:52 Uhr:
>
>> > If I understand
>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=384593,
>> > this functionality was disabled by default on Debian-based systems for
>> > security reasons.
>>
>> Ah, indeed I see in "man xterm" that allowWindowOps defaults to false
>> and that disallowedWindowOps includes both GetSelection and SetSelection.
>> If I try
>>
>>    xterm -xrm '*.allowWindowOps: true'
>>
>> Then things work.  Yay!
>>
>> I don't see why SetSelection would be a serious security issue (tho
>> I guess if a program does the right SetSelection at the right time, you
>> could end up pasting dangerous commands into a shell).
>> For GetSelection, the problem can show up if you view "raw data" without
>> going though a pager, but if your terminal is busy running Emacs you're
>> safe ;-)
>>
>
> I think the attack vector is: you can trust SSH to not destroy or leak
> data on your machine, so you can SSH into arbitrary untrusted machines and
> run arbitrary programs there. This trust is broken if the program can
> initiate a read of the clipboard of the local machine (the clipboard could
> contain confidential information). So I can see why terminal emulator
> authors would want to disable/omit this function. Agreed that it wouldn't
> be an issue to support it on Emacs's side. I'll try to get my patch working.
>

I don't recall what exactly happened after this discussion, but it seems
that the emacs-25 branch now has support for getSelection and setSelection.
Thanks.

[-- Attachment #2: Type: text/html, Size: 2411 bytes --]