bug#72983: 29.4; Inconsistent parameter types sent to GUI selection converters

bug#72983: 29.4; Inconsistent parameter types sent to GUI selection converters

[Derek Upham via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Existing code
-------------

We'll have to go into some obscure areas of the GUI selection 
code.
Let's start with xselect-convert-to-targets (select.el).

  (defun xselect-convert-to-targets (selection _type value)
    ;; Return a vector of atoms, but remove duplicates first.
    (if (eq selection 'XdndSelection)
        ;; This isn't required by the XDND protocol, and sure 
        enough no
        ;; clients seem to dependent on it, but Emacs implements 
        the
        ;; receiver side of the Motif drop protocol by looking at 
        the
        ;; initiator selection's TARGETS target (which Motif 
        provides)
        ;; instead of the target table on the drag window, so it 
        seems
        ;; plausible for other clients to rely on that as well.
        (apply #'vector (mapcar #'intern x-dnd-targets-list))
      (apply #'vector
             (delete-dups
              `( TIMESTAMP MULTIPLENIL
                 . ,(delq '_EMACS_INTERNAL
                          (mapcar (lambda (conv)
                                    (if (or (not (consp (cdr 
                                    conv)))
                                            (funcall (cadr conv) 
                                            selection
                                                     (car conv) 
                                                     value))
                                        (car conv)
                                      '_EMACS_INTERNAL))
                                  selection-converter-alist)))))))

This function evaluates each converter in 
selection-converter-alist
against the selection value, and returns the labels of any 
converters
that return non-NIL.  The goal here is to filter out targets that 
Emacs
can't vend for the current value.  The converters are responsible 
for
noticing and rejecting inputs that they can't support.

Be aware that the "value" parameter may be a string with text
properties.  The "gui-set-selection" Info documentation mentions 
this:

     If DATA is a string, then its text properties can specify 
     values
     used for individual data types.  For example, if DATA has a
     property named ‘text/uri-list’, then a call to 
     ‘gui-get-selection’
     with the data type ‘text/uri-list’ will result in the value 
     of that
     property being used instead of DATA itself.

Now compare the xselect-convert-to-targets function with the code 
in
x_get_local_selection (xselect.c, excerpted).

      CHECK_SYMBOL (target_type);
      handler_fn = CDR (Fassq (target_type, 
      Vselection_converter_alist));

      if (CONSP (handler_fn))
	handler_fn = XCDR (handler_fn);

      if (!need_alternate)
	tem = XCAR (XCDR (local_value));
      else
	tem = XCAR (XCDR (XCDR (XCDR (XCDR (local_value)))));

      if (STRINGP (tem))
	{
	  local_value = Fget_text_property (make_fixnum (0),
					    target_type, tem);

	  if (!NILP (local_value))
	    tem = local_value;
	}

      if (!NILP (handler_fn))
	value = call3 (handler_fn, selection_symbol,
		       ((local_request
			 && NILP 
			 (Vx_treat_local_requests_remotely))
			? Qnil
			: target_type),
		       tem);
      else
	value = Qnil;

The caller (possibly another X client) provides the target, which
defines the converter to use.  If tem is a string, then we check 
for a
property that matches the target type.  If such a property exists, 
we
clobber the existing string with the associated property's object. 
Then
we call the converter.


Problem
-------

This discrepancy trips up potential HTML support.

A typical application like Firefox or LibreOffice vends both 
text/html
and text/plain content.  Clients will ask for the targets, then 
ask
for the text/html value if available, falling back to text/plain. 
For
example, we might want to support an italiced "foo", while falling
back to the underlying word.

  #("foo" 0 3 (text/html "<i>foo</i>"))

We want to advertise a text/html target only when our value has a
text/html property.  We can do that with new 
"xselect-convert-to-html"
function in selection-converter-alist.

  (text/html . xselect-convert-to-html)

The function returns true if the input is a string with a 
text/html
property.  But if the client then *asks* for the text/html, the C 
code
will send the same function a plain string “<i>foo</i>” without 
the
property.  The function bails out with NIL.  Most clients will 
then fall
back and ask for the text/plain target.

In broad terms, we can’t distinguish between regular text and HTML 
text
from first principles.  We need guidance from upstream.  Also note 
that
if we write the HTML converter function such that it doesn’t test 
for
and require that text/html property, then Emacs will happily vend 
the
plain text strings to text/html requesters.


Possible fixes
--------------

The current implementation doesn't nail down the protocol and the 
data types.

There are a couple of potential fixes; some are more invasive than 
others.

1. We can define that, if we have a string, then the string is 
always
   implicitly a variant type that we pass the converters.  Just 
   take out
   the local_value clobbering in the C code.  The HTML converter 
   and all
   other converters can then consistently look for and extract 
   their
   relevant property from the string.  This is a breaking behavior
   change, but for already-broken behavior.  And the built-in 
   converters
   in select.el don’t seem to care about those properties.

2. We can put the properties back in.  Once we extract the 
property
   string local_value, copy the properties of the original string 
   tem
   into local_value.  Then overwrite tem.  The rule for handlers 
   is then
   to *look* for the property, but it can use property’s string or 
   the
   underlying string.

3. We can declare that callers have to add type tags to the 
property objects.

     #("foo" 0 3 (text/html (html . "<i>foo</i>")))

   Then the converters are responsible for receiving that string 
   *or*
   (html . "<i>foo</i>"), depending on which function calls them,
   handling both inputs.  This is ugly, but it works for a 
   prototype
   HTML converter on top of the existing v29.4 code.

-- 
Derek Upham
derek_upham@mailfence.com

bug#72983: 29.4; Inconsistent parameter types sent to GUI selection converters

[Eli Zaretskii]

> Date: Mon, 02 Sep 2024 11:15:40 -0700
> From:  Derek Upham via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> 
> Existing code
> -------------
> 
> We'll have to go into some obscure areas of the GUI selection 
> code.
> Let's start with xselect-convert-to-targets (select.el).
> 
>   (defun xselect-convert-to-targets (selection _type value)
>     ;; Return a vector of atoms, but remove duplicates first.
>     (if (eq selection 'XdndSelection)
>         ;; This isn't required by the XDND protocol, and sure 
>         enough no
>         ;; clients seem to dependent on it, but Emacs implements 
>         the
>         ;; receiver side of the Motif drop protocol by looking at 
>         the
>         ;; initiator selection's TARGETS target (which Motif 
>         provides)
>         ;; instead of the target table on the drag window, so it 
>         seems
>         ;; plausible for other clients to rely on that as well.
>         (apply #'vector (mapcar #'intern x-dnd-targets-list))
>       (apply #'vector
>              (delete-dups
>               `( TIMESTAMP MULTIPLENIL
>                  . ,(delq '_EMACS_INTERNAL
>                           (mapcar (lambda (conv)
>                                     (if (or (not (consp (cdr 
>                                     conv)))
>                                             (funcall (cadr conv) 
>                                             selection
>                                                      (car conv) 
>                                                      value))
>                                         (car conv)
>                                       '_EMACS_INTERNAL))
>                                   selection-converter-alist)))))))
> 
> This function evaluates each converter in 
> selection-converter-alist
> against the selection value, and returns the labels of any 
> converters
> that return non-NIL.  The goal here is to filter out targets that 
> Emacs
> can't vend for the current value.  The converters are responsible 
> for
> noticing and rejecting inputs that they can't support.
> 
> Be aware that the "value" parameter may be a string with text
> properties.  The "gui-set-selection" Info documentation mentions 
> this:
> 
>      If DATA is a string, then its text properties can specify 
>      values
>      used for individual data types.  For example, if DATA has a
>      property named ‘text/uri-list’, then a call to 
>      ‘gui-get-selection’
>      with the data type ‘text/uri-list’ will result in the value 
>      of that
>      property being used instead of DATA itself.
> 
> Now compare the xselect-convert-to-targets function with the code 
> in
> x_get_local_selection (xselect.c, excerpted).
> 
>       CHECK_SYMBOL (target_type);
>       handler_fn = CDR (Fassq (target_type, 
>       Vselection_converter_alist));
> 
>       if (CONSP (handler_fn))
> 	handler_fn = XCDR (handler_fn);
> 
>       if (!need_alternate)
> 	tem = XCAR (XCDR (local_value));
>       else
> 	tem = XCAR (XCDR (XCDR (XCDR (XCDR (local_value)))));
> 
>       if (STRINGP (tem))
> 	{
> 	  local_value = Fget_text_property (make_fixnum (0),
> 					    target_type, tem);
> 
> 	  if (!NILP (local_value))
> 	    tem = local_value;
> 	}
> 
>       if (!NILP (handler_fn))
> 	value = call3 (handler_fn, selection_symbol,
> 		       ((local_request
> 			 && NILP 
> 			 (Vx_treat_local_requests_remotely))
> 			? Qnil
> 			: target_type),
> 		       tem);
>       else
> 	value = Qnil;
> 
> The caller (possibly another X client) provides the target, which
> defines the converter to use.  If tem is a string, then we check 
> for a
> property that matches the target type.  If such a property exists, 
> we
> clobber the existing string with the associated property's object. 
> Then
> we call the converter.
> 
> 
> Problem
> -------
> 
> This discrepancy trips up potential HTML support.
> 
> A typical application like Firefox or LibreOffice vends both 
> text/html
> and text/plain content.  Clients will ask for the targets, then 
> ask
> for the text/html value if available, falling back to text/plain. 
> For
> example, we might want to support an italiced "foo", while falling
> back to the underlying word.
> 
>   #("foo" 0 3 (text/html "<i>foo</i>"))
> 
> We want to advertise a text/html target only when our value has a
> text/html property.  We can do that with new 
> "xselect-convert-to-html"
> function in selection-converter-alist.
> 
>   (text/html . xselect-convert-to-html)
> 
> The function returns true if the input is a string with a 
> text/html
> property.  But if the client then *asks* for the text/html, the C 
> code
> will send the same function a plain string “<i>foo</i>” without 
> the
> property.  The function bails out with NIL.  Most clients will 
> then fall
> back and ask for the text/plain target.
> 
> In broad terms, we can’t distinguish between regular text and HTML 
> text
> from first principles.  We need guidance from upstream.  Also note 
> that
> if we write the HTML converter function such that it doesn’t test 
> for
> and require that text/html property, then Emacs will happily vend 
> the
> plain text strings to text/html requesters.
> 
> 
> Possible fixes
> --------------
> 
> The current implementation doesn't nail down the protocol and the 
> data types.
> 
> There are a couple of potential fixes; some are more invasive than 
> others.
> 
> 1. We can define that, if we have a string, then the string is 
> always
>    implicitly a variant type that we pass the converters.  Just 
>    take out
>    the local_value clobbering in the C code.  The HTML converter 
>    and all
>    other converters can then consistently look for and extract 
>    their
>    relevant property from the string.  This is a breaking behavior
>    change, but for already-broken behavior.  And the built-in 
>    converters
>    in select.el don’t seem to care about those properties.
> 
> 2. We can put the properties back in.  Once we extract the 
> property
>    string local_value, copy the properties of the original string 
>    tem
>    into local_value.  Then overwrite tem.  The rule for handlers 
>    is then
>    to *look* for the property, but it can use property’s string or 
>    the
>    underlying string.
> 
> 3. We can declare that callers have to add type tags to the 
> property objects.
> 
>      #("foo" 0 3 (text/html (html . "<i>foo</i>")))
> 
>    Then the converters are responsible for receiving that string 
>    *or*
>    (html . "<i>foo</i>"), depending on which function calls them,
>    handling both inputs.  This is ugly, but it works for a 
>    prototype
>    HTML converter on top of the existing v29.4 code.

Po Lu, any comments?

bug#72983: 29.4; Inconsistent parameter types sent to GUI selection converters

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> Po Lu, any comments?

The facilities that currently exist for customizing
selection-converter-alist and the data types returned for a conversion
from a string to any particular target are only designed for drag and
drop operations, unfortunately.  If you remind me of this report after
the 30.1 release (or maybe in a number of weeks) I will probably find
enough time to address it.  At the moment, I'm only willing to attend to
issues that affect new functionality introduced in Emacs 30, or
absolutely critical issues.

Sorry to disappoint.