Converting compositions to strings

Converting compositions to strings

[Clément Pit-Claudel]

Hi emacs-devel,

Many compositions can be represented as strings: for example, prettify-symbols-mode composes "lambda" into "λ", and nameless prettifies the name of the current package as ":" or "::".

How can I reconstruct these strings from the value of the 'composition' text property?  Small experiments suggest that for single-character compositions the property has the shape (_ _ [?λ]) and for multi-character compositions (_ _ [?x _ ?y _ …]), but I also found instances of the shape ((_ ?x _ _)), and ((_ . ?x)) (I think).

Is there a recipe that I can use to construct a string representation of a composition? (I realize that many compositions can't adequately be represented as plain strings; I'm only interested in those that can).

Thanks!
Clément.

Re: Converting compositions to strings

[Stefan Monnier]

> How can I reconstruct these strings from the value of the 'composition' text
> property?

This value is documented explicitly as being "internal! don't look at it!".


        Stefan

Re: Converting compositions to strings

[Ted Zlatanov]

On Mon, 30 Jan 2017 00:13:34 -0500 Clément Pit-Claudel <cpitclaudel@gmail.com> wrote: 

CP> Many compositions can be represented as strings: for example, prettify-symbols-mode composes "lambda" into "λ", and nameless prettifies the name of the current package as ":" or "::".

CP> How can I reconstruct these strings from the value of the 'composition' text
CP> property?

Can you explain why you need this?

Ted

Re: Converting compositions to strings

[Eli Zaretskii]

> From: Clément Pit-Claudel <cpitclaudel@gmail.com>
> Date: Mon, 30 Jan 2017 00:13:34 -0500
> 
> Many compositions can be represented as strings: for example, prettify-symbols-mode composes "lambda" into "λ", and nameless prettifies the name of the current package as ":" or "::".
> 
> How can I reconstruct these strings from the value of the 'composition' text property?  Small experiments suggest that for single-character compositions the property has the shape (_ _ [?λ]) and for multi-character compositions (_ _ [?x _ ?y _ …]), but I also found instances of the shape ((_ ?x _ _)), and ((_ . ?x)) (I think).

The information about this can be found in the doc strong of
compose-region, under COMPONENTS.

> Is there a recipe that I can use to construct a string representation of a composition? (I realize that many compositions can't adequately be represented as plain strings; I'm only interested in those that can).

I don't really understand what you mean by "string representation of a
composition".  In general, a composition doesn't represent a string,
and even those that are represented by characters or strings in the
property are many times something other than a string.  The ones that
seem to be relevant to what you are looking for (I'm guessing) are a
small class, and I don't think you can discern them from the others
unless you look at the "rules" parts of a composition.

Re: Converting compositions to strings

[Clément Pit-Claudel]

On 2017-01-30 08:34, Stefan Monnier wrote:
>> How can I reconstruct these strings from the value of the 'composition' text
>> property?
> 
> This value is documented explicitly as being "internal! don't look at it!".

I saw that :) Are you offering to implement a high-level primitive that does what I need without looking at it? ;)

Re: Converting compositions to strings

[Clément Pit-Claudel]

On 2017-01-30 09:24, Ted Zlatanov wrote:
> Can you explain why you need this?

Sure :) I need it for (an equivalent of) htmlfontify.  In the current version, exporting a buffer to HTML ignores compositions entirely (including prettify-symbols-mode).  I posted a patch that extends htmlfontify to render compositions a while ago, but the approach that I used there wasn't very principled (https://lists.gnu.org/archive/html/emacs-devel/2016-01/msg00845.html).

Clément.

Re: Converting compositions to strings

[Clément Pit-Claudel]

On 2017-01-30 10:18, Eli Zaretskii wrote:
> The information about this can be found in the doc strong of 
> compose-region, under COMPONENTS.

Thanks.  This covers only "relative" compositions, right? Is there more information on non-relative compositions somewhere?

>> Is there a recipe that I can use to construct a string 
>> representation of a composition? (I realize that many compositions 
>> can't adequately be represented as plain strings; I'm only 
>> interested in those that can).
> 
> The ones that seem to be relevant to what you are looking for (I'm 
> guessing) are a small class, and I don't think you can discern them 
> from the others unless you look at the "rules" parts of a 
> composition.

Looking at the rules sounds fine; thanks! I think these compositions form a small class in terms of what Emacs supports, but that class seems to most if not all uses of composition that I see in my daily use of Emacs (where are other compositions used, beyond character spacing in what-cursor-position?).

Based on your pointers, the following seems to work OK for me (I guess it could be extended to cover the (Bl . Br) case, but I haven't seen that pop up yet in practice).

(defun esh--parse-composition (components)
  "Translate composition COMPONENTS into a string."
  (let ((chars (list (aref components 0)))
        (nrules (/ (length components) 2)))
    (dotimes (nrule nrules)
      (let* ((rule (aref components (+ 1 (* 2 nrule))))
             (char (aref components (+ 2 (* 2 nrule)))))
        (pcase rule
          (`(Br . Bl) (push char chars))
          (_ (error "Unsupported composition COMPONENTS")))))
    (concat chars)))

Thanks,
Clément.

Re: Converting compositions to strings

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Clément Pit-Claudel <cpitclaudel@gmail.com>
> Date: Mon, 30 Jan 2017 11:39:09 -0500
> 
> On 2017-01-30 10:18, Eli Zaretskii wrote:
> > The information about this can be found in the doc strong of 
> > compose-region, under COMPONENTS.
> 
> Thanks.  This covers only "relative" compositions, right? Is there more information on non-relative compositions somewhere?

No, AFAIK this documents both relative and rule-based compositions.

> > The ones that seem to be relevant to what you are looking for (I'm 
> > guessing) are a small class, and I don't think you can discern them 
> > from the others unless you look at the "rules" parts of a 
> > composition.
> 
> Looking at the rules sounds fine; thanks! I think these compositions form a small class in terms of what Emacs supports, but that class seems to most if not all uses of composition that I see in my daily use of Emacs (where are other compositions used, beyond character spacing in what-cursor-position?).
> 
> Based on your pointers, the following seems to work OK for me (I guess it could be extended to cover the (Bl . Br) case, but I haven't seen that pop up yet in practice).
> 
> (defun esh--parse-composition (components)
>   "Translate composition COMPONENTS into a string."
>   (let ((chars (list (aref components 0)))
>         (nrules (/ (length components) 2)))
>     (dotimes (nrule nrules)
>       (let* ((rule (aref components (+ 1 (* 2 nrule))))
>              (char (aref components (+ 2 (* 2 nrule)))))
>         (pcase rule
>           (`(Br . Bl) (push char chars))
>           (_ (error "Unsupported composition COMPONENTS")))))
>     (concat chars)))

I think if we want to be able to produce text with all the text
properties and overlays expressed explicitly as text, we need to write
a back-end for xdisp.c that "displays" to a buffer.  It shouldn't be
hard, the number of interface routines a back-end needs to support is
not large; see 'struct redisplay_interface' in dispextern.h.

Re: Converting compositions to strings

[Ted Zlatanov]

On Mon, 30 Jan 2017 11:27:09 -0500 Clément Pit-Claudel <cpitclaudel@gmail.com> wrote: 

CP> On 2017-01-30 09:24, Ted Zlatanov wrote:
>> Can you explain why you need this?

CP> Sure :) I need it for (an equivalent of) htmlfontify. In the current version,
CP> exporting a buffer to HTML ignores compositions entirely (including
CP> prettify-symbols-mode). I posted a patch that extends htmlfontify to render
CP> compositions a while ago, but the approach that I used there wasn't very
CP> principled
CP> (https://lists.gnu.org/archive/html/emacs-devel/2016-01/msg00845.html).

I see, thanks for explaining. That sounds fun, I would use it.

Ted

Re: Converting compositions to strings

[Stefan Monnier]

> I think if we want to be able to produce text with all the text
> properties and overlays expressed explicitly as text, we need to write
> a back-end for xdisp.c that "displays" to a buffer.  It shouldn't be
> hard, the number of interface routines a back-end needs to support is
> not large; see 'struct redisplay_interface' in dispextern.h.

And IIRC it was mentioned that it would be helpful for some ERT tests
(and even more so, if it can render the whole screen, so we can test
the appearance of the modeline and headerline as well).


        Stefan