Re: Render a buffer or string to a simpler string?

Re: Render a buffer or string to a simpler string?

[Barry Margolin]

In article <mailman.381.1369510626.22516.help-gnu-emacs@gnu.org>,
 Dmitry Gutov <dgutov@yandex.ru> wrote:

> I'm currently fixing Company's popup overlay positioning and display
> interacts with org-indent-mode. The latter adds `line-prefix' text
> properties everywhere in the buffer, and they influence the offset of
> each line of the popup "rectangle".
> 
> Is there an easy way to convert a string with `line-prefix' and,
> optionally, `prefix' and `display' properties to a string without those
> properties corresponding to how the former string is going to be
> displayed?
> 
> Or, instead of a string, convert a buffer?

(remove-list-of-properties 0 (length string)
                           '(line-prefix prefix display)
                           string)

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Render a buffer or string to a simpler string?

[Dmitry Gutov]

I'm currently fixing Company's popup overlay positioning and display
interacts with org-indent-mode. The latter adds `line-prefix' text
properties everywhere in the buffer, and they influence the offset of
each line of the popup "rectangle".

Is there an easy way to convert a string with `line-prefix' and,
optionally, `prefix' and `display' properties to a string without those
properties corresponding to how the former string is going to be
displayed?

Or, instead of a string, convert a buffer?

Somewhat relatedly, is there a way to make `count-screen-lines' work in
batch mode (i.e. not return crazy numbers)? It would be nice to be able
to run code that calls it on an integration server.

Re: Render a buffer or string to a simpler string?

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 25 May 2013 23:36:52 +0400
> 
> Is there an easy way to convert a string with `line-prefix' and,
> optionally, `prefix' and `display' properties to a string without those
> properties corresponding to how the former string is going to be
> displayed?

Why do you need that?  What do you need to accomplish that needs such
a strange feature?

Re: Render a buffer or string to a simpler string?

[Dmitry Gutov]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Dmitry Gutov <dgutov@yandex.ru>
>> Date: Sat, 25 May 2013 23:36:52 +0400
>> 
>> Is there an easy way to convert a string with `line-prefix' and,
>> optionally, `prefix' and `display' properties to a string without those
>> properties corresponding to how the former string is going to be
>> displayed?
>
> Why do you need that?  What do you need to accomplish that needs such
> a strange feature?

Write an automated test (with ERT) that would be easy to read and modify
later.

I could, for example, compare against a string artificially constructed
with `ert-propertized-string', but it requires more effort, it's easy to
get wrong, and it would not express the intended feature.

To put it differently, the goal is to make the buffer look a certain way,
so I'd like to be able to check that it does look that way.

(For my current purpose, it would be fine if that render-to-string
function disregarded eyecandy text-properties, like `face', but in the
general case, being able to compare them, too, would be handy.)

Re: Render a buffer or string to a simpler string?

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Cc: help-gnu-emacs@gnu.org
> Date: Sun, 26 May 2013 01:46:52 +0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Dmitry Gutov <dgutov@yandex.ru>
> >> Date: Sat, 25 May 2013 23:36:52 +0400
> >> 
> >> Is there an easy way to convert a string with `line-prefix' and,
> >> optionally, `prefix' and `display' properties to a string without those
> >> properties corresponding to how the former string is going to be
> >> displayed?
> >
> > Why do you need that?  What do you need to accomplish that needs such
> > a strange feature?
> 
> Write an automated test (with ERT) that would be easy to read and modify
> later.

We lack such a feature currently.  Display rendering is a C-level
operation, whose result is not a string, but an array of structures
called "glyphs" which are passed to the terminal back end for drawing
on the screen.  So you need primitives (which don't exist) to produce
Lisp strings out of those glyphs.

> To put it differently, the goal is to make the buffer look a certain way,
> so I'd like to be able to check that it does look that way.

That's hard to do automatically, unless you use external software that
grabs screen portions.  Rendering is just one of the aspects, there's
also alignment, decorations, remnants from previous redisplay cycles,
etc.

> (For my current purpose, it would be fine if that render-to-string
> function disregarded eyecandy text-properties, like `face', but in the
> general case, being able to compare them, too, would be handy.)

Removing properties from strings is a feature we do have.

Re: Render a buffer or string to a simpler string?

[Dmitry Gutov]

Eli Zaretskii <eliz@gnu.org> writes:
> We lack such a feature currently.  Display rendering is a C-level
> operation, whose result is not a string, but an array of structures
> called "glyphs" which are passed to the terminal back end for drawing
> on the screen.  So you need primitives (which don't exist) to produce
> Lisp strings out of those glyphs.

That's what I figured, but had to ask anyway. Thanks for the reply.
Anyway, instead of writing a hacky fix + a bunch of new tests, I ended
up finding a better fix instead (and changing existing tests).

>> To put it differently, the goal is to make the buffer look a certain way,
>> so I'd like to be able to check that it does look that way.
>
> That's hard to do automatically, unless you use external software that
> grabs screen portions.  Rendering is just one of the aspects, there's
> also alignment, decorations, remnants from previous redisplay cycles,
> etc.

I think it's a shame, because out of all (?) text editors, Emacs is the
best positioned to enable human-readable UI tests, because of how often
people use text properties to do visuals.

Testing UI look in graphical applications has to involve screen grabs,
at least on some level.

>> (For my current purpose, it would be fine if that render-to-string
>> function disregarded eyecandy text-properties, like `face', but in the
>> general case, being able to compare them, too, would be handy.)
>
> Removing properties from strings is a feature we do have.

Sure. That's not what I was asking. If render-to-string included those
text properties, that would be fine, too.

Re: Render a buffer or string to a simpler string?

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Cc: help-gnu-emacs@gnu.org
> Date: Sun, 26 May 2013 07:38:30 +0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> > We lack such a feature currently.  Display rendering is a C-level
> > operation, whose result is not a string, but an array of structures
> > called "glyphs" which are passed to the terminal back end for drawing
> > on the screen.  So you need primitives (which don't exist) to produce
> > Lisp strings out of those glyphs.
> 
> That's what I figured, but had to ask anyway. Thanks for the reply.

Btw, the general problem you described -- "render to a string" -- is
not necessarily solvable even in principle.  E.g., how do you express
in a string variable fonts and other face attributes that don't change
the text, only how it is displayed? what about pixel-granular
alignment we do with :align-to and similar display properties? what to
do with images and sounds? etc. etc.

> >> To put it differently, the goal is to make the buffer look a certain way,
> >> so I'd like to be able to check that it does look that way.
> >
> > That's hard to do automatically, unless you use external software that
> > grabs screen portions.  Rendering is just one of the aspects, there's
> > also alignment, decorations, remnants from previous redisplay cycles,
> > etc.
> 
> I think it's a shame, because out of all (?) text editors, Emacs is the
> best positioned to enable human-readable UI tests, because of how often
> people use text properties to do visuals.

I don't understand this reasoning at all.  Why do you think Emacs is
best positioned for this?  And what have human-readable UI tests to do
with this; I thought you were talking about _automated_ tests?

> Testing UI look in graphical applications has to involve screen grabs,
> at least on some level.

When Emacs runs on a graphical terminal, it acts as a graphical
application.

Re: Render a buffer or string to a simpler string?

[Dmitry Gutov]

Eli Zaretskii <eliz@gnu.org> writes:
> Btw, the general problem you described -- "render to a string" -- is
> not necessarily solvable even in principle.  E.g., how do you express
> in a string variable fonts and other face attributes that don't change
> the text, only how it is displayed? what about pixel-granular
> alignment we do with :align-to and similar display properties? what to
> do with images and sounds? etc. etc.

That's true, but it can be solved well enough to be useful for testing
some classes of packages.

Pixel-granular alignment would probably be ignored, `face' properties
would be either ignored, or just kept as-is in the rendered string.

>> I think it's a shame, because out of all (?) text editors, Emacs is the
>> best positioned to enable human-readable UI tests, because of how often
>> people use text properties to do visuals.
>
> I don't understand this reasoning at all.  Why do you think Emacs is
> best positioned for this?  And what have human-readable UI tests to do
> with this; I thought you were talking about _automated_ tests?

A human has to write these tests, and then come back later, be able to
read and modify them. Try doing that efficiently with screen grabs.

>> Testing UI look in graphical applications has to involve screen grabs,
>> at least on some level.
>
> When Emacs runs on a graphical terminal, it acts as a graphical
> application.

The only means of building interface widgets exposed to Elisp are text
properties, AFAIK. They do include some graphical-only features, but
those aren't used as often as one might expect.

Re: Render a buffer or string to a simpler string?

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Cc: help-gnu-emacs@gnu.org
> Date: Sun, 26 May 2013 19:40:01 +0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> > Btw, the general problem you described -- "render to a string" -- is
> > not necessarily solvable even in principle.  E.g., how do you express
> > in a string variable fonts and other face attributes that don't change
> > the text, only how it is displayed? what about pixel-granular
> > alignment we do with :align-to and similar display properties? what to
> > do with images and sounds? etc. etc.
> 
> That's true, but it can be solved well enough to be useful for testing
> some classes of packages.

"Some classes" might not be enough for testing display.

> Pixel-granular alignment would probably be ignored, `face' properties
> would be either ignored, or just kept as-is in the rendered string.

Keeping the faces makes sense if they are automatically generated, but
not if they are put by explicit Lisp code, since then you just get
whatever the code put.

> >> I think it's a shame, because out of all (?) text editors, Emacs is the
> >> best positioned to enable human-readable UI tests, because of how often
> >> people use text properties to do visuals.
> >
> > I don't understand this reasoning at all.  Why do you think Emacs is
> > best positioned for this?  And what have human-readable UI tests to do
> > with this; I thought you were talking about _automated_ tests?
> 
> A human has to write these tests, and then come back later, be able to
> read and modify them. Try doing that efficiently with screen grabs.

UI testing people do that all the time: you compare the grab with some
baseline.

> >> Testing UI look in graphical applications has to involve screen grabs,
> >> at least on some level.
> >
> > When Emacs runs on a graphical terminal, it acts as a graphical
> > application.
> 
> The only means of building interface widgets exposed to Elisp are text
> properties, AFAIK. They do include some graphical-only features, but
> those aren't used as often as one might expect.

Try looking at Customize, and you will see a different story.

Re: Render a buffer or string to a simpler string?

[Dmitry Gutov]

Eli Zaretskii <eliz@gnu.org> writes:
>> Pixel-granular alignment would probably be ignored, `face' properties
>> would be either ignored, or just kept as-is in the rendered string.
>
> Keeping the faces makes sense if they are automatically generated, but
> not if they are put by explicit Lisp code, since then you just get
> whatever the code put.

There's no easy way to compare faces, save for rendering to some
external format like images or html. So yeah, this mechanism by itself
would be unsuitable for testing i.e. syntax highlighting. If you recall,
I gave some examples of text properties that *could* be tested with it.

I'll admit that `before-string', `display', etc, are not used too much
by packages everywhere, but they do get used, and other packages
sometimes have to interoperate with those that do. (*) (**)

>> A human has to write these tests, and then come back later, be able to
>> read and modify them. Try doing that efficiently with screen grabs.
>
> UI testing people do that all the time: you compare the grab with some
> baseline.

I'm not saying it can't be done with grabs.

http://en.wikipedia.org/wiki/Efficiency

And if you're suggesting just comparing screenshots, even font rendering
is very different across platforms, and it's affected by many settings,
so each such reference image would be platform- and settings-specific.
Good luck writing a portable test suite.

>> The only means of building interface widgets exposed to Elisp are text
>> properties, AFAIK. They do include some graphical-only features, but
>> those aren't used as often as one might expect.
>
> Try looking at Customize, and you will see a different story.

Customize is just one package.

By "often", I meant the percentage of packages that do, compared to the
total amount, not "number of interactions by users per hour".

But Customize still works with -nw, right? It might be helpful to test
just how it looks in non-graphical terminal, and verify any additional
features separately.

(*) Two examples:

- `outlne-minor-mode' hides a portion of buffer text, our overlay covers
  it (https://github.com/company-mode/company-mode/issues/16), we'd like
  to make sure the rectangle is still still drawn in the right shape.

- `org-indent-mode' uses `line-prefix' and `wrap-prefix' text
  properties, and they seep into strings we collect from the buffer
  before drawing over them
  (https://github.com/company-mode/company-mode/issues/24).
  We'd like to make sure the lines still look fine, even when there are
  such text properties inside the overlay's `before-string'.

I ended up removing `line-prefix' from all collected lines, prepending
the values physically to the strings that get collected, but it's not
fun that I need to look up the property value at the beginning of the
line and act on it, instead of just asking "gimme the line like it
looks!" (actually, I should search the whole line, but that's more work,
so I'm not doing that until someone else asks). I don't handle
`wrap-prefix' here at all because it's one more special case, and
luckily, org-mode buffers truncate lines by default.

Another thing I found lacking is a version of `current-column' that's
aware of those text properties.

(**) Another feature that would render better to strings is visual line
folding (when truncation is off). For example, the candidates popup in
Company needs to work with visual lines, not physical ones, so writing a
test that ensures that popup overlay took the requires lines and painted
over them, without jumping over folded lines, woult be useful.

Although, in this case, the harder part is setting up the test data the
right way to make sure the line does get wrapped where it should.

Re: Render a buffer or string to a simpler string?

[esabof]

A couple of notes:

It shouldn't be very hard to create such a function in elisp - the subset of display properties you need is probably not very large.

Also, how hard it would be to put a single transparent image on top of a frame? This would allow more robust pop-ups, as well as other UI elements by using SVG.

Evgeni

Re: Render a buffer or string to a simpler string?

[Dmitry Gutov]

esabof@gmail.com writes:

> It shouldn't be very hard to create such a function in elisp - the subset of display properties you need is probably not very large.

Doing that independent of rendering engine wouldn't be very durable, the
tests using the result wouldn't be protected against display engine bugs
and changes in behavior. And there are fine details like `line-prefix'
values priorities (what if we have several such properties on one line?)
that I think only expressed at C level.

> Also, how hard it would be to put a single transparent image on top of a frame? This would allow more robust pop-ups, as well as other UI elements by using SVG.

There was a discussion a little while ago about generalizing
`x-show-tip' so it could be used to render popups of this kind. It's a
nice idea, although it would benefit only the users of future Emacs
releases.

Re: Render a buffer or string to a simpler string?

[Stefan Monnier]

> We lack such a feature currently.  Display rendering is a C-level
> operation, whose result is not a string, but an array of structures
> called "glyphs" which are passed to the terminal back end for drawing
> on the screen.  So you need primitives (which don't exist) to produce
> Lisp strings out of those glyphs.

Maybe we could provide some way to "render to Elisp", which would
provide access to the glyph matrix somehow (probably exporting it as,
say, an array of "glyphs").  I'd imagine an interface along the lines of
"(render-to-lisp START END &optional WINDOW)" to render a particular
region of the current buffer as it would be displayed in WINDOW.

Obviously, some ERT tests could use such a feature, but things like
htmlfontify could probably also make use of it.


        Stefan

Re: Render a buffer or string to a simpler string?

[Dmitry Gutov]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> We lack such a feature currently.  Display rendering is a C-level
>> operation, whose result is not a string, but an array of structures
>> called "glyphs" which are passed to the terminal back end for drawing
>> on the screen.  So you need primitives (which don't exist) to produce
>> Lisp strings out of those glyphs.
>
> Maybe we could provide some way to "render to Elisp", which would
> provide access to the glyph matrix somehow (probably exporting it as,
> say, an array of "glyphs").  I'd imagine an interface along the lines of
> "(render-to-lisp START END &optional WINDOW)" to render a particular
> region of the current buffer as it would be displayed in WINDOW.

Right, I should be able to use that almost directly, even though I've no
idea what kind of values are contained in "glyphs". I'd just render to
glyphs both the buffer under test and a reference buffer, and compare
results.

Re: Render a buffer or string to a simpler string?

[esabof]

> Maybe we could provide some way to "render to Elisp", which would
> 
> provide access to the glyph matrix somehow (probably exporting it as,
> 
> say, an array of "glyphs").  I'd imagine an interface along the lines of
> 
> "(render-to-lisp START END &optional WINDOW)" to render a particular
> 
> region of the current buffer as it would be displayed in WINDOW.
> 
> 
> 
> Obviously, some ERT tests could use such a feature, but things like
> 
> htmlfontify could probably also make use of it.

Another primitive which I think doesn't exist, is given a position, return it's pixel coordinates. From this one could construct things like region-pixel-width and region-pixel-height.

It might also be useful to be able to store frame renderings as images and compare them. This would also allow a somewhat grotesque way to do transparency. (Save the image (which I think is already possible), overlay the buffer with a SVG, include the rendering in the SVG and render whatever necessary ontop of it).

Evgeni

Re: Render a buffer or string to a simpler string?

[Eli Zaretskii]

> Date: Sun, 26 May 2013 10:56:03 -0700 (PDT)
> From: esabof@gmail.com
> 
> Another primitive which I think doesn't exist, is given a position, return it's pixel coordinates.


You mean, like posn-at-point?

Re: Render a buffer or string to a simpler string?

[Dmitry Gutov]

esabof@gmail.com writes:
>> Obviously, some ERT tests could use such a feature, but things like
>> 
>> htmlfontify could probably also make use of it.
>
> Another primitive which I think doesn't exist, is given a position,
> return it's pixel coordinates. From this one could construct things
> like region-pixel-width and region-pixel-height.

See `posn-at-point'.

> It might also be useful to be able to store frame renderings as images
> and compare them. This would also allow a somewhat grotesque way to do
> transparency. (Save the image (which I think is already possible),
> overlay the buffer with a SVG, include the rendering in the SVG and
> render whatever necessary ontop of it).

The main problem as I see it, is how to output the difference
information in case of mismatch. Saying "these pics are different, go
look at them yourself" isn't very helpful for test-driven development.

This would extend to arrays of glyphs, too, maybe even to a larger
degree (haven't thought of that before).

Re: Render a buffer or string to a simpler string?

[esabof]

On Monday, May 27, 2013 3:47:12 AM UTC+1, Eli Zaretskii wrote:
> You mean, like posn-at-point?

Yes

Re: Render a buffer or string to a simpler string?

[esabof]

On Monday, May 27, 2013 3:48:54 AM UTC+1, Dmitry Gutov wrote:

> The main problem as I see it, is how to output the difference
> 
> information in case of mismatch. Saying "these pics are different, go
> 
> look at them yourself" isn't very helpful for test-driven development.
> 
> 
> 
> This would extend to arrays of glyphs, too, maybe even to a larger
> 
> degree (haven't thought of that before).

If you want to quickly become aware of differences between two images, you can switch between them - human eyes are very good at detecting movement.

Further, some image formats are quite straight forward (like BMP, if I'm not mistaken). It should be possible to implement a cropping function. It should also be possible to only display the rectangle which is different (with some padding). I guess this could be made into an extension to ERT, and images could be dumped along with other errors.

Evgeni