Better ways to inspect text properties?

Better ways to inspect text properties?

[Eduardo Ochs]

Hi list...

suppose that I create a file /tmp/foo.org whose contents are just
this:

  a[[http://b/][c]]d

and then I visit /tmp/foo.org, and I pretty-print the result of
(buffer-substring (point-min) (point-max)). The result is:

#("a[[http://b/][c]]d\n" 0 1
  (fontified t)
  1 13
  (face org-link font-lock-multiline t keymap
        (keymap
         (follow-link . mouse-face)
         (mouse-3 . org-find-file-at-mouse)
         (mouse-2 . org-open-at-mouse))
        mouse-face highlight invisible org-link htmlize-link
        (:uri "http://b/")
        help-echo "LINK: http://b/" fontified t)
  13 14
  (face org-link font-lock-multiline t keymap
        (keymap
         (follow-link . mouse-face)
         (mouse-3 . org-find-file-at-mouse)
         (mouse-2 . org-open-at-mouse))
        mouse-face highlight invisible org-link rear-nonsticky
        (mouse-face highlight keymap invisible intangible help-echo
org-linked-text htmlize-link)
        htmlize-link
        (:uri "http://b/")
        help-echo "LINK: http://b/" fontified t)
  14 15
  (font-lock-multiline t keymap
                       (keymap
                        (follow-link . mouse-face)
                        (mouse-3 . org-find-file-at-mouse)
                        (mouse-2 . org-open-at-mouse))
                       mouse-face highlight face org-link rear-nonsticky
                       (mouse-face highlight keymap invisible
intangible help-echo org-linked-text htmlize-link)
                       htmlize-link
                       (:uri "http://b/")
                       help-echo "LINK: http://b/" fontified t)
  15 16
  (font-lock-multiline t keymap
                       (keymap
                        (follow-link . mouse-face)
                        (mouse-3 . org-find-file-at-mouse)
                        (mouse-2 . org-open-at-mouse))
                       mouse-face highlight invisible org-link face
org-link htmlize-link
                       (:uri "http://b/")
                       help-echo "LINK: http://b/" fontified t)
  16 17
  (font-lock-multiline t keymap
                       (keymap
                        (follow-link . mouse-face)
                        (mouse-3 . org-find-file-at-mouse)
                        (mouse-2 . org-open-at-mouse))
                       mouse-face highlight invisible org-link face
org-link htmlize-link
                       (:uri "http://b/")
                       help-echo "LINK: http://b/" rear-nonsticky
                       (mouse-face highlight keymap invisible
intangible help-echo org-linked-text htmlize-link)
                       fontified t)
  17 18
  (rear-nonsticky t fontified t)
  18 19
  (fontified t))

This is not very human-friendly (to me). Can you recommend me ways to
optimize that output?

  Thanks in advance,
    Eduardo Ochs
    http://angg.twu.net/eepitch.html

Re: Better ways to inspect text properties?

[Emanuel Berg]

Eduardo Ochs wrote:

> This is not very human-friendly (to me). Can you recommend
> me ways to optimize that output?

Looks like if you don't have a full-blown and very good
interface to do that, it'll still be better just to look at
it ...

-- 
underground experts united
https://dataswamp.org/~incal

Re: Better ways to inspect text properties?

[Eli Zaretskii]

> From: Eduardo Ochs <eduardoochs@gmail.com>
> Date: Wed, 26 Oct 2022 20:19:59 -0300
> 
> suppose that I create a file /tmp/foo.org whose contents are just
> this:
> 
>   a[[http://b/][c]]d
> 
> and then I visit /tmp/foo.org, and I pretty-print the result of
> (buffer-substring (point-min) (point-max)). The result is:
> 
> #("a[[http://b/][c]]d\n" 0 1
>   (fontified t)
>   1 13
>   (face org-link font-lock-multiline t keymap
>         (keymap
>          (follow-link . mouse-face)
>          (mouse-3 . org-find-file-at-mouse)
>          (mouse-2 . org-open-at-mouse))
>         mouse-face highlight invisible org-link htmlize-link
>         (:uri "http://b/")
>         help-echo "LINK: http://b/" fontified t)
>   13 14
>   (face org-link font-lock-multiline t keymap
>         (keymap
>          (follow-link . mouse-face)
>          (mouse-3 . org-find-file-at-mouse)
>          (mouse-2 . org-open-at-mouse))
>         mouse-face highlight invisible org-link rear-nonsticky
>         (mouse-face highlight keymap invisible intangible help-echo
> org-linked-text htmlize-link)
>         htmlize-link
>         (:uri "http://b/")
>         help-echo "LINK: http://b/" fontified t)
>   14 15
>   (font-lock-multiline t keymap
>                        (keymap
>                         (follow-link . mouse-face)
>                         (mouse-3 . org-find-file-at-mouse)
>                         (mouse-2 . org-open-at-mouse))
>                        mouse-face highlight face org-link rear-nonsticky
>                        (mouse-face highlight keymap invisible
> intangible help-echo org-linked-text htmlize-link)
>                        htmlize-link
>                        (:uri "http://b/")
>                        help-echo "LINK: http://b/" fontified t)
>   15 16
>   (font-lock-multiline t keymap
>                        (keymap
>                         (follow-link . mouse-face)
>                         (mouse-3 . org-find-file-at-mouse)
>                         (mouse-2 . org-open-at-mouse))
>                        mouse-face highlight invisible org-link face
> org-link htmlize-link
>                        (:uri "http://b/")
>                        help-echo "LINK: http://b/" fontified t)
>   16 17
>   (font-lock-multiline t keymap
>                        (keymap
>                         (follow-link . mouse-face)
>                         (mouse-3 . org-find-file-at-mouse)
>                         (mouse-2 . org-open-at-mouse))
>                        mouse-face highlight invisible org-link face
> org-link htmlize-link
>                        (:uri "http://b/")
>                        help-echo "LINK: http://b/" rear-nonsticky
>                        (mouse-face highlight keymap invisible
> intangible help-echo org-linked-text htmlize-link)
>                        fontified t)
>   17 18
>   (rear-nonsticky t fontified t)
>   18 19
>   (fontified t))
> 
> This is not very human-friendly (to me). Can you recommend me ways to
> optimize that output?

Please tell which part(s) is/are not very human-friendly in the above.
Otherwise it's hard to suggest improvements, because to me the above
is quite friendly and easy to read and understand.

Re: Better ways to inspect text properties?

[Eduardo Ochs]

On Thu, 27 Oct 2022 at 02:48, Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Eduardo Ochs <eduardoochs@gmail.com>
> > Date: Wed, 26 Oct 2022 20:19:59 -0300
> >
> > suppose that I create a file /tmp/foo.org whose contents are just
> > this:
> >
> >   a[[http://b/][c]]d
> >
> > and then I visit /tmp/foo.org, and I pretty-print the result of
> > (buffer-substring (point-min) (point-max)). The result is:
> >
> > #("a[[http://b/][c]]d\n" 0 1
> >   (fontified t)
> >   1 13
> >   (face org-link font-lock-multiline t keymap
> >         (keymap
> >          (follow-link . mouse-face)
> >          (mouse-3 . org-find-file-at-mouse)
> >          (mouse-2 . org-open-at-mouse))
> >         mouse-face highlight invisible org-link htmlize-link
> >         (:uri "http://b/")
> >         help-echo "LINK: http://b/" fontified t)
> >   13 14
> >   (face org-link font-lock-multiline t keymap
> >         (keymap
> >          (follow-link . mouse-face)
> >          (mouse-3 . org-find-file-at-mouse)
> >          (mouse-2 . org-open-at-mouse))
> >         mouse-face highlight invisible org-link rear-nonsticky
> >         (mouse-face highlight keymap invisible intangible help-echo
> > org-linked-text htmlize-link)
> >         htmlize-link
> >         (:uri "http://b/")
> >         help-echo "LINK: http://b/" fontified t)
> >   14 15
> >   (font-lock-multiline t keymap
> >                        (keymap
> >                         (follow-link . mouse-face)
> >                         (mouse-3 . org-find-file-at-mouse)
> >                         (mouse-2 . org-open-at-mouse))
> >                        mouse-face highlight face org-link rear-nonsticky
> >                        (mouse-face highlight keymap invisible
> > intangible help-echo org-linked-text htmlize-link)
> >                        htmlize-link
> >                        (:uri "http://b/")
> >                        help-echo "LINK: http://b/" fontified t)
> >   15 16
> >   (font-lock-multiline t keymap
> >                        (keymap
> >                         (follow-link . mouse-face)
> >                         (mouse-3 . org-find-file-at-mouse)
> >                         (mouse-2 . org-open-at-mouse))
> >                        mouse-face highlight invisible org-link face
> > org-link htmlize-link
> >                        (:uri "http://b/")
> >                        help-echo "LINK: http://b/" fontified t)
> >   16 17
> >   (font-lock-multiline t keymap
> >                        (keymap
> >                         (follow-link . mouse-face)
> >                         (mouse-3 . org-find-file-at-mouse)
> >                         (mouse-2 . org-open-at-mouse))
> >                        mouse-face highlight invisible org-link face
> > org-link htmlize-link
> >                        (:uri "http://b/")
> >                        help-echo "LINK: http://b/" rear-nonsticky
> >                        (mouse-face highlight keymap invisible
> > intangible help-echo org-linked-text htmlize-link)
> >                        fontified t)
> >   17 18
> >   (rear-nonsticky t fontified t)
> >   18 19
> >   (fontified t))
> >
> > This is not very human-friendly (to me). Can you recommend me ways to
> > optimize that output?
>
> Please tell which part(s) is/are not very human-friendly in the above.
> Otherwise it's hard to suggest improvements, because to me the above
> is quite friendly and easy to read and understand.

Ooops, sorry! My problem is that that output is too big, and many
parts are repeated... I was wondering that there could be functions
around that would split the original string into substrings with
constant properties, and also would rewrite that output in other
formats, putting one property-value pair in each line, sorting these
pairs by property name, and maybe doing something nice with the
keymaps... I am trying to write something like that myself, but I
thought that maybe I could be reinventing the wheel...

  Cheers,
    Eduardo Ochs
    http://angg.twu.net/eepitch.html

Re: Better ways to inspect text properties?

[Emanuel Berg]

Eduardo Ochs wrote:

> My problem is that that output is too big, and many parts
> are repeated... I was wondering that there could be
> functions around that would split the original string [...]

What is it that you want outputted? And how?

my-data: is stable, sometimes?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Better ways to inspect text properties?

[Eduardo Ochs]

On Thu, 27 Oct 2022 at 02:48, Eli Zaretskii <eliz@gnu.org> wrote:
> Please tell which part(s) is/are not very human-friendly in the above.
> Otherwise it's hard to suggest improvements, because to me the above
> is quite friendly and easy to read and understand.

Hi Eli,

...problem solved. The main function that I am using is this one,

  (defun ee-string-intervals (str)
    "This is similar to `object-intervals', but uses another output format."
    (let ((pair< (lambda (pair1 pair2)
                   (string< (symbol-name (car pair1))
                            (symbol-name (car pair2))))))
      (cl-loop for (b e props) in (object-intervals str)
               for s = (substring-no-properties str b e)
               for pairs = (cl-loop for (x y) on props by 'cddr
                                    collect (list x y))
               collect (list s (sort pairs pair<)))))

and it can be tested with:

  (setq mystr (concat (propertize "a" 'p1 'v1 'p2 'v2)
      (propertize "b" 'p3 'v3 'p4 'v4)))

  (ee-string-intervals mystr)

I am experimenting with functions that extract the region with
buffer-substring, use `ee-string-intervals' to convert that
string with properties to something that I find better to work
with, and then deletes the uninteresting properties and
pretty-prints the result. I'll see if that helps me to understand
dired and org and then make a blog-ish post about that - now
without much fear that I am reinventing the wheel and making it
square...

  Cheers =),
    Eduardo Ochs
    http://angg.twu.net/eepitch.html

Re: Better ways to inspect text properties?

[Emanuel Berg]

Eduardo Ochs wrote:

>   (defun ee-string-intervals (str)
>     "This is similar to `object-intervals', but uses another output format."
>     (let ((pair< (lambda (pair1 pair2)
>                    (string< (symbol-name (car pair1))
>                             (symbol-name (car pair2))))))
>       (cl-loop for (b e props) in (object-intervals str)
>                for s = (substring-no-properties str b e)
>                for pairs = (cl-loop for (x y) on props by 'cddr
>                                     collect (list x y))
>                collect (list s (sort pairs pair<)))))

A Common Lisp-style loop with three clauses - the first one
a for loop, the second one an assignment to a variable, and
the third one an assignment to a variable the result of a for
loop - which makes a collection based on certain aspects as
explicitely extracted - and then it is all sorted according to
a user-defined anonymous function to impose a total order.

Hm ... maybe we can use this piece of code in our next book
aimed at absolute beginners, maybe to replace the initial
"hello world" example?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Better ways to inspect text properties?

[Eduardo Ochs]

On Fri, 28 Oct 2022 at 11:18, Emanuel Berg <incal@dataswamp.org> wrote:
 >
> A Common Lisp-style loop with three clauses - the first one
> a for loop, the second one an assignment to a variable, and
> the third one an assignment to a variable the result of a for
> loop - which makes a collection based on certain aspects as
> explicitely extracted - and then it is all sorted according to
> a user-defined anonymous function to impose a total order.
>
> Hm ... maybe we can use this piece of code in our next book
> aimed at absolute beginners, maybe to replace the initial
> "hello world" example?

Feel free! =)
  Cheers, E.

Re: Better ways to inspect text properties?

[Samuel Wales]

a good question imo!  tangentially, by coincidence i was wanting some
similar things to understand better.

showing the output prettily would be great.  c-u c-x = is just a bit
limited.  some kind of inspector?

what if we could show all text properties in a buffer kind of like visible-mode?

is it possible to diff two buffers completely?  i.e. not only the text
as shown [or folded] but the properties too?

that would allow regression testing of changes that affect only text
properties.  i don't mean displaying colors or so as a bespoke
non-traditional diff but rather the text property differnces as text.

can a buffer with text properties be saved to a file and reloaded
preserving properties?

what about visibility and folding?  could they be preserved like that
also?  my regression testing could be improved.

i also still struggle with visibility and folding and the fucntios for
getting visible-only parts of a buffer, or copying and inserting in
buffers, or looping only visible lines etc.  i keep getting results i
do not expect.  [cannot debug now.]  e.g. i am in magit and it is
partly folded and i want to do various programmatic copy or operate
type operations on visible.

i have not even yet found a good protocol for pp and frineds yet to
insert nicely into buffer at point, when called after a pp expression
with c-x c-=.  even for a paren-containing sexp, not only a string
with text properties.

so idk it is probably just me and not finding documentation that suits
me well for it, but it seems a ripe area.

On 10/26/22, Eduardo Ochs <eduardoochs@gmail.com> wrote:
> Hi list...
>
> suppose that I create a file /tmp/foo.org whose contents are just
> this:
>
>   a[[http://b/][c]]d
>
> and then I visit /tmp/foo.org, and I pretty-print the result of
> (buffer-substring (point-min) (point-max)). The result is:
>
> #("a[[http://b/][c]]d\n" 0 1
>   (fontified t)
>   1 13
>   (face org-link font-lock-multiline t keymap
>         (keymap
>          (follow-link . mouse-face)
>          (mouse-3 . org-find-file-at-mouse)
>          (mouse-2 . org-open-at-mouse))
>         mouse-face highlight invisible org-link htmlize-link
>         (:uri "http://b/")
>         help-echo "LINK: http://b/" fontified t)
>   13 14
>   (face org-link font-lock-multiline t keymap
>         (keymap
>          (follow-link . mouse-face)
>          (mouse-3 . org-find-file-at-mouse)
>          (mouse-2 . org-open-at-mouse))
>         mouse-face highlight invisible org-link rear-nonsticky
>         (mouse-face highlight keymap invisible intangible help-echo
> org-linked-text htmlize-link)
>         htmlize-link
>         (:uri "http://b/")
>         help-echo "LINK: http://b/" fontified t)
>   14 15
>   (font-lock-multiline t keymap
>                        (keymap
>                         (follow-link . mouse-face)
>                         (mouse-3 . org-find-file-at-mouse)
>                         (mouse-2 . org-open-at-mouse))
>                        mouse-face highlight face org-link rear-nonsticky
>                        (mouse-face highlight keymap invisible
> intangible help-echo org-linked-text htmlize-link)
>                        htmlize-link
>                        (:uri "http://b/")
>                        help-echo "LINK: http://b/" fontified t)
>   15 16
>   (font-lock-multiline t keymap
>                        (keymap
>                         (follow-link . mouse-face)
>                         (mouse-3 . org-find-file-at-mouse)
>                         (mouse-2 . org-open-at-mouse))
>                        mouse-face highlight invisible org-link face
> org-link htmlize-link
>                        (:uri "http://b/")
>                        help-echo "LINK: http://b/" fontified t)
>   16 17
>   (font-lock-multiline t keymap
>                        (keymap
>                         (follow-link . mouse-face)
>                         (mouse-3 . org-find-file-at-mouse)
>                         (mouse-2 . org-open-at-mouse))
>                        mouse-face highlight invisible org-link face
> org-link htmlize-link
>                        (:uri "http://b/")
>                        help-echo "LINK: http://b/" rear-nonsticky
>                        (mouse-face highlight keymap invisible
> intangible help-echo org-linked-text htmlize-link)
>                        fontified t)
>   17 18
>   (rear-nonsticky t fontified t)
>   18 19
>   (fontified t))
>
> This is not very human-friendly (to me). Can you recommend me ways to
> optimize that output?
>
>   Thanks in advance,
>     Eduardo Ochs
>     http://angg.twu.net/eepitch.html
>
>


-- 
The Kafka Pandemic

A blog about science, health, human rights, and misopathy:
https://thekafkapandemic.blogspot.com

Re: Better ways to inspect text properties?

[Eduardo Ochs]

On Sat, 29 Oct 2022 at 20:46, Samuel Wales <samologist@gmail.com> wrote:
>
> a good question imo!  tangentially, by coincidence i was wanting some
> similar things to understand better.
>
> showing the output prettily would be great.  c-u c-x = is just a bit
> limited.  some kind of inspector?
>
> what if we could show all text properties in a buffer kind of like visible-mode?
>
> is it possible to diff two buffers completely?  i.e. not only the text
> as shown [or folded] but the properties too?
>
> that would allow regression testing of changes that affect only text
> properties.  i don't mean displaying colors or so as a bespoke
> non-traditional diff but rather the text property differnces as text.
>
> can a buffer with text properties be saved to a file and reloaded
> preserving properties?
>
> what about visibility and folding?  could they be preserved like that
> also?  my regression testing could be improved.
>
> i also still struggle with visibility and folding and the fucntios for
> getting visible-only parts of a buffer, or copying and inserting in
> buffers, or looping only visible lines etc.  i keep getting results i
> do not expect.  [cannot debug now.]  e.g. i am in magit and it is
> partly folded and i want to do various programmatic copy or operate
> type operations on visible.
>
> i have not even yet found a good protocol for pp and frineds yet to
> insert nicely into buffer at point, when called after a pp expression
> with c-x c-=.  even for a paren-containing sexp, not only a string
> with text properties.
>
> so idk it is probably just me and not finding documentation that suits
> me well for it, but it seems a ripe area.

Hi Samuel!
By "an inspector" do you mean like what this package does?

  https://github.com/mmontone/emacs-inspector

I have just tried this

  (defun inspector-inspect-region (b e)
    (interactive "r")
    (inspector-inspect-expression (buffer-substring b e)))

in a grep buffer, and it seems that the current version of inspector
doesn't have support for treating strings with text properties as
trees... and then I tried this on the same grep buffer,

  (defun inspector-inspect-region (b e)
    (interactive "r")
    (inspector-inspect-expression
     '(ee-string-intervals
       (buffer-substring b e))))

where ee-string-intervals is the function defined here,

  (find-eev "eev-blinks.el" "find-eregionpp")
  http://angg.twu.net/eev-current/eev-blinks.el.html#find-eregionpp

and it worked very well - it was even able to show that certain
#s(...) objects are structs of the kind "compilation--message", and it
showed its slots.

I have added inspector's author - Mariano Montone - to the list of
recipients of this message, let's see what happens... =)

  Cheers,
    Eduardo Ochs
    http://angg.twu.net/eepitch.html

Re: Better ways to inspect text properties?

[Samuel Wales]

more below.

On 10/29/22, Eduardo Ochs <eduardoochs@gmail.com> wrote:
> On Sat, 29 Oct 2022 at 20:46, Samuel Wales <samologist@gmail.com> wrote:
>>
>> a good question imo!  tangentially, by coincidence i was wanting some
>> similar things to understand better.
>>
>> showing the output prettily would be great.  c-u c-x = is just a bit
>> limited.  some kind of inspector?
>>
>> what if we could show all text properties in a buffer kind of like
>> visible-mode?
>>
>> is it possible to diff two buffers completely?  i.e. not only the text
>> as shown [or folded] but the properties too?
>>
>> that would allow regression testing of changes that affect only text
>> properties.  i don't mean displaying colors or so as a bespoke
>> non-traditional diff but rather the text property differnces as text.
>>
>> can a buffer with text properties be saved to a file and reloaded
>> preserving properties?
>>
>> what about visibility and folding?  could they be preserved like that
>> also?  my regression testing could be improved.
>>
>> i also still struggle with visibility and folding and the fucntios for
>> getting visible-only parts of a buffer, or copying and inserting in
>> buffers, or looping only visible lines etc.  i keep getting results i
>> do not expect.  [cannot debug now.]  e.g. i am in magit and it is
>> partly folded and i want to do various programmatic copy or operate
>> type operations on visible.
>>
>> i have not even yet found a good protocol for pp and frineds yet to
>> insert nicely into buffer at point, when called after a pp expression
>> with c-x c-=.  even for a paren-containing sexp, not only a string
>> with text properties.
>>
>> so idk it is probably just me and not finding documentation that suits
>> me well for it, but it seems a ripe area.
>
> Hi Samuel!
> By "an inspector" do you mean like what this package does?
>
>   https://github.com/mmontone/emacs-inspector

i can't tell.  white-background images are very hard on me so i can't
look a them.  i occasionally used symbolics and  commercial unix cl
inspectors, but i also meant inspector in a more generic and vague
"let user explore" sense.

the idea being to help the user understand all that behind hte scenes
stuff in emacs buffers and strings, namely text properties, more
richly than c-u c-x =.  [never mind overlays for the time being.]
great an inspector package exists though.

... and diff buffers by text properties and pp and serialize and have
a full set of functions to do only visible stuff in various operations
like in magit or org buffers and all the rest of the things one might
want to do with text properties and folding.

idk about anybody else but i often put comments above lisp code with
some expression and then eval it in the .el buffer to show what the
function does or so.  and pp of some type should be useful for that.

eev, i'm afraid, i have heard about and skimmed for many many years,
but it has its terminology and sensibilities and never caught on for
me for whatever reason and atm i can't investigate it.

perhaps someday i will.  [i have burning need for specific link and
anchor functionality, specifically id markers, that are partly, but
not completely, roughly in, but not covered by, embark, eev,
hyperbole.  and that would be the hook for me.]

>
> I have just tried this
>
>   (defun inspector-inspect-region (b e)
>     (interactive "r")
>     (inspector-inspect-expression (buffer-substring b e)))
>
> in a grep buffer, and it seems that the current version of inspector
> doesn't have support for treating strings with text properties as
> trees... and then I tried this on the same grep buffer,
>
>   (defun inspector-inspect-region (b e)
>     (interactive "r")
>     (inspector-inspect-expression
>      '(ee-string-intervals
>        (buffer-substring b e))))
>
> where ee-string-intervals is the function defined here,
>
>   (find-eev "eev-blinks.el" "find-eregionpp")
>   http://angg.twu.net/eev-current/eev-blinks.el.html#find-eregionpp
>
> and it worked very well - it was even able to show that certain
> #s(...) objects are structs of the kind "compilation--message", and it
> showed its slots.

sounds ineresting.

>
> I have added inspector's author - Mariano Montone - to the list of
> recipients of this message, let's see what happens... =)

hehe.  :)

>
>   Cheers,
>     Eduardo Ochs
>     http://angg.twu.net/eepitch.html
>


-- 
The Kafka Pandemic

A blog about science, health, human rights, and misopathy:
https://thekafkapandemic.blogspot.com