Usage of org-element api

Usage of org-element api

[Dieter Van Eessen]

[-- Attachment #1: Type: text/plain, Size: 3370 bytes --]

Hello,

New to the list. Got some questions regarding the use of the org-element
api. I've already learned alot by reading the emacs help and org-developers
section. and the online org-element-docstrings have been very helpful. I'm
not a highly skilled lisp-programmer, but got good grip on the basics and I
learn fast :)

Still a few questions remain:

1) How to use org-element-content? Returns nil when used on elements parsed
by org-element-<element>-parser (because they operate locally)  When used
globally, it only seems to remove the car of the list (org-data ....)

2) What is actually a normal workflow if you wish to interactive manipulate
only some of the elements? Is it something like:
 a)  Define the region in which you wish to manipulate things (using :begin
and :extract from  org-element-at-point)
 b) Parse the region: Get TREE and CONTENT
     (Are they always separated for manipulation?)
 c) Manipulate tree AND/OR manipulate content
 d) Interpret (the org-element-<element>interpreters seem to require
<element> and content whilst the org-element-interpret-data only requires a
single 'data'. Why?)
 e) See the manipulated stuff appear in the buffer.

3) How can the output of (org-element-parse-secondary-string ...) be used.
When I give a heading and bit of text as input (output of
buffer-substring), it looks like it returns the 'content' of the region.
Though I can't seem to use it anyway as 'CONTENT' for the functions
requiring this.

4) How to use org-element--parse-elements? Whilst it is running i notice
that it uses (org-element--current-element) and some of the
(org-element-<element>-parser) functions. Thought it could be nice to use
this one, but no matter how use it, all I get is nil. For example:

 (setq element-parse (org-element--parse-elements (org-element-property
':begin (org-element-at-point)) (org-element-property ':end
(org-element-at-point)) 'section nil 'headline nil nil))

When I try such this ANYWHERE in the following text, i get nil.
* boe
* argh
** lol
    Something funny
** woot

5) What is org-element--current-element for? It also seems to be called by
org-element--parse-element.The properties :begin, :end and :title  seem
different than when parsing with org-element-at-point. Org-element-contents
also nill when applied on the output. Why can't I use
    this function (org-element--current-element) on plainlists/items
(returns error with point within a plain-list/item)?


I know the basic answer to most of these question is: Why don't you use
(org-element-parse-buffer). Well simply because I don't need everything. in
first implementation I only need:
OR HEADLINE under point and it's subtree
  (HEADLINE (plainlist (item,item,item)),(subheadline),...)
OR the headline of an ITEM under point and subtree
 (headline (plainlist (item, ITEM,item)),(subheadline),...).

Manipulating these trees won't be too hard. I don't neccesarily need the
contents, I can (buffer-substring) them using the :contents-begin and
:contents-end properties.
BUT I think I will need them to write back to the buffer after having
manipulated some things in the tree. Haven't investigated
(org-element-interpret-data) yet, nor all the
(org-element-<element>-interpreters) it calls.


It's a long mail, please don't yell at me if I overlooked stuff on the web,
it's quite a huge pool of information...

Dieter Van Eessen

[-- Attachment #2: Type: text/html, Size: 4041 bytes --]

Re: Usage of org-element api

[Rasmus]

Hi Dieter,

Nicolas will probably reply at some point and he has much greater (∞ more)
insight in this topics.  None the less, I hope the below message will
help a bit.

Dieter Van Eessen <dieter.van.eessen@gmail.com> writes:

> 1) How to use org-element-content? Returns nil when used on elements parsed
> by org-element-<element>-parser (because they operate locally)  When used
> globally, it only seems to remove the car of the list (org-data ....)

Don't use org-element-<element>-parser directly.

Use org-element-at-point or org-element-contents.  The former is for
elements, the latter is for contents, such as scripts, bold, etc.  See the
head of org-element.el.

> 2) What is actually a normal workflow if you wish to interactive manipulate
> only some of the elements? Is it something like:
>  a)  Define the region in which you wish to manipulate things (using :begin
> and :extract from  org-element-at-point)

Org-element is a parser and manipulation might not be super efficient with
org-element, but it can be done.

OTOH Org-syntax is great for manipulation.  E.g. to insert a heading you
can do (insert (format "* %s" "my-heading")).

>  b) Parse the region: Get TREE and CONTENT
>      (Are they always separated for manipulation?)

See org-element-map for operating on a subset of the consents.  You could
also use narrowing to operate only on a subset of the buffer.

>  c) Manipulate tree AND/OR manipulate content


Manipulate whatever you have as you want.  Org-elements are plists.  Check
org-element-type, org-element-property, org-element-contents,
org-element-put-property, org-element-set-contents.  See also
";;; Accessors and Setters" in org-element.el.

org-element-interpret-data is the way to go from an element to
org-syntax.  Here's an example:

             http://emacs.stackexchange.com/questions/2869/turn-a-list-or-data-structure-into-an-org-document/

>  d) Interpret (the org-element-<element>interpreters seem to require
> <element> and content whilst the org-element-interpret-data only requires a
> single 'data'. Why?)

Don't use org-element-<element>interpreters.  Use
org-element-interpret-data for transforming org-element→org-syntax.

>  e) See the manipulated stuff appear in the buffer.

There's insert for that.

> 3) How can the output of (org-element-parse-secondary-string ...) be used.
> When I give a heading and bit of text as input (output of
> buffer-substring), it looks like it returns the 'content' of the region.
> Though I can't seem to use it anyway as 'CONTENT' for the functions
> requiring this.

I don't get this.

> 4) How to use org-element--parse-elements? Whilst it is running i notice
> that it uses (org-element--current-element) and some of the
> (org-element-<element>-parser) functions. Thought it could be nice to use
> this one, but no matter how use it, all I get is nil. For example:

In Emacs-lisp typically "--" indicates that it's a private function.
Don't use it.

> 5) What is org-element--current-element for? It also seems to be called by
> org-element--parse-element.The properties :begin, :end and :title  seem
> different than when parsing with org-element-at-point. Org-element-contents
> also nill when applied on the output. Why can't I use
>     this function (org-element--current-element) on plainlists/items
> (returns error with point within a plain-list/item)?

See above.

> I know the basic answer to most of these question is: Why don't you use
> (org-element-parse-buffer). Well simply because I don't need everything. in
> first implementation I only need:
> OR HEADLINE under point and it's subtree
>   (HEADLINE (plainlist (item,item,item)),(subheadline),...)
> OR the headline of an ITEM under point and subtree
>  (headline (plainlist (item, ITEM,item)),(subheadline),...).

So use org-element-map and org-element-parse-buffer.

Hope it helps,
Rasmus

-- 
The second rule of Fight Club is: You do not talk about Fight Club

Re: Usage of org-element api

[Dieter Van Eessen]

[-- Attachment #1: Type: text/plain, Size: 7409 bytes --]

Hello Rasmus,

Thank you for the fast reply, the link you've given on interpreting is very
useful ! Also didn't know there existed such thing as the org-dp library to
manipulate org-elements, I'll sure check it out.

More about question number 3:
>> 3) How can the output of (org-element-parse-secondary-string ...) be
used.
>> When I give a heading and bit of text as input (output of
>> buffer-substring), it looks like it returns the 'content' of the region.
>> Though I can't seem to use it anyway as 'CONTENT' for the functions
>> requiring this.

The reason I've tried this (and the internal org-element--parse-elements)
is because I'd prefer not having to parse the whole buffer and still get
the contents. The local parsing functions (org-element-at-point) and
(org-element-context) don't contain content (stated in the org-element api,
also tried it ).

Now I'm not sure IF I really NEED it? I could actually get the contents
using the  :content-begin and :content-end and other properties from
(org-elemen-at-point)  BUT I don't know the exact syntax the content should
have and how to merge it with the element-list I get from
(org-element-at-point) before feeding it to the org-element-interpret-data.

I don't actually know how 'content' looks like...  When applying a small
function
" (defun test
()

    (interactive)

(debug)

    (setq content (org-element-contents (org-element-parse-buffer))))"
on a simple headline with some text,
" * woot
     And a bit of text "
I can see in debugger (returned values) that 'content' is actually
(org-data nil ('content'). See output below. Thus concluding that content
is also just an <element>. Then why do all the
org-element-<element>-interpreters require both the <element>  AND
'content', since to me, it seems logical that all this information is
already inside <element>.

At first I thought that the things behind #  were 'content' (for example in
the output below. These don't show up in (org-element-at-point), thus
explaining why they returned nil when asked for content. The
org-element-parse-secondary-string also returns all things behind #. If
this is NOT content, then how to manipulate this data (behind the #) whilst
assuring that syntax and position remains valid for
org-element-interpret-data to understand?

OUTPUT:
 (org-data nil (headline (:raw-value "woot" :begin 1 :end 29 :pre-blank 0
:contents-begin 8 ...) (section (:begin 8 :end 29 :contents-begin
  8 :contents-end 26 :post-blank 2 ...) (paragraph (:begin 8 :end 26
:contents-begin 8 :contents-end 26 :post-blank 0 ...) #("And a bit of
  text  " 0 18 (:parent #3))))))

  ((headline (:raw-value "woot" :begin 1 :end 29 :pre-blank 0
:contents-begin 8 ...) (section (:begin 8 :end 29 :contents-begin 8
  :contents-end 26 :post-blank 2 ...) (paragraph (:begin 8 :end 26
:contents-begin 8 :contents-end 26 :post-blank 0 ...) #("And a bit of text
  " 0 18 (:parent #3))))))

Perhaps my view is completely wrong, so please correct me if possible.
Thanks for the time,
Dieter





On Fri, Jan 16, 2015 at 8:35 PM, Rasmus <rasmus@gmx.us> wrote:

> Hi Dieter,
>
> Nicolas will probably reply at some point and he has much greater (∞ more)
> insight in this topics.  None the less, I hope the below message will
> help a bit.
>
> Dieter Van Eessen <dieter.van.eessen@gmail.com> writes:
>
> > 1) How to use org-element-content? Returns nil when used on elements
> parsed
> > by org-element-<element>-parser (because they operate locally)  When used
> > globally, it only seems to remove the car of the list (org-data ....)
>
> Don't use org-element-<element>-parser directly.
>
> Use org-element-at-point or org-element-contents.  The former is for
> elements, the latter is for contents, such as scripts, bold, etc.  See the
> head of org-element.el.
>
> > 2) What is actually a normal workflow if you wish to interactive
> manipulate
> > only some of the elements? Is it something like:
> >  a)  Define the region in which you wish to manipulate things (using
> :begin
> > and :extract from  org-element-at-point)
>
> Org-element is a parser and manipulation might not be super efficient with
> org-element, but it can be done.
>
> OTOH Org-syntax is great for manipulation.  E.g. to insert a heading you
> can do (insert (format "* %s" "my-heading")).
>
> >  b) Parse the region: Get TREE and CONTENT
> >      (Are they always separated for manipulation?)
>
> See org-element-map for operating on a subset of the consents.  You could
> also use narrowing to operate only on a subset of the buffer.
>
> >  c) Manipulate tree AND/OR manipulate content
>
>
> Manipulate whatever you have as you want.  Org-elements are plists.  Check
> org-element-type, org-element-property, org-element-contents,
> org-element-put-property, org-element-set-contents.  See also
> ";;; Accessors and Setters" in org-element.el.
>
> org-element-interpret-data is the way to go from an element to
> org-syntax.  Here's an example:
>
>
> http://emacs.stackexchange.com/questions/2869/turn-a-list-or-data-structure-into-an-org-document/
>
> >  d) Interpret (the org-element-<element>interpreters seem to require
> > <element> and content whilst the org-element-interpret-data only
> requires a
> > single 'data'. Why?)
>
> Don't use org-element-<element>interpreters.  Use
> org-element-interpret-data for transforming org-element→org-syntax.
>
> >  e) See the manipulated stuff appear in the buffer.
>
> There's insert for that.
>
> > 3) How can the output of (org-element-parse-secondary-string ...) be
> used.
> > When I give a heading and bit of text as input (output of
> > buffer-substring), it looks like it returns the 'content' of the region.
> > Though I can't seem to use it anyway as 'CONTENT' for the functions
> > requiring this.
>
> I don't get this.
>
> > 4) How to use org-element--parse-elements? Whilst it is running i notice
> > that it uses (org-element--current-element) and some of the
> > (org-element-<element>-parser) functions. Thought it could be nice to use
> > this one, but no matter how use it, all I get is nil. For example:
>
> In Emacs-lisp typically "--" indicates that it's a private function.
> Don't use it.
>
> > 5) What is org-element--current-element for? It also seems to be called
> by
> > org-element--parse-element.The properties :begin, :end and :title  seem
> > different than when parsing with org-element-at-point.
> Org-element-contents
> > also nill when applied on the output. Why can't I use
> >     this function (org-element--current-element) on plainlists/items
> > (returns error with point within a plain-list/item)?
>
> See above.
>
> > I know the basic answer to most of these question is: Why don't you use
> > (org-element-parse-buffer). Well simply because I don't need everything.
> in
> > first implementation I only need:
> > OR HEADLINE under point and it's subtree
> >   (HEADLINE (plainlist (item,item,item)),(subheadline),...)
> > OR the headline of an ITEM under point and subtree
> >  (headline (plainlist (item, ITEM,item)),(subheadline),...).
>
> So use org-element-map and org-element-parse-buffer.
>
> Hope it helps,
> Rasmus
>
> --
> The second rule of Fight Club is: You do not talk about Fight Club
>
>
>


-- 
gtz,
Dieter VE

[-- Attachment #2: Type: text/html, Size: 10109 bytes --]

Re: Usage of org-element api

[Nicolas Goaziou]

Hello,

Dieter Van Eessen <dieter.van.eessen@gmail.com> writes:

> The reason I've tried this (and the internal org-element--parse-elements)
> is because I'd prefer not having to parse the whole buffer and still get
> the contents. The local parsing functions (org-element-at-point) and
> (org-element-context) don't contain content (stated in the org-element api,
> also tried it ).

`org-element-parse-buffer' takes into consideration narrowing. IOW,
narrow to the interesting part, then parse it.


Regards,

-- 
Nicolas Goaziou

Re: Usage of org-element api

[Rasmus]

Hi Dieter,

Dieter Van Eessen <dieter.van.eessen@gmail.com> writes:

> Hello Rasmus,
>
> Thank you for the fast reply, the link you've given on interpreting is very
> useful ! Also didn't know there existed such thing as the org-dp library to
> manipulate org-elements, I'll sure check it out.

I don't know org-dp myself, but Thorsten posts here regularly.

> More about question number 3:
>>> 3) How can the output of (org-element-parse-secondary-string ...) be
>>> used.
>>> When I give a heading and bit of text as input (output of
>>> buffer-substring), it looks like it returns the 'content' of the region.
>>> Though I can't seem to use it anyway as 'CONTENT' for the functions
>>> requiring this.
>
> The reason I've tried this (and the internal org-element--parse-elements)
> is because I'd prefer not having to parse the whole buffer and still get
> the contents. The local parsing functions (org-element-at-point) and
> (org-element-context) don't contain content (stated in the org-element api,
> also tried it).

But all elements contain :begin and :end and most :contens-begin
and :contents-end (maybe sans an 's').

> Now I'm not sure IF I really NEED it? I could actually get the contents
> using the  :content-begin and :content-end and other properties from
> (org-elemen-at-point)  BUT I don't know the exact syntax the content should
> have and how to merge it with the element-list I get from
> (org-element-at-point) before feeding it to the org-element-interpret-data.

Maybe it would be easier if you state plainly what your desired goal is?
It's all a bit abstract.  You can write pretty sophisticated things using
just (org-element-at-point) (e.g. I have a function that escapes
*math-su{b,per}script* on double space at appropriate places).

> [...]
> At first I thought that the things behind #  were 'content' (for example in
> the output below. These don't show up in (org-element-at-point), thus
> explaining why they returned nil when asked for content. The
> org-element-parse-secondary-string also returns all things behind #. If
> this is NOT content, then how to manipulate this data (behind the #) whilst
> assuring that syntax and position remains valid for
> org-element-interpret-data to understand?

If you want to manipulate the buffer text use the great functions.  You
can condition on the element under point or whatever you desire and then
use the regular functions you'd otherwise use.  I'm probably wrong, but it
seems as if you trying to shoot flies with canons, or however the saying
goes in English.  Also, recall there is no such thing as wrong Org-syntax
(but there is unexpected outcomes).

Check Org.el if you want.

Cheers,
Rasmus

-- 
Not everything that goes around comes back around, you know

Re: Usage of org-element api

[Dieter Van Eessen]

[-- Attachment #1: Type: text/plain, Size: 5002 bytes --]

Hello,

Nicolas pointed me in the right direction! It was so obvious that I looked
right passed it:
Should just create a temp buffer with the text (headline+plainlist+text) i
wish to parse, then parse that temp-buffer... so obvious, sorry for the
waste of time.

Just to give an idea of what I'm trying to accomplish:
The first implementation remains simple: I'll create 2 interactive
functions,

1: Having point on headline, (concept-expand) will call
(concept-expand-headline) in which a level 1 headline containing a link is
expanded based on the content of that link. Subheadline becomes +
plainlist, +plainlist becomes -plainlist. For example:
* readme
   Some text...
** goal
    Anything...
** [[*something][something]] step
    + woot
* [[*readme][readme]] This is a readme

Expanding headline 'this is a readme' would result in
* [[*readme][readme]] This is a readme
    + goal
    + [[*something][something]] step

2: Having point on item, (concept-expand) will call  (concept-expand-item)
in which items get expanded
For example: expanding item 'step results in
* [[*readme][readme]] This is a readme
   + goal
   + [[*something][something]] step
** [[link of choice or no link at all]] Do this first         :step:

Now this does seem quite stupid. In a second and third implementation I'll
try to increase the locations where concepts may be found (project,
personal, system-wide), in files (based on filename and directory) or
subheadlines (instead of headlines), expanding subheadlines and files
(based on filename and directory),...

I'm kind of looking for a way to create some abstractions/concepts, without
ever being tied to a static model/template. People must never be forced to
use the system. It remains a free choice whether you use concepts, create
your own or just choose to write anything. Any document always remains
human readable plain text.  Some concepts will always be very divers,
others will survive the test of time and stabilize.

What do you think? Waste of time? :)

kind regards,
Dieter


On Sat, Jan 17, 2015 at 1:33 AM, Rasmus <rasmus@gmx.us> wrote:

> Hi Dieter,
>
> Dieter Van Eessen <dieter.van.eessen@gmail.com> writes:
>
> > Hello Rasmus,
> >
> > Thank you for the fast reply, the link you've given on interpreting is
> very
> > useful ! Also didn't know there existed such thing as the org-dp library
> to
> > manipulate org-elements, I'll sure check it out.
>
> I don't know org-dp myself, but Thorsten posts here regularly.
>
> > More about question number 3:
> >>> 3) How can the output of (org-element-parse-secondary-string ...) be
> >>> used.
> >>> When I give a heading and bit of text as input (output of
> >>> buffer-substring), it looks like it returns the 'content' of the
> region.
> >>> Though I can't seem to use it anyway as 'CONTENT' for the functions
> >>> requiring this.
> >
> > The reason I've tried this (and the internal org-element--parse-elements)
> > is because I'd prefer not having to parse the whole buffer and still get
> > the contents. The local parsing functions (org-element-at-point) and
> > (org-element-context) don't contain content (stated in the org-element
> api,
> > also tried it).
>
> But all elements contain :begin and :end and most :contens-begin
> and :contents-end (maybe sans an 's').
>
> > Now I'm not sure IF I really NEED it? I could actually get the contents
> > using the  :content-begin and :content-end and other properties from
> > (org-elemen-at-point)  BUT I don't know the exact syntax the content
> should
> > have and how to merge it with the element-list I get from
> > (org-element-at-point) before feeding it to the
> org-element-interpret-data.
>
> Maybe it would be easier if you state plainly what your desired goal is?
> It's all a bit abstract.  You can write pretty sophisticated things using
> just (org-element-at-point) (e.g. I have a function that escapes
> *math-su{b,per}script* on double space at appropriate places).
>
> > [...]
> > At first I thought that the things behind #  were 'content' (for example
> in
> > the output below. These don't show up in (org-element-at-point), thus
> > explaining why they returned nil when asked for content. The
> > org-element-parse-secondary-string also returns all things behind #. If
> > this is NOT content, then how to manipulate this data (behind the #)
> whilst
> > assuring that syntax and position remains valid for
> > org-element-interpret-data to understand?
>
> If you want to manipulate the buffer text use the great functions.  You
> can condition on the element under point or whatever you desire and then
> use the regular functions you'd otherwise use.  I'm probably wrong, but it
> seems as if you trying to shoot flies with canons, or however the saying
> goes in English.  Also, recall there is no such thing as wrong Org-syntax
> (but there is unexpected outcomes).
>
> Check Org.el if you want.
>
> Cheers,
> Rasmus
>
> --
> Not everything that goes around comes back around, you know
>
>
>


-- 
gtz,
Dieter VE

[-- Attachment #2: Type: text/html, Size: 6506 bytes --]