Expose value-begin and value-end instead of just value in org-element API

Expose value-begin and value-end instead of just value in org-element API

[Somelauw .]

[-- Attachment #1: Type: text/plain, Size: 975 bytes --]

Some org-mode elements expose "contents-begin" and "contents-end"
properties as buffer positions, whereas other elements such as verbatim and
org-src-block expose a "value" property in the form of a string.

I think it would be preferable to also expose the value by beginning and
ending buffer positions for the following reasons:
- Consistency with elements that expose contents-begin and contents-end.
- More powerful. In my evil-org plugin I want to be able to mark the value
property of the org element at point (so the user can do stuff like easily
copy the code of the current code block), but to do so I need the beginning
and ending position in the buffer of "value". The org-element API does
currently not provide clean way to retrieve these positions.
- It's usually more efficient to return the beginning and ending positions
than to retrieve the substring that contains the value, which may require a
large buffer partition to be copied.

Kind regards,
Somelauw

[-- Attachment #2: Type: text/html, Size: 1044 bytes --]

Re: Expose value-begin and value-end instead of just value in org-element API

[John Kitchin]

[-- Attachment #1: Type: text/plain, Size: 1726 bytes --]

+1 on this.

I also have some janky code to do things like go to the beginning/end of
the value in a src block. Here is my solution to mark the code in a src
block.

(defun ob-ipython-mark-code ()
  "Mark the code in the block."
  (interactive)
  (org-edit-special)
  (let ((p0 (point-min))
(p1 (point-max)))
    (goto-char p0)
    (org-edit-src-exit)
    (set-mark (point))
    (goto-char (+ (point) (- p1 2)))))

John

-----------------------------------
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu


On Mon, Feb 19, 2018 at 3:29 PM, Somelauw . <somelauw@gmail.com> wrote:

> Some org-mode elements expose "contents-begin" and "contents-end"
> properties as buffer positions, whereas other elements such as verbatim and
> org-src-block expose a "value" property in the form of a string.
>
> I think it would be preferable to also expose the value by beginning and
> ending buffer positions for the following reasons:
> - Consistency with elements that expose contents-begin and contents-end.
> - More powerful. In my evil-org plugin I want to be able to mark the value
> property of the org element at point (so the user can do stuff like easily
> copy the code of the current code block), but to do so I need the beginning
> and ending position in the buffer of "value". The org-element API does
> currently not provide clean way to retrieve these positions.
> - It's usually more efficient to return the beginning and ending positions
> than to retrieve the substring that contains the value, which may require a
> large buffer partition to be copied.
>
> Kind regards,
> Somelauw

[-- Attachment #2: Type: text/html, Size: 2531 bytes --]

Re: Expose value-begin and value-end instead of just value in org-element API

[Nicolas Goaziou]

Hello,

John Kitchin <jkitchin@andrew.cmu.edu> writes:

> +1 on this.
>
> I also have some janky code to do things like go to the beginning/end of
> the value in a src block. Here is my solution to mark the code in a src
> block.
>
> (defun ob-ipython-mark-code ()
>   "Mark the code in the block."
>   (interactive)
>   (org-edit-special)
>   (let ((p0 (point-min))
> (p1 (point-max)))
>     (goto-char p0)
>     (org-edit-src-exit)
>     (set-mark (point))
>     (goto-char (+ (point) (- p1 2)))))

You get the beginning of the code of a source block with

  (org-with-point-at (org-element-property :post-affiliated element)
    (line-beginning-position 2))

and its end with

  (org-with-point-at (org-element-property :end element)
    (line-beginning-position (- (org-element-property :post-blank element))))

From there, you can easily construct something that doesn't rely on
`org-edit-special'.

>> I think it would be preferable to also expose the value by beginning and
>> ending buffer positions for the following reasons:
>> - Consistency with elements that expose contents-begin and contents-end.

The point is terminal elements are not consistent with non-terminal
ones, and should not be.

>> - More powerful. In my evil-org plugin I want to be able to mark the value
>> property of the org element at point (so the user can do stuff like easily
>> copy the code of the current code block), but to do so I need the beginning
>> and ending position in the buffer of "value". The org-element API does
>> currently not provide clean way to retrieve these positions.

See above. It is quite simple to extract this information from the parse
tree.

>> - It's usually more efficient to return the beginning and ending positions
>> than to retrieve the substring that contains the value, which may require a
>> large buffer partition to be copied.

Efficiency is a moot point because you still need to store the value of
the block. If you remove it, the parse tree is no longer an equivalent
representation of the document.

Regards,

-- 
Nicolas Goaziou

Re: Expose value-begin and value-end instead of just value in org-element API

[Somelauw .]

Thank you for your response.

2018-02-21 12:17 GMT+01:00 Nicolas Goaziou <mail@nicolasgoaziou.fr>:
> You get the beginning of the code of a source block with
>
>   (org-with-point-at (org-element-property :post-affiliated element)
>     (line-beginning-position 2))
>
> and its end with
>
>   (org-with-point-at (org-element-property :end element)
>     (line-beginning-position (- (org-element-property :post-blank element))))

This works well to find the :value of code blocks, but doesn't work
for other org elements such as clock, babel-call and comment. What I'm
looking for is a generic way to find the beginning and end of the
:value of the element at point that works on all elements and is
future-proof.

>>> - More powerful. In my evil-org plugin I want to be able to mark the value
>>> property of the org element at point (so the user can do stuff like easily
>>> copy the code of the current code block), but to do so I need the beginning
>>> and ending position in the buffer of "value". The org-element API does
>>> currently not provide clean way to retrieve these positions.
>
> See above. It is quite simple to extract this information from the parse
> tree.

Something I would like to have is a command to mark the "useful inner
part" of the element that the point is currently on. For recursive
elements this is usually the part between :contents-begin and
:contents-end and for non-recursive elements this is usually :value.
The org-element-parser was probably not written for the specific goal
I have in mind, but the :value of org-element gives me almost what I
want, except that I need the buffer positions.

Kind regards,
Somelauw

Re: Expose value-begin and value-end instead of just value in org-element API

[John Kitchin]

Nicolas Goaziou writes:

> Hello,
>
> John Kitchin <jkitchin@andrew.cmu.edu> writes:
>
>> +1 on this.
>>
>> I also have some janky code to do things like go to the beginning/end of
>> the value in a src block. Here is my solution to mark the code in a src
>> block.
>>
>> (defun ob-ipython-mark-code ()
>>   "Mark the code in the block."
>>   (interactive)
>>   (org-edit-special)
>>   (let ((p0 (point-min))
>> (p1 (point-max)))
>>     (goto-char p0)
>>     (org-edit-src-exit)
>>     (set-mark (point))
>>     (goto-char (+ (point) (- p1 2)))))
>
> You get the beginning of the code of a source block with
>
>   (org-with-point-at (org-element-property :post-affiliated element)
>     (line-beginning-position 2))
>
> and its end with
>
>   (org-with-point-at (org-element-property :end element)
>     (line-beginning-position (- (org-element-property :post-blank element))))
>

Wow. I would not have guessed either one of these! Thanks for sharing
them. Is that documented somewhere? I gather that given something like

#+header: :var a=5
#+name: test
#+BEGIN_SRC ipython

print(a)

#+END_SRC

which parses like (from org-element-context):

(src-block
 (:language "ipython" :switches nil :parameters nil :begin 362 :end 436 :number-lines nil :preserve-indent nil :retain-labels t :use-labels t :label-fmt nil :value "\nprint(a)\n\n" :post-blank 1 :post-affiliated 394 :header
	    (":var a=5")
	    :name "test" :parent nil))

In this:

:begin refers to the point at the beginning of the #+header line.

whereas

:post-affiliated is the point after all the "affiliated" header/name lines, i.e. the
beginning of #+BEGIN_SRC.

:end refers to the point where the end of the element is, which may
include blank lines.

:post-blank is the number of blank lines (including lines that are just
spaces) after the element.

is that the right interpretation?

For elements with a :contents-begin where does :post-affiliated come in?


> From there, you can easily construct something that doesn't rely on
> `org-edit-special'.
>
>>> I think it would be preferable to also expose the value by beginning and
>>> ending buffer positions for the following reasons:
>>> - Consistency with elements that expose contents-begin and contents-end.
>
> The point is terminal elements are not consistent with non-terminal
> ones, and should not be.
>
>>> - More powerful. In my evil-org plugin I want to be able to mark the value
>>> property of the org element at point (so the user can do stuff like easily
>>> copy the code of the current code block), but to do so I need the beginning
>>> and ending position in the buffer of "value". The org-element API does
>>> currently not provide clean way to retrieve these positions.
>
> See above. It is quite simple to extract this information from the parse
> tree.

Once you get the idea, maybe, but this approach seems specific to
src-blocks (maybe any block) where there are delimiting lines. It
doesn't work on all elements (which to be fair was not claimed). I think
the OP was interested in something more consistent, which I am
sympathetic to.

Some things aren't clear to me what should happen though, especially in
composite elements like tables and plain lists. E.g. To just select a
table without the affiliated lines, one can use :contents-begin and
:contents-end, once you get the table element (e.g. by walking up the
:parent chain if you are in a cell or row).

In the absence of a single way, maybe there could be a small number of
ways to do this? How many cases do you think there are?

- blocks (which have :value)
- composite elements (which have :contents-begin/end and/or non-nil :parents)
- regular elements (which have :contents-begin/end)

Are there any others?

>
>>> - It's usually more efficient to return the beginning and ending positions
>>> than to retrieve the substring that contains the value, which may require a
>>> large buffer partition to be copied.
>
> Efficiency is a moot point because you still need to store the value of
> the block. If you remove it, the parse tree is no longer an equivalent
> representation of the document.

I can see this argument, but I am still unclear on which elements need a
value, and which don't. For example, src-blocks have a value, but a
paragraph doesn't, nor do items in a plain list, at least from
(org-element-context).

I guess my confusion comes from these being different than what comes
out of (org-element-parse-buffer).

>
> Regards,


--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu

Re: Expose value-begin and value-end instead of just value in org-element API

[Nicolas Goaziou]

Hello,

John Kitchin <jkitchin@andrew.cmu.edu> writes:

> Wow. I would not have guessed either one of these! Thanks for sharing
> them. Is that documented somewhere? 

[...]

> For elements with a :contents-begin where does :post-affiliated come in?

See <https://orgmode.org/worg/dev/org-element-api.html>

> Once you get the idea, maybe, but this approach seems specific to
> src-blocks (maybe any block) where there are delimiting lines. It
> doesn't work on all elements (which to be fair was not claimed). I think
> the OP was interested in something more consistent, which I am
> sympathetic to.

Some elements are very different from others. What would be the innards
of an horizontal line or a planning line?

> Some things aren't clear to me what should happen though, especially in
> composite elements like tables and plain lists. E.g. To just select a
> table without the affiliated lines, one can use :contents-begin and
> :contents-end, once you get the table element (e.g. by walking up the
> :parent chain if you are in a cell or row).
>
> In the absence of a single way, maybe there could be a small number of
> ways to do this? How many cases do you think there are?
>
> - blocks (which have :value)
> - composite elements (which have :contents-begin/end and/or non-nil :parents)
> - regular elements (which have :contents-begin/end)

There is no difference between regular elements and composite elements.
Also, all blocks do not have a value (e.g., center blocks).

> I can see this argument, but I am still unclear on which elements need a
> value, and which don't. For example, src-blocks have a value, but a
> paragraph doesn't, nor do items in a plain list, at least from
> (org-element-context).

In a parse tree, some elements are terminal (i.e., leaves), others are
not. Non-terminal elements have :contents-begin and :contents-end
properties. Others have :value, if it makes sense, or nothing (e.g.,
planning lines).

Source blocks are terminal elements. Paragraphs, lists and items are
not.

Regards,

-- 
Nicolas Goaziou