Dynamic block tables: adding prefix of "id:" to %ID

Dynamic block tables: adding prefix of "id:" to %ID

[Karl Voit]

Hi,

I do have a dynamic block table like this:

#+BEGIN: columnview :id global :match "+project-focus/!+STARTED|+NEXT|+WAITING" :format "%TODO(State) %ITEM(What) %ID(ID) %TAGS(Tags)"
| State   | What               | ID  | Tags          |
|---------+--------------------+-----+---------------|
| STARTED | [12/14] proj foo   | foo | :foo:project: |
| NEXT    | [0/0] proj bar     | bar | :bar:project: |
| STARTED | [191/191] proj baz | baz | :baz:project: |
#+END:

Is there a way to get the ID column with working ID links such as:

#+BEGIN: columnview :id global :match "+project-focus/!+STARTED|+NEXT|+WAITING" :format "%TODO(State) %ITEM(What) %ID(ID) %TAGS(Tags)"
| State   | What               | ID     | Tags          |
|---------+--------------------+--------+---------------|
| STARTED | [12/14] proj foo   | id:foo | :foo:project: |
| NEXT    | [0/0] proj bar     | id:bar | :bar:project: |
| STARTED | [191/191] proj baz | id:baz | :baz:project: |
#+END:

Thanks!

-- 
get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <
Personal Information Management > http://Karl-Voit.at/tags/pim/
Emacs-related > http://Karl-Voit.at/tags/emacs/

Re: Dynamic block tables: adding prefix of "id:" to %ID

[Kristian Grönberg]

[-- Attachment #1.1: Type: text/plain, Size: 2715 bytes --]

hi Karl,

I had to solve this by putting the link inside of the heading, like so:

* STARTED proj foo
:PROPERTIES:
:ID: [[foo][id]]
:END:

However this is not optimal since viewing the same ID property with "org-agenda-custom-commands" includes the link as well (which for me is not good since I have long links).

Perhaps someone else has a better solution.

/Kris

On Friday, July 23rd, 2021 at 11:46, Karl Voit <devnull@Karl-Voit.at> wrote:

> Hi,
> 

> I do have a dynamic block table like this:
> 

> #+BEGIN: columnview :id global :match "+project-focus/!+STARTED|+NEXT|+WAITING" :format "%TODO(State) %ITEM(What) %ID(ID) %TAGS(Tags)"
> 

> | State | What | ID | Tags |
> 

> |---------+--------------------+-----+---------------|
> 

> | STARTED | [12/14] proj foo | foo | :foo:project: |
> 

> | NEXT | [0/0] proj bar | bar | :bar:project: |
> 

> | STARTED | [191/191] proj baz | baz | :baz:project: |
> 

> #+END:
> 

> Is there a way to get the ID column with working ID links such as:
> 

> #+BEGIN: columnview :id global :match "+project-focus/!+STARTED|+NEXT|+WAITING" :format "%TODO(State) %ITEM(What) %ID(ID) %TAGS(Tags)"
> 

> | State | What | ID | Tags |
> 

> |---------+--------------------+--------+---------------|
> 

> | STARTED | [12/14] proj foo | id:foo | :foo:project: |
> 

> | NEXT | [0/0] proj bar | id:bar | :bar:project: |
> 

> | STARTED | [191/191] proj baz | id:baz | :baz:project: |
> 

> #+END:
> 

> Thanks!
> 

> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> 

> get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode:
> 

> > get Memacs from https://github.com/novoid/Memacs <
> 

> Personal Information Management > http://Karl-Voit.at/tags/pim/
> 

> Emacs-related > http://Karl-Voit.at/tags/emacs/

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 509 bytes --]

Re: Dynamic block tables: adding prefix of "id:" to %ID

[Karl Voit]

Hi Kristian,

* Kristian Grönberg <kristian@gronberg.org> wrote:
>
> hi Karl,
>
> I had to solve this by putting the link inside of the heading, like so:
>
> * STARTED proj foo
>:PROPERTIES:
>:ID: [[foo][id]]
>:END:

I can't follow you here.

How does a working link to your heading look like?

Here are some of my tests:

- id:bar → works

- id:foo → not working

- id:baz → not working

- id:myfoo → not working
- id:myid → not working

*** barheading
:PROPERTIES:
:ID:       bar
:END:

*** fooheading
:PROPERTIES:
:ID:       [[foo]]
:END:

*** bazheading
:PROPERTIES:
:ID:       id:baz
:END:

*** example
:PROPERTIES:
:ID: [[myfoo][myid]]
:END:

> However this is not optimal since viewing the same ID property with "org-a=
> genda-custom-commands" includes the link as well (which for me is not good=
>  since I have long links).
>
> Perhaps someone else has a better solution.
>
> /Kris
>
> On Friday, July 23rd, 2021 at 11:46, Karl Voit <devnull@Karl-Voit.at> wrot=
> e:
>
>> Hi,
>> =
>
>
>> I do have a dynamic block table like this:
>> =
>
>
>> #+BEGIN: columnview :id global :match "+project-focus/!+STARTED|+NEXT|+W=
> AITING" :format "%TODO(State) %ITEM(What) %ID(ID) %TAGS(Tags)"
>> =
>
>
>> | State | What | ID | Tags |
>> =
>
>
>> |---------+--------------------+-----+---------------|
>> =
>
>
>> | STARTED | [12/14] proj foo | foo | :foo:project: |
>> =
>
>
>> | NEXT | [0/0] proj bar | bar | :bar:project: |
>> =
>
>
>> | STARTED | [191/191] proj baz | baz | :baz:project: |
>> =
>
>
>> #+END:
>> =
>
>
>> Is there a way to get the ID column with working ID links such as:
>> =
>
>
>> #+BEGIN: columnview :id global :match "+project-focus/!+STARTED|+NEXT|+W=
> AITING" :format "%TODO(State) %ITEM(What) %ID(ID) %TAGS(Tags)"
>> =
>
>
>> | State | What | ID | Tags |
>> =
>
>
>> |---------+--------------------+--------+---------------|
>> =
>
>
>> | STARTED | [12/14] proj foo | id:foo | :foo:project: |
>> =
>
>
>> | NEXT | [0/0] proj bar | id:bar | :bar:project: |
>> =
>
>
>> | STARTED | [191/191] proj baz | id:baz | :baz:project: |
>> =
>
>
>> #+END:
>> =
>
>
>> Thanks!
>> =
>
>
>> ------------------------------------------------------------------------=
> --------------------------------------------------------------------------=
> --------------------------------------------------------------------------=
> --------------------------------------------------------------------------=
> --------------------------------------------------------------------------=
> --------------------------------------------------------------------------=
> --------------------------------------------------------------------------=
> --------------------------------------------------------------------------=
> --------------------------------------------------------------------------=
> --------------------------------------------------------------------------=
> --------------------------------------------------------------------------=
> --------------------------------------------------------------------------=
> -----------------------------
>> =
>
>
>> get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mod=
> e:
>> =
>
>
>> > get Memacs from https://github.com/novoid/Memacs <
>> =
>
>
>> Personal Information Management > http://Karl-Voit.at/tags/pim/
>> =
>
>
>> Emacs-related > http://Karl-Voit.at/tags/emacs/
> -----------------------a8d906ea5b9eed33670cb7379910ea33--
>
> -----------------------34bede87e6eb1c0d1b98c778939fee68
> Content-Type: application/pgp-signature; name="signature.asc"
> Content-Description: OpenPGP digital signature
> Content-Disposition: attachment; filename="signature.asc"
>
>
>
> -----------------------34bede87e6eb1c0d1b98c778939fee68--
>
>
>


-- 
get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <
Personal Information Management > http://Karl-Voit.at/tags/pim/
Emacs-related > http://Karl-Voit.at/tags/emacs/

Re: Dynamic block tables: adding prefix of "id:" to %ID

[Kristian Grönberg]

[-- Attachment #1.1: Type: text/plain, Size: 260 bytes --]

> I can't follow you here.
> 

> How does a working link to your heading look like?


I didn't read your question properly.
Would it work for you to use the "<<id-link>>"?

*** <<bar>> barheading
:PROPERTIES:
:ID: [[bar][barheading]]
:END:

/Kris

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 509 bytes --]

Re: Dynamic block tables: adding prefix of "id:" to %ID

[Karl Voit]

Hi,

* Kristian Grönberg <kristian@gronberg.org> wrote:
>
> I didn't read your question properly.
> Would it work for you to use the "<<id-link>>"?
>
> *** <<bar>> barheading
>:PROPERTIES:
>:ID: [[bar][barheading]]
>:END:
>
> /Kris

I'm afraid not because the dynamic block would duplicate the
target definition and not generating a link. 

-- 
get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <
Personal Information Management > http://Karl-Voit.at/tags/pim/
Emacs-related > http://Karl-Voit.at/tags/emacs/

Re: Dynamic block tables: adding prefix of "id:" to %ID

[Kristian Grönberg]

[-- Attachment #1.1: Type: text/plain, Size: 437 bytes --]

Hi Karl,

> > Would it work for you to use the "<<id-link>>"?
> > 

> > *** <<bar>> barheading
> > :PROPERTIES:
> > :ID: [[bar][barheading]]
> > :END:
> > /Kris
> 

> I'm afraid not because the dynamic block would duplicate the
> target definition and not generating a link.

OK, I see.
If you do find a solution in the future, I would be interested in getting to know it as well. Feel free to keep me posted.
Thank you.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 509 bytes --]

Re: Dynamic block tables: adding prefix of "id:" to %ID

[Jens Neuhalfen]

 Hi Karl,

 I use a wrapper around org-map-entries to create custom tables.

 Calling the wrapper takes a configuration for each table column. This is quite nice for other use cases as well, to e.g. convert properties (":TASK_PROGRESS: 75") into graphical representations like harvey balls.

 Each column takes the following configuration values:

 - colname :: The name of the column
 - entry :: the property value of the column
 - map :: a lambda function called with the value (optional)
 - default :: default value if nothing yields a non nil value (optional)

 (defun neuhalje/org-generate-table-from-sections-mapped (query default scope mapping add-headline)
   "See my library of Babel."
   (let* (
          (org-use-tag-inheritance nil)
          (title-row (mapcar '(lambda (cfg) (cdr (assoc "colname" cfg))) mapping))
          (rows (org-map-entries #'(lambda ()
                                     (mapcar '(lambda (column) (let* ((default-value (or (cdr (assoc "default" column)) default))
                                                                 (entry (cdr (assoc "entry" column)))
                                                                 (mapping (cdr (assoc "map" column)))
                                                                 (raw (org-entry-get nil entry))
                                                                 (mapped (if mapping (funcall mapping raw) raw)))
                                                            (or  mapped default-value))) mapping)) query scope)))
     (if add-headline (append  `(,title-row hline) rows) rows)))

 #+NAME: generate-table-from-sections-mapped
 #+BEGIN_SRC emacs-lisp :exports results :results table :var query='() :var default="N/A" scope='file :var mapping='() :var add-headline='t  :cache no
 (neuhalje/org-generate-table-from-sections-mapped query default scope mapping add-headline)
 #+end_src

 I call it from my org files like this:

 #+name: demoprops2
 #+begin_src emacs-lisp
 '(
   (("colname" . "ROW") ("entry" . "ITEM"))
   (("colname" . "default") ("entry" . "x-prop-xxx"))
   (("colname" . "custom default") ("entry" . "x-prop-b") ("default" . "not there"))
   (("colname" . "link") ("entry" . "CUSTOM_ID") ("map" . (lambda (value) (if value (format "[[#%s][%s]]" value value) "<default>"))))
   (("colname" . "linked item") ("entry" . "CUSTOM_ID") ("map" . (lambda (value) (if value (format "[[#%s][%s]]" value (org-entry-get nil "ITEM")) (org-entry-get nil "ITEM")))))
  )
 #+end_src

 #+CALL: generate-table-from-sections-mapped(scope='file, mapping=demoprops2, query="+mytag-exclude")

 Regards
 Jens

> Hi,
>
> * Kristian Grönberg <kristian@gronberg.org> wrote:
>>
>> I didn't read your question properly.
>> Would it work for you to use the "<<id-link>>"?
>>
>> *** <<bar>> barheading
>>:PROPERTIES:
>>:ID: [[bar][barheading]]
>>:END:
>>
>> /Kris
>
> I'm afraid not because the dynamic block would duplicate the
> target definition and not generating a link.
>
> --
> get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode:
>        > get Memacs from https://github.com/novoid/Memacs <
> Personal Information Management > http://Karl-Voit.at/tags/pim/
> Emacs-related > http://Karl-Voit.at/tags/emacs/

Re: Dynamic block tables: adding prefix of "id:" to %ID

[Ihor Radchenko]

Karl Voit <devnull@Karl-Voit.at> writes:

> I do have a dynamic block table like this:
> ...
> | NEXT    | [0/0] proj bar     | bar | :bar:project: |
>
> Is there a way to get the ID column with working ID links such as:
> ...
> | NEXT    | [0/0] proj bar     | id:bar | :bar:project: |

You may customise org-columns-modify-value-for-display-function:

(defun yant/org-columns-custom-formatter (column-title value)
  "Format column values for columns with ID-LINK title as proper Org mode id: link."
  (pcase column-title
    ("ID-LINK"
     (format "[[id:%s][%s]]"
	     value
	     (org-with-point-at (org-id-find value 'marker)
               (org-get-heading t t t t))))
    (_ nil)))
(setq org-columns-modify-value-for-display-function #'yant/org-columns-custom-formatter)

Then, you can use the following dblock. Note that I renamed the column
name to "ID-LINK"

#+BEGIN: columnview :id global :match "+project-focus/!+STARTED|+NEXT|+WAITING" :format "%TODO(State) %ITEM(What) %ID(ID-LINK) %TAGS(Tags)"
#+END:

Best,
Ihor

Re: Dynamic block tables: adding prefix of "id:" to %ID

[Karl Voit]

Hi Ihor,

* Ihor Radchenko <yantar92@gmail.com> wrote:
> Karl Voit <devnull@Karl-Voit.at> writes:
>
>> I do have a dynamic block table like this:
>> ...
>> | NEXT    | [0/0] proj bar     | bar | :bar:project: |
>>
>> Is there a way to get the ID column with working ID links such as:
>> ...
>> | NEXT    | [0/0] proj bar     | id:bar | :bar:project: |
>
> You may customise org-columns-modify-value-for-display-function:
>
> (defun yant/org-columns-custom-formatter (column-title value)
>   "Format column values for columns with ID-LINK title as proper Org mode id: link."
>   (pcase column-title
>     ("ID-LINK"
>      (format "[[id:%s][%s]]"
> 	     value
> 	     (org-with-point-at (org-id-find value 'marker)
>                (org-get-heading t t t t))))
>     (_ nil)))
> (setq org-columns-modify-value-for-display-function #'yant/org-columns-custom-formatter)
>
> Then, you can use the following dblock. Note that I renamed the column
> name to "ID-LINK"
>
> #+BEGIN: columnview :id global :match "+project-focus/!+STARTED|+NEXT|+WAITING" :format "%TODO(State) %ITEM(What) %ID(ID-LINK) %TAGS(Tags)"
> #+END:

Thanks, this sounds clever and I think I understand the code.
Although I would have preferred not to overwrite a function. I
always have a fear that this leads to nasty side-effects with future
updates.

Without deeper knowledge, I was astonished that C-h f
org-columns-modify-value-for-display-function did not lead to a
matching function and C-h v ... to a matching variable.

However, I set up following test file and it worked:

    #+BEGIN: columnview :id global :match "+project+focus/!+STARTED|+NEXT|+WAITING" :format "%TODO(State) %ITEM(What) %ID(ID-LINK) %TAGS(Tags)"
    | State | What           | ID-LINK        | Tags            |
    |-------+----------------+----------------+-----------------|
    | NEXT  | [/] focus proj | [[id:2021-07-27-focus-proj][[/] focus proj]] | :focus:project: |
    #+END:
    
    * Test
    
    ** NEXT [/] focus proj                                                               :focus:project:
    :PROPERTIES:
    :CREATED:  [2021-07-27 Tue 13:59]
    :COOKIE_DATA: todo recursive
    :ID:       2021-07-27-focus-proj
    :END:
    
    ** NEXT [/] non-focus proj                                                                          :project:
    :PROPERTIES:
    :CREATED:  [2021-07-27 Tue 13:59]
    :COOKIE_DATA: todo recursive
    :ID:       2021-07-27-non-focus-proj
    :END:

When I applied the new change to update a table in a file of 71k
lines of org, I had to cancel the process after over two hours
without a result. Before the change, updating this table took
roughly 20 minutes.

Therefore, this somehow makes a poor performance even worse.

-- 
get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <
Personal Information Management > http://Karl-Voit.at/tags/pim/
Emacs-related > http://Karl-Voit.at/tags/emacs/

Re: Dynamic block tables: adding prefix of "id:" to %ID

[Ihor Radchenko]

Karl Voit <devnull@Karl-Voit.at> writes:
> Thanks, this sounds clever and I think I understand the code.
> Although I would have preferred not to overwrite a function. I
> always have a fear that this leads to nasty side-effects with future
> updates.

That's not overwriting. org-columns-modify-value-for-display-function is
not a function, but a custom variable defaulting to nil. You have all
rights to set it to whatever you wish.

> Without deeper knowledge, I was astonished that C-h f
> org-columns-modify-value-for-display-function did not lead to a
> matching function and C-h v ... to a matching variable.

Hmm. You are right. This should be a bug. It happens because
org-colview.el is only loaded after you actually run column view or
corresponding dblock. Before that, Emacs is not aware about this
variable (unless you require org-colview manually in your config).

> When I applied the new change to update a table in a file of 71k
> lines of org, I had to cancel the process after over two hours
> without a result. Before the change, updating this table took
> roughly 20 minutes.

That's probably because the function I provided tries to compute the
description part of the link by querying the headline for each result.
You may get much better performance using the following version:

(defun yant/org-columns-custom-formatter (column-title value)
  "Format column values for columns with ID-LINK title as proper Org mode id: link."
  (pcase column-title
    ("ID-LINK"
     (format "[[id:%s]]" value))
    (_ nil)))

Best,
Ihor

Re: Dynamic block tables: adding prefix of "id:" to %ID

[Karl Voit]

* Ihor Radchenko <yantar92@gmail.com> wrote:
> Karl Voit <devnull@Karl-Voit.at> writes:
>> Thanks, this sounds clever and I think I understand the code.
>> Although I would have preferred not to overwrite a function. I
>> always have a fear that this leads to nasty side-effects with future
>> updates.
>
> That's not overwriting. org-columns-modify-value-for-display-function is
> not a function, but a custom variable defaulting to nil. You have all
> rights to set it to whatever you wish.

Oh, this is reassuring. Thank you!

>> Without deeper knowledge, I was astonished that C-h f
>> org-columns-modify-value-for-display-function did not lead to a
>> matching function and C-h v ... to a matching variable.
>
> Hmm. You are right. This should be a bug. It happens because
> org-colview.el is only loaded after you actually run column view or
> corresponding dblock. Before that, Emacs is not aware about this
> variable (unless you require org-colview manually in your config).
>
>> When I applied the new change to update a table in a file of 71k
>> lines of org, I had to cancel the process after over two hours
>> without a result. Before the change, updating this table took
>> roughly 20 minutes.
>
> That's probably because the function I provided tries to compute the
> description part of the link by querying the headline for each result.
> You may get much better performance using the following version:
>
> (defun yant/org-columns-custom-formatter (column-title value)
>   "Format column values for columns with ID-LINK title as proper Org mode id: link."
>   (pcase column-title
>     ("ID-LINK"
>      (format "[[id:%s]]" value))
>     (_ nil)))

I can confirm that this version does not have any negative impact
compared to the version without your code.

If anybody is interested why this is taking roughly 20min to update
one table:

| - command-execute                                              553233  93%
|  - call-interactively                                          553233  93%
|   - funcall-interactively                                      553233  93%
|    - org-ctrl-c-ctrl-c                                         553233  93%
|     - org-update-dblock                                        553228  93%
|      - org-dblock-write:columnview                             553228  93%
|       - org-columns--capture-view                              553212  93%
|        - org-columns                                           540042  91%
|         - org-columns--display-here                            536947  90%
|          - face-remap-add-relative                             527000  88%
|           - face-remap-order                                   526004  88%
|            - sort                                              521312  88%
|               face-attrs-more-relative-p                       458136  77%
|          + org-columns--new-overlay                              2008   0%
|            face-attribute                                         205   0%
|          + org-columns--overlay-text                               44   0%
|            org-get-level-face                                       4   0%
|         + org-map-entries                                        1506   0%
|         + org-columns-compute-all                                1441   0%
|         + org-columns--set-widths                                   8   0%
|        + org-map-entries                                        12126   2%
|        + org-columns-quit                                          16   0%
|       + org-table-align                                            12   0%
|       + org-columns--clean-item                                     4   0%
|     + org-element-context                                           5   0%
| + ...                                                           38956   6%
| + timer-event-handler                                             166   0%
| + redisplay_internal (C function)                                  28   0%


> Best,
> Ihor

Thank Ihor, you have improved yet another part of my Emacs life!

-- 
get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <
Personal Information Management > http://Karl-Voit.at/tags/pim/
Emacs-related > http://Karl-Voit.at/tags/emacs/