Is there a better (built-in) way to insert an org link with title as description?

Is there a better (built-in) way to insert an org link with title as description?

[Arthur Miller]

Hello Org experts,

I want to auto insert a title from an HTML page as description for an org link in
my notes. I stumbled upon some old message by Miro Bezjak on this list:

https://lists.gnu.org/archive/html/emacs-orgmode/2012-09/msg01435.html

I have seen the replies, but I am not sure how to use
org-make-link-description-function, so I coded my own version of Miros idea:

#+begin_src emacs-lisp
(defun org-link-from-clipboard ()
  "Insert an org link into current buffer from an URL in clipboard."
  (interactive)
  (let ((marker (point-marker))
        (url
         (if (string-match-p "^\\(http\\|https\\)://" (current-kill 0))
             (current-kill 0)
           (read-string "URL: ")))
        (title nil))
    (when url
      (url-retrieve url
       (lambda (buffer)
         (goto-char (point-min))
         (when (re-search-forward "<title>\\(.*\\)</title>" nil t)
           (setq title (string-trim (match-string-no-properties 1))))
         (with-current-buffer (marker-buffer marker)
           (save-excursion
             (goto-char (marker-position marker))
             (org-insert-link
              nil url (or title (read-string "Description: "))))))
       nil t t))))
#+end_src

While my function is not very big, I would still like to learn how to use the
suggested built-in stuff from that discussion. I also dislike the interactive
fallback in asynchronous callback if the title is not found, but I would also
dislike to have a bunch of "No description found" strings in my notes too, so I
am not sure which one is less evil there.

Thankful for any help and advice.

Re: Is there a better (built-in) way to insert an org link with title as description?

[Ihor Radchenko]

Arthur Miller <arthur.miller@live.com> writes:

> I want to auto insert a title from an HTML page as description for an org link in
> my notes. I stumbled upon some old message by Miro Bezjak on this list:
>
> https://lists.gnu.org/archive/html/emacs-orgmode/2012-09/msg01435.html
>
> I have seen the replies, but I am not sure how to use
> org-make-link-description-function

Since that time, Org got a capability to set description function
per link type. Just use

(org-link-set-parameters "http" :insert-description #'your-function)
(org-link-set-parameters "https" :insert-description #'your-function)

The calling convention is the same as
`org-link-make-description-function'.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Is there a better (built-in) way to insert an org link with title as description?

[Arthur Miller]

Ihor Radchenko <yantar92@posteo.net> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
>> I want to auto insert a title from an HTML page as description for an org link in
>> my notes. I stumbled upon some old message by Miro Bezjak on this list:
>>
>> https://lists.gnu.org/archive/html/emacs-orgmode/2012-09/msg01435.html
>>
>> I have seen the replies, but I am not sure how to use
>> org-make-link-description-function
>
> Since that time, Org got a capability to set description function
> per link type. Just use
>
> (org-link-set-parameters "http" :insert-description #'your-function)
> (org-link-set-parameters "https" :insert-description #'your-function)

Thanks, after some thinkering I got it:

#+begin_src emacs-lisp
(defun my-org-insert-link ()
  "Insert org link where default description is set to html title."
  (interactive)
  (let* ((url (or (current-kill 0) (read-string "URL: "))))
    (org-insert-link nil url)))

(defun org-desc-from-clipboard (url _desc)
  "Insert an org link into current buffer from an URL in clipboard."
  (with-current-buffer (url-retrieve-synchronously url t)
    (goto-char (point-min))
    (let ((title "<title>\\(.*\\)\\(/>\\|</title>\\)"))
      (if (re-search-forward title nil t)
          (string-trim (match-string-no-properties 1))
        url))))

(org-link-set-parameters "http" :insert-description #'org-desc-from-clipboard)
(org-link-set-parameters "https" :insert-description #'org-desc-from-clipboard)
#+end_src

And I can do it async too, *but*; this will affect all insertions of links,
right?

I am not sure if it is safe/possible always to access the internet or do
it asynchronously, so I'll abandon the ship and revert to home-cooked one just
for the precautios measures:

#+begin_src emacs-lisp
(defun org-link-from-clipboard ()
  "Insert an org link into current buffer from an URL in clipboard."
  (interactive)
  (let ((marker (point-marker))
        (url (or (current-kill 0) (read-string "URL: "))))
    (url-retrieve
     url
     (lambda (_status title)
       (goto-char (point-min))
       (when (re-search-forward title nil t)
         (setq title (string-trim (match-string-no-properties 1))))
       (with-current-buffer (marker-buffer marker)
         (save-excursion
           (goto-char (marker-position marker))
           (org-insert-link
            nil url (or title url)))))
     '("<title>\\(.*\\)\\(/>\\|</title>\\)") t t)))
#end_src

But it was a bit of learning, thanks for pointing me in the right direction.

Re: Is there a better (built-in) way to insert an org link with title as description?

[Ihor Radchenko]

Arthur Miller <arthur.miller@live.com> writes:

>> (org-link-set-parameters "http" :insert-description #'your-function)
>> (org-link-set-parameters "https" :insert-description #'your-function)
>
> Thanks, after some thinkering I got it:
>
> ...
> (org-link-set-parameters "http" :insert-description #'org-desc-from-clipboard)
> (org-link-set-parameters "https" :insert-description #'org-desc-from-clipboard)
> #+end_src
>
> And I can do it async too, *but*; this will affect all insertions of links,
> right?

Indeed. By design, `org-insert-link' is synchronous - it expects the
link and description to be available upon request.

> I am not sure if it is safe/possible always to access the internet or do
> it asynchronously, so I'll abandon the ship and revert to home-cooked one just
> for the precautios measures:

What you can do is (1) make url descriptions be something like <title to
be retrieved>; (2) add an :after advice for `org-insert-link' that will
queue asynchronous url fetching; (3) replace <title to be retrieved>
with the fetched title upon finishing the request. If the request fails,
the description will remain <title to be retrieved>.

Or you can run description retrieval independently, as a minor mode
that will search for <title to be retrieved> marks and try to fetch
them.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Is there a better (built-in) way to insert an org link with title as description?

[Arthur Miller]

Ihor Radchenko <yantar92@posteo.net> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
>>> (org-link-set-parameters "http" :insert-description #'your-function)
>>> (org-link-set-parameters "https" :insert-description #'your-function)
>>
>> Thanks, after some thinkering I got it:
>>
>> ...
>> (org-link-set-parameters "http" :insert-description #'org-desc-from-clipboard)
>> (org-link-set-parameters "https" :insert-description #'org-desc-from-clipboard)
>> #+end_src
>>
>> And I can do it async too, *but*; this will affect all insertions of links,
>> right?
>
> Indeed. By design, `org-insert-link' is synchronous - it expects the
> link and description to be available upon request.
>
>> I am not sure if it is safe/possible always to access the internet or do
>> it asynchronously, so I'll abandon the ship and revert to home-cooked one just
>> for the precautios measures:
>
> What you can do is (1) make url descriptions be something like <title to
> be retrieved>; (2) add an :after advice for `org-insert-link' that will
> queue asynchronous url fetching; (3) replace <title to be retrieved>
> with the fetched title upon finishing the request. If the request fails,
> the description will remain <title to be retrieved>.

Yes of course, placeholders could work. Another option is to use url as
a placeholder, so if the retrieval failed the url would still be visible
which is a bit more informative than some generic placholder.

> Or you can run description retrieval independently, as a minor mode
> that will search for <title to be retrieved> marks and try to fetch
> them.

True. I could also run an idle timer on my notes file and try to patch
all urls without descriptions. Would need to put something as a marker
into a desription for dead links, or just remove them, so they are not
fetched over and over again.

However, I am ok with doing it on the request only, when I actually
create a note :).

But if someone adds something similar to org, I'll gladly use it :).

Thanks and sorry the  late response. My memory is like a gold fish,
sometimes when GNUS remove a mail from my view I totally forget about
it.

/a

Re: Is there a better (built-in) way to insert an org link with title as description?

[Max Nikulin]

On 19/07/2023 19:06, Arthur Miller wrote:
> I want to auto insert a title from an HTML page as description for an org link in
> my notes.
> (defun org-link-from-clipboard ()
...
>        (url-retrieve url
>         (lambda (buffer)
>           (goto-char (point-min))
>           (when (re-search-forward "<title>\\(.*\\)</title>" nil t)

What are origins of your links?

If it is an URL opened in browser then `org-capture' or 
org-protocol:/store-link/ may be used. There are a number of browser 
extensions for that.

More metadata sometimes desired and just page title is not enough. For 
extracting it within Emacs see e.g. Ihor's
https://github.com/yantar92/org-capture-ref
Search for its discussions on this mailing lists.

Some complications:
- titles may have &...; entities
- Not all pages have <title>, so heuristics have to be used.
- Some HTML files contains nothing besides JavaScript to load actual content
- Some URLs are from minifiers or obfuscated by Outlook "protection", 
trampolines to prevent leaking of data through the Referer header, etc. 
Likely redirection target should be saved, not original URL.
- Some sites like GitHub have API that allows to get metadata in JSON 
format. It is better than parsing HTML with regexp.

Anyway I suggest to split non-interactive part of the command to allow 
code reuse (for drag and drop, etc.).

Re: Is there a better (built-in) way to insert an org link with title as description?

[Arthur Miller]

Max Nikulin <manikulin@gmail.com> writes:

Hi thank you for the thorough and well-informed answer.

> On 19/07/2023 19:06, Arthur Miller wrote:
>> I want to auto insert a title from an HTML page as description for an org link in
>> my notes.
>> (defun org-link-from-clipboard ()
> ...
>>        (url-retrieve url
>>         (lambda (buffer)
>>           (goto-char (point-min))
>>           (when (re-search-forward "<title>\\(.*\\)</title>" nil t)
>
> What are origins of your links?

> If it is an URL opened in browser then `org-capture' or
> org-protocol:/store-link/ may be used. There are a number of browser extensions
> for that.

I do use org-protocol, and I do have it in my FFX, so I am aware of it. But
sometimes I copy a link from a readme file or a piece of code or elsewhere and
wish to stash it away in a note but not necessary open in a browser. You know,
"todo" to come back later for it :).

> More metadata sometimes desired and just page title is not enough. For
> extracting it within Emacs see e.g. Ihor's
> https://github.com/yantar92/org-capture-ref
> Search for its discussions on this mailing lists.
>
> Some complications:
> - titles may have &...; entities
> - Not all pages have <title>, so heuristics have to be used.
Yepp, I am aware, the goal was not to be 100% fool proof. I had experienced
sometimes a couple of characters that Emacs can't dissambiguate, but it is not a
problem and yes, in case of no title it will prompt; the other strategy I used
was to return just url itself or "no description". Perhaps I should revert to
just the url.

> - Some HTML files contains nothing besides JavaScript to load actual content
> - Some URLs are from minifiers or obfuscated by Outlook "protection",
>   trampolines to prevent leaking of data through the Referer header, etc. Likely
>  redirection target should be saved, not original URL.
> - Some sites like GitHub have API that allows to get metadata in JSON format. It
>  is better than parsing HTML with regexp.

Yes, I am aware and completely agree with you!

Luckely I am getting quite old by now and don't visit too many sites or sites of
dubious JS character, so for my needs IDC :).

Miros idea served me well for several years now, I just improved it a bit the
other day to skip prmpting for the URL and used asynchornous download to skip
that slight second or two of delay in some links.

> Anyway I suggest to split non-interactive part of the command to allow code
> reuse (for drag and drop, etc.).

Tell me more here? Can I drag a link from one buffer into a note buffer, or how
can I use it?

Thank you for the answer.

Re: Is there a better (built-in) way to insert an org link with title as description?

[Max Nikulin]

On 21/07/2023 20:04, Arthur Miller wrote:
> I copy a link from a readme file or a piece of code or elsewhere and
> wish to stash it away in a note but not necessary open in a browser. You know,
> "todo" to come back later for it :).

If a file is opened in Emacs then it sounds like a case for 
`org-capture' or `org-store-link'. Perhaps storing link is better 
compatible with `url-retrive' callback and it makes delay for 
`org-insert-link' due to network roundtrip less probable.

When I add a link to my notes, often I mention where I noticed it.

>> Anyway I suggest to split non-interactive part of the command to allow code
>> reuse (for drag and drop, etc.).
> 
> Tell me more here? Can I drag a link from one buffer into a note buffer, or how
> can I use it?

My idea is to handle dropping of text into Org buffers. Currently it 
causes opening links in eww. I think, there are better options: convert 
from html to Org using pandoc and insert text, `org-capture', 
`org-store-link'. I have not tried to implement it. Currently I have 
just a couple of hints in my notes: `dnd-protocol-alist' and 
https://github.com/abo-abo/org-download/blob/master/org-download.el as 
an example of code.

Re: Is there a better (built-in) way to insert an org link with title as description?

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> My idea is to handle dropping of text into Org buffers. Currently it 
> causes opening links in eww. I think, there are better options: convert 
> from html to Org using pandoc and insert text, `org-capture', 
> `org-store-link'. I have not tried to implement it. Currently I have 
> just a couple of hints in my notes: `dnd-protocol-alist' and 
> https://github.com/abo-abo/org-download/blob/master/org-download.el as 
> an example of code.

Supporting dnd will certainly be welcome.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>