[DISCUSSION] Possible inclusion of org-capture.el into Emacs core (was: [ELPA] New package: jami-bot and org-jami-bot)

[DISCUSSION] Possible inclusion of org-capture.el into Emacs core (was: [ELPA] New package: jami-bot and org-jami-bot)

[Ihor Radchenko]

Richard Stallman <rms@gnu.org> writes:

> Dependence of various features on Org mode causes problems in two ways:
>
> * It makes those features hard to use for those that don't use Org mode.
>
> * It adds the need to learn Org mode to the need to learn Emacs itself.
>
> Therefore, new features which make sense indepdently of Org mode
> should be usable without Org mode.

This reminds me of "Keep Org mode submodules in Emacs, please!" request:
https://list.orgmode.org/E1kIPh1-0001Lu-Rg@fencepost.gnu.org/

Similar to jami-bot providing a dbus interface for Jami to interact with
Emacs, Org mode has org-protocol.el providing command line and URL
handler interface that is nearly independent of Org mode itself in its
core function:

;; Usage:
;; ------
;;
;;   1.) Add this to your init file (.emacs probably):
;;
;;       (require 'org-protocol)
;;
;;   3.) Ensure emacs-server is up and running.
;;   4.) Try this from the command line (adjust the URL as needed):
;;
;;       $ emacsclient \
;;         "org-protocol://store-link?url=http:%2F%2Flocalhost%2Findex.html&title=The%20title"
;;
;;   5.) Optionally add custom sub-protocols and handlers:
;;
;;       (setq org-protocol-protocol-alist
;;             '(("my-protocol"
;;                :protocol "my-protocol"
;;                :function my-protocol-handler-function)))
;;
;;       A "sub-protocol" will be found in URLs like this:
;;
;;           org-protocol://sub-protocol?key=val&key2=val2
;;
;; If it works, you can now setup other applications for using this feature.

With org-protocol, one can also make Emacs receive data from browser and
perform any action defined by custom handler function - all fully
configurable and not necessarily related to Org mode.
    
    17.16.3 The open-source protocol
    ... The open-source handler is designed to help with editing local
    sources when reading a document.. To that effect, you can use a
    bookmark with the following location:
    
    javascript:location.href='org-protocol://open-source?&url='+
          encodeURIComponent(location.href)

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

[DISCUSSION] org-capture.el vs remember.el (was: [ELPA] New package: jami-bot and org-jami-bot)

[Ihor Radchenko]

Recent emacs-devel discussion brought up some concerns about org-capture
being very specific to Org mode and not integrated with the rest of
Emacs.

However, org-capture was once built on top of the built-in remember.el
before it was refactored into independent code by Carsten:
https://list.orgmode.org/orgmode/C46F10DC-DE51-43D4-AFFE-F71E440D1E1F@gmail.com/

The refactoring de-coupled what used to be org-remember.el into
completely rewritten org-capture.el that added important features that
could not be implemented within remember.el framework:

1. org-capture arranges writing the text to remember directly into the
   target location where it is to be stored. This addressed an important
   problem of remember.el - remember.el always uses temporary buffer for
   user input and does not save it until remember-finalize is executed
   explicitly. A number of users sometimes forgot to call
   remember-finalize, leading to annoying data losses.

2. remember.el only allows a single text input at a time. org-capture
   makes it possible for users to write multiple notes in parallel.

I am wondering if it would be possible to have the above two key
features supported by remember.el. Then, org-capture.el could go back to
re-using remember.el infrastructure and thus reduce code duplication.

Richard Stallman <rms@gnu.org> writes:

>   > org-jami-bot builds upon jami-bot and extends it with Org mode capture
>   > functionality for text messages and images.
>
> This feature will be useful.  I am not concretely sure what "capture
> functionality" includes, but if it provides higher-level communication
> between Emacs and Jami, that is a good thing.
>
> At the same time, it seems to be implemented in a way that imposes an
> evitable restriction -- requiring use of Org mode.
>
> Dependence of various features on Org mode causes problems in two ways:
>
> * It makes those features hard to use for those that don't use Org mode.
>
> * It adds the need to learn Org mode to the need to learn Emacs itself.
>
> Therefore, new features which make sense indepdently of Org mode
> should be usable without Org mode.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [DISCUSSION] org-capture.el vs remember.el (was: [ELPA] New package: jami-bot and org-jami-bot)

[Stefan Kangas]

Ihor Radchenko <yantar92@posteo.net> writes:

> The refactoring de-coupled what used to be org-remember.el into
> completely rewritten org-capture.el that added important features that
> could not be implemented within remember.el framework:
>
> 1. org-capture arranges writing the text to remember directly into the
>    target location where it is to be stored. This addressed an important
>    problem of remember.el - remember.el always uses temporary buffer for
>    user input and does not save it until remember-finalize is executed
>    explicitly. A number of users sometimes forgot to call
>    remember-finalize, leading to annoying data losses.
>
> 2. remember.el only allows a single text input at a time. org-capture
>    makes it possible for users to write multiple notes in parallel.
>
> I am wondering if it would be possible to have the above two key
> features supported by remember.el. Then, org-capture.el could go back to
> re-using remember.el infrastructure and thus reduce code duplication.

This is, of course, desirable.  The real question is how much work it
will take and who will volunteer to do it.

On the face of things, remember.el does not comprise a huge amount of
code, and I don't see why the above two features couldn't be retrofitted
into it.

Patches welcome.

Re: [DISCUSSION] Possible inclusion of org-capture.el into Emacs core (was: [ELPA] New package: jami-bot and org-jami-bot)

[Stefan Kangas]

Ihor Radchenko <yantar92@posteo.net> writes:

> With org-protocol, one can also make Emacs receive data from browser and
> perform any action defined by custom handler function - all fully
> configurable and not necessarily related to Org mode.

I don't have much to add besides giving support to the idea.  It would
be useful.  I'm sure it would find its use in other types of note taking
and productivity systems, of which we have several.

Re: [DISCUSSION] org-capture.el vs remember.el (was: [ELPA] New package: jami-bot and org-jami-bot)

[João Távora]

On Sat, Dec 30, 2023 at 5:21 PM Stefan Kangas <stefankangas@gmail.com> wrote:
>
> Ihor Radchenko <yantar92@posteo.net> writes:
>
> > The refactoring de-coupled what used to be org-remember.el into
> > completely rewritten org-capture.el that added important features that
> > could not be implemented within remember.el framework:
> >
> > 1. org-capture arranges writing the text to remember directly into the
> >    target location where it is to be stored. This addressed an important
> >    problem of remember.el - remember.el always uses temporary buffer for
> >    user input and does not save it until remember-finalize is executed
> >    explicitly. A number of users sometimes forgot to call
> >    remember-finalize, leading to annoying data losses.
> >
> > 2. remember.el only allows a single text input at a time. org-capture
> >    makes it possible for users to write multiple notes in parallel.
> >
> > I am wondering if it would be possible to have the above two key
> > features supported by remember.el. Then, org-capture.el could go back to
> > re-using remember.el infrastructure and thus reduce code duplication.
>
> This is, of course, desirable.  The real question is how much work it
> will take and who will volunteer to do it.

Not volunteering but those two goals seems good, especially
the first one

> On the face of things, remember.el does not comprise a huge amount of
> code, and I don't see why the above two features couldn't be retrofitted
> into it.

Seems like the first one could be done with narrowing.
But would also need some kind of indirect buffer, too,
right?  How does org-capture.el do it?  It would
also mean "commiting to remember" is just C-x C-s and file
recovery works as normal, which is how it should be IMO.

Maybe after-save-hook in this indirect buffer specifically
would have the effect of quitting the window displaying the
indirect buffer? ... Well I just checked and the hook approach
doesn't work, because save-buffer always saves the base buffer.
But surely some other hook or some remapped key would do the
trick.

If this works (big if since given all this is vapourware), we'll
finally have found good use for indirect buffers :-)

The second one is just multiple such setups, which the
window-management code should in theory already allow.

João

Re: [DISCUSSION] org-capture.el vs remember.el (was: [ELPA] New package: jami-bot and org-jami-bot)

[João Távora]

On Sat, Dec 30, 2023 at 7:16 PM João Távora <joaotavora@gmail.com> wrote:

> > On the face of things, remember.el does not comprise a huge amount of
> > code, and I don't see why the above two features couldn't be retrofitted
> > into it.
>
> Seems like the first one could be done with narrowing.
> But would also need some kind of indirect buffer, too,
> right?  How does org-capture.el do it?

Answering myself, with indirect buffers :-)

João

Re: [DISCUSSION] org-capture.el vs remember.el (was: [ELPA] New package: jami-bot and org-jami-bot)

[Adam Porter]

> If this works (big if since given all this is vapourware), we'll
> finally have found good use for indirect buffers :-)

FWIW, I use indirect buffers constantly through 
`org-tree-to-indirect-buffer', which is also called from several of my 
tools like org-ql, org-bookmark-heading, etc.  Largely, I treat my ~/org 
directory as a database, and when I open an entry from a search result 
or a bookmark in an indirect buffer, I don't even care what file/buffer 
it's actually in; the indirect buffer offers the illusion that it's in a 
file of its own, without any other content around it.

Re: [DISCUSSION] Possible inclusion of org-capture.el into Emacs core (was: [ELPA] New package: jami-bot and org-jami-bot)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Similar to jami-bot providing a dbus interface for Jami to interact with
  > Emacs, Org mode has org-protocol.el providing command line and URL
  > handler interface that is nearly independent of Org mode itself in its
  > core function:

Could you please explain in a few lines what sort of jobs org-protocol.el does?
The doc you sent

;;   1.) Add this to your init file (.emacs probably):
;;
;;       (require 'org-protocol)
;;
;;   3.) Ensure emacs-server is up and running.
...

describes in a low-level way _how_ to use it,
but in order to think about it, the first thing I need to know
is _what it can do_.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [DISCUSSION] Possible inclusion of org-capture.el into Emacs core (was: [ELPA] New package: jami-bot and org-jami-bot)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

NOTE: Is it absolute;y necessary to cross post to emacs-devel and
emacs-orgmode?

  > Similar to jami-bot providing a dbus interface for Jami to interact with
  > Emacs, Org mode has org-protocol.el providing command line and URL
  > handler interface that is nearly independent of Org mode itself in its
  > core function:

Could you please explain in a few lines what sort of jobs org-protocol.el does?
The doc you sent

;;   1.) Add this to your init file (.emacs probably):
;;
;;       (require 'org-protocol)
;;
;;   3.) Ensure emacs-server is up and running.
...

describes in a low-level way _how_ to use it, but in order to think
about that, the first thing I need to know is _what jobs it can do_.

  > With org-protocol, one can also make Emacs receive data from browser and
  > perform any action defined by custom handler function - all fully
  > configurable and not necessarily related to Org mode.

Could you state a few examples?

Also, how is this related to Org mode?  Why call it "org-protocol" rather
tham some other name?

      > 17.16.3 The open-source protocol
      > ... The open-source handler is designed to help with editing local
      > sources when reading a document..

What document does that come from?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [DISCUSSION] Possible inclusion of org-capture.el into Emacs core (was: [ELPA] New package: jami-bot and org-jami-bot)

[Ihor Radchenko]

Richard Stallman <rms@gnu.org> writes:

> NOTE: Is it absolute;y necessary to cross post to emacs-devel and
> emacs-orgmode?

This branch of the discussion is also about Org mode development. And we
discuss Org mode development in our mailing list. Not all Org mode users
and contributors are subscribed to emacs-devel, so they would miss this
thread if I did not cross-post.

> Could you please explain in a few lines what sort of jobs org-protocol.el does?
> The doc you sent
> ...
>
> describes in a low-level way _how_ to use it, but in order to think
> about that, the first thing I need to know is _what jobs it can do_.

1. Send web pages (urls, page title, text selection) from browser to
   Emacs buffers. This is often used to "bookmark" pages using Emacs.

2. Save web links from browser/command line into Org link kill ring.

3. Locate and open local source of web pages in Emacs.
   For example, when users have a local git clone of Org wiki pages
   (https://orgmode.org/worg/), they can open Org mode file
   corresponding to any Org wiki page opened in browser in Emacs.

All the above can also be done from command line or by invoking system
URI handler - from any application that knows how to open custom URI
schema.
        
(1) and (2) are specific to Org mode and will be left as a part of Org
package. But the core framework and maybe (3) might be upstreamed.

The above can also be achieved by invoking emacsclient --eval, but
org-protocol (1) makes it safer as only pre-defined Elisp code will be
evaluated, preventing security issues; (2) integrates with OS URI
handlers.

>   > With org-protocol, one can also make Emacs receive data from browser and
>   > perform any action defined by custom handler function - all fully
>   > configurable and not necessarily related to Org mode.
>
> Could you state a few examples?

In addition to the above 3 examples there are many possibilities.

Say, one can take some proprietary web page like reddit, and make it
open using Emacs reddit frontend.

Or save a quick note right from terminal outside Emacs
emacsclient "org-protocol://capture?title=A quick note for later"

> Also, how is this related to Org mode?  Why call it "org-protocol" rather
> tham some other name?

(1) is using org-capture library to put the received
url/title/annotation into user-defined template and save the filled
template into Org files.

(2) is using `org-store-link' to save url/title as Org mode link.

>       > 17.16.3 The open-source protocol
>       > ... The open-source handler is designed to help with editing local
>       > sources when reading a document..
>
> What document does that come from?

Org mode manual.
You can evaluate (info "(org)Protocols") to get there in Emacs.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [DISCUSSION] Possible inclusion of org-capture.el into Emacs core

[Max Nikulin]

I have dropped emacs-devel from Cc.

On 30/12/2023 19:15, Ihor Radchenko wrote:
> With org-protocol, one can also make Emacs receive data from browser and
> perform any action defined by custom handler function - all fully
> configurable and not necessarily related to Org mode.

and

Ihor Radchenko [SUMMARY] #8 [[bbb:OrgMeetup]] on Wed, June 12, 19:00 
UTC+3. Sat, 22 Jun 2024 08:32:40 +0000.
<https://list.orgmode.org/87h6dlidhz.fsf@localhost>
>   - Splitting Org into multiple small libraries is also a step forward
>     toward more modular Org and to converting some Org-specific
>     libraries to proper Emacs libraries, as requested by RMS:
>     https://list.orgmode.org/E1kIkxv-0007iy-Av@fencepost.gnu.org/
[...]
>       - org-protocol :: Processing external input into Emacs (from
>         command line, from clicked Urls as url handler). Unlike
>         running commands directly (which is unsafe), =org-protocol= can
>         be used for safe interaction with external processes

I have nothing against adding org-capture to Emacs core, but I think 
org-protocol should be just obsoleted. Since `server-eval-args-left' has 
been added to emacs-30, hacks from org-protocol have a little value. The 
new feature allows to implement handlers of URL schemes in a safe way 
and to avoid backslash hell similar to emacsclient-mail.desktop.

A tiny package may be created for convenience to have single .desktop 
file for all URI protocols (mailto:, org-protocol:). Leaving aside 
tricks with --display, handler should be like

      emacsclient --eval "(handle-url)" %u

or %U for multiple URLs. Packages just register supported protocols in 
e.g. an alist and the dispatcher calls a function for matched scheme 
with an argument from `server-eval-args-left'.