[ELPA] New package: jami-bot and org-jami-bot

[ELPA] New package: jami-bot and org-jami-bot

[Hanno Perrey]

Hej,

I would like to propose two tightly-coupled packages for inclusion into
ELPA: jami-bot and org-jami-bot.

jami-bot is an extendable chat bot for the distributed, private
messenger Jami. It interacts with the locally-installed Jami daemon via
D-Bus and reacts to both plain text messages and file transfers sent to
local accounts.

org-jami-bot builds upon jami-bot and extends it with Org mode capture
functionality for text messages and images. It allows to schedule agenda
items at specific dates, compose multi-measure captures and capture
images -- all by sending a message via the GNU Jami messenger.

I would like to keep maintaining both packages at their current
repositories:

- https://gitlab.com/hperrey/jami-bot
- https://gitlab.com/hperrey/org-jami-bot

I have signed the copyright paperwork with the FSF already and changed
the copyright in the respective files for both packages.

As this is the first time I am proposing a package for ELPA, any
guidance on the process and feedback would be highly welcome!

Thanks and cheers,
Hanno

--
Hanno Perrey
https://hoowl.se

Re: [ELPA] New package: jami-bot and org-jami-bot

[Stefan Kangas]

Hanno Perrey <hanno@hoowl.se> writes:

> I would like to propose two tightly-coupled packages for inclusion into
> ELPA: jami-bot and org-jami-bot.
>
> jami-bot is an extendable chat bot for the distributed, private
> messenger Jami. It interacts with the locally-installed Jami daemon via
> D-Bus and reacts to both plain text messages and file transfers sent to
> local accounts.
>
> org-jami-bot builds upon jami-bot and extends it with Org mode capture
> functionality for text messages and images. It allows to schedule agenda
> items at specific dates, compose multi-measure captures and capture
> images -- all by sending a message via the GNU Jami messenger.
>
> I would like to keep maintaining both packages at their current
> repositories:
>
> - https://gitlab.com/hperrey/jami-bot
> - https://gitlab.com/hperrey/org-jami-bot
>
> I have signed the copyright paperwork with the FSF already and changed
> the copyright in the respective files for both packages.

Sounds great.  Since Jami is a GNU project, it's a no-brainer that we should
have support for it in a GNU ELPA package.  I enjoyed your blog posts
about it too, and could easily see myself using something like this myself.

In other words, I don't see any reason why it couldn't go in.

> As this is the first time I am proposing a package for ELPA, any
> guidance on the process and feedback would be highly welcome!

Just some stylistic remarks:

1. Add .elpaignore files to both repositories containing, on a line of
   its own:

LICENSE

2. Here are some proposed changes, for your consideration.  This makes
   the package better follow some of our conventions:


diff --git a/org-jami-bot.el b/org-jami-bot.el
index 9cee94f..6d2f515 100644
--- a/org-jami-bot.el
+++ b/org-jami-bot.el
@@ -1,7 +1,7 @@
 ;;; org-jami-bot.el --- Capture GNU Jami messages as notes and todos
in Org mode -*- lexical-binding: t; -*-
-;;
+
 ;; Copyright (C) 2023 Free Software Foundation, Inc.
-;;
+
 ;; Author: Hanno Perrey <http://gitlab.com/hperrey>
 ;; Maintainer: Hanno Perrey <hanno@hoowl.se>
 ;; Created: April 16, 2023
@@ -10,46 +10,46 @@
 ;; Keywords: comm, outlines, org-capture, jami
 ;; Homepage: https://gitlab.com/hperrey/org-jami-bot
 ;; Package-Requires: ((emacs "28.1") (jami-bot "0.0.1"))
-;;
+
 ;; This file is not part of GNU Emacs.
 ;;
-;;    This program is free software: you can redistribute it and/or modify
-;;    it under the terms of the GNU General Public License as published by
-;;    the Free Software Foundation, either version 3 of the License, or
-;;    (at your option) any later version.
+;; This program is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
 ;;
-;;    This program is distributed in the hope that it will be useful,
-;;    but WITHOUT ANY WARRANTY; without even the implied warranty of
-;;    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-;;    GNU General Public License for more details.
-;;
-;;    You should have received a copy of the GNU General Public License
-;;    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
 ;;
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
 ;;; Commentary:
 ;;
 ;; `org-jami-bot' builds upon `jami-bot' and extends it with Org mode capture
-;; functionality for text messages and images. It allows to schedule agenda
+;; functionality for text messages and images.  It allows to schedule agenda
 ;; items at specific dates, compose multi-measure captures and
capture images --
 ;; all by sending a message via the GNU Jami messenger.
 ;;
 ;; `org-jami-bot' provides multi-message capture from within the Jami messenger
 ;; app -- that is, a capture process that consists of several messages and can
-;; include even images and other files. The process is started by sending the
+;; include even images and other files.  The process is started by sending the
 ;; command "!start" followed by the title of the capture and finished
by sending
-;; "!done". Once the multi-message capture session is started, every following
-;; message is simply added. This includes images which will be downloaded and
-;; stored locally. A reference in the form of a link will be included in the
+;; "!done".  Once the multi-message capture session is started, every following
+;; message is simply added.  This includes images which will be downloaded and
+;; stored locally.  A reference in the form of a link will be included in the
 ;; notes.
 ;;
 ;; Every command consists of an exclamation mark and a single word,
for example:
 ;; "!help" which shows the available commands or "!today" which captures the
-;; remainder of the message as a todo entry scheduled today. Everything else is
+;; remainder of the message as a todo entry scheduled today.
Everything else is
 ;; treated as a normal message (and captured verbatim).
 ;;
 ;; Files sent separately as a single message are captured as links to the
-;; locally downloaded file and tagged as =FILE=. In principle,
further automatic
-;; processing (e.g. OCR) could easily be integrated. Any received
file will also
+;; locally downloaded file and tagged as =FILE=.  In principle,
further automatic
+;; processing (e.g. OCR) could easily be integrated.  Any received
file will also
 ;; be added to the variable =org-stored-links= and can then be easily inserted
 ;; as link in any Org mode document using =C-c C-l=.
 ;;
@@ -113,8 +113,8 @@ CONVERSATION and ACCOUNT.
 Further plain text messages processed by
 `org-jami-bot--capture-plain-messsage' or files received by
 `org-jami-bot--capture-file' will be added to this capture
-buffer. The actual capture needs to happen through a separate
-function, e.g. `org-jami-bot--command-function-done'. Return a
+buffer.  The actual capture needs to happen through a separate
+function, e.g. `org-jami-bot--command-function-done'.  Return a
 reply string informing correspondent about how to finish capture
 by sending '!done'."
   (let* ((buf (format "*jami-capture-%s-%s*" account conversation))
@@ -203,7 +203,7 @@ CONVERSATION for jami ACCOUNT."
 (defun org-jami-bot--command-function-today (_account _conversation msg)
   "Capture body of message as todo entry scheduled today.

- Returns a reply string as confirmation. MSG is the full message
+ Returns a reply string as confirmation.  MSG is the full message
  in CONVERSATION id for ACCOUNT id."
   (let* ((body (cadr (assoc-string "body" msg)))
          (lines (string-lines body))
@@ -223,10 +223,10 @@ CONVERSATION for jami ACCOUNT."
   "Capture body as todo entry and schedule it on the date given after
the command.

 The entry will be scheduled according to the first line of the
-MSG body immediately following the command string. The date will
+MSG body immediately following the command string.  The date will
 be parsed through `org-read-date' and supports the same
-string-to-date conversations. Returns a reply string as
-confirmation. ACCOUNT and CONVERSATION are not used."
+string-to-date conversations.  Returns a reply string as
+confirmation.  ACCOUNT and CONVERSATION are not used."
   (let* ((body (cadr (assoc-string "body" msg)))
          (lines (string-lines body))
          (swhen (org-read-date nil nil (car lines)))
@@ -245,9 +245,9 @@ confirmation. ACCOUNT and CONVERSATION are not used."
 (defun org-jami-bot-default-setup ()
   "Set up `org-jami-bot' with default values.

-        Create a capture template, extend `jami-bot' commands via
-        `jami-bot-command-function-alist' and add hooks to
-        `jami-bot'."
+Create a capture template, extend `jami-bot' commands via
+`jami-bot-command-function-alist' and add hooks to
+`jami-bot'."
   (if (assoc org-jami-bot-capture-key org-capture-templates)
       (message "Capture template referred to by \"%s\" key already defined!"
                org-jami-bot-capture-key)




diff --git a/jami-bot.el b/jami-bot.el
index c453469..32e13f1 100644
--- a/jami-bot.el
+++ b/jami-bot.el
@@ -1,7 +1,7 @@
 ;;; jami-bot.el --- An extendable chat bot for the private messenger
GNU Jami -*- lexical-binding: t; -*-
-;;
+
 ;; Copyright (C) 2023 Free Software Foundation, Inc.
-;;
+
 ;; Author: Hanno Perrey <http://gitlab.com/hperrey>
 ;; Maintainer: Hanno Perrey <hanno@hoowl.se>
 ;; Created: April 15, 2023
@@ -10,27 +10,27 @@
 ;; Keywords: comm, jami, messenger, chat bot, dbus
 ;; Homepage: https://gitlab.com/hperrey/jami-bot
 ;; Package-Requires: ((emacs "27.1"))
-;;
+
 ;; This file is not part of GNU Emacs.
 ;;
-;;    This program is free software: you can redistribute it and/or modify
-;;    it under the terms of the GNU General Public License as published by
-;;    the Free Software Foundation, either version 3 of the License, or
-;;    (at your option) any later version.
-;;
-;;    This program is distributed in the hope that it will be useful,
-;;    but WITHOUT ANY WARRANTY; without even the implied warranty of
-;;    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-;;    GNU General Public License for more details.
+;; This program is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
 ;;
-;;    You should have received a copy of the GNU General Public License
-;;    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
 ;;
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
 ;;; Commentary:
 ;;
-;; An extendable chat bot for the distributed, private messenger Jami. It
+;; An extendable chat bot for the distributed, private messenger Jami.  It
 ;; interacts with the locally-installed Jami daemon via D-Bus and reacts to
-;; both plain text messages and file transfers sent to local accounts. Further
+;; both plain text messages and file transfers sent to local accounts.  Further
 ;; processing of either or both can be configured by adding functions to the
 ;; abnormal hooks, `jami-bot-text-message-functions' and
 ;; `jami-bot-data-transfer-functions', respectively.
@@ -40,7 +40,7 @@
 ;; Further commands than the ones included can be configured by mapping them to
 ;; functions through `jami-bot-command-function-alist'.
 ;;
-;; Set up `jami-bot' by executing `jami-bot-register'. This will set up the
+;; Set up `jami-bot' by executing `jami-bot-register'.  This will set up the
 ;; message handler, `jami-bot--messageReceived-handler', to be called on the
 ;; `messageReceived' D-Bus signal.
 ;;
@@ -51,11 +51,11 @@
 (defvar jami-bot-account-user-names nil
   "List of account user names that `jami-bot' handles messages for.

-        If set to nil then `jami-bot' will react to any message
-        send to a local account. The user name is also sometimes
-        referred to as address in Jami and should be a 40
-        character has such as
-        \"badac18e13ec1a6e1266600e457859afebfb9c46\".")
+If set to nil then `jami-bot' will react to any message
+send to a local account.  The user name is also sometimes
+referred to as address in Jami and should be a 40
+character has such as
+\"badac18e13ec1a6e1266600e457859afebfb9c46\".")

 (defvar jami-bot-command-function-alist
   '(("!ping" . jami-bot--command-function-ping)
@@ -63,15 +63,15 @@
   "Alist mapping command strings in message body to functions to be executed.

 Each command needs to start with an exclamation mark '!' and
-consist of a single (lowercase) word. The corresponding function
needs to accept
-the account id, the conversation id and the message alist as
+consist of a single (lowercase) word.  The corresponding function needs to
+accept the account id, the conversation id and the message alist as
 arguments and return a string (that is sent as reply to the original
message).")

 (defvar jami-bot-text-message-functions nil
   "A list of functions that will be called when processing a plain
text message.

 Functions must take the ACCOUNT and CONVERSATION ids as well as
-the actual MSG as arguments. Their return value will be ignored.")
+the actual MSG as arguments.  Their return value will be ignored.")

 (defvar jami-bot-download-path "~/jami/"
 "Path in which to store files downloaded from conversations.
@@ -83,21 +83,21 @@ Will be created if not existing yet.")

 Functions must take the ACCOUNT and CONVERSATION ids as well as
 the actual MSG and the local downloaded file name, DLNAME, as
-arguments. Their return value will be ignored.")
+arguments.  Their return value will be ignored.")

 (defvar jami-bot--jami-local-account-ids nil
   "List of `jami' local accounts user ids and name pairs.

-Caches output of dbus-methods 'getAccountList' and
-'getAccountDetails'. For internal use in `jami-bot'.")
+Caches output of dbus-methods \"getAccountList\" and
+\"getAccountDetails\".  For internal use in `jami-bot'.")

 (defun jami-bot--messageReceived-handler (account conversation msg)
   "Handle messages from Jami's `messageReceived' D-Bus signal.

-  ACCOUNT and CONVERSATION are the corresponding ids to which the
-  MSG belongs to. The latter contains additional fields such as
-  `author' and `body'. The field `type' is used to identify which
-  function to call for further processing."
+ACCOUNT and CONVERSATION are the corresponding ids to which the
+MSG belongs to.  The latter contains additional fields such as
+`author' and `body'.  The field `type' is used to identify which
+function to call for further processing."
   ;; make sure we are not reacting to messages sent from our own local
   ;; account(s) or accounts we are not to monitor
   (unless jami-bot--jami-local-account-ids
@@ -160,7 +160,7 @@ Caches output of dbus-methods 'getAccountList' and
   "Ping the Jami daemon and register `jami-bot' handler for receiving
messages."
   (interactive)
   (or (dbus-ping :session "cx.ring.Ring")
-      (error "Jami Daemon (jamid) not available through dbus. Please
check Jami installation"))
+      (error "Jami Daemon (jamid) not available through dbus.  Please
check Jami installation"))
   (dbus-register-signal :session "cx.ring.Ring"
                       "/cx/ring/Ring/ConfigurationManager"
                       "cx.ring.Ring.ConfigurationManager"
@@ -171,7 +171,7 @@ Caches output of dbus-methods 'getAccountList' and
   "Process plain text messages and parse the message body for commands.

   ACCOUNT and CONVERSATION are the corresponding ids to which the
-  message MSG belongs to. Messages containing commands must start
+  message MSG belongs to.  Messages containing commands must start
   with an exclamation mark (\"!\") followed by the single-word
   command. Each command is mapped to a function via
   `jami-bot-command-function-alist' which will be executed when
@@ -210,10 +210,10 @@ Caches output of dbus-methods 'getAccountList' and
                           account conversation msg))))

 (defun jami-bot--command-function-ping (_account _conversation msg)
-  "Return the string 'pong!' followed by the message's body.
+  "Return the string \"pong!\" followed by the message body.

-Example for a basic jami bot command handling function. Acts on MSG
-received via _ACCOUNT in _CONVERSATION. The latter two are unused."
+Example for a basic jami bot command handling function.  Acts on MSG
+received via _ACCOUNT in _CONVERSATION.  The latter two are unused."
   (let ((body (cadr (assoc-string "body" msg))))
     (format "pong! %s" body)))

@@ -230,10 +230,10 @@ which are used."
 (defun jami-bot--process-data-transfer (account conversation msg)
   "Process data transfer from received messages.

-  Downloads files to the path given by `jami-bot-download-path'
-  and calls the abnormal hook `jami-bot-data-transfer-functions'
-  for further processing. ACCOUNT and CONVERSATION are the
-  corresponding ids to which the message MSG belongs to."
+Downloads files to the path given by `jami-bot-download-path'
+and calls the abnormal hook `jami-bot-data-transfer-functions'
+for further processing.  ACCOUNT and CONVERSATION are the
+corresponding ids to which the message MSG belongs to."
   (let* ((id (cadr (assoc-string "id" msg)))
          (fileid (cadr (assoc-string "fileId" msg)))
          (filename (cadr (assoc-string "displayName" msg)))
@@ -254,7 +254,7 @@ which are used."
                     ,method ,@(when args args))))

 (defun jami-bot-send-message (account conversation text &optional reply)
-  "Add TEXT to CONVERSATION via ACCOUNT. REPLY specifies a message id."
+  "Add TEXT to CONVERSATION via ACCOUNT.  REPLY specifies a message id."
   (jami-bot--dbus-cfgmgr-call-method "sendMessage"
                                      account
                                      conversation

Re: [ELPA] New package: jami-bot and org-jami-bot

[Hanno Perrey]

Hej Stefan,

> Sounds great.  Since Jami is a GNU project, it's a no-brainer that we should
> have support for it in a GNU ELPA package.  I enjoyed your blog posts
> about it too, and could easily see myself using something like this myself.

thank you for the kind words! If you happen to give those packages a
try, I would love to hear how it goes! :)

> In other words, I don't see any reason why it couldn't go in.

That is great to hear!

> Just some stylistic remarks:
>
> 1. Add .elpaignore files to both repositories containing, on a line of
>    its own:
> LICENSE

Done.

> 2. Here are some proposed changes, for your consideration.  This makes
>    the package better follow some of our conventions:

Thanks for the detailed style vetting! I have applied the patches and made
some minor adjustment in a couple of other places -- hope I didn't miss
anything.

Thanks and cheers,
Hanno

Re: [ELPA] New package: jami-bot and org-jami-bot

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > jami-bot is an extendable chat bot for the distributed, private
  > messenger Jami. It interacts with the locally-installed Jami daemon via
  > D-Bus and reacts to both plain text messages and file transfers sent to
  > local accounts.

That sounds very useful.  (It might make a difference to how easily
I can use Jami.)

  > org-jami-bot builds upon jami-bot and extends it with Org mode capture
  > functionality for text messages and images.

This feature will be useful.  I am not concretely sure what "capture
functionality" includes, but if it provides higher-level communication
between Emacs and Jami, that is a good thing.

At the same time, it seems to be implemented in a way that imposes an
evitable restriction -- requiring use of Org mode.

Dependence of various features on Org mode causes problems in two ways:

* It makes those features hard to use for those that don't use Org mode.

* It adds the need to learn Org mode to the need to learn Emacs itself.

Therefore, new features which make sense indepdently of Org mode
should be usable without Org mode.

It is fine for this feature to support an Org mode interface, but it
should be equally convenient to use the feature with standard Emacs
formats (and not using Org).

So, before you install this, would you please add support for the same
features using the formats of standard Emacs, sans Org?

It would be good to change the name too, to a name that does not
presume use of Org (though it could work with Org also).  Perhaps
jami-message?

Thank you for working on this.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] New package: jami-bot and org-jami-bot

[Hanno Perrey]

Hej Richard,

>   > org-jami-bot builds upon jami-bot and extends it with Org mode capture
>   > functionality for text messages and images.
>
> This feature will be useful.  I am not concretely sure what "capture
> functionality" includes, but if it provides higher-level communication
> between Emacs and Jami, that is a good thing.

glad to hear you are interested in the package and thanks for your
comments!

Just to clarify: jami-bot is not intended as a high-level interface,
that is to say a client for Jami; it is instead a "bot", a tool to send
automatic messages to predefined recipients or to respond to incoming
messages in a flexible way depending on message content.

Using jami-bot, it would be straightforward to e.g. send a message to
someone when a specific document has been edited or chat with the Emacs
doctor -- along those lines. My personal itch was note taking and
accessing information that I only keep in Emacs.

> At the same time, it seems to be implemented in a way that imposes an
> evitable restriction -- requiring use of Org mode.

Not quite, as this proposal is about two distinct packages, jami-bot and
org-jami-bot. The latter depends on the former and is the one that
depends on Org mode. It can also be seen as a demonstration of how
jami-bot can be used to interface to different aspects of Emacs.

As these packages are so tightly related, I made the decision to propose
them for inclusion in Elpa in one go; in retrospect, that might have
made it less clear where one ends and the other starts.

> Therefore, new features which make sense indepdently of Org mode
> should be usable without Org mode.

All the base functionality (e.g. sending messages, parsing incoming
messages and adding hooks for replies depending on any command found in
the message) is implemented independent of Org mode. Only the
note-taking (i.e. Org mode-specific) parts require Org mode. But the
principle is the same and could be extended to other packages as well.

> It would be good to change the name too, to a name that does not
> presume use of Org (though it could work with Org also).  Perhaps
> jami-message?

The base package is called jami-bot which I hope emphasizes that it is
not really a user-interface to Jami itself, but a framework for
automatic message sending and processing. In itself it is only useful
for someone familiar with Elisp but it can be used for packages (such as
org-jami-bot) that provide easy-to-configure functionality out of the box.

A full Jami UI would require a lot more work, however. If anyone wants
to go in this direction, the parts of jami-bot preparing the Dbus calls
could be refactorized as a starting point. I have documented my
exploration of Jami's Dbus interface here:
http://www.hoowl.se/exploring_dbus_from_emacs.html


Thanks and cheers,
Hanno

Re: [ELPA] New package: jami-bot and org-jami-bot

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > At the same time, it seems to be implemented in a way that imposes an
  > > evitable restriction -- requiring use of Org mode.

  > Not quite, as this proposal is about two distinct packages, jami-bot and
  > org-jami-bot.

I undersand that.  I'm taling about org-jami-bot.  Here's what you said
about it:

>   > org-jami-bot builds upon jami-bot and extends it with Org mode capture
>   > functionality for text messages and images.

I commented on that:

> This feature will be useful.  I am not concretely sure what "capture
> functionality" includes, but if it provides higher-level communication
> between Emacs and Jami, that is a good thing.

> At the same time, it seems to be implemented in a way that imposes an
> evitable restriction -- requiring use of Org mode.

All that was meant to be about org-jami-bot.

Thanks for giving more info about jami-bot itself.  It sounds useful,
and I am glad it does not depend on Org mode.

I'm not proposing we write a full UI for Jami implemented in Emacs.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

[DISCUSSION] Possible inclusion of org-capture.el into Emacs core (was: [ELPA] New package: jami-bot and org-jami-bot)

[Ihor Radchenko]

Richard Stallman <rms@gnu.org> writes:

> Dependence of various features on Org mode causes problems in two ways:
>
> * It makes those features hard to use for those that don't use Org mode.
>
> * It adds the need to learn Org mode to the need to learn Emacs itself.
>
> Therefore, new features which make sense indepdently of Org mode
> should be usable without Org mode.

This reminds me of "Keep Org mode submodules in Emacs, please!" request:
https://list.orgmode.org/E1kIPh1-0001Lu-Rg@fencepost.gnu.org/

Similar to jami-bot providing a dbus interface for Jami to interact with
Emacs, Org mode has org-protocol.el providing command line and URL
handler interface that is nearly independent of Org mode itself in its
core function:

;; Usage:
;; ------
;;
;;   1.) Add this to your init file (.emacs probably):
;;
;;       (require 'org-protocol)
;;
;;   3.) Ensure emacs-server is up and running.
;;   4.) Try this from the command line (adjust the URL as needed):
;;
;;       $ emacsclient \
;;         "org-protocol://store-link?url=http:%2F%2Flocalhost%2Findex.html&title=The%20title"
;;
;;   5.) Optionally add custom sub-protocols and handlers:
;;
;;       (setq org-protocol-protocol-alist
;;             '(("my-protocol"
;;                :protocol "my-protocol"
;;                :function my-protocol-handler-function)))
;;
;;       A "sub-protocol" will be found in URLs like this:
;;
;;           org-protocol://sub-protocol?key=val&key2=val2
;;
;; If it works, you can now setup other applications for using this feature.

With org-protocol, one can also make Emacs receive data from browser and
perform any action defined by custom handler function - all fully
configurable and not necessarily related to Org mode.
    
    17.16.3 The open-source protocol
    ... The open-source handler is designed to help with editing local
    sources when reading a document.. To that effect, you can use a
    bookmark with the following location:
    
    javascript:location.href='org-protocol://open-source?&url='+
          encodeURIComponent(location.href)

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [DISCUSSION] Possible inclusion of org-capture.el into Emacs core (was: [ELPA] New package: jami-bot and org-jami-bot)

[Stefan Kangas]

Ihor Radchenko <yantar92@posteo.net> writes:

> With org-protocol, one can also make Emacs receive data from browser and
> perform any action defined by custom handler function - all fully
> configurable and not necessarily related to Org mode.

I don't have much to add besides giving support to the idea.  It would
be useful.  I'm sure it would find its use in other types of note taking
and productivity systems, of which we have several.

Re: [DISCUSSION] Possible inclusion of org-capture.el into Emacs core (was: [ELPA] New package: jami-bot and org-jami-bot)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Similar to jami-bot providing a dbus interface for Jami to interact with
  > Emacs, Org mode has org-protocol.el providing command line and URL
  > handler interface that is nearly independent of Org mode itself in its
  > core function:

Could you please explain in a few lines what sort of jobs org-protocol.el does?
The doc you sent

;;   1.) Add this to your init file (.emacs probably):
;;
;;       (require 'org-protocol)
;;
;;   3.) Ensure emacs-server is up and running.
...

describes in a low-level way _how_ to use it,
but in order to think about it, the first thing I need to know
is _what it can do_.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [DISCUSSION] Possible inclusion of org-capture.el into Emacs core (was: [ELPA] New package: jami-bot and org-jami-bot)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

NOTE: Is it absolute;y necessary to cross post to emacs-devel and
emacs-orgmode?

  > Similar to jami-bot providing a dbus interface for Jami to interact with
  > Emacs, Org mode has org-protocol.el providing command line and URL
  > handler interface that is nearly independent of Org mode itself in its
  > core function:

Could you please explain in a few lines what sort of jobs org-protocol.el does?
The doc you sent

;;   1.) Add this to your init file (.emacs probably):
;;
;;       (require 'org-protocol)
;;
;;   3.) Ensure emacs-server is up and running.
...

describes in a low-level way _how_ to use it, but in order to think
about that, the first thing I need to know is _what jobs it can do_.

  > With org-protocol, one can also make Emacs receive data from browser and
  > perform any action defined by custom handler function - all fully
  > configurable and not necessarily related to Org mode.

Could you state a few examples?

Also, how is this related to Org mode?  Why call it "org-protocol" rather
tham some other name?

      > 17.16.3 The open-source protocol
      > ... The open-source handler is designed to help with editing local
      > sources when reading a document..

What document does that come from?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [DISCUSSION] Possible inclusion of org-capture.el into Emacs core (was: [ELPA] New package: jami-bot and org-jami-bot)

[Ihor Radchenko]

Richard Stallman <rms@gnu.org> writes:

> NOTE: Is it absolute;y necessary to cross post to emacs-devel and
> emacs-orgmode?

This branch of the discussion is also about Org mode development. And we
discuss Org mode development in our mailing list. Not all Org mode users
and contributors are subscribed to emacs-devel, so they would miss this
thread if I did not cross-post.

> Could you please explain in a few lines what sort of jobs org-protocol.el does?
> The doc you sent
> ...
>
> describes in a low-level way _how_ to use it, but in order to think
> about that, the first thing I need to know is _what jobs it can do_.

1. Send web pages (urls, page title, text selection) from browser to
   Emacs buffers. This is often used to "bookmark" pages using Emacs.

2. Save web links from browser/command line into Org link kill ring.

3. Locate and open local source of web pages in Emacs.
   For example, when users have a local git clone of Org wiki pages
   (https://orgmode.org/worg/), they can open Org mode file
   corresponding to any Org wiki page opened in browser in Emacs.

All the above can also be done from command line or by invoking system
URI handler - from any application that knows how to open custom URI
schema.
        
(1) and (2) are specific to Org mode and will be left as a part of Org
package. But the core framework and maybe (3) might be upstreamed.

The above can also be achieved by invoking emacsclient --eval, but
org-protocol (1) makes it safer as only pre-defined Elisp code will be
evaluated, preventing security issues; (2) integrates with OS URI
handlers.

>   > With org-protocol, one can also make Emacs receive data from browser and
>   > perform any action defined by custom handler function - all fully
>   > configurable and not necessarily related to Org mode.
>
> Could you state a few examples?

In addition to the above 3 examples there are many possibilities.

Say, one can take some proprietary web page like reddit, and make it
open using Emacs reddit frontend.

Or save a quick note right from terminal outside Emacs
emacsclient "org-protocol://capture?title=A quick note for later"

> Also, how is this related to Org mode?  Why call it "org-protocol" rather
> tham some other name?

(1) is using org-capture library to put the received
url/title/annotation into user-defined template and save the filled
template into Org files.

(2) is using `org-store-link' to save url/title as Org mode link.

>       > 17.16.3 The open-source protocol
>       > ... The open-source handler is designed to help with editing local
>       > sources when reading a document..
>
> What document does that come from?

Org mode manual.
You can evaluate (info "(org)Protocols") to get there in Emacs.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [DISCUSSION] Possible inclusion of org-capture.el into Emacs core

[Max Nikulin]

I have dropped emacs-devel from Cc.

On 30/12/2023 19:15, Ihor Radchenko wrote:
> With org-protocol, one can also make Emacs receive data from browser and
> perform any action defined by custom handler function - all fully
> configurable and not necessarily related to Org mode.

and

Ihor Radchenko [SUMMARY] #8 [[bbb:OrgMeetup]] on Wed, June 12, 19:00 
UTC+3. Sat, 22 Jun 2024 08:32:40 +0000.
<https://list.orgmode.org/87h6dlidhz.fsf@localhost>
>   - Splitting Org into multiple small libraries is also a step forward
>     toward more modular Org and to converting some Org-specific
>     libraries to proper Emacs libraries, as requested by RMS:
>     https://list.orgmode.org/E1kIkxv-0007iy-Av@fencepost.gnu.org/
[...]
>       - org-protocol :: Processing external input into Emacs (from
>         command line, from clicked Urls as url handler). Unlike
>         running commands directly (which is unsafe), =org-protocol= can
>         be used for safe interaction with external processes

I have nothing against adding org-capture to Emacs core, but I think 
org-protocol should be just obsoleted. Since `server-eval-args-left' has 
been added to emacs-30, hacks from org-protocol have a little value. The 
new feature allows to implement handlers of URL schemes in a safe way 
and to avoid backslash hell similar to emacsclient-mail.desktop.

A tiny package may be created for convenience to have single .desktop 
file for all URI protocols (mailto:, org-protocol:). Leaving aside 
tricks with --display, handler should be like

      emacsclient --eval "(handle-url)" %u

or %U for multiple URLs. Packages just register supported protocols in 
e.g. an alist and the dispatcher calls a function for matched scheme 
with an argument from `server-eval-args-left'.

[DISCUSSION] org-capture.el vs remember.el (was: [ELPA] New package: jami-bot and org-jami-bot)

[Ihor Radchenko]

Recent emacs-devel discussion brought up some concerns about org-capture
being very specific to Org mode and not integrated with the rest of
Emacs.

However, org-capture was once built on top of the built-in remember.el
before it was refactored into independent code by Carsten:
https://list.orgmode.org/orgmode/C46F10DC-DE51-43D4-AFFE-F71E440D1E1F@gmail.com/

The refactoring de-coupled what used to be org-remember.el into
completely rewritten org-capture.el that added important features that
could not be implemented within remember.el framework:

1. org-capture arranges writing the text to remember directly into the
   target location where it is to be stored. This addressed an important
   problem of remember.el - remember.el always uses temporary buffer for
   user input and does not save it until remember-finalize is executed
   explicitly. A number of users sometimes forgot to call
   remember-finalize, leading to annoying data losses.

2. remember.el only allows a single text input at a time. org-capture
   makes it possible for users to write multiple notes in parallel.

I am wondering if it would be possible to have the above two key
features supported by remember.el. Then, org-capture.el could go back to
re-using remember.el infrastructure and thus reduce code duplication.

Richard Stallman <rms@gnu.org> writes:

>   > org-jami-bot builds upon jami-bot and extends it with Org mode capture
>   > functionality for text messages and images.
>
> This feature will be useful.  I am not concretely sure what "capture
> functionality" includes, but if it provides higher-level communication
> between Emacs and Jami, that is a good thing.
>
> At the same time, it seems to be implemented in a way that imposes an
> evitable restriction -- requiring use of Org mode.
>
> Dependence of various features on Org mode causes problems in two ways:
>
> * It makes those features hard to use for those that don't use Org mode.
>
> * It adds the need to learn Org mode to the need to learn Emacs itself.
>
> Therefore, new features which make sense indepdently of Org mode
> should be usable without Org mode.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [DISCUSSION] org-capture.el vs remember.el (was: [ELPA] New package: jami-bot and org-jami-bot)

[Stefan Kangas]

Ihor Radchenko <yantar92@posteo.net> writes:

> The refactoring de-coupled what used to be org-remember.el into
> completely rewritten org-capture.el that added important features that
> could not be implemented within remember.el framework:
>
> 1. org-capture arranges writing the text to remember directly into the
>    target location where it is to be stored. This addressed an important
>    problem of remember.el - remember.el always uses temporary buffer for
>    user input and does not save it until remember-finalize is executed
>    explicitly. A number of users sometimes forgot to call
>    remember-finalize, leading to annoying data losses.
>
> 2. remember.el only allows a single text input at a time. org-capture
>    makes it possible for users to write multiple notes in parallel.
>
> I am wondering if it would be possible to have the above two key
> features supported by remember.el. Then, org-capture.el could go back to
> re-using remember.el infrastructure and thus reduce code duplication.

This is, of course, desirable.  The real question is how much work it
will take and who will volunteer to do it.

On the face of things, remember.el does not comprise a huge amount of
code, and I don't see why the above two features couldn't be retrofitted
into it.

Patches welcome.

Re: [DISCUSSION] org-capture.el vs remember.el (was: [ELPA] New package: jami-bot and org-jami-bot)

[João Távora]

On Sat, Dec 30, 2023 at 5:21 PM Stefan Kangas <stefankangas@gmail.com> wrote:
>
> Ihor Radchenko <yantar92@posteo.net> writes:
>
> > The refactoring de-coupled what used to be org-remember.el into
> > completely rewritten org-capture.el that added important features that
> > could not be implemented within remember.el framework:
> >
> > 1. org-capture arranges writing the text to remember directly into the
> >    target location where it is to be stored. This addressed an important
> >    problem of remember.el - remember.el always uses temporary buffer for
> >    user input and does not save it until remember-finalize is executed
> >    explicitly. A number of users sometimes forgot to call
> >    remember-finalize, leading to annoying data losses.
> >
> > 2. remember.el only allows a single text input at a time. org-capture
> >    makes it possible for users to write multiple notes in parallel.
> >
> > I am wondering if it would be possible to have the above two key
> > features supported by remember.el. Then, org-capture.el could go back to
> > re-using remember.el infrastructure and thus reduce code duplication.
>
> This is, of course, desirable.  The real question is how much work it
> will take and who will volunteer to do it.

Not volunteering but those two goals seems good, especially
the first one

> On the face of things, remember.el does not comprise a huge amount of
> code, and I don't see why the above two features couldn't be retrofitted
> into it.

Seems like the first one could be done with narrowing.
But would also need some kind of indirect buffer, too,
right?  How does org-capture.el do it?  It would
also mean "commiting to remember" is just C-x C-s and file
recovery works as normal, which is how it should be IMO.

Maybe after-save-hook in this indirect buffer specifically
would have the effect of quitting the window displaying the
indirect buffer? ... Well I just checked and the hook approach
doesn't work, because save-buffer always saves the base buffer.
But surely some other hook or some remapped key would do the
trick.

If this works (big if since given all this is vapourware), we'll
finally have found good use for indirect buffers :-)

The second one is just multiple such setups, which the
window-management code should in theory already allow.

João

Re: [DISCUSSION] org-capture.el vs remember.el (was: [ELPA] New package: jami-bot and org-jami-bot)

[João Távora]

On Sat, Dec 30, 2023 at 7:16 PM João Távora <joaotavora@gmail.com> wrote:

> > On the face of things, remember.el does not comprise a huge amount of
> > code, and I don't see why the above two features couldn't be retrofitted
> > into it.
>
> Seems like the first one could be done with narrowing.
> But would also need some kind of indirect buffer, too,
> right?  How does org-capture.el do it?

Answering myself, with indirect buffers :-)

João

Re: [DISCUSSION] org-capture.el vs remember.el (was: [ELPA] New package: jami-bot and org-jami-bot)

[Adam Porter]

> If this works (big if since given all this is vapourware), we'll
> finally have found good use for indirect buffers :-)

FWIW, I use indirect buffers constantly through 
`org-tree-to-indirect-buffer', which is also called from several of my 
tools like org-ql, org-bookmark-heading, etc.  Largely, I treat my ~/org 
directory as a database, and when I open an entry from a search result 
or a bookmark in an indirect buffer, I don't even care what file/buffer 
it's actually in; the indirect buffer offers the illusion that it's in a 
file of its own, without any other content around it.

Re: [ELPA] New package: jami-bot and org-jami-bot

[Stefan Kangas]

Hanno Perrey <hanno@hoowl.se> writes:

> I would like to propose two tightly-coupled packages for inclusion into
> ELPA: jami-bot and org-jami-bot.
>
> jami-bot is an extendable chat bot for the distributed, private
> messenger Jami. It interacts with the locally-installed Jami daemon via
> D-Bus and reacts to both plain text messages and file transfers sent to
> local accounts.
>
> org-jami-bot builds upon jami-bot and extends it with Org mode capture
> functionality for text messages and images. It allows to schedule agenda
> items at specific dates, compose multi-measure captures and capture
> images -- all by sending a message via the GNU Jami messenger.
>
> I would like to keep maintaining both packages at their current
> repositories:
>
> - https://gitlab.com/hperrey/jami-bot
> - https://gitlab.com/hperrey/org-jami-bot
>
> I have signed the copyright paperwork with the FSF already and changed
> the copyright in the respective files for both packages.
>
> As this is the first time I am proposing a package for ELPA, any
> guidance on the process and feedback would be highly welcome!

Philip, could you please help add these packages to GNU ELPA?

The copyright assignment is all taken care of.

Re: [ELPA] New package: jami-bot and org-jami-bot

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Please do install jami-bit, but DO NOT install org-jami-bot without
some changes.  Before that is installed, it needs to be changed not to
require or presuppose Org format.

We have not adopted Org format as the standard Emacs format for text,
because it would not be adequate for that.  Designing additional
packages to impose the use of Org format, even though they have
nothing to do with Org, pushes Emacs in a direction that is not a good
design.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] New package: jami-bot and org-jami-bot

[Ihor Radchenko]

Richard Stallman <rms@gnu.org> writes:

> Please do install jami-bit, but DO NOT install org-jami-bot without
> some changes.  Before that is installed, it needs to be changed not to
> require or presuppose Org format.

org-jami-bot's description is the following:

Capture GNU Jami messages as notes and todos in Org mode
...
 `org-jami-bot' builds upon `jami-bot' and extends it with Org mode capture
;; functionality for text messages and images.  It allows to schedule agenda
;; items at specific dates, compose multi-measure captures and capture images --
;; all by sending a message via the GNU Jami messenger.

In other words, org-jami-bot package is an integration of jami-bot with
org-capture API. Its name clearly indicates that it has something to do
with Org mode.

> We have not adopted Org format as the standard Emacs format for text,
> because it would not be adequate for that.  Designing additional
> packages to impose the use of Org format, even though they have
> nothing to do with Org, pushes Emacs in a direction that is not a good
> design.

I am not sure what is the problem you are trying to avoid, but I hope
that you did not imply that packages that provide integration with Org
mode are not allowed on ELPA.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [ELPA] New package: jami-bot and org-jami-bot

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: hanno@hoowl.se, emacs-devel@gnu.org
> Date: Mon, 08 Jan 2024 16:08:08 -0500
> 
> Please do install jami-bit, but DO NOT install org-jami-bot without
> some changes.  Before that is installed, it needs to be changed not to
> require or presuppose Org format.

If org-jami-bot is specifically for doing something on top of Org, I
see no problem in installing it as it is.

> We have not adopted Org format as the standard Emacs format for text,
> because it would not be adequate for that.  Designing additional
> packages to impose the use of Org format, even though they have
> nothing to do with Org, pushes Emacs in a direction that is not a good
> design.

AFAIU, org-jami-bot _does_ have something to do with Org.

Re: [ELPA] New package: jami-bot and org-jami-bot

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Richard Stallman <rms@gnu.org>
>> Cc: hanno@hoowl.se, emacs-devel@gnu.org
>> Date: Mon, 08 Jan 2024 16:08:08 -0500
>>
>> Please do install jami-bit, but DO NOT install org-jami-bot without
>> some changes.  Before that is installed, it needs to be changed not to
>> require or presuppose Org format.
>
> If org-jami-bot is specifically for doing something on top of Org, I
> see no problem in installing it as it is.

Yes, indeed.  Without the org parts, there's nothing there.  To change
it "not to require org" amounts to writing another package from scratch.

>> We have not adopted Org format as the standard Emacs format for text,
>> because it would not be adequate for that.  Designing additional
>> packages to impose the use of Org format, even though they have
>> nothing to do with Org, pushes Emacs in a direction that is not a good
>> design.
>
> AFAIU, org-jami-bot _does_ have something to do with Org.

We are also not discussing adding this package to core Emacs, but to GNU
ELPA.  There, we do not just allow but in fact actively encourage
extensions that are likely to be useful only in niche use cases.

There are no plans to adopt Org as the standard Emacs format for text.

Re: [ELPA] New package: jami-bot and org-jami-bot

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > We are also not discussing adding this package to core Emacs, but to GNU
  > ELPA.

GNU ELPA is part of Emacs.  We move packages between GNU ELPA and core
Emacs as a matter of how it is most convenient to distribute each
individual package.  It was all planned for this.

This issue is not about distribution of one package, it is about the
overall direction of Emacs development.

  > Yes, indeed.  Without the org parts, there's nothing there.  To change
  > it "not to require org" amounts to writing another package from scratch.

Are you saying that the code to do the job of org-jami-bot in a
non-Org-specific way is trivial?  Does the jami-bot package do the
whole of that general job already?

If that is so, I will stand corrected.  So I'd like to check the code.
Stefan, would you please email me the source code of org-jami-bot?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] New package: jami-bot and org-jami-bot

[Stefan Kangas]

Richard Stallman <rms@gnu.org> writes:

> Are you saying that the code to do the job of org-jami-bot in a
> non-Org-specific way is trivial?  Does the jami-bot package do the
> whole of that general job already?
>
> If that is so, I will stand corrected.  So I'd like to check the code.

My understanding is that the separation you ask for is already
essentially done, indeed.

> Stefan, would you please email me the source code of org-jami-bot?

File sent off-list.

Re: [ELPA] New package: jami-bot and org-jami-bot

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > If org-jami-bot is specifically for doing something on top of Org, I
  > see no problem in installing it as it is.

The problem is that that package restricts a general facility
(communicating data with Jami) by implementing it to be usable only
with Org format.

There is no reason for it to have that limitation.  It should be
implemented to work with a more general interface.  People could
then implement support for Org format on that base.

The right thing to do with org-jami-bot is to generalize it to remove
this limit, rename it, and install that instead.

Emacs already shows the effects of many years of the bad practice of
integrating various new features specifically with Org, not with Emacs
generally.  As a result, they are available only to those who use Org
mode.  There are no efforts to generalize these features so they are
not limited to Org.

We must therefore insist that every new feature XYZ which CAN be
modularly separate from Org, be implemented as modularly separate from
Org.  People who want to use XYZ and Org together should be able to do
so -- but people who want to use the one without the other should also
be able to do so.

In other words, if XYZ doesn't inherently _have_ to be limited to Org,
it must not be limited to Org.

Please join me in insisting on this modularity.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

ELPA packages and Org mode integration (was: [ELPA] New package: jami-bot and org-jami-bot)

[Ihor Radchenko]

Richard Stallman <rms@gnu.org> writes:

> Emacs already shows the effects of many years of the bad practice of
> integrating various new features specifically with Org, not with Emacs
> generally.  As a result, they are available only to those who use Org
> mode.  There are no efforts to generalize these features so they are
> not limited to Org.
>
> We must therefore insist that every new feature XYZ which CAN be
> modularly separate from Org, be implemented as modularly separate from
> Org.  People who want to use XYZ and Org together should be able to do
> so -- but people who want to use the one without the other should also
> be able to do so.
>
> In other words, if XYZ doesn't inherently _have_ to be limited to Org,
> it must not be limited to Org.
>
> Please join me in insisting on this modularity.

I have no issue with a push towards better support of Emacs-wide
facilities. However, I strongly oppose the idea to reject ELPA
submissions for packages that utilize the existing Org mode interfaces.

Rejecting ELPA is not a solution - if implementing non-Org interface is
hard, this will not encourage package authors to do the work, but
instead push them towards other package repositories like MELPA.
A side effect of more MELPA submissions will be fewer authors with
copyright assignment - this is a co-lateral damage.

> In other words, if XYZ doesn't inherently _have_ to be limited to Org,
> it must not be limited to Org.

This statement only makes sense for the Org mode features that should
better be a part of Emacs proper. Rather than prohibiting ELPA packages
from using these features, please encourage contributors to work towards
extracting them into Emacs code. This will be more constructive.

Do not place barriers, encourage improvements.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [ELPA] New package: jami-bot and org-jami-bot

[Philip Kaludercic]

Stefan Kangas <stefankangas@gmail.com> writes:

> Hanno Perrey <hanno@hoowl.se> writes:
>
>> I would like to propose two tightly-coupled packages for inclusion into
>> ELPA: jami-bot and org-jami-bot.
>>
>> jami-bot is an extendable chat bot for the distributed, private
>> messenger Jami. It interacts with the locally-installed Jami daemon via
>> D-Bus and reacts to both plain text messages and file transfers sent to
>> local accounts.
>>
>> org-jami-bot builds upon jami-bot and extends it with Org mode capture
>> functionality for text messages and images. It allows to schedule agenda
>> items at specific dates, compose multi-measure captures and capture
>> images -- all by sending a message via the GNU Jami messenger.
>>
>> I would like to keep maintaining both packages at their current
>> repositories:
>>
>> - https://gitlab.com/hperrey/jami-bot
>> - https://gitlab.com/hperrey/org-jami-bot
>>
>> I have signed the copyright paperwork with the FSF already and changed
>> the copyright in the respective files for both packages.
>>
>> As this is the first time I am proposing a package for ELPA, any
>> guidance on the process and feedback would be highly welcome!
>
> Philip, could you please help add these packages to GNU ELPA?
>
> The copyright assignment is all taken care of.

Sure, but first Hanno could you bump the version header, so that the
ELPA build system uses a newer commit, that includes the stylistic
changes + the updated copyright line?

A few comments from my side:

- It seems like a few variable (`jami-bot-account-user-names'?) would be
  nice to have as proper user options, defined with `defcustom' and a
  type.

- Try to sharp-quote (#') function symbols.

- If possible, try to join file paths using functions like
  `expand-file-name' with `default-directory' set, instead of using
  "low-level" string-manipulation.

- There seem to be some indentation issues (jami-bot-send-message), try
  running `indent-region' on the entire buffer.

- There are a few minor checkdoc issues, consider resolving them.

Re: [ELPA] New package: jami-bot and org-jami-bot

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > >> org-jami-bot builds upon jami-bot and extends it with Org mode capture
  > >> functionality for text messages and images. It allows to schedule agenda
  > >> items at specific dates, compose multi-measure captures and capture
  > >> images -- all by sending a message via the GNU Jami messenger.

When I read that, and looking at it again now. I understand this as
saying that this package is a feature for certain kinds of
communication from Emacs via Jami.  That seems to be the job that the
package does.

By contrast, the connection with Org mode seems to be a secondary
matter -- that this feature interfaces to Emacs: only via Org mode.
At least, that is what these words imply to me.

If that is true, there is a problem here: a feature which could (and
should) be more general has been implemented in a limited way -- to
use the Jami capture feature, you are compelled to do it via Org mode.

It would not be the first time.  There are various other features that
were integrated into Org mode and there is no way to use them from
Emacs except via Org mode.  It is a general design flaw, taking Emacs
overall in a bad direction.

Stefan Kangas said I was mistaken, that this package is not really an
instance of that problem.  Did I get the wrong idea of the package
from the words I cited above?  He said that the Org part was
"trivial".  Maybe my understanding of the description is mistaken.

But it isn't clear yet, because "trivial" can have various gradations.
I could be mistaken, or Stefan and I could be miscommunicating, or
both.  I had better make sure of how things really are.

The right way to implement Jami capture for Emacs is to integrate it
in a general way, which does not depend on Org mode.  Then, if you
like, add another layer to access Jami capture conveniently from Org
mode.  Doing a good job of this means offering the same festures at
the same levels, with or without Org mode,

If this is in fact what has been done, that's fine.  I will be glad to
find that the problem I thought I saw does not exist.  But I should
make sure of this by looking at the code.

Yesterday evening I asked someone to send me or point me at the source
code for org-jami-bot.  (At that time I collected the incoming emails
that arrived yesterday.)  If someone has sent me that info since
yesterday evening, thank you, and I will see that response soon.

Otherwise, would someone please send me the source of org-jami-bot?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] New package: jami-bot and org-jami-bot

[Stefan Kangas]

Richard Stallman <rms@gnu.org> writes:

> It would not be the first time.  There are various other features that
> were integrated into Org mode and there is no way to use them from
> Emacs except via Org mode.  It is a general design flaw, taking Emacs
> overall in a bad direction.

Sure, but you might be preaching to the choir here.  There is already
broad agreement on this point.  The Org maintainers themselves proposed
some features (org-protocol and org-capture) that are good candidates to
separate from Org in this very thread.

AFAIK, the mistakes that have been made in this regard are historical,
and the only reason we haven't rectified them yet is that no one has
stepped up to do the work.

It might make sense to add this to etc/TODO.

> But it isn't clear yet, because "trivial" can have various gradations.
> I could be mistaken, or Stefan and I could be miscommunicating, or
> both.  I had better make sure of how things really are.

I never used the word "trivial" in this thread, I don't think.

This is what I said:

    "Without the org parts, there's nothing there.  To change it 'not to
    require org' amounts to writing another package from scratch."

> Yesterday evening I asked someone to send me or point me at the source
> code for org-jami-bot.  (At that time I collected the incoming emails
> that arrived yesterday.)  If someone has sent me that info since
> yesterday evening, thank you, and I will see that response soon.

I sent it to you on Tue, 09 Jan 2024 21:35:20.  I'm assuming that you
will see that email before this one, but I'm saying this so that others
don't need to worry about sending it also.

Re: [ELPA] New package: jami-bot and org-jami-bot

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Thu, 11 Jan 2024 12:12:20 -0800
> Cc: hanno@hoowl.se, emacs-devel@gnu.org
> 
> Richard Stallman <rms@gnu.org> writes:
> 
> > It would not be the first time.  There are various other features that
> > were integrated into Org mode and there is no way to use them from
> > Emacs except via Org mode.  It is a general design flaw, taking Emacs
> > overall in a bad direction.
> 
> Sure, but you might be preaching to the choir here.  There is already
> broad agreement on this point.  The Org maintainers themselves proposed
> some features (org-protocol and org-capture) that are good candidates to
> separate from Org in this very thread.

What about ox-* packages?  Could they be converted to more
generally-useful export capabilities for Emacs?

In general, I feel that the various useful capabilities in Org were
generalized only to the level of Org, but not beyond that.  Look at
ob-* or oc-* or ol-* -- these could be the basis of Emacs features
useful in other places, but they are written in Org-specific ways that
make using them elsewhere difficult and sometimes even nonsensical.

I think patches to bring at least some of that into Emacs will be very
welcome, in the same way we nowadays use some originally Gnus-only
features in Emacs outside of Gnus (the most prominent example is
message.el, but there are others).

Re: [ELPA] New package: jami-bot and org-jami-bot

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> ...  The Org maintainers themselves proposed
>> some features (org-protocol and org-capture) that are good candidates to
>> separate from Org in this very thread.
>
> What about ox-* packages?  Could they be converted to more
> generally-useful export capabilities for Emacs?

Kind of. ox-* packages mostly rely on ox.el capabilities + Org mode
parsed representation. ox.el is also relying on the parsed
representation, except a few places.

If someone writes a parser that can understand alternative, non-Org
markup of source files, and produce org-element-ast.el-compatible list
structure, most of the ox-* exporters should remain functional.

> In general, I feel that the various useful capabilities in Org were
> generalized only to the level of Org, but not beyond that.  Look at
> ob-* or oc-* or ol-* -- these could be the basis of Emacs features
> useful in other places, but they are written in Org-specific ways that
> make using them elsewhere difficult and sometimes even nonsensical.
>
> I think patches to bring at least some of that into Emacs will be very
> welcome, in the same way we nowadays use some originally Gnus-only
> features in Emacs outside of Gnus (the most prominent example is
> message.el, but there are others).

oc-* is a part of the ox-* family.

ob-* for code evaluation libraries are not very modular currently - they
use a lot of ad-hoc regexp matching that is very difficult to
generalize. Much more difficult compared to exporter libraries
(although, not impossible).

If someone is up to extracting things from Org mode, and choosing
between babel and export parts, I recommend trying export first.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [ELPA] New package: jami-bot and org-jami-bot

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: Stefan Kangas <stefankangas@gmail.com>, rms@gnu.org, philipk@posteo.net,
>  hanno@hoowl.se, emacs-devel@gnu.org
> Date: Fri, 12 Jan 2024 12:38:07 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> ...  The Org maintainers themselves proposed
> >> some features (org-protocol and org-capture) that are good candidates to
> >> separate from Org in this very thread.
> >
> > What about ox-* packages?  Could they be converted to more
> > generally-useful export capabilities for Emacs?
> 
> Kind of. ox-* packages mostly rely on ox.el capabilities + Org mode
> parsed representation. ox.el is also relying on the parsed
> representation, except a few places.

Yes, I know.  And that is the problem I'm pointing to: export
capabilities should ideally not depend on a specific parser, let alone
an Org-specific one.  Because export capabilities can be beneficial in
much more places in Emacs than just Org buffers.

> If someone writes a parser that can understand alternative, non-Org
> markup of source files, and produce org-element-ast.el-compatible list
> structure, most of the ox-* exporters should remain functional.

It would be better to allow alternative data structures to be input to
the export functionality, rather than expect other features to mimic
the Org parser and its outputs.

Re: [ELPA] New package: jami-bot and org-jami-bot

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> If someone writes a parser that can understand alternative, non-Org
>> markup of source files, and produce org-element-ast.el-compatible list
>> structure, most of the ox-* exporters should remain functional.
>
> It would be better to allow alternative data structures to be input to
> the export functionality, rather than expect other features to mimic
> the Org parser and its outputs.

One way or another, there should be some structure to the parser output.
Org parser output is just a nested list.
org-element-ast.el defines the details of what can be inside that list -
independent of the details of Org markup, and compatible with the
existing ox-* implementations.

Moving away from the nested list is not feasible. Not unless you also
want to force all the third-party ox-* libraries to be rewritten or
remain incompatible.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [ELPA] New package: jami-bot and org-jami-bot

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: stefankangas@gmail.com, rms@gnu.org, philipk@posteo.net, hanno@hoowl.se,
>  emacs-devel@gnu.org
> Date: Fri, 12 Jan 2024 14:13:50 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> If someone writes a parser that can understand alternative, non-Org
> >> markup of source files, and produce org-element-ast.el-compatible list
> >> structure, most of the ox-* exporters should remain functional.
> >
> > It would be better to allow alternative data structures to be input to
> > the export functionality, rather than expect other features to mimic
> > the Org parser and its outputs.
> 
> One way or another, there should be some structure to the parser output.
> Org parser output is just a nested list.
> org-element-ast.el defines the details of what can be inside that list -
> independent of the details of Org markup, and compatible with the
> existing ox-* implementations.

I don't see org-element-ast.el in the Emacs tree, so I cannot see what
kind of structures it defines.

Re: [ELPA] New package: jami-bot and org-jami-bot

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> One way or another, there should be some structure to the parser output.
>> Org parser output is just a nested list.
>> org-element-ast.el defines the details of what can be inside that list -
>> independent of the details of Org markup, and compatible with the
>> existing ox-* implementations.
>
> I don't see org-element-ast.el in the Emacs tree, so I cannot see what
> kind of structures it defines.

Check out top comment in
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/lisp/org-element-ast.el

It is a part of the Org mode development branch.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [ELPA] New package: jami-bot and org-jami-bot

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: stefankangas@gmail.com, rms@gnu.org, philipk@posteo.net, hanno@hoowl.se,
>  emacs-devel@gnu.org
> Date: Fri, 12 Jan 2024 14:45:44 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> One way or another, there should be some structure to the parser output.
> >> Org parser output is just a nested list.
> >> org-element-ast.el defines the details of what can be inside that list -
> >> independent of the details of Org markup, and compatible with the
> >> existing ox-* implementations.
> >
> > I don't see org-element-ast.el in the Emacs tree, so I cannot see what
> > kind of structures it defines.
> 
> Check out top comment in
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/lisp/org-element-ast.el
> 
> It is a part of the Org mode development branch.

It's a beginning, but I somehow doubt that any list that matches the
description in org-element-ast.el will be automatically exportable by
ox-*.el.

Re: [ELPA] New package: jami-bot and org-jami-bot

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> Check out top comment in
>> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/lisp/org-element-ast.el
>> 
>> It is a part of the Org mode development branch.
>
> It's a beginning, but I somehow doubt that any list that matches the
> description in org-element-ast.el will be automatically exportable by
> ox-*.el.

It will, as long as ox-*.el library implements transcoders for the
node types present in a given AST and the AST match some expectations
about node properties. For example, it is common for export backends to
expect that "headline" nodes have :title property containing the
headline title text.

I am sceptical about generalizing the requirement to have properties
like :title present. Although, most of the other Org-specific properties
are optional.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [ELPA] New package: jami-bot and org-jami-bot

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I sent it to you on Tue, 09 Jan 2024 21:35:20.  I'm assuming that you
  > will see that email before this one, but I'm saying this so that others
  > don't need to worry about sending it also.

Actually no, your message was among the 200 or so that got backlogged
this week.  Your reminder was helpful since it told me to go look for
that one.

Thanks.
-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] New package: jami-bot and org-jami-bot

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I never used the word "trivial" in this thread, I don't think.

  > This is what I said:

  >     "Without the org parts, there's nothing there.

I apologize for misquoting you.  I didn't realize I was doing so.
(I still don't entirely understand the difference.)

Maybe you're saying that it is that the package's Org-related parts
are spread all through it?

I will look at the code today and I suppose I will then get the point.

                                                        To change it 'not to
  >     require org' amounts to writing another package from scratch."

Maybe that's what's needed, then.  The crucial thing is that this
feature "Jami capture" not be limited to Org mode.  But any reasonable way
of doing it is ok -- that level is not what the issue is about.

  > > It would not be the first time.  There are various other features that
  > > were integrated into Org mode and there is no way to use them from
  > > Emacs except via Org mode.  It is a general design flaw, taking Emacs
  > > overall in a bad direction.

  > Sure, but you might be preaching to the choir here.  There is already
  > broad agreement on this point.

That sounds like great news.  I have not seen any of it yet, so I hope
I will see it in last week's backlog of mail.

I spent many hours writing the code for cond*, which is mostly
working.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] New package: jami-bot and org-jami-bot

[Stefan Kangas]

Richard Stallman <rms@gnu.org> writes:

>                                                         To change it 'not to
>   >     require org' amounts to writing another package from scratch."
>
> Maybe that's what's needed, then.  The crucial thing is that this
> feature "Jami capture" not be limited to Org mode.  But any reasonable way
> of doing it is ok -- that level is not what the issue is about.

We can't start blocking Org specific features until they are rewritten
to work without Org too, because that puts an undue burden on
contributors.  People are scratching their own itches first and
foremost.  Note that we don't put up any such roadblocks in other cases.

It is fine to suggest it, of course.  But we can't force people.
Some might consider this a moral issue, but it is actually even more so
a practical one: they will simply go away (for example to MELPA).

When discussing with package authors how their package might better fit
into the broader Emacs environment, we should prefer nudging people in
the right direction, and be careful to inspire rather than demotivate.
I'm concerned that this subthread is not doing that, since it started
out with the proposal that the package shouldn't even be accepted.

What's more, the only thing that stands out about this particular
package is that its core functionality is already separated out into
`jami-bot', which others can use to build things for their own favorite
non-Org modes.

Re: [ELPA] New package: jami-bot and org-jami-bot

[Emanuel Berg]

Stefan Kangas wrote:

>> Maybe that's what's needed, then. The crucial thing is that
>> this feature "Jami capture" not be limited to Org mode.
>> But any reasonable way of doing it is ok -- that level is
>> not what the issue is about.
>
> We can't start blocking Org specific features until they are
> rewritten to work without Org too, because that puts an
> undue burden on contributors. People are scratching their
> own itches first and foremost. Note that we don't put up any
> such roadblocks in other cases.

1) The Org people should re-write everything useful they have
   so that is available to us without Org;

2) after that, hey, why do we even need Org for?

-- 
underground experts united
https://dataswamp.org/~incal

Re: [ELPA] New package: jami-bot and org-jami-bot

[Ihor Radchenko]

Emanuel Berg <incal@dataswamp.org> writes:

> 1) The Org people should re-write everything useful they have
>    so that is available to us without Org;
>
> 2) after that, hey, why do we even need Org for?

This sounds like reductio ad absurdum. Nobody is suggesting to "re-write
everything useful". What is being suggested is to re-write parts that
can be useful in more general scope than just Org mode. Such parts are
not "everything". Far from it.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [ELPA] New package: jami-bot and org-jami-bot

[Emanuel Berg]

Stefan Kangas wrote:

>> Maybe that's what's needed, then. The crucial thing is that
>> this feature "Jami capture" not be limited to Org mode.
>> But any reasonable way of doing it is ok -- that level is
>> not what the issue is about.
>
> We can't start blocking Org specific features until they are
> rewritten to work without Org too, because that puts an
> undue burden on contributors. People are scratching their
> own itches first and foremost.

Their own itches - in this case, the contents of lisp/org
which is shipped with core Emacs and contains 127 .el files
all with a familiar comment:

;; This file is part of GNU Emacs.

-- 
underground experts united
https://dataswamp.org/~incal

Re: [ELPA] New package: jami-bot and org-jami-bot

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

When a feature is implemented to depend unnecessarily on Org mode,
assuming it is ok in all other respects, we must say to the
contributors, "We like this feature, but before we install it we want
you to fix this problem: avoidable dependence on Org mode.  Keep the
functionality of working with Org mode, but make the same features
work also eithout Org mode."

We already take this stance regarding various other kinds of flaws.
The usual result, when we do that, is that the flaws are fixed.  But
at least it avoids piling up increasing numbers of flaws.

We must do the same thing when a package has the flaw of avoidable
dependence on Org mode.

  > When discussing with package authors how their package might better fit
  > into the broader Emacs environment, we should prefer nudging people in
  > the right direction, and be careful to inspire rather than demotivate.

I agree.  In our communication with the contributors, we should
encourage them and even offer them help in fixing the flaw.  We should
explain what to do in a clear way, and show examples.  I hope that
jami-bot will soon be an example we can show of the right way to do
things.

But that doesn't mean we should install the package with a flaw that
might exist uncorrected for many years.




-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] New package: jami-bot and org-jami-bot

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

When I lookd at the code for org-jami-bot, I saw that it consisted of
various jami-bot operations together with calls to org-capture.  It is
not very complex.  If I understood the calling conventions for
jami-bot and the calling conventions for org-capture, rather than
guessing them, I might be able to quickly develop something comparable
to org-jami-bot which does not depend on Org.

But I don't know them, and practically speaking I would need to study
a lot of code before I could possibly writ such a thing.
Someone who doesn't want to adopt Org and simply wants a way to set
up responses to Jami messages would have to give up.

The code of org-jami-bot is very clean, and I presume jami-bot is too.
But they do have a problem of a different kind: a gap.  Using Org for
this feature should be optional, not practically required.

I observed another gap: org-jami-bot does not contain documentation
that answers the basic questions, "What does this do, and how do you use it?"
Its user options and function arguments were carefully documented in detail;
what was missing was the overall framework to understand it all.

Would people please fill these gaps before instelling the code in GNU
ELPA?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] New package: jami-bot and org-jami-bot

[Hanno Perrey]

[-- Attachment #1: Type: text/plain, Size: 3958 bytes --]


Hej,

> If I understood the calling conventions for
> jami-bot and the calling conventions for org-capture, rather than
> guessing them, I might be able to quickly develop something comparable
> to org-jami-bot which does not depend on Org.

I am really glad for your interest in jami-bot and happy to help you
with the above!

> I observed another gap: org-jami-bot does not contain documentation
> that answers the basic questions, "What does this do, and how do you use it?"
> Its user options and function arguments were carefully documented in detail;
> what was missing was the overall framework to understand it all.

You have a point in that the documentation has been quite sparse in the
code. I have started to close that gap and extended the documentation in
both jami-bot and org-jami-bot. The latter refers to the former where
appropriate.

It might also be useful to summarize the usage of org-jami-bot here with
a focus to clarify its function to any non-Org mode users:

As a user, the setup of org-jami-bot is done by calling
`org-jami-bot-default-setup' which is now highlighted more explicitly in
the commentary to the package. This function does the following:

- set up a Org mode capture template. Think of this as a combined format
  string and file name to save anything "captured".

- register a hook each to `jami-bot-text-message-functions' and
  `jami-bot-data-transfer-functions'. Both use the capture template to
  save any plain text message and data transfer, respectively, to the
  default notes file. Both message types are essentially just appended
  (under their own Org mode header as the template adds a leading '*'),
  but file transfers have an automatically generated header and include
  a link to the file already downloaded by jami-bot as well as some
  additional meta information such as the timestamp.

- add four entries to `jami-bot-command-function-alist'. jami-bot
  handles plain text messages that start with an exclamation mark
  followed by a single word as a "command" which is mapped to a function
  by said alist. The function then can parse the message or perform any
  other action. The return string of the function is sent as a reply to
  the original message. So this registers four new "commands" in
  jami-bot that are for further, more refined captures (explained below).

The four commands are as follows:

- "!today" and "!schedule" (`org-jami-bot--command-function-today' and
  `org-jami-bot--command-function-schedule', respectively) simply add
  today's date or a date specified in the message, respectively, to the
  Org mode header together with the "SCHEDULED:" keyword. Within Org
  mode, this makes this particular entry appear e.g. in time-based
  searches or the Org mode agenda which summarizes TODO entries for each
  day.

- "!start" and "!done" (`org-jami-bot--command-function-start' and
  `org-jami-bot--command-function-done', respectively) are used to
  redirect captures to a temporary buffer created when sending "!start"
  and finally captured/saved when sending "!done". The idea is that one
  might want to have several messages appended to a single Org mode
  header. So this is only necessary as each message by default (due to
  the capture template format) is added under its own header.

For a user that does not want to modify this behavior, only the call to
`org-jami-bot-default-setup' is needed, everything else is called via
jami-bot when sending messages.

A version without Org mode would only need to adjust the two hooks for
plain text and file transfer handling and store the messages in some
other format to a file. Whether or not the commands introduced by
org-jami-bot make sense in that context depends on the chosen format I
guess.


I attach the new versions of jami-bot and org-jami-bot with the extended
documentation. Please let me know whether or not these close any gap for
you and -- if not -- what to improve.


Thanks and cheers,
Hanno


[-- Attachment #2: jami-bot.el --]
[-- Type: application/emacs-lisp, Size: 15618 bytes --]

[-- Attachment #3: org-jami-bot.el --]
[-- Type: application/emacs-lisp, Size: 15236 bytes --]

Re: [ELPA] New package: jami-bot and org-jami-bot

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I read through the code of org-jami-bot.el, not studying in detail
since it calls lots of functions I don't know the definitions of.  I
see it consists mostly of interaction with jami-bot and other things
related to Jami, plus calls to org-capture.

I don't know what org-capture does, but I can guess it is responsible
for sending the Jami data to Org mode.

I get the feeling that the Org interaction is now more modular, and
that it would be easy to make a modified versio which calls some
general hooks.  One version of those hooks would be for Org mode.
This is exactly the sort of separation I have asked for.

WDYT?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] New package: jami-bot and org-jami-bot

[Hanno Perrey]

Hej,

I appreciate the discussion on generalizing features offered by Org mode
that my request triggered. However, I am not really in a position to
comment on this -- instead, I would like to clarify some aspects of the
two *jami-bot packages without commenting on the broader direction of
Emacs development.

>   > >> org-jami-bot builds upon jami-bot and extends it with Org mode capture
>   > >> functionality for text messages and images. It allows to schedule agenda
>   > >> items at specific dates, compose multi-measure captures and capture
>   > >> images -- all by sending a message via the GNU Jami messenger.
>
> When I read that, and looking at it again now. I understand this as
> saying that this package is a feature for certain kinds of
> communication from Emacs via Jami.  That seems to be the job that the
> package does.
>
> By contrast, the connection with Org mode seems to be a secondary
> matter -- that this feature interfaces to Emacs: only via Org mode.
> At least, that is what these words imply to me.

I would say that this is a mostly correct interpretation. However, the
package description might not be technically accurate enough for this
discussion: jami-bot deals with all the actual text message and
file/image handling. The latter, for example, will be automatically
downloaded into a customizable directory. The former are parsed for
command strings and functions called to handle these (via abnormal
hooks). Similarly, downloaded files can be further processed by adding
hooks.

This is what org-jami-bot does: it adds hooks to jami-bot for Org
mode-specific features, such as using the capture mechanism. For files,
for example, an Org mode heading is created via the capture mechanism
and a Org mode-style link to the downloaded file is added.

jami-bot therefore offers more of a framework to trigger actions via
Jami messages but it only implements a basic feature set. org-jami-bot,
on the other hand, only adds hooks for a couple of Org mode features but
otherwise relies on the handling provided by jami-bot. It does not
really create any unique features itself.

> If that is true, there is a problem here: a feature which could (and
> should) be more general has been implemented in a limited way -- to
> use the Jami capture feature, you are compelled to do it via Org mode.

While this is technically true, the capture functionality is only a hook
added to jami-bot. It would be very easy to "capture" messages into
plain text files or other formats with only a few lines of code and only
requiring jami-bot.

> Stefan Kangas said I was mistaken, that this package is not really an
> instance of that problem.  Did I get the wrong idea of the package
> from the words I cited above?  He said that the Org part was
> "trivial".  Maybe my understanding of the description is mistaken.

I would not call org-jami-bot "trivial" but it is very light-weight as
all the heavy lifting is done either in jami-bot or (maybe more so) Org
mode.

> The right way to implement Jami capture for Emacs is to integrate it
> in a general way, which does not depend on Org mode.  Then, if you
> like, add another layer to access Jami capture conveniently from Org
> mode.  Doing a good job of this means offering the same festures at
> the same levels, with or without Org mode,

This is exactly what I intended to do with jami-bot and org-jami-bot.
However, with the latter I have not implemented any actual capture
feature, only the "glue" to Org mode's.

I also want to add that note capture is only one option for interacting
with Emacs via Jami and jami-bot; one could use as well have e.g. a chat
with doctor or process messages via calc or anything like that. Only
org-jami-bot has a note-taking focus due to the Org mode features being
used.


Thanks and cheers,
Hanno

Re: [ELPA] New package: jami-bot and org-jami-bot

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > While this is technically true, the capture functionality is only a hook
  > added to jami-bot. It would be very easy to "capture" messages into
  > plain text files or other formats with only a few lines of code and only
  > requiring jami-bot.

This might be "very easy" for someone who understands jami-bot, but it
is NOT as easy for a USER as simply using org-jami-bot.  Even if the
code to do so is rather simple, there is a big practical diffeence for
a user between "Just load this file" and "You need to write 50 lines
of code."

I'd like it to be _exactly_ very easy for a non-programmer.

Can you do this easy job, so that using jami-bot without Org is 100% as
easy as using it with Org?

  > However, with the latter I have not implemented any actual capture
  > feature, only the "glue" to Org mode's.

I don't understand this part at all.  What is a "capture" feature?
What does it do?  What does org-capture do?

I have never used Org mode.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] New package: jami-bot and org-jami-bot

[Hanno Perrey]

Hej,

>   > While this is technically true, the capture functionality is only a hook
>   > added to jami-bot. It would be very easy to "capture" messages into
>   > plain text files or other formats with only a few lines of code and only
>   > requiring jami-bot.
>
> This might be "very easy" for someone who understands jami-bot, but it
> is NOT as easy for a USER as simply using org-jami-bot.  Even if the
> code to do so is rather simple, there is a big practical diffeence for
> a user between "Just load this file" and "You need to write 50 lines
> of code."

I completely get your point and I think we are talking about different
things. I was thinking of developers or power-users, who I think are the
main target user-base for jami-bot as it does not come with much
built-in functionality for the user. It is more of a library to make
automatic messaging through Jami easier. User features would be added by
packages building on top of it, such as org-jami-bot.

While I stand by my point that implementing a specific use case is
straightforward (I should not have said "very easy") for someone
familiar with Elisp, creating a package that replicates Org mode
features but without Org mode and is simple to use and configure *is*
work.

> Can you do this easy job, so that using jami-bot without Org is 100% as
> easy as using it with Org?

No. Currently, I simply do not have the spare time.

What is more, I am a heavy Org mode user and do not have a clear
understanding of the use case without Org mode. Who are the users and
what are their needs? What should the format be? What needs to be
configurable?

If anyone is interested in working on such a package or even just
scratching their own itch and creating a hook for a specific use case,
then I am more than happy to advice and help out.

Alternatively, if a generic capture framework for Emacs is created, I am
happy to target that instead of Org mode capture.

>   > However, with the latter I have not implemented any actual capture
>   > feature, only the "glue" to Org mode's.
>
> I don't understand this part at all.  What is a "capture" feature?
> What does it do?  What does org-capture do?

"capturing" refers to the process of storing new information in your
collection of notes. org-capture does this very quickly and
unobtrusively. It is highly configurable and a framework that I use for
a lot of different knowledge-storing actions, from todos to
bibliographic references. For me, it is the natural entry point for new
information as it is automatically stored where I can find it again.

> I have never used Org mode.

And I am similarly unfamiliar with your workflow and needs for storing
notes (this is what we talk about here as org-jami-bot does just that
through org-capture). By targeting org-capture, I had hoped to create a
tool that would be useful for anyone who is familiar with its mechanism.
I understand that this alienates anyone who does not use Org mode but to
the best of my knowledge, there is currently no more generic way in
Emacs.

Best wishes,
Hanno

Re: [ELPA] New package: jami-bot and org-jami-bot

[Hanno Perrey]

Hej,

>> Philip, could you please help add these packages to GNU ELPA?
>
> Sure, but first Hanno could you bump the version header, so that the
> ELPA build system uses a newer commit, that includes the stylistic
> changes + the updated copyright line?
>
> A few comments from my side:
> [..]

thanks for the comments! I hope I will be able to address them all over
the weekend.

My computer access is very limited right now, however, and answers/code
fixes from my side are much slower than I would like them to be.

Thanks and cheers,
Hanno

Re: [ELPA] New package: jami-bot and org-jami-bot

[Hanno Perrey]

Hej Philip,

I have now modified the code according to your suggestions and pushed
the changes. Some more detailed responses follow below:

> Sure, but first Hanno could you bump the version header, so that the
> ELPA build system uses a newer commit, that includes the stylistic
> changes + the updated copyright line?

Done!

> - It seems like a few variable (`jami-bot-account-user-names'?) would be
>   nice to have as proper user options, defined with `defcustom' and a
>   type.

Good idea! I have implemented that now and the types pass the
compilation, so I hope I did that correctly in all instances.

> - Try to sharp-quote (#') function symbols.

This could just be my limited Elisp-foo, but I could not find any
further places in the code where sharp-quotes seem to be appropriate. If
this was not meant as a general comment but aimed at a specific bit of
code, I would really appreciate a pointer!

> - If possible, try to join file paths using functions like
>   `expand-file-name' with `default-directory' set, instead of using
>   "low-level" string-manipulation.

I looked through the code and could not find an instance where that
wasn't already the case. Do you remember where you saw string
manipulation being used for path construction? My eyes could simply be
too tired to spot that...

> - There seem to be some indentation issues (jami-bot-send-message), try
>   running `indent-region' on the entire buffer.

Good catch, fixed!

> - There are a few minor checkdoc issues, consider resolving them.

Also resolved!

Thanks again for the feedback!

Best wishes,
Hanno

Re: [ELPA] New package: jami-bot and org-jami-bot

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 297 bytes --]

Hanno Perrey <hanno@hoowl.se> writes:

> Hej Philip,
>
> I have now modified the code according to your suggestions and pushed
> the changes. 

[...]


The sharp-quoting was my mistake, I misread the hooks as functions.
Other than that, here is a diff with the remaining changes I would
propose:


[-- Attachment #2: Type: text/plain, Size: 2710 bytes --]

diff --git a/jami-bot.el b/jami-bot.el
index 8ad3ed0..5ea9aad 100644
--- a/jami-bot.el
+++ b/jami-bot.el
@@ -1,6 +1,6 @@
 ;;; jami-bot.el --- An extendable chat bot for the private messenger GNU Jami -*- lexical-binding: t; -*-
 
-;; Copyright (C) 2023 Free Software Foundation, Inc.
+;; Copyright (C) 2023, 2024 Free Software Foundation, Inc.
 
 ;; Author: Hanno Perrey <http://gitlab.com/hperrey>
 ;; Maintainer: Hanno Perrey <hanno@hoowl.se>
@@ -73,14 +73,14 @@ consist of a single (lowercase) word.  The corresponding function needs to
 accept the account id, the conversation id and the message alist as
 arguments and return a string (that is sent as reply to the original message)."
   :group 'jami-bot
-  :type '(alist :key-type 'string :value-type 'function))
+  :type '(alist :key-type string :value-type function))
 
 (defcustom jami-bot-text-message-functions nil
   "A list of functions that will be called when processing a plain text message.
 Functions must take the ACCOUNT and CONVERSATION ids as well as
 the actual MSG as arguments.  Their return value will be ignored."
   :group 'jami-bot
-  :type '(group 'function))
+  :type '(group function))
 
 (defcustom jami-bot-download-path "~/jami/"
   "Path in which to store files downloaded from conversations.
@@ -95,9 +95,9 @@ Functions must take the ACCOUNT and CONVERSATION ids as well as
 the actual MSG and the local downloaded file name, DLNAME, as
 arguments.  Their return value will be ignored."
   :group 'jami-bot
-  :type '(group 'function))
+  :type '(group function))
 
-;; Internal variables
+;;;; Internal variables
 
 (defvar jami-bot--jami-local-account-ids nil
   "List of `jami' local accounts user ids and name pairs.
@@ -105,7 +105,7 @@ arguments.  Their return value will be ignored."
 Caches output of dbus-methods \"getAccountList\" and
 \"getAccountDetails\". For internal use in `jami-bot'.")
 
-;; Functions
+;;;; Functions
 
 (defun jami-bot--messageReceived-handler (account conversation msg)
   "Handle messages from Jami's `messageReceived' D-Bus signal.
@@ -254,7 +254,9 @@ corresponding ids to which the message MSG belongs to."
          (filename (cadr (assoc-string "displayName" msg)))
          (dlpath (file-name-as-directory
                   (expand-file-name jami-bot-download-path)))
-         (dlname (concat dlpath (format-time-string "%Y%m%d-%H%M") "_" filename)))
+         (dlname (expand-file-name
+		  (concat (format-time-string "%Y%m%d-%H%M") "_" filename)
+		  dlpath)))
     (unless (file-directory-p dlpath) (make-directory dlpath 't))
     (message "jami-bot: downloading file %s" dlname)
     (jami-bot--dbus-cfgmgr-call-method "downloadFile" account conversation id fileid dlname)

[-- Attachment #3: Type: text/plain, Size: 60 bytes --]



> Thanks again for the feedback!
>
> Best wishes,
> Hanno

Re: [ELPA] New package: jami-bot and org-jami-bot

[Hanno Perrey]

Dear Philip,

> Other than that, here is a diff with the remaining changes I would
> propose:

applied and pushed, thanks for the detailed comb-through! :)

Best wishes,

Hanno

Re: [ELPA] New package: jami-bot and org-jami-bot

[Philip Kaludercic]

Hanno Perrey <hanno@hoowl.se> writes:

> Dear Philip,
>
>> Other than that, here is a diff with the remaining changes I would
>> propose:
>
> applied and pushed, thanks for the detailed comb-through! :)

OK, looks like it works now, so I have pushed the package specification.
One more thing I noticed is that you have this in your package header,

;; Author: Hanno Perrey <http://gitlab.com/hperrey>

which is not a valid Email address.  The idea is to have an email
address here, that is used to generate a compose buffer when the user is
reading about the package from a package description buffer.

> Best wishes,
>
> Hanno

Re: [ELPA] New package: jami-bot and org-jami-bot

[Hanno Perrey]

Hej Philip,

> OK, looks like it works now, so I have pushed the package specification.
> One more thing I noticed is that you have this in your package header,
>
> ;; Author: Hanno Perrey <http://gitlab.com/hperrey>
>
> which is not a valid Email address.  The idea is to have an email
> address here, that is used to generate a compose buffer when the user is
> reading about the package from a package description buffer.

thanks, fixed that in both the packages and my yas-snippet for new
packages!

Best wishes,
Hanno