bug#45435: Additional libraries required by transient and magit manuals

bug#45435: Additional libraries required by transient and magit manuals

[Jonas Bernoulli]

I wrote magit's info manual using an org file before org-mode itself
switched the input format of its own manual from texinfo to org.  At
the time it was necessary to improve upon "ox-texinfo.el" to make it
suitable for a large manual.

While that may not be the cases anymore, I have kept using my extended
exporter implemented in "ox-texinfo+.el" and would like to keep doing
so.

I am guessing that [non]gnu elpa currently use the version of org that
comes with Emacs.  It would be nice if we could also use the libraries
in org's "contrib/lisp/" directory.

Transient's manual needs "ox-extra.el" library from that directory.
Magit's additionally needs "org-man.el".

"ox-texinfo+.el" is not part of org's "contrib/lisp/" directory and
would have to be added separately to the machines that build the gnu
and nongnu and elpas.

I would need it for the following three features:

;; 1. Create `@deffn' and similar definition items by writing list
;;    items in Org that look similar to what they will look like in
;;    Info.  To enable this, add:
;;
;;      #+TEXINFO_DEFFN: t
;;
;;    to your Org file.  After doing that, you can create definition
;;    items like so:
;;
;;      - Command: magit-section-show
;;
;;        Show the body of the current section.
;;
;;      - Function: magit-git-exit-code &rest args
;;      - Macro: magit-insert-section &rest args
;;      - Variable: magit-display-buffer-noselect
;;      - User Option: magit-display-buffer-function
;;      - Key: q, magit-mode-bury-buffer

;; 2. Optionally share a section's node with some or all of its child
;;    sections.  By default every section on every level gets its own
;;    node, and `ox-texinfo' provides no mechanism for changing that.
;;    To place a section in the same node as its parent section, do
;;    this:
;;
;;      **** Log Performance
;;      :PROPERTIES:
;;      :NONODE: t
;;      :END:

;; 3. Optionally modify the Org file before exporting it.  This is
;;    implemented using a hook that can be set using the `BIND'
;;    property:
;;
;;      #+BIND: ox-texinfo+-before-export-hook some-function
;;      #+BIND: ox-texinfo+-before-export-hook another-function
;;
;;    The function `ox-texinfo+-update-version-strings' is provided
;;    as an example.  It makes some assumptions that might not be
;;    appropriate for your manuals, so you might have to define your
;;    own variant.

(The third feature can also be invoked using a make target and this needs
some love to work properly with "elpa-admin.el", but that's a detail.)

You can use git or a browser to get "ox-texinfo+.el" from
https://github.com/tarsius/ox-texinfo-plus.

Please consider making "org/contrib/lisp/" and "ox-texinfo+.el"
available to the elpas.

     Jonas

bug#45435: Additional libraries required by transient and magit manuals

[Stefan Monnier]

> I am guessing that [non]gnu elpa currently use the version of org that
> comes with Emacs.

Indeed, more specifically with the Emacs distributed in Debian stable,
i.e. Emacs-26 currently.

> Please consider making "org/contrib/lisp/" and "ox-texinfo+.el"
> available to the elpas.

IIUC Bastien is working on (or planning to soon work on) adding org-contrib
to NonGNU ELPA.  As for `ox-texinfo+.el` (or any other package you
fancy), feel free to add them to `elpa.git` or `nongnu.git` (depending
on their copyright paperwork status, mostly).

But the main point you raise is the use of extra packages when building
(Non)GNU ELPA packages, such as for the needs of building the
Info manual.

There are mostly two issues:

1- The philosophical issue of relying on packages which we don't distribute.
   I think we should try and only use ELisp packages which we
   distribute, either as part of Emacs or GNU ELPA or NonGNU ELPA.
   But this should be easy to fix: just add the package to
   (Non)GNU ELPA.

2- Making use of those extra packages while building your own (Non)GNU
   ELPA package.  This is a technical issue and I'm not completely sure
   how best to solve it.

I think point 2 is the only relevant problem here, so I suggest we focus
on this in the bug#45435.

Currently, when building a GNU ELPA package, the `:make` rule has read
access to the whole of `elpa.git`, and similarly while building a NonGNU
ELPA package, the `:make` rule has read access to the whole of
`nongnu.git`.

There are several problem, tho:
1- GNU ELPA Packages aren't readable while building NonGNU ELPA
   packages, and vice-versa.
2- While there is read access to the source code of other packages,
   these aren't "prepared" to be activated (as by
   `package-activate-all`), e.g. their [PKG]-pkg.el and more importantly
   [PKG]-autoloads.el files haven't been built (and they haven't been
   byte-compiled either).
3- Of course, the code available is (usually) that of the head of their
   respective branch, which may be in a temporarily broken state.

So maybe rather than look for the solution by re-using the code we
already have lying around, we should "manually" add the handful of extra
packages to the builder's `~/.emacs.d/elpa` ?

The downside would be that it requires a manual step from someone with
access to `elpa.gnu.org`.  Or maybe we could keep the contents of that
`~/.emacs.d/elpa` in a separate branch/directory and just make it
available to `:make` targets, so anyone with write access to the Git
repository can add (installed) packages in there.

Hmm...


        Stefan

bug#45435: Additional libraries required by transient and magit manuals

[Jonas Bernoulli]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> So maybe rather than look for the solution by re-using the code we
> already have lying around, we should "manually" add the handful of extra
> packages to the builder's `~/.emacs.d/elpa` ?

> The downside would be that it requires a manual step from someone with
> access to `elpa.gnu.org`.

That's what I had in mind and if it were you who kept that up-to-date,
as opposed to some fsf admin, then that could work.

> Or maybe we could keep the contents of that
> `~/.emacs.d/elpa` in a separate branch/directory and just make it
> available to `:make` targets, so anyone with write access to the Git
> repository can add (installed) packages in there.

That is probably the better approach.

Would you want check a copy of these libraries into the "main" branch?
We do that for "package-build.el" in the Melpa repository like so:

pull-package-build:
	git subtree pull --squash -P package-build package-build master

I am not really a fan of that approach but it has the benefit that it is
robust.

Alternatively you could generalize the code that makes "elpa-admin.el"
available.  "./admin" could then serve a similar purpose as "./package",
i.e. "./admin/elpa-admin" would become the worktree that checks out the
"elpa-admin" branch.

Maybe that branch should be renamed to "admin/elpa-admin" and other
admin tools could use "admin/<name>" as well.  Gnu elpa would end up
with two identical branches for "ox-texinfo+": "externals/ox-texinfo+"
and "admin/ox-texinfo+", nongnu elpa however would only feature the
latter.

     Jonas

bug#45435: Additional libraries required by transient and magit manuals

[Stefan Monnier]

>> So maybe rather than look for the solution by re-using the code we
>> already have lying around, we should "manually" add the handful of extra
>> packages to the builder's `~/.emacs.d/elpa` ?
>> The downside would be that it requires a manual step from someone with
>> access to `elpa.gnu.org`.
> That's what I had in mind and if it were you who kept that up-to-date,
> as opposed to some fsf admin, then that could work.

I'd most likely be the one keeping it up-to-date (the fsf admins have
better things to do than try and understand what we've setup inside the
VM ;-).

And yes, that would work but I'd rather not be so "indispensable".
Also it'd make it yet harder to approximate on your own machine what
happens on `elpa.gnu.org`.

>> Or maybe we could keep the contents of that
>> `~/.emacs.d/elpa` in a separate branch/directory and just make it
>> available to `:make` targets, so anyone with write access to the Git
>> repository can add (installed) packages in there.
> That is probably the better approach.

Yes, the more I think about it, the more I'm convinced that it's
obviously superior.

> Would you want check a copy of these libraries into the "main" branch?

I'm thinking of keeping it in a separate branch that can be shared
between `elpa.git` and `nongnu.git`.

> Maybe that branch should be renamed to "admin/elpa-admin" and other
> admin tools could use "admin/<name>" as well.  Gnu elpa would end up
> with two identical branches for "ox-texinfo+": "externals/ox-texinfo+"
> and "admin/ox-texinfo+", nongnu elpa however would only feature the
> latter.

I'm rather thinking that the new branch would keep the equivalent of
`~/.emacs.d/elpa`, i.e. code that's in an "installed" state, e.g.
byte compiled, so not "two identical branches".


        Stefan