[bug#57280] [PATCH 0/3] Add documentation-files argument to emacs build system.

[bug#57280] [PATCH 0/3] Add documentation-files argument to emacs build system.

[Andrew Tropin]

[-- Attachment #1.1: Type: text/plain, Size: 527 bytes --]


This patch adds a handy way for generating info documentation for emacs
packages from texinfo or org files.

Andrew Tropin (3):
  build-system: emacs: Add documentation-files argument.
  gnu: emacs-orderless: Use documentation-files argument.
  gnu: emacs-consult: Use documentation-files argument.

 gnu/packages/emacs-xyz.scm        | 11 +++--------
 guix/build-system/emacs.scm       | 11 +++++++++++
 guix/build/emacs-build-system.scm | 17 +++++++++++++++++
 3 files changed, 31 insertions(+), 8 deletions(-)

-- 
2.37.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-build-system-emacs-Add-documentation-files-argument.patch --]
[-- Type: text/x-patch, Size: 4079 bytes --]

From 74b671b94d16db2f21c1df02672fef0b5228a08a Mon Sep 17 00:00:00 2001
From: Andrew Tropin <andrew@trop.in>
Date: Thu, 18 Aug 2022 17:43:14 +0300
Subject: [PATCH 1/3] build-system: emacs: Add documentation-files argument.

Allows to build info files from texinfo or org.

* guix/build-system/emacs.scm (default-texinfo): New variable.
* guix/build-system/emacs.scm (lower): New arguments.
* guix/build/emacs-build-system.scm (generate-docs): New variable.
---
 guix/build-system/emacs.scm       | 11 +++++++++++
 guix/build/emacs-build-system.scm | 17 +++++++++++++++++
 2 files changed, 28 insertions(+)

diff --git a/guix/build-system/emacs.scm b/guix/build-system/emacs.scm
index 3df68789ff..632ba2ddb3 100644
--- a/guix/build-system/emacs.scm
+++ b/guix/build-system/emacs.scm
@@ -56,8 +56,16 @@ (define (default-emacs)
   (let ((emacs-mod (resolve-interface '(gnu packages emacs))))
     (module-ref emacs-mod 'emacs-minimal)))
 
+(define (default-texinfo)
+  "Return the default texinfo package."
+  ;; Lazily resolve the binding to avoid a circular dependency.
+  (let ((texinfo-mod (resolve-interface '(gnu packages texinfo))))
+    (module-ref texinfo-mod 'texinfo)))
+
 (define* (lower name
                 #:key source inputs native-inputs outputs system target
+                documentation-files
+                (texinfo (default-texinfo))
                 (emacs (default-emacs))
                 #:allow-other-keys
                 #:rest arguments)
@@ -77,6 +85,7 @@ (define private-keywords
                         ;; Keep the standard inputs of 'gnu-build-system'.
                         ,@(standard-packages)))
          (build-inputs `(("emacs" ,emacs)
+                         ,@(if (null? documentation-files) '() `(("texinfo" ,texinfo)))
                          ,@native-inputs))
          (outputs outputs)
          (build emacs-build)
@@ -87,6 +96,7 @@ (define* (emacs-build name inputs
                       (tests? #f)
                       (parallel-tests? #t)
                       (test-command ''("make" "check"))
+                      (documentation-files ''())
                       (phases '%standard-phases)
                       (outputs '("out"))
                       (include (quote %default-include))
@@ -109,6 +119,7 @@ (define builder
                        #:test-command #$test-command
                        #:tests? #$tests?
                        #:parallel-tests? #$parallel-tests?
+                       #:documentation-files #$documentation-files
                        #:phases #$phases
                        #:outputs #$(outputs->gexp outputs)
                        #:include #$include
diff --git a/guix/build/emacs-build-system.scm b/guix/build/emacs-build-system.scm
index 6a6918bfdd..08c61ddfd8 100644
--- a/guix/build/emacs-build-system.scm
+++ b/guix/build/emacs-build-system.scm
@@ -274,6 +274,22 @@ (define (match-stripped-file action regex)
                            (install-file? file stat #:verbose? #t)))
       #f))))
 
+(define* (generate-docs #:key outputs documentation-files #:allow-other-keys)
+  "Convert texinfo or org files specified in DOCUMENTATION-FILES argument to
+info files."
+  (map
+   (lambda (path)
+     (if (or (string-suffix? ".texi" path)
+             (string-suffix? ".texinfo" path)
+             (string-suffix? ".txi" path))
+         (invoke "makeinfo" path)
+         (emacs-batch-script ; else org file
+          `(progn
+            (require 'ox-texinfo)
+            (find-file ,path)
+            (org-texinfo-export-to-info)))))
+   documentation-files))
+
 (define* (move-doc #:key outputs #:allow-other-keys)
   "Move info files from the ELPA package directory to the info directory."
   (let* ((out (assoc-ref outputs "out"))
@@ -343,6 +359,7 @@ (define %standard-phases
   (modify-phases gnu:%standard-phases
     (replace 'unpack unpack)
     (add-after 'unpack 'expand-load-path expand-load-path)
+    (add-after 'expand-load-path 'generate-docs generate-docs)
     (delete 'bootstrap)
     (delete 'configure)
     (delete 'build)
-- 
2.37.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.3: 0002-gnu-emacs-orderless-Use-documentation-files-argument.patch --]
[-- Type: text/x-patch, Size: 1283 bytes --]

From 6c8fb7d173c24ef6c00aca5a7697cf14d1353f37 Mon Sep 17 00:00:00 2001
From: Andrew Tropin <andrew@trop.in>
Date: Thu, 18 Aug 2022 17:49:18 +0300
Subject: [PATCH 2/3] gnu: emacs-orderless: Use documentation-files argument.

* gnu/packages/emacs-xyz.scm (emacs-orderless): Use documentation-files argument.
---
 gnu/packages/emacs-xyz.scm | 9 +--------
 1 file changed, 1 insertion(+), 8 deletions(-)

diff --git a/gnu/packages/emacs-xyz.scm b/gnu/packages/emacs-xyz.scm
index 811e293c1d..f3d515b3c6 100644
--- a/gnu/packages/emacs-xyz.scm
+++ b/gnu/packages/emacs-xyz.scm
@@ -9045,14 +9045,7 @@ (define-public emacs-orderless
        (file-name (git-file-name name version))))
     (build-system emacs-build-system)
     (arguments
-     `(#:phases
-       (modify-phases %standard-phases
-         (add-after 'install 'makeinfo
-           (lambda* (#:key outputs #:allow-other-keys)
-             (invoke "makeinfo" "orderless.texi")
-             (install-file "orderless.info"
-                           (string-append (assoc-ref outputs "out")
-                                          "/share/info")))))))
+     (list #:documentation-files #~'("orderless.texi")))
     (native-inputs
      (list texinfo))
     (home-page "https://github.com/oantolin/orderless")
-- 
2.37.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.4: 0003-gnu-emacs-consult-Use-documentation-files-argument.patch --]
[-- Type: text/x-patch, Size: 985 bytes --]

From d3ad4d4446ba4275bec5f9ed2aaa7e74289727f2 Mon Sep 17 00:00:00 2001
From: Andrew Tropin <andrew@trop.in>
Date: Thu, 18 Aug 2022 17:50:00 +0300
Subject: [PATCH 3/3] gnu: emacs-consult: Use documentation-files argument.

* gnu/packages/emacs-xyz.scm (emacs-consult): Use documentation-files argument.
---
 gnu/packages/emacs-xyz.scm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/gnu/packages/emacs-xyz.scm b/gnu/packages/emacs-xyz.scm
index f3d515b3c6..cab1ad9dee 100644
--- a/gnu/packages/emacs-xyz.scm
+++ b/gnu/packages/emacs-xyz.scm
@@ -9071,6 +9071,8 @@ (define-public emacs-consult
         (base32 "0sy4rn1vjk1g50r8z14hzj8lds6s7ij2zkjqfi6mfash5il75wnq"))
        (file-name (git-file-name name version))))
     (build-system emacs-build-system)
+    (arguments
+     (list #:documentation-files #~'("README.org")))
     (propagated-inputs (list emacs-compat))
     (home-page "https://github.com/minad/consult")
     (synopsis "Consulting completing-read")
-- 
2.37.1


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[bug#57280] [PATCH 0/3] Add documentation-files argument to emacs build system.

[Liliana Marie Prikler]

Hi Andrew,

again for the mailing list.

Am Donnerstag, dem 18.08.2022 um 17:50 +0300 schrieb Andrew Tropin:
> 
> This patch adds a handy way for generating info documentation for
> emacs packages from texinfo or org files.
> 
> Andrew Tropin (3):
>   build-system: emacs: Add documentation-files argument.
>   gnu: emacs-orderless: Use documentation-files argument.
>   gnu: emacs-consult: Use documentation-files argument.
Is it just those two packages that require this phase?  If so, what
value is there in making it a "standard" phase?

> +(define (default-texinfo)
> +  "Return the default texinfo package."
> +  ;; Lazily resolve the binding to avoid a circular dependency.
> +  (let ((texinfo-mod (resolve-interface '(gnu packages texinfo))))
> +    (module-ref texinfo-mod 'texinfo)))
> +
>  (define* (lower name
>                  #:key source inputs native-inputs outputs system
> target
> +                documentation-files
I don't think hard-coding this list is useful.  Instead, it would be
nice if we simply used find-files with the right pattern, and use a
binary switch as in meson-build-systems #:glib-or-gtk?
> +                (texinfo (default-texinfo))
>                  (emacs (default-emacs))
>                  #:allow-other-keys
>                  #:rest arguments)
> @@ -77,6 +85,7 @@ (define private-keywords
>                          ;; Keep the standard inputs of 'gnu-build-
> system'.
>                          ,@(standard-packages)))
>           (build-inputs `(("emacs" ,emacs)
> +                         ,@(if (null? documentation-files) '()
> `(("texinfo" ,texinfo)))
>                           ,@native-inputs))
We should probably append rather than prepend implicit inputs.  In
fact, doing so for emacs itself also means that people could prepend
their own emacs if emacs-minimal is not enough rather than needing a
transformer.

> +(define* (generate-docs #:key outputs documentation-files #:allow-
> other-keys)
> +  "Convert texinfo or org files specified in DOCUMENTATION-FILES
> argument to
> +info files."
> +  (map
> +   (lambda (path)
> +     (if (or (string-suffix? ".texi" path)
> +             (string-suffix? ".texinfo" path)
> +             (string-suffix? ".txi" path))
> +         (invoke "makeinfo" path)
> +         (emacs-batch-script ; else org file
> +          `(progn
> +            (require 'ox-texinfo)
> +            (find-file ,path)
> +            (org-texinfo-export-to-info)))))
> +   documentation-files))
(ice-9 match) is your friend.

Cheers

[bug#57280] [PATCH 0/3] Add documentation-files argument to emacs build system.

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 3665 bytes --]

On 2022-08-18 20:31, Liliana Marie Prikler wrote:

> Hi Andrew,
>
> if this ought to have went to a mailing list, it didn't.

Yep, I missed To:, resent it yesterday.

>
> Am Donnerstag, dem 18.08.2022 um 17:50 +0300 schrieb Andrew Tropin:
>> 
>> This patch adds a handy way for generating info documentation for
>> emacs packages from texinfo or org files.
>> 
>> Andrew Tropin (3):
>>   build-system: emacs: Add documentation-files argument.
>>   gnu: emacs-orderless: Use documentation-files argument.
>>   gnu: emacs-consult: Use documentation-files argument.
> Is it just those two packages that require this phase?  If so, what
> value is there in making it a "standard" phase?
>

It's just two examples, I think there is much more packages.

>> +(define (default-texinfo)
>> +  "Return the default texinfo package."
>> +  ;; Lazily resolve the binding to avoid a circular dependency.
>> +  (let ((texinfo-mod (resolve-interface '(gnu packages texinfo))))
>> +    (module-ref texinfo-mod 'texinfo)))
>> +
>>  (define* (lower name
>>                  #:key source inputs native-inputs outputs system
>> target
>> +                documentation-files
> I don't think hard-coding this list is useful.  Instead, it would be
> nice if we simply used find-files with the right pattern, and use a
> binary switch as in meson-build-systems #:glib-or-gtk?

It's not clear how to find a documentation file heuristically, it can be
README, DOCUMENTATION, README.org, docs/MANUAL.org docs/PACKAGE.texi or
anything else, morevover a few of them can be present at the same time
and I'm afraid it will be a very tough task to understand which of them
to use.

The idea is inspired by :doc keyword from elpa and the fact that some of
emacs-xyz packages either miss documentation or have custom build phases
for it:
https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/elpa-packages#n781

>> +                (texinfo (default-texinfo))
>>                  (emacs (default-emacs))
>>                  #:allow-other-keys
>>                  #:rest arguments)
>> @@ -77,6 +85,7 @@ (define private-keywords
>>                          ;; Keep the standard inputs of 'gnu-build-
>> system'.
>>                          ,@(standard-packages)))
>>           (build-inputs `(("emacs" ,emacs)
>> +                         ,@(if (null? documentation-files) '()
>> `(("texinfo" ,texinfo)))
>>                           ,@native-inputs))
> We should probably append rather than prepend implicit inputs.  In
> fact, doing so for emacs itself also means that people could prepend
> their own emacs if emacs-minimal is not enough rather than needing a
> transformer.
>

I thought #:emacs and #:texinfo arguments are enough to specify custom
emacs/texinfo inputs.

>> +(define* (generate-docs #:key outputs documentation-files #:allow-
>> other-keys)
>> +  "Convert texinfo or org files specified in DOCUMENTATION-FILES
>> argument to
>> +info files."
>> +  (map
>> +   (lambda (path)
>> +     (if (or (string-suffix? ".texi" path)
>> +             (string-suffix? ".texinfo" path)
>> +             (string-suffix? ".txi" path))
>> +         (invoke "makeinfo" path)
>> +         (emacs-batch-script ; else org file
>> +          `(progn
>> +            (require 'ox-texinfo)
>> +            (find-file ,path)
>> +            (org-texinfo-export-to-info)))))
>> +   documentation-files))
> (ice-9 match) is your friend.

That's right, I thought about it when was writting this code :)  Will
wait for a few more comments and will refactor in the next revision.

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[bug#57280] [PATCH 0/3] Add documentation-files argument to emacs build system.

[Liliana Marie Prikler]

Am Freitag, dem 19.08.2022 um 06:33 +0300 schrieb Andrew Tropin:
> On 2022-08-18 20:31, Liliana Marie Prikler wrote:
> [...]
> > Am Donnerstag, dem 18.08.2022 um 17:50 +0300 schrieb Andrew Tropin:
> > 
> [...]
> > > +(define (default-texinfo)
> > > +  "Return the default texinfo package."
> > > +  ;; Lazily resolve the binding to avoid a circular dependency.
> > > +  (let ((texinfo-mod (resolve-interface '(gnu packages texinfo))))
> > > +    (module-ref texinfo-mod 'texinfo)))
> > > +
> > >  (define* (lower name
> > >                  #:key source inputs native-inputs outputs system
> > > target
> > > +                documentation-files
> > I don't think hard-coding this list is useful.  Instead, it would be
> > nice if we simply used find-files with the right pattern, and use a
> > binary switch as in meson-build-systems #:glib-or-gtk?
> 
> It's not clear how to find a documentation file heuristically, it can
> be README, DOCUMENTATION, README.org, docs/MANUAL.org docs/PACKAGE.texi
> or anything else, morevover a few of them can be present at the same
> time and I'm afraid it will be a very tough task to understand which of
> them to use.
I think it's possible to cover most of those with heuristics.  For the
rest, we can still override the phase or just rename the file to
something our heuristics handle.

> > 
> > > +                (texinfo (default-texinfo))
> > >                  (emacs (default-emacs))
> > >                  #:allow-other-keys
> > >                  #:rest arguments)
> > > @@ -77,6 +85,7 @@ (define private-keywords
> > >                          ;; Keep the standard inputs of 'gnu-
> > > build-
> > > system'.
> > >                          ,@(standard-packages)))
> > >           (build-inputs `(("emacs" ,emacs)
> > > +                         ,@(if (null? documentation-files) '()
> > > `(("texinfo" ,texinfo)))
> > >                           ,@native-inputs))
> > We should probably append rather than prepend implicit inputs.
> > In fact, doing so for emacs itself also means that people could
> > prepend their own emacs if emacs-minimal is not enough rather than
> > needing a transformer.
> > 
> 
> I thought #:emacs and #:texinfo arguments are enough to specify
> custom emacs/texinfo inputs.
And what if any of the documentations needs emacs-org rather than the
org included by emacs-minimal?  Spamming keywords is not helpful.

Cheers
> > 

[bug#57280] [PATCH 0/3] Add documentation-files argument to emacs build system.

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 3085 bytes --]

On 2022-08-19 06:19, Liliana Marie Prikler wrote:

> Am Freitag, dem 19.08.2022 um 06:33 +0300 schrieb Andrew Tropin:
>> On 2022-08-18 20:31, Liliana Marie Prikler wrote:
>> [...]
>> > Am Donnerstag, dem 18.08.2022 um 17:50 +0300 schrieb Andrew Tropin:
>> > 
>> [...]
>> > > +(define (default-texinfo)
>> > > +  "Return the default texinfo package."
>> > > +  ;; Lazily resolve the binding to avoid a circular dependency.
>> > > +  (let ((texinfo-mod (resolve-interface '(gnu packages texinfo))))
>> > > +    (module-ref texinfo-mod 'texinfo)))
>> > > +
>> > >  (define* (lower name
>> > >                  #:key source inputs native-inputs outputs system
>> > > target
>> > > +                documentation-files
>> > I don't think hard-coding this list is useful.  Instead, it would be
>> > nice if we simply used find-files with the right pattern, and use a
>> > binary switch as in meson-build-systems #:glib-or-gtk?
>> 
>> It's not clear how to find a documentation file heuristically, it can
>> be README, DOCUMENTATION, README.org, docs/MANUAL.org docs/PACKAGE.texi
>> or anything else, morevover a few of them can be present at the same
>> time and I'm afraid it will be a very tough task to understand which of
>> them to use.
> I think it's possible to cover most of those with heuristics.  For the
> rest, we can still override the phase or just rename the file to
> something our heuristics handle.
>

If there is an info file(s) do nothing.
If there are texinfo file build them.
If there are no texinfo files build README.org or README.

Something like that?

Will play around with it a little bit and will publish v2 next week.

>> > 
>> > > +                (texinfo (default-texinfo))
>> > >                  (emacs (default-emacs))
>> > >                  #:allow-other-keys
>> > >                  #:rest arguments)
>> > > @@ -77,6 +85,7 @@ (define private-keywords
>> > >                          ;; Keep the standard inputs of 'gnu-
>> > > build-
>> > > system'.
>> > >                          ,@(standard-packages)))
>> > >           (build-inputs `(("emacs" ,emacs)
>> > > +                         ,@(if (null? documentation-files) '()
>> > > `(("texinfo" ,texinfo)))
>> > >                           ,@native-inputs))
>> > We should probably append rather than prepend implicit inputs.
>> > In fact, doing so for emacs itself also means that people could
>> > prepend their own emacs if emacs-minimal is not enough rather than
>> > needing a transformer.
>> > 
>> 
>> I thought #:emacs and #:texinfo arguments are enough to specify
>> custom emacs/texinfo inputs.
> And what if any of the documentations needs emacs-org rather than the
> org included by emacs-minimal?  Spamming keywords is not helpful.

This is a good point.  I'll reorder inputs.

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[bug#57280] [PATCH 0/3] Add documentation-files argument to emacs build system.

[Liliana Marie Prikler]

Am Freitag, dem 19.08.2022 um 09:21 +0300 schrieb Andrew Tropin:
> On 2022-08-19 06:19, Liliana Marie Prikler wrote:
> > I think it's possible to cover most of those with heuristics.  For
> > the rest, we can still override the phase or just rename the file
> > to something our heuristics handle.
> > 
> 
> If there is an info file(s) do nothing.
> If there are texinfo file build them.
> If there are no texinfo files build README.org or README.
> 
> Something like that?
> 
> Will play around with it a little bit and will publish v2 next week.
I'd word those in terms of for-each, i.e. "build all texinfo files and
org-mode files".  Don't trust already compiled sources, i.e. if there's
both README.info and README.org, you still want to generate README.info
from README.org (though "README" doesn't sound like a particular good
heuristic for an org-file to makeinfo from).

> > > 
Cheers

[bug#57280] [PATCH 0/3] Add documentation-files argument to emacs build system.

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 2047 bytes --]

On 2022-08-19 17:39, Liliana Marie Prikler wrote:

> Am Freitag, dem 19.08.2022 um 09:21 +0300 schrieb Andrew Tropin:
>> On 2022-08-19 06:19, Liliana Marie Prikler wrote:
>> > I think it's possible to cover most of those with heuristics.  For
>> > the rest, we can still override the phase or just rename the file
>> > to something our heuristics handle.
>> > 
>> 
>> If there is an info file(s) do nothing.
>> If there are texinfo file build them.
>> If there are no texinfo files build README.org or README.
>> 
>> Something like that?
>> 
>> Will play around with it a little bit and will publish v2 next week.
> I'd word those in terms of for-each, i.e. "build all texinfo files and
> org-mode files".  Don't trust already compiled sources, i.e. if there's
> both README.info and README.org, you still want to generate README.info
> from README.org (though "README" doesn't sound like a particular good
> heuristic for an org-file to makeinfo from).
>
>> > > 
> Cheers

I went through a few popular packages and came up with conclusion that
it's hard to make good heuristic for automatical documentation build:

1. I tried (find-files "." "\\.(texi|txi|texinfo)$") with consequent
for-each and it doesn't work in general case because it will build files
intended for inclusion, not standalone building.  And it's not fixable
with auxiliary build phase.  Examples: geiser, dash.  It seems that we
need to decide manually for each package, which documentation files to
build.

2. Adding automatic documentation build phase also means that almost all
emacs packages will be rebuild and we don't know what documentation will
be shipped (if it useful doc compiled from texinfo or almost empty
README.org).

It seems that manual approach is more precise, less intrusive and helps
to get rid of many custom and non-uniform documentation build phases.

I'll check a few more emacs packages I use and will send updated
implementation of #:documentation-files argument.

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[bug#57280] [PATCH 0/3] Add documentation-files argument to emacs build system.

[Liliana Marie Prikler]

Am Freitag, dem 26.08.2022 um 17:33 +0300 schrieb Andrew Tropin:

> > > > 
> > Cheers
> 
> I went through a few popular packages and came up with conclusion
> that it's hard to make good heuristic for automatical documentation
> build:
> 
> 1. I tried (find-files "." "\\.(texi|txi|texinfo)$") with consequent
> for-each and it doesn't work in general case because it will build
> files intended for inclusion, not standalone building.
Fair enough, there's probably similar issues with org etc.  That said,
wouldn't the top-level info/org/whatever file share the package name?

> 2. Adding automatic documentation build phase also means that almost
> all emacs packages will be rebuild
That's why I'm currently delaying native-comp until all other changes
to emacs-build-system are done.

> It seems that manual approach is more precise, less intrusive and
> helps to get rid of many custom and non-uniform documentation build
> phases.
If you're going for a "manual" approach, I'd suggest instead making a
curried ((build-documentation #:texinfo-files #:texinfo-regexp ...)
#:outputs ...) so that the files can be written directly into the (add-
after ...) syntax.

Cheers

[bug#57280] [PATCH 0/3] Add documentation-files argument to emacs build system.

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 2554 bytes --]

On 2022-08-29 18:38, Liliana Marie Prikler wrote:

> Am Freitag, dem 26.08.2022 um 17:33 +0300 schrieb Andrew Tropin:
>
>> > > > 
>> > Cheers
>> 
>> I went through a few popular packages and came up with conclusion
>> that it's hard to make good heuristic for automatical documentation
>> build:
>> 
>> 1. I tried (find-files "." "\\.(texi|txi|texinfo)$") with consequent
>> for-each and it doesn't work in general case because it will build
>> files intended for inclusion, not standalone building.
> Fair enough, there's probably similar issues with org etc.  That said,
> wouldn't the top-level info/org/whatever file share the package name?
>

In many cases, yes it would, but not always.

magit: ("docs/magit.texi" "docs/magit-section.texi")
geiser: ("doc/geiser.texi")
geiser-guile: ("geiser-guile.texi")
dash: ("dash.texi")
orderless: ("orderless.texi")
consult/cape/tempel: ("README.org")
cider: ("doc/modules/ROOT/nav.adoc")
all-the-icons: ("README.md")
citar: ("README.org")
org-roam: ("doc/org-roam.texi")
debbugs: ("debbugs.texi" "debbugs-ug.texi")
modus-themes: ("doc/modus-themes.org")

>> 2. Adding automatic documentation build phase also means that almost
>> all emacs packages will be rebuild
> That's why I'm currently delaying native-comp until all other changes
> to emacs-build-system are done.
>
>> It seems that manual approach is more precise, less intrusive and
>> helps to get rid of many custom and non-uniform documentation build
>> phases.
> If you're going for a "manual" approach, I'd suggest instead making a
> curried ((build-documentation #:texinfo-files #:texinfo-regexp ...)
> #:outputs ...) so that the files can be written directly into the (add-
> after ...) syntax.

Do you mean to make a helper function, which can be used to generate a
closure of build phase, which can be added with replace/add-after?

Another idea is to make a separate functions for different documentation
type: build-{texinfo,markdown,org,etc}-documentation.  Also, it seems
useful outside of emacs-build-system as well.

In such case user will need to accomplish following steps: 1. add
texinfo/pandoc/emacs-org dependency 2. use modify-phases to add
(build-{texinfo,whatever}-documentation #:texinfo-files
'("doc/manual.{texi,whatever}")), seems a little less convenient than
simple #:documentation-files #~(list "manual.{texi,whatever}"), but also
work, at least documentation will be built more uniformly for different
packages.

WDYT?

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[bug#57280] [PATCH 0/3] Add documentation-files argument to emacs build system.

[Liliana Marie Prikler]

Am Dienstag, dem 30.08.2022 um 11:15 +0300 schrieb Andrew Tropin:
> On 2022-08-29 18:38, Liliana Marie Prikler wrote:
> 
> > Am Freitag, dem 26.08.2022 um 17:33 +0300 schrieb Andrew Tropin:
> > 
> > > > > > 
> > > > Cheers
> > > 
> > > I went through a few popular packages and came up with conclusion
> > > that it's hard to make good heuristic for automatical
> > > documentation
> > > build:
> > > 
> > > 1. I tried (find-files "." "\\.(texi|txi|texinfo)$") with
> > > consequent
> > > for-each and it doesn't work in general case because it will
> > > build
> > > files intended for inclusion, not standalone building.
> > Fair enough, there's probably similar issues with org etc.  That
> > said,
> > wouldn't the top-level info/org/whatever file share the package
> > name?
> > 
> 
> In many cases, yes it would, but not always.
> 
> magit: ("docs/magit.texi" "docs/magit-section.texi")
Is magit-section a top-level file?

> geiser: ("doc/geiser.texi")
I think trying docs?/whatever is good praxis, so I count that as a hit.

> geiser-guile: ("geiser-guile.texi")
Hit.

> dash: ("dash.texi")
Hit.

> orderless: ("orderless.texi")
Hit.

> consult/cape/tempel: ("README.org")
Hit for README.whatever

> cider: ("doc/modules/ROOT/nav.adoc")
Miss.

> all-the-icons: ("README.md")
Hit for README.whatever

> citar: ("README.org")
Hit for README.whatever

> org-roam: ("doc/org-roam.texi")
Hit.

> debbugs: ("debbugs.texi" "debbugs-ug.texi")
Is debbugs-ug a top-level file?

> modus-themes: ("doc/modus-themes.org")
Hit.

> > 
> > 
> > > It seems that manual approach is more precise, less intrusive and
> > > helps to get rid of many custom and non-uniform documentation
> > > build
> > > phases.
> > If you're going for a "manual" approach, I'd suggest instead making
> > a curried ((build-documentation #:texinfo-files #:texinfo-regexp
> > ...)
> > #:outputs ...) so that the files can be written directly into the
> > (add-after ...) syntax.
> 
> Do you mean to make a helper function, which can be used to generate
> a closure of build phase, which can be added with replace/add-after?
> 
> Another idea is to make a separate functions for different
> documentation
> type: build-{texinfo,markdown,org,etc}-documentation.  Also, it seems
> useful outside of emacs-build-system as well.
Hmm, if we wanted to make that even more generic than just emacs, it'd
go to core-updates.  

> In such case user will need to accomplish following steps: 1. add
> texinfo/pandoc/emacs-org dependency 2. use modify-phases to add
> (build-{texinfo,whatever}-documentation #:texinfo-files
> '("doc/manual.{texi,whatever}")), seems a little less convenient than
> simple #:documentation-files #~(list "manual.{texi,whatever}"), but
> also work, at least documentation will be built more uniformly for
> different packages.
> 
> WDYT?
I think if we want to go this more generic route, we'd have to redesign
this a little.  For instance, (build-texinfo-documentation) should take
regular expressions as remaining arguments.  As for the native-inputs
required, there has already been a precedent set with bash-minimal that
anything requiring extraneous inputs needs to declare them explicitly.

Cheers 

[bug#57280] [PATCH 0/3] Add documentation-files argument to emacs build system.

[Andrew Tropin]

[-- Attachment #1.1: Type: text/plain, Size: 4013 bytes --]

On 2022-08-30 10:28, Liliana Marie Prikler wrote:

> Am Dienstag, dem 30.08.2022 um 11:15 +0300 schrieb Andrew Tropin:
>> On 2022-08-29 18:38, Liliana Marie Prikler wrote:
>> 
>> > Am Freitag, dem 26.08.2022 um 17:33 +0300 schrieb Andrew Tropin:
>> > 
>> > > > > > 
>> > > > Cheers
>> > > 
>> > > I went through a few popular packages and came up with conclusion
>> > > that it's hard to make good heuristic for automatical
>> > > documentation
>> > > build:
>> > > 
>> > > 1. I tried (find-files "." "\\.(texi|txi|texinfo)$") with
>> > > consequent
>> > > for-each and it doesn't work in general case because it will
>> > > build
>> > > files intended for inclusion, not standalone building.
>> > Fair enough, there's probably similar issues with org etc.  That
>> > said,
>> > wouldn't the top-level info/org/whatever file share the package
>> > name?
>> > 
>> 
>> In many cases, yes it would, but not always.
>> 
>> magit: ("docs/magit.texi" "docs/magit-section.texi")
> Is magit-section a top-level file?

Yes.  In 3.3.0 it's Documentation/magit.texi and
Documentation/magit-section.texi, but in recent not yet released
version it's ("docs/magit.texi" "docs/magit-section.texi"), there are
also org counterparts of magit and magit-section files, but they are in
sync with texi files.

>
>> geiser: ("doc/geiser.texi")
> I think trying docs?/whatever is good praxis, so I count that as a hit.
>
>> geiser-guile: ("geiser-guile.texi")
> Hit.
>
>> dash: ("dash.texi")
> Hit.
>
>> orderless: ("orderless.texi")
> Hit.
>
>> consult/cape/tempel: ("README.org")
> Hit for README.whatever
>
>> cider: ("doc/modules/ROOT/nav.adoc")
> Miss.
>
>> all-the-icons: ("README.md")
> Hit for README.whatever
>
>> citar: ("README.org")
> Hit for README.whatever
>
>> org-roam: ("doc/org-roam.texi")
> Hit.
>
>> debbugs: ("debbugs.texi" "debbugs-ug.texi")
> Is debbugs-ug a top-level file?

Yes, the second one is User Guide.

>
>> modus-themes: ("doc/modus-themes.org")
> Hit.
>
>> > 
>> > 
>> > > It seems that manual approach is more precise, less intrusive and
>> > > helps to get rid of many custom and non-uniform documentation
>> > > build
>> > > phases.
>> > If you're going for a "manual" approach, I'd suggest instead making
>> > a curried ((build-documentation #:texinfo-files #:texinfo-regexp
>> > ...)
>> > #:outputs ...) so that the files can be written directly into the
>> > (add-after ...) syntax.
>> 
>> Do you mean to make a helper function, which can be used to generate
>> a closure of build phase, which can be added with replace/add-after?
>> 
>> Another idea is to make a separate functions for different
>> documentation
>> type: build-{texinfo,markdown,org,etc}-documentation.  Also, it seems
>> useful outside of emacs-build-system as well.
> Hmm, if we wanted to make that even more generic than just emacs, it'd
> go to core-updates.  
>
>> In such case user will need to accomplish following steps: 1. add
>> texinfo/pandoc/emacs-org dependency 2. use modify-phases to add
>> (build-{texinfo,whatever}-documentation #:texinfo-files
>> '("doc/manual.{texi,whatever}")), seems a little less convenient than
>> simple #:documentation-files #~(list "manual.{texi,whatever}"), but
>> also work, at least documentation will be built more uniformly for
>> different packages.
>> 
>> WDYT?
> I think if we want to go this more generic route, we'd have to redesign
> this a little.  For instance, (build-texinfo-documentation) should take
> regular expressions as remaining arguments.  

What can be a good place (module) for such build phases?

> As for the native-inputs required, there has already been a precedent
> set with bash-minimal that anything requiring extraneous inputs needs
> to declare them explicitly.

I think it will work, need to experiment with (build-*-documentation) to
get the feeling.


Attaching the latest version of the documentation-files patch I have:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: v2-0001-build-system-emacs-Add-documentation-files-argume.patch --]
[-- Type: text/x-patch, Size: 4749 bytes --]

From a1534b2158c97986e1048379661ee9d250ad6c02 Mon Sep 17 00:00:00 2001
From: Andrew Tropin <andrew@trop.in>
Date: Thu, 18 Aug 2022 17:43:14 +0300
Subject: [PATCH v2 1/2] build-system: emacs: Add documentation-files argument.

Allows to build info files from texinfo or org.

* guix/build-system/emacs.scm (default-texinfo): New variable.
* guix/build-system/emacs.scm (lower): New arguments.
* guix/build/emacs-build-system.scm (generate-docs): New variable.
---
 guix/build-system/emacs.scm       | 16 ++++++++++++++--
 guix/build/emacs-build-system.scm | 22 ++++++++++++++++++++++
 2 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/guix/build-system/emacs.scm b/guix/build-system/emacs.scm
index 3df68789ff..02379ee54c 100644
--- a/guix/build-system/emacs.scm
+++ b/guix/build-system/emacs.scm
@@ -56,8 +56,16 @@ (define (default-emacs)
   (let ((emacs-mod (resolve-interface '(gnu packages emacs))))
     (module-ref emacs-mod 'emacs-minimal)))
 
+(define (default-texinfo)
+  "Return the default texinfo package."
+  ;; Lazily resolve the binding to avoid a circular dependency.
+  (let ((texinfo-mod (resolve-interface '(gnu packages texinfo))))
+    (module-ref texinfo-mod 'texinfo)))
+
 (define* (lower name
                 #:key source inputs native-inputs outputs system target
+                documentation-files
+                (texinfo (default-texinfo))
                 (emacs (default-emacs))
                 #:allow-other-keys
                 #:rest arguments)
@@ -76,8 +84,10 @@ (define private-keywords
 
                         ;; Keep the standard inputs of 'gnu-build-system'.
                         ,@(standard-packages)))
-         (build-inputs `(("emacs" ,emacs)
-                         ,@native-inputs))
+         (build-inputs `(,@native-inputs
+                         ("emacs" ,emacs)
+                         ;; ,@(if (null? documentation-files) '() )
+                         ("texinfo" ,texinfo)))
          (outputs outputs)
          (build emacs-build)
          (arguments (strip-keyword-arguments private-keywords arguments)))))
@@ -87,6 +97,7 @@ (define* (emacs-build name inputs
                       (tests? #f)
                       (parallel-tests? #t)
                       (test-command ''("make" "check"))
+                      (documentation-files ''())
                       (phases '%standard-phases)
                       (outputs '("out"))
                       (include (quote %default-include))
@@ -109,6 +120,7 @@ (define builder
                        #:test-command #$test-command
                        #:tests? #$tests?
                        #:parallel-tests? #$parallel-tests?
+                       #:documentation-files #$documentation-files
                        #:phases #$phases
                        #:outputs #$(outputs->gexp outputs)
                        #:include #$include
diff --git a/guix/build/emacs-build-system.scm b/guix/build/emacs-build-system.scm
index 6a6918bfdd..3ffa196862 100644
--- a/guix/build/emacs-build-system.scm
+++ b/guix/build/emacs-build-system.scm
@@ -274,6 +274,27 @@ (define (match-stripped-file action regex)
                            (install-file? file stat #:verbose? #t)))
       #f))))
 
+(define* (generate-documentation
+          #:key outputs documentation-files
+          #:allow-other-keys)
+  "Convert texinfo or org files specified in DOCUMENTATION-FILES argument to
+info files.  And move info files site-lisp directory."
+  (for-each (lambda (f)
+              (if (regexp-exec
+                   (make-regexp "\\.(txi|texi|texinfo)" regexp/icase)
+                   f)
+                  (invoke "makeinfo" f)
+                  (emacs-batch-script ; else org file
+                   `(progn
+                     (require 'ox-texinfo)
+                     (find-file ,f)
+                     (org-texinfo-export-to-info)))))
+            documentation-files)
+  (for-each (lambda (f)
+              (install-file f (string-append (assoc-ref outputs "out")
+                                             %install-dir)))
+            (find-files "." "\\.info$")))
+
 (define* (move-doc #:key outputs #:allow-other-keys)
   "Move info files from the ELPA package directory to the info directory."
   (let* ((out (assoc-ref outputs "out"))
@@ -343,6 +364,7 @@ (define %standard-phases
   (modify-phases gnu:%standard-phases
     (replace 'unpack unpack)
     (add-after 'unpack 'expand-load-path expand-load-path)
+    (add-after 'expand-load-path 'generate-documentation generate-documentation)
     (delete 'bootstrap)
     (delete 'configure)
     (delete 'build)
-- 
2.37.2


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.3: v2-0002-gnu-emacs-xyz-Add-documentation-files-example-usa.patch --]
[-- Type: text/x-patch, Size: 4223 bytes --]

From 8adbef898ef80851753ba9d64b31eed727bb34de Mon Sep 17 00:00:00 2001
From: Andrew Tropin <andrew@trop.in>
Date: Wed, 31 Aug 2022 12:16:10 +0300
Subject: [PATCH v2 2/2] gnu: emacs-xyz: Add documentation-files example usage.

* gnu/packages/emacs-xyz.scm (emacs-geiser, emacs-geiser-guile, emacs-magit,
emacs-dash, emacs-consult, emacs-tempel): Add documentation-files example
usage.
---
 gnu/packages/emacs-xyz.scm | 26 ++++++++++----------------
 1 file changed, 10 insertions(+), 16 deletions(-)

diff --git a/gnu/packages/emacs-xyz.scm b/gnu/packages/emacs-xyz.scm
index 90ee485f1e..df0570a4a1 100644
--- a/gnu/packages/emacs-xyz.scm
+++ b/gnu/packages/emacs-xyz.scm
@@ -262,7 +262,8 @@ (define-public emacs-geiser
         (base32 "1pm33zlcq84h61xhplmrlicckrax1pv39zrmv8ryzhi9mqrb6bdg"))))
     (build-system emacs-build-system)
     (arguments
-     '(#:phases
+     '(#:documentation-files (list "doc/geiser.texi")
+       #:phases
        (modify-phases %standard-phases
          ;; Move the source files to the top level, which is included in
          ;; the EMACSLOADPATH.
@@ -271,12 +272,7 @@ (define-public emacs-geiser
              (let ((el-files (find-files "./elisp" ".*\\.el$")))
                (for-each (lambda (f)
                            (rename-file f (basename f)))
-                         el-files))))
-         (add-before 'install 'make-info
-           (lambda _
-             (with-directory-excursion "doc"
-               (invoke "makeinfo" "--no-split"
-                       "-o" "geiser.info" "geiser.texi")))))))
+                         el-files)))))))
     (native-inputs
      (list texinfo))
     (home-page "https://www.nongnu.org/geiser/")
@@ -311,6 +307,7 @@ (define-public emacs-geiser-guile
     (arguments
      (list
       #:include #~(cons "^src/" %default-include)
+      #:documentation-files #~(list "geiser-guile.texi")
       #:phases
       #~(modify-phases %standard-phases
           (add-after 'unpack 'patch-geiser-guile-binary
@@ -978,17 +975,11 @@ (define-public emacs-magit
       #:exclude #~(cons* "magit-libgit.el"
                          "magit-libgit-pkg.el"
                          %default-exclude)
+      #:documentation-files #~(list "Documentation/magit.texi"
+                                    "Documentation/magit-section.texi")
       #:phases
       #~(modify-phases %standard-phases
-          (add-after 'unpack 'build-info-manual
-            (lambda _
-              (invoke "make" "info")
-              ;; Copy info files to the lisp directory, which acts as
-              ;; the root of the project for the emacs-build-system.
-              (for-each (lambda (f)
-                          (install-file f "lisp"))
-                        (find-files "Documentation" "\\.info$"))))
-          (add-after 'build-info-manual 'set-magit-version
+          (add-after 'unpack 'set-magit-version
             (lambda _
               (make-file-writable "lisp/magit.el")
               (emacs-substitute-variables "lisp/magit.el"
@@ -3909,6 +3900,7 @@ (define-public emacs-dash
     (build-system emacs-build-system)
     (arguments
      (list #:tests? #t
+           #:documentation-files #~(list "dash.texi")
            #:phases
            #~(modify-phases %standard-phases
                (add-after 'unpack 'disable-byte-compile-error-on-warn
@@ -9188,6 +9180,7 @@ (define-public emacs-consult
        (sha256
         (base32 "0sy4rn1vjk1g50r8z14hzj8lds6s7ij2zkjqfi6mfash5il75wnq"))
        (file-name (git-file-name name version))))
+    (arguments (list #:documentation-files #~(list "README.org")))
     (build-system emacs-build-system)
     (propagated-inputs (list emacs-compat))
     (home-page "https://github.com/minad/consult")
@@ -14145,6 +14138,7 @@ (define-public emacs-tempel
                (base32
                 "0iyh6wxchqg83gpwvg6lz4qy4c2qh25iqjpjm56kif52346a99d2"))))
     (build-system emacs-build-system)
+    (arguments (list #:documentation-files #~(list "README.org")))
     (home-page "https://github.com/minad/tempel")
     (synopsis "Simple templates for Emacs")
     (description
-- 
2.37.2


[-- Attachment #1.4: Type: text/plain, Size: 37 bytes --]


-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[bug#57280] [PATCH 0/3] Add documentation-files argument to emacs build system.

[Liliana Marie Prikler]

Am Mittwoch, dem 31.08.2022 um 12:36 +0300 schrieb Andrew Tropin:
> > I think if we want to go this more generic route, we'd have to
> > redesign this a little.  For instance, (build-texinfo-
> > documentation) should take
> > regular expressions as remaining arguments.  
> 
> What can be a good place (module) for such build phases?
I was thinking (guix build utils).  Of course, one could introduce a
new module, but doing that would come with even more downsides in terms
of UX (or PX if we're pedantic).

> Attaching the latest version of the documentation-files patch I have
Looking at this patch, perhaps we'd also have to allow customizing
command line options.  Also, as for installing, I think this should be
handled by the install phase, which already has includes
"^[^/]*\\.info$" and "^doc/.*\\.info$" by default.  Thus, you only need
to build documentation before the install phase.

Cheers

[bug#57280] [PATCH 0/3] Add documentation-files argument to emacs build system.

[Andrew Tropin]

[-- Attachment #1.1: Type: text/plain, Size: 643 bytes --]

On 2022-08-31 12:07, Liliana Marie Prikler wrote:

> Am Mittwoch, dem 31.08.2022 um 12:36 +0300 schrieb Andrew Tropin:
>> > I think if we want to go this more generic route, we'd have to
>> > redesign this a little.  For instance, (build-texinfo-
>> > documentation) should take
>> > regular expressions as remaining arguments.  
>> 
>> What can be a good place (module) for such build phases?
> I was thinking (guix build utils).  Of course, one could introduce a
> new module, but doing that would come with even more downsides in terms
> of UX (or PX if we're pedantic).

Ok, I prepared more generic version of the patch.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: v3-0001-build-system-emacs-Add-build-documentation-phase.patch --]
[-- Type: text/x-patch, Size: 10640 bytes --]

From 4a706908491daafc0493ab15297665eb2b9fce4e Mon Sep 17 00:00:00 2001
From: Andrew Tropin <andrew@trop.in>
Date: Fri, 2 Sep 2022 08:23:18 +0300
Subject: [PATCH v3] build-system: emacs: Add build-documentation phase.

Allows to build info files from texinfo, org or other markup files.

* guix/build-system/emacs.scm (default-texinfo, default-pandoc): New variable.
(lower)[build-inputs]: Add texinfo and pandoc.
* guix/build/emacs-build-system.scm (build-documentantion): New variable.
* guix/build/emacs-utils.scm (build-documentantion-texinfo,
build-documentation-org, convert-documentation): New variable.
* gnu/packages/emacs-xyz.scm (emacs-orderless)[arguments]: Remove custom
documentation build phase.
(emacs-org)[arguments]: Remove build-documentation as it already builds
documentation with make.
---
 gnu/packages/emacs-xyz.scm        | 21 ++++++-------
 guix/build-system/emacs.scm       | 16 +++++++++-
 guix/build/emacs-build-system.scm | 42 +++++++++++++++++++++++++-
 guix/build/emacs-utils.scm        | 49 +++++++++++++++++++++++++++++++
 4 files changed, 114 insertions(+), 14 deletions(-)

diff --git a/gnu/packages/emacs-xyz.scm b/gnu/packages/emacs-xyz.scm
index 074d4d1c4c..f6fb76258b 100644
--- a/gnu/packages/emacs-xyz.scm
+++ b/gnu/packages/emacs-xyz.scm
@@ -2974,6 +2974,7 @@ (define-public emacs-auctex
        #:exclude '("^tests/" "^latex/README")
        #:phases
        (modify-phases %standard-phases
+         (delete 'build-documentation)
          (add-after 'unpack 'configure
            (lambda* (#:key inputs #:allow-other-keys)
              (emacs-substitute-variables "preview.el"
@@ -9155,17 +9156,6 @@ (define-public emacs-orderless
         (base32 "0m9nyz80j0qnn14drbgk8vn5yr7sv0z6yiz8w95ahcw2qwlgyjs7"))
        (file-name (git-file-name name version))))
     (build-system emacs-build-system)
-    (arguments
-     `(#:phases
-       (modify-phases %standard-phases
-         (add-after 'install 'makeinfo
-           (lambda* (#:key outputs #:allow-other-keys)
-             (invoke "makeinfo" "orderless.texi")
-             (install-file "orderless.info"
-                           (string-append (assoc-ref outputs "out")
-                                          "/share/info")))))))
-    (native-inputs
-     (list texinfo))
     (home-page "https://github.com/oantolin/orderless")
     (synopsis "Emacs completion style that matches multiple regexps in any order")
     (description "This package provides an orderless completion style that
@@ -13250,6 +13240,7 @@ (define-public emacs-org
        #:phases
        (modify-phases %standard-phases
          (delete 'build)
+         (delete 'build-documentation)
          (add-before 'check 'make
            (lambda _
              (invoke "make" (string-append "ORGVERSION=" ,version))))
@@ -17846,7 +17837,13 @@ (define-public emacs-esxml
      `(#:emacs ,emacs                   ;need libxml
        ;; XXX: Only the two following files are meant to be packaged.
        ;; Byte-compiling the others Elisp files leads to build errors anyway.
-       #:include (list "esxml.el" "esxml-query.el")))
+       #:include (list "esxml.el" "esxml-query.el")
+       #:phases (modify-phases %standard-phases
+                  (add-before 'build-documentation 'fix-readme-org
+                    (lambda _
+                      (substitute* "README.org"
+                        ;; Fix malformed src block
+                        (("^#\\+BEGIN_SRC\\s$") "#+BEGIN_SRC html")))))))
     (propagated-inputs
      (list emacs-kv))
     (home-page "https://github.com/tali713/esxml/")
diff --git a/guix/build-system/emacs.scm b/guix/build-system/emacs.scm
index 3df68789ff..61746f26a5 100644
--- a/guix/build-system/emacs.scm
+++ b/guix/build-system/emacs.scm
@@ -56,6 +56,18 @@ (define (default-emacs)
   (let ((emacs-mod (resolve-interface '(gnu packages emacs))))
     (module-ref emacs-mod 'emacs-minimal)))
 
+(define (default-texinfo)
+  "Return the default texinfo package."
+  ;; Lazily resolve the binding to avoid a circular dependency.
+  (let ((texinfo-mod (resolve-interface '(gnu packages texinfo))))
+    (module-ref texinfo-mod 'texinfo)))
+
+(define (default-pandoc)
+  "Return the default pandoc package."
+  ;; Lazily resolve the binding to avoid a circular dependency.
+  (let ((pandoc-mod (resolve-interface '(gnu packages haskell-xyz))))
+    (module-ref pandoc-mod 'pandoc)))
+
 (define* (lower name
                 #:key source inputs native-inputs outputs system target
                 (emacs (default-emacs))
@@ -77,7 +89,9 @@ (define private-keywords
                         ;; Keep the standard inputs of 'gnu-build-system'.
                         ,@(standard-packages)))
          (build-inputs `(("emacs" ,emacs)
-                         ,@native-inputs))
+                         ,@native-inputs
+                         ("pandoc" ,(default-pandoc))
+                         ("texinfo" ,(default-texinfo))))
          (outputs outputs)
          (build emacs-build)
          (arguments (strip-keyword-arguments private-keywords arguments)))))
diff --git a/guix/build/emacs-build-system.scm b/guix/build/emacs-build-system.scm
index 6a6918bfdd..fe03fa975c 100644
--- a/guix/build/emacs-build-system.scm
+++ b/guix/build/emacs-build-system.scm
@@ -274,6 +274,45 @@ (define (match-stripped-file action regex)
                            (install-file? file stat #:verbose? #t)))
       #f))))
 
+(define* (build-documentation #:key outputs #:allow-other-keys)
+  (let* ((out (assoc-ref outputs "out"))
+         (library-root (with-directory-excursion (elpa-directory out)
+                         (find-root-library-file
+                          (store-directory->elpa-name-version out))))
+         (texi-file-name (if library-root
+                             (string-append library-root ".texi")
+                             #f))
+
+         (readme-regexp (make-regexp "README.*" regexp/icase))
+         (readme-files (find-files "." readme-regexp))
+         (readme-file (match readme-files
+                        ((file . _) file)
+                        (_ #f)))
+
+         (texi-regexp (string-append library-root "\\.texi"))
+         (texinfo-files (find-files "." texi-regexp)))
+
+    (if (null? texinfo-files)
+        (cond
+         ((and readme-file (string-suffix? ".org" readme-file))
+          ((build-documentation-org #:files (list readme-file))))
+         ((and readme-file texi-file-name)
+          ((convert-documentation #:mapping
+                                  `((,readme-file . ,texi-file-name))))
+          ((build-documentation-texinfo
+            #:files (list texi-file-name)
+            ;; Some README.md can have missing levels of subheadings or
+            ;; incorrect links, this is why --force is used.  Examples of such
+            ;; READMEs: emacs-avy, emacs-git-gutter, emacs-f.
+            #:command  '("makeinfo" "--no-split" "--force"))))
+         (else #t))
+
+        ((build-documentation-texinfo #:files texinfo-files)))
+
+        (for-each (lambda (f)
+                    (install-file f (string-append out %install-dir)))
+                  (find-files "." "\\.info$"))))
+
 (define* (move-doc #:key outputs #:allow-other-keys)
   "Move info files from the ELPA package directory to the info directory."
   (let* ((out (assoc-ref outputs "out"))
@@ -357,7 +396,8 @@ (define %standard-phases
     ;; The .el files are byte compiled directly in the store.
     (add-after 'ensure-package-description 'build build)
     (add-after 'build 'validate-compiled-autoloads validate-compiled-autoloads)
-    (add-after 'validate-compiled-autoloads 'move-doc move-doc)))
+    (add-after 'validate-compiled-autoloads 'move-doc move-doc)
+    (add-before 'move-doc 'build-documentation build-documentation)))
 
 (define* (emacs-build #:key inputs (phases %standard-phases)
                       #:allow-other-keys #:rest args)
diff --git a/guix/build/emacs-utils.scm b/guix/build/emacs-utils.scm
index 8ee547f2b3..b7e0a3e939 100644
--- a/guix/build/emacs-utils.scm
+++ b/guix/build/emacs-utils.scm
@@ -36,6 +36,10 @@ (define-module (guix build emacs-utils)
             emacs-batch-error?
             emacs-batch-error-message
 
+            build-documentation-texinfo
+            build-documentation-org
+            convert-documentation
+
             emacs-generate-autoloads
             emacs-byte-compile-directory
             emacs-header-parse
@@ -100,6 +104,51 @@ (define (emacs-batch-script expr)
                          (message (read-string (car error-pipe)))))))
     output))
 
+\f
+;;;
+;;; Helpers for generating frequently used phases.
+;;;
+
+(define* (build-documentation-texinfo
+          #:key
+          (files '())
+          (command '("makeinfo" "--no-split")))
+  "Don't forget to add texinfo into list of inputs for the package."
+  (lambda* (#:key outputs #:allow-other-keys)
+    (for-each (lambda (f) (apply invoke (append command (list f)))) files)))
+
+(define* (build-documentation-org
+          #:key
+          (files '()))
+  "This is a preferred way over pandoc, because it keeps the meta information
+from in-buffer org settings.  Don't forget to add texinfo and emacs into list
+of inputs for the package."
+  (lambda* (#:key outputs #:allow-other-keys)
+    (for-each (lambda (f)
+                (emacs-batch-script
+                 `(progn
+                   (require 'ox-texinfo)
+                   (find-file ,f)
+                   (let ((org-export-use-babel nil)
+                         (org-texinfo-info-process
+                          '("makeinfo --force --no-split %f")))
+                     (org-texinfo-export-to-info))))) files)))
+
+(define* (convert-documentation
+          #:key
+          (mapping '())
+          (command '("pandoc")))
+  "Don't forget to add pandoc into list of inputs for the package."
+  (lambda* (#:key outputs #:allow-other-keys)
+    (for-each (lambda (p) (apply invoke
+                                 (append command
+                                         (list (car p) "-o" (cdr p)))))
+              mapping)))
+
+;;;
+;;; Helpers for generating frequently used phases ends here.
+;;;
+
 (define (emacs-generate-autoloads name directory)
   "Generate autoloads for Emacs package NAME placed in DIRECTORY."
   (let* ((file (string-append directory "/" name "-autoloads.el"))
-- 
2.37.2


[-- Attachment #1.3: Type: text/plain, Size: 2188 bytes --]


Picked (guix build emacs-utils) for now, it's done to avoid huge
rebuilds, while testing, later we can move it to (guix build utils).

Also, temporary added pandoc and texinfo to native-inputs for
emacs-build-system, otherwise I would need to update inputs for almost
every emacs-* package.  Need to figure out what to do with this.

>
>> Attaching the latest version of the documentation-files patch I have
> Looking at this patch, perhaps we'd also have to allow customizing
> command line options.  Also, as for installing, I think this should be
> handled by the install phase, which already has includes
> "^[^/]*\\.info$" and "^doc/.*\\.info$" by default.  Thus, you only need
> to build documentation before the install phase.

That's right, but in the new iteration (v3) of build-documentation phase
I use find-root-library-file, which expects to be executed when elpa
directory is already available, so I can't do it before install phase
and need to manually install info files.

Also, current build phases order is a little confusing, a lot of builds
happens after install phase, directly in output directory:

`set-SOURCE-DATE-EPOCH'
`set-paths'
`install-locale'
`unpack'
`expand-load-path'
`patch-usr-bin-file'
`patch-source-shebangs'
`patch-generated-file-shebangs'
`check'
`install'
`make-autoloads'
`enable-autoloads-compilation'
`patch-el-files'
`ensure-package-description'
`build'
`validate-compiled-autoloads'
`build-documentation'
`move-doc'
`patch-shebangs'
`strip'
`validate-runpath'
`validate-documentation-location'
`delete-info-dir-file'
`patch-dot-desktop-files'
`make-dynamic-linker-cache'
`install-license-files'
`reset-gzip-timestamps'
`compress-documentation'

What if instead of install phase we will use
create-tmp-lisp-and-documentation-directories phase (or something more
meaningful) to make a temporary directory, where we will build all the
stuff and after that, at the end of the build process will install
everything from this temporary directory to the store?  This way
emacs-build-system will become more usual and easier to understand and
predict.

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[bug#57280] [PATCH 0/3] Add documentation-files argument to emacs build system.

[Liliana Marie Prikler]

Hi,

Am Freitag, dem 02.09.2022 um 17:02 +0300 schrieb Andrew Tropin:
> Picked (guix build emacs-utils) for now, it's done to avoid huge
> rebuilds, while testing, later we can move it to (guix build utils).
> 
> Also, temporary added pandoc and texinfo to native-inputs for
> emacs-build-system, otherwise I would need to update inputs for
> almost every emacs-* package.  Need to figure out what to do with
> this.
I still don't like that we need pandoc and texinfo as implicit inputs.
There ought to be a better solution than this.

> > 
> > > Attaching the latest version of the documentation-files patch I
> > > have
> > Looking at this patch, perhaps we'd also have to allow customizing
> > command line options.  Also, as for installing, I think this should
> > be
> > handled by the install phase, which already has includes
> > "^[^/]*\\.info$" and "^doc/.*\\.info$" by default.  Thus, you only
> > need
> > to build documentation before the install phase.
> 
> That's right, but in the new iteration (v3) of build-documentation
> phase I use find-root-library-file, which expects to be executed when
> elpa directory is already available, so I can't do it before install
> phase and need to manually install info files.
> 
> Also, current build phases order is a little confusing, a lot of
> builds happens after install phase, directly in output directory:
> 
> `set-SOURCE-DATE-EPOCH'
> `set-paths'
> `install-locale'
> `unpack'
> `expand-load-path'
> `patch-usr-bin-file'
> `patch-source-shebangs'
> `patch-generated-file-shebangs'
> `check'
> `install'
> `make-autoloads'
> `enable-autoloads-compilation'
> `patch-el-files'
> `ensure-package-description'
> `build'
> `validate-compiled-autoloads'
> `build-documentation'
> `move-doc'
> `patch-shebangs'
> `strip'
> `validate-runpath'
> `validate-documentation-location'
> `delete-info-dir-file'
> `patch-dot-desktop-files'
> `make-dynamic-linker-cache'
> `install-license-files'
> `reset-gzip-timestamps'
> `compress-documentation'
> 
> What if instead of install phase we will use create-tmp-lisp-and-
> documentation-directories phase (or something
> more meaningful) to make a temporary directory, where we will build
> all the stuff and after that, at the end of the build process will
> install everything from this temporary directory to the store?  This
> way emacs-build-system will become more usual and easier to
> understand andpredict.
I don't think we would need to do staged installations.  As for why we
don't build in the temporary directory, I do not know, I merely
inherited that code.

> +(define* (build-documentation-texinfo
> +          #:key
> +          (files '())
> +          (command '("makeinfo" "--no-split")))
> +  "Don't forget to add texinfo into list of inputs for the package."
> +  (lambda* (#:key outputs #:allow-other-keys)
> +    (for-each (lambda (f) (apply invoke (append command (list f))))
> files)))
This is hardly specific to emacs, is it?
Also, append is usually a code smell.  Can't we (apply invoke
"makeinfo" file options)?

> +(define* (convert-documentation
> +          #:key
> +          (mapping '())
> +          (command '("pandoc")))
> +  "Don't forget to add pandoc into list of inputs for the package."
> +  (lambda* (#:key outputs #:allow-other-keys)
> +    (for-each (lambda (p) (apply invoke
> +                                 (append command
> +                                         (list (car p) "-o" (cdr
> p)))))
> +              mapping)))
As above.

> +(define* (build-documentation-org
> +          #:key
> +          (files '()))
This one is emacs-specific due to emacs-batch-script and can remain
there.

> +    (add-after 'validate-compiled-autoloads 'move-doc move-doc)
> +    (add-before 'move-doc 'build-documentation
> build-documentation)))
I don't think that we'll have to add this phase once we've added the
helpers.  And as previously discussed, we'd have to build the
documentation before install.

Cheers

bug#57280: [PATCH 0/3] Add documentation-files argument to emacs build system.

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 4569 bytes --]

On 2022-09-02 16:52, Liliana Marie Prikler wrote:

> Hi,
>
> Am Freitag, dem 02.09.2022 um 17:02 +0300 schrieb Andrew Tropin:
>> Picked (guix build emacs-utils) for now, it's done to avoid huge
>> rebuilds, while testing, later we can move it to (guix build utils).
>> 
>> Also, temporary added pandoc and texinfo to native-inputs for
>> emacs-build-system, otherwise I would need to update inputs for
>> almost every emacs-* package.  Need to figure out what to do with
>> this.
> I still don't like that we need pandoc and texinfo as implicit inputs.
> There ought to be a better solution than this.
>
>> > 
>> > > Attaching the latest version of the documentation-files patch I
>> > > have
>> > Looking at this patch, perhaps we'd also have to allow customizing
>> > command line options.  Also, as for installing, I think this should
>> > be
>> > handled by the install phase, which already has includes
>> > "^[^/]*\\.info$" and "^doc/.*\\.info$" by default.  Thus, you only
>> > need
>> > to build documentation before the install phase.
>> 
>> That's right, but in the new iteration (v3) of build-documentation
>> phase I use find-root-library-file, which expects to be executed when
>> elpa directory is already available, so I can't do it before install
>> phase and need to manually install info files.
>> 
>> Also, current build phases order is a little confusing, a lot of
>> builds happens after install phase, directly in output directory:
>> 
>> `set-SOURCE-DATE-EPOCH'
>> `set-paths'
>> `install-locale'
>> `unpack'
>> `expand-load-path'
>> `patch-usr-bin-file'
>> `patch-source-shebangs'
>> `patch-generated-file-shebangs'
>> `check'
>> `install'
>> `make-autoloads'
>> `enable-autoloads-compilation'
>> `patch-el-files'
>> `ensure-package-description'
>> `build'
>> `validate-compiled-autoloads'
>> `build-documentation'
>> `move-doc'
>> `patch-shebangs'
>> `strip'
>> `validate-runpath'
>> `validate-documentation-location'
>> `delete-info-dir-file'
>> `patch-dot-desktop-files'
>> `make-dynamic-linker-cache'
>> `install-license-files'
>> `reset-gzip-timestamps'
>> `compress-documentation'
>> 
>> What if instead of install phase we will use create-tmp-lisp-and-
>> documentation-directories phase (or something
>> more meaningful) to make a temporary directory, where we will build
>> all the stuff and after that, at the end of the build process will
>> install everything from this temporary directory to the store?  This
>> way emacs-build-system will become more usual and easier to
>> understand andpredict.
> I don't think we would need to do staged installations.  As for why we
> don't build in the temporary directory, I do not know, I merely
> inherited that code.
>
>> +(define* (build-documentation-texinfo
>> +          #:key
>> +          (files '())
>> +          (command '("makeinfo" "--no-split")))
>> +  "Don't forget to add texinfo into list of inputs for the package."
>> +  (lambda* (#:key outputs #:allow-other-keys)
>> +    (for-each (lambda (f) (apply invoke (append command (list f))))
>> files)))
> This is hardly specific to emacs, is it?
> Also, append is usually a code smell.  Can't we (apply invoke
> "makeinfo" file options)?
>
>> +(define* (convert-documentation
>> +          #:key
>> +          (mapping '())
>> +          (command '("pandoc")))
>> +  "Don't forget to add pandoc into list of inputs for the package."
>> +  (lambda* (#:key outputs #:allow-other-keys)
>> +    (for-each (lambda (p) (apply invoke
>> +                                 (append command
>> +                                         (list (car p) "-o" (cdr
>> p)))))
>> +              mapping)))
> As above.
>
>> +(define* (build-documentation-org
>> +          #:key
>> +          (files '()))
> This one is emacs-specific due to emacs-batch-script and can remain
> there.
>
>> +    (add-after 'validate-compiled-autoloads 'move-doc move-doc)
>> +    (add-before 'move-doc 'build-documentation
>> build-documentation)))
> I don't think that we'll have to add this phase once we've added the
> helpers.  And as previously discussed, we'd have to build the
> documentation before install.
>
> Cheers

I think we can consider this patch series as an thought experiment.

Manually adding documentation build phases works good enough and
probably there is no need to do something more generic like this.

Closing the ticket without merging.

Liliana, thank you for you time!

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]