procedure for creating online Emacs manual?

procedure for creating online Emacs manual?

[Paul Eggert]

Today I noticed that the online Emacs manual 
<https://www.gnu.org/software/emacs/manual/> is still using version 
25.2. What's the procedure for generating the online manual from the 
released version? I can volunteer to turn the crank, but I don't know 
where the crank is.

Re: procedure for creating online Emacs manual?

[Glenn Morris]

Paul Eggert wrote:

> Today I noticed that the online Emacs manual
> <https://www.gnu.org/software/emacs/manual/> is still using version
> 25.2.

I don't think there were any (?) changes to the manual between 25.2
and the latest release (ie 25.3).

> What's the procedure for generating the online manual from the
> released version?

It is summarized at the end of admin/make-tarball.txt.
Basically it uses M-x make-manuals from admin/admin.el.

Re: procedure for creating online Emacs manual?

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 2457 bytes --]

On 03/30/2018 12:32 PM, Glenn Morris wrote:
> I don't think there were any (?) changes to the manual between 25.2
> and the latest release (ie 25.3).

OK. It is confusing that the latest release is 25.3 but the online 
manuals say 25.2, but we can live with it if it's too much trouble to fix.

>> What's the procedure for generating the online manual from the
>> released version?
> It is summarized at the end of admin/make-tarball.txt.
> Basically it uses M-x make-manuals from admin/admin.el.

Thanks. I tried to follow that procedure but came up dry. I fixed the 
problems that I found (e.g., we don't put refcards up on the web any 
more) by installing the attached into 'master', and left in a FIXME for 
the stuff I couldn't figure out. Plus, I altered the instructions so 
that the tarballs are reproducible. (It wouldn't hurt to go reproducible 
even for Emacs 26 but I was leery of changing distribution instructions 
in the emacs-26 branch.)

I couldn't build the refcards on Fedora 27, due to some problem with 
larm1000 (see below). Maybe we should stop maintaining the refcards 
since we no longer put them on the web? I doubt whether they're used 
much any more and the whole idea of a refcard is so 1980s anyway. Stefan 
hinted at doing that here:

https://lists.gnu.org/archive/html/emacs-devel/2016-11/msg00715.html


Here's the failed refcard build on Fedora:

$ make -C etc/refcards pdf ps
...
pdflatex ru-refcard.tex
This is pdfTeX, Version 3.14159265-2.6-1.40.17 (TeX Live 2016) 
(preloaded format=pdflatex)
  restricted \write18 enabled.
entering extended mode
(./ru-refcard.tex
LaTeX2e <2016/03/31>
Babel <3.9r> and hyphenation patterns for 22 language(s) loaded.
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
...
kpathsea: Running mktextfm larm1000
/usr/share/texlive/texmf-dist/web2c/mktexnam: Could not map source 
abbreviation  for larm1000.
/usr/share/texlive/texmf-dist/web2c/mktexnam: Need to update ?
mktextfm: Running mf-nowin -progname=mf \mode:=ljfour; mag:=1; 
nonstopmode; input larm1000
This is METAFONT, Version 2.7182818 (TeX Live 2016) (preloaded base=mf)


kpathsea: Running mktexmf larm1000
! I can't find file `larm1000'.
<*> ...ljfour; mag:=1; nonstopmode; input larm1000

...
Transcript written on ru-refcard.log.
make: *** [Makefile:262: ru-refcard.pdf] Error 1


[-- Attachment #2: 0001-Improve-doc-for-web-pages-reproducible-tarballs.patch --]
[-- Type: text/x-patch, Size: 4090 bytes --]

From ae0187535bec4ff114e994c9da83be07b1f7ff4f Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Fri, 30 Mar 2018 16:11:45 -0700
Subject: [PATCH] Improve doc for web pages; reproducible tarballs

* admin/make-tarball.txt: Make the tarballs more reproducible.
Fix instructions for web pages as best I can (they are still
incomplete).
* make-dist (default_gzip): Add --no-name for gzip.
(taropt): Add options to make the build more reproducible.
---
 admin/make-tarball.txt | 34 +++++++++++++++++++++++++---------
 make-dist              |  8 ++++----
 2 files changed, 29 insertions(+), 13 deletions(-)

diff --git a/admin/make-tarball.txt b/admin/make-tarball.txt
index ac6d15d6ce..19edeb79e6 100644
--- a/admin/make-tarball.txt
+++ b/admin/make-tarball.txt
@@ -123,7 +123,7 @@ General steps (for each step, check for possible errors):
 
 9. Decide what compression schemes to offer.
     For a release, at least gz and xz:
-      gzip --best -c emacs-NEW.tar > emacs-NEW.tar.gz
+      gzip --best --no-name -c emacs-NEW.tar > emacs-NEW.tar.gz
       xz -c emacs-NEW.tar > emacs-NEW.tar.xz
     For pretests, just xz is probably fine (saves bandwidth).
 
@@ -197,7 +197,6 @@ The pages to update are:
 
 emacs.html (for a new major release, a more thorough update is needed)
 history.html
-add the new NEWS file as news/NEWS.xx.y
 
 For every new release, a banner is displayed on top of the emacs.html
 page.  Uncomment and the release banner in emacs.html.  Keep it on the
@@ -210,15 +209,32 @@ manual/html_node directory, delete any old manual pages that are no
 longer present.
 
 Tar up the generated html_node/emacs/ and elisp/ directories and update
-the files manual/elisp.html_node.tar.gz and emacs.html_node.tar.gz.
+the files  manual/elisp.html_node.tar.gz and emacs.html_node.tar.gz.
+Use GNU Tar as follows so that the tarballs are reproducible:
 
-Use M-x make-manuals-dist from admin/admin.el to update the
-manual/texi/ tarfiles.
-
-Add compressed copies of the main info pages from the tarfile to manual/info/.
+cd manual
+tar='tar --numeric-owner --owner=0 --group=0 --mode=go+u,go-w --sort=name'
+gzip='gzip --best --no-name'
+$tar -cf - html_node/emacs | $gzip >emacs.html_node.tar.gz
+$tar -cf - html_node/elisp | $gzip >elisp.html_node.tar.gz
 
-Update the refcards/pdf/ and ps/ directories, and also
-refcards/emacs-refcards.tar.gz (use make -C etc/refcards pdf ps dist).
+Use M-x make-manuals-dist from admin/admin.el to update the
+manual/*.tar files.
+
+Add compressed copies of the main info pages from the tarfile to manual/info/
+as follows:
+
+cd manual
+mkdir info
+gzip --best --no-name <../info/eintr.info >info/eintr.info.gz
+gzip --best --no-name <../info/elisp.info >info/elisp.info.gz
+gzip --best --no-name <../info/emacs.info >info/emacs.info.gz
+
+FIXME: The above instructions are not quite complete, as they do not
+specify how to copy the generated files in the 'manual' directory to
+the corresponding web files.  Also, they are missing some files, e.g.,
+they generate manual/html_mono/ada-mode.html but do not generate the
+top-level ada-mode.html file for the one-node-per-page version.
 
 Browsing <https://web.cvs.savannah.gnu.org/viewvc/?root=emacs> is one
 way to check for any files that still need updating.
diff --git a/make-dist b/make-dist
index 26247b37bc..48c7fb4fb7 100755
--- a/make-dist
+++ b/make-dist
@@ -639,14 +639,14 @@ files=
   case "${default_gzip}" in
     bzip2) gzip_extension=.bz2 ;;
     xz)  gzip_extension=.xz ;;
-    gzip)  gzip_extension=.gz ; default_gzip="gzip --best";;
+    gzip)  gzip_extension=.gz ; default_gzip="gzip --best --no-name";;
        *)  gzip_extension= ;;
   esac
   echo "Creating tar file"
-  taropt=
-  [ "$verbose" = "yes" ] && taropt=v
+  taropt='--numeric-owner --owner=0 --group=0 --mode=go+u,go-w --sort=name'
+  [ "$verbose" = "yes" ] && taropt="$taropt --verbose"
 
-  (cd ${tempparent} ; tar c${taropt}f - ${emacsname} ) \
+  (cd ${tempparent} ; tar $taropt -cf - ${emacsname} ) \
     | ${default_gzip} \
     > ${emacsname}.tar${gzip_extension}
 fi
-- 
2.14.3

Re: procedure for creating online Emacs manual?

[Glenn Morris]

Paul Eggert wrote:

> OK. It is confusing that the latest release is 25.3 but the online
> manuals say 25.2, but we can live with it if it's too much trouble to
> fix.

Anyone should free to fix it, it's not difficult IMO.

> Thanks. I tried to follow that procedure but came up dry. I fixed the
> problems that I found (e.g., we don't put refcards up on the web any
> more)

Yes we do?
https://www.gnu.org/software/emacs/refcards/index.html

> by installing the attached into 'master', and left in a FIXME for the
> stuff I couldn't figure out.

Re this:

> FIXME: The above instructions are not quite complete, as they do not
> specify how to copy the generated files in the 'manual' directory to
> the corresponding web files.

Not scripted, copy by hand. Might be good to script. Need to cvs remove
any pages that are no longer present, cvs add new ones, etc.

> Also, they are missing some files, e.g., they generate
> manual/html_mono/ada-mode.html but do not generate the top-level
> ada-mode.html file for the one-node-per-page version.

If you mean eg
https://www.gnu.org/software/emacs/manual/ada-mode.html
then you never need to change that page. If a new texinfo manual is
added, just copy an existing index page and change by hand as needed.

> I couldn't build the refcards on Fedora 27, due to some problem with
> larm1000 (see below).

It used to work for me on Debian, once I installed the relevant TeX
language packages, but I haven't tried in a while.

Re: procedure for creating online Emacs manual?

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 905 bytes --]

On 03/30/2018 05:58 PM, Glenn Morris wrote:
>> I fixed the
>> problems that I found (e.g., we don't put refcards up on the web any
>> more)
> Yes we do?
> https://www.gnu.org/software/emacs/refcards/index.html

Ouch. These were not in my copy of the CVS repository of the Emacs web pages, 
even after I did a "cvs update". Apparently my copy (which has been around for a 
while) was incorrect, or was updated incorrectly, or something. I removed my bad 
copy and got a fresh one from Savannah. Sorry about the false alarm. I restored 
the refcards commentary to admin/make-tarball.txt on master (see attached).

I tried to take your comments into account in the attached further patch. Thanks.

It's still not clear that the refcards are worth the maintenance hassle. I can't 
build them on either of my main build platforms right now, so I can't completely 
update the Emacs web pages.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Further-improve-advice-in-make-tarball.txt.patch --]
[-- Type: text/x-patch; name="0001-Further-improve-advice-in-make-tarball.txt.patch", Size: 2228 bytes --]

From 7bedc8812bd7ca1d9cf36636322068b28b690a85 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Sun, 1 Apr 2018 14:16:48 -0700
Subject: [PATCH] Further improve advice in make-tarball.txt

See comments by Glenn Morris in:
https://lists.gnu.org/r/emacs-devel/2018-03/msg00968.html
* admin/make-tarball.txt: Bring back refcard advice.
---
 admin/make-tarball.txt | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/admin/make-tarball.txt b/admin/make-tarball.txt
index f9ad217..9b9c964 100644
--- a/admin/make-tarball.txt
+++ b/admin/make-tarball.txt
@@ -87,6 +87,11 @@ General steps (for each step, check for possible errors):
      make -C etc/refcards
      make -C etc/refcards clean
 
+    If etc/refcards does not build you may need to downgrade or
+    upgrade your TeX installation, or do that part of the build by
+    hand.  For clues, search for the string "refcard" in the file
+    admin/release-process.
+
 5.  Copy lisp/loaddefs.el to lisp/ldefs-boot.el.
 
     Commit ChangeLog.N, etc/AUTHORS, lisp/ldefs-boot.el, and the
@@ -231,11 +236,16 @@ as follows:
   gzip --best --no-name <../info/elisp.info >info/elisp.info.gz
   gzip --best --no-name <../info/emacs.info >info/emacs.info.gz
 
+Update the refcards/pdf/ and ps/ directories, and also
+refcards/emacs-refcards.tar.gz (use make -C etc/refcards pdf ps dist).
+
 FIXME: The above instructions are not quite complete, as they do not
-specify how to copy the generated files in the 'manual' directory to
-the corresponding web files.  Also, they are missing some files, e.g.,
+specify the manual procuedure used to copy the generated files in the
+'manual' directory to the corresponding web files, and to cvs remove
+and add files as needed.  Also, they are missing some files, e.g.,
 they generate manual/html_mono/ada-mode.html but do not generate the
-top-level ada-mode.html file for the one-node-per-page version.
+top-level ada-mode.html file for the one-node-per-page version; this
+is currently done by hand.
 
 Browsing <https://web.cvs.savannah.gnu.org/viewvc/?root=emacs> is one
 way to check for any files that still need updating.
-- 
2.7.4

Re: procedure for creating online Emacs manual?

[Glenn Morris]

Paul Eggert wrote:

> I can't build them on either of my main build platforms right now, so
> I can't completely update the Emacs web pages.

ru-refcard builds fine for me on Debian. I think the relevant package is 
texlive-fonts-extra.

It also builds fine for me on RHEL7 if I install texlive-lh from EPEL.
(It seems a packaging bug to me that installing texlive-cyrillic does
not bring in the fonts package as a dependency.)