Re: Why wasn't the 25.3 release based on the then-head of the emacs-25 branch?

[Eli Zaretskii <eliz@gnu.org>]

> Cc: rgm@gnu.org, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Sun, 17 Sep 2017 15:44:36 -0700
> 
>     That time included the time to make the tarball and test it.
>
> If making the tarball and testing it takes 1.5 days, then that was 20% of the overall delay last time, and there is good opportunity for speeding up the process. Such a process should take minutes, not hours.

The process of producing the tarball once you invoke make-dist is
indeed very short.  But the process of checking-out from Git, setting
the version number, committing the changes and the corresponding tag,
making the tarball itself, verifying its correctness, uploading it to
the GNU servers, then downloading it to several different systems, and
verifying the build--that process takes longer, especially if done by
several people who have their lives and live in different timezones.

Much of this involves tedious manual procedures, and is thus
error-prone.  It would be nice to automate at least some of that
(tarball content verification comes to mind), which could shorten the
time and eliminate the potential for errors.  For example, in the case
of Emacs 25.3, the original tarball included 2 minor mistakes, which
required to re-upload it after fixing them.

But whatever improvements we introduce, I think it's naïve to assume
time frames of minutes for these activities, unless we want to give up
QA.  And giving up QA would be a mistake, IMO, since emergency
releases that fix grave problems must not themselves be problematic;
if they are, that would require additional fixup releases and prolong
the time even more.

>     How can people outside of
>     the project be charged with reviewing our bugs and patches?
> 
> These are people quite knowledgeable about security and software maintenance. They can be a good source for security reviews. It's another set of eyes, with an outside perspective, and that is helpful.
> 
>     why wouldn't those people speak up here
>     and work with us within our procedures?
> 
> They're busy. Also, we haven't exactly been soliciting or welcoming their input. The most recent emergency release had a bit of an NIH feel about it.

I still don't understand what you are suggesting, in practical terms,
and how will this be different from the current procedures, where
anyone could raise a security issue and propose a solution.

>     No one was arguing for additional bureaucracy.  What we need is data
>     and procedures
> 
> Whatever it's called it's more work, and we lack the resources to do it. Maybe we can look at two disparate releases (Debian and Fedora, say). Above that there are diminishing returns. Outside reviewers could help here (some are Fedora experts). 

Are Debian and Fedora indeed enough?  What about Red Hat?  What about
Arch Linux?  I don't have the answers, but we should decide on the
representative sample of the distributions based on some principles we
agree upon, and we should do it ahead of time.

The next question is, given the sample of distributions, how does one
figure out which ones of them include a given Emacs changeset, in
which versions of Emacs, and since what time.  Asking some experts
about this is possible, but it runs the risk that those experts are
unavailable or incommunicado when we need them, so if some databases
can be searched instead, that's better.

And of course all of that should be recorded in some file in admin/,
so that no one has to remember any of this when the time comes.