Generate NEWS.XX.Y.html file for Emacs website

Generate NEWS.XX.Y.html file for Emacs website

[Stefan Kangas]

[I'm creating a new thread for generating HTML versions of etc/NEWS for
 the website.  This is continued from the discussion starting here:
 https://lists.gnu.org/r/emacs-devel/2022-09/msg00759.html ]

Eli Zaretskii <eliz@gnu.org> writes:

> [P]lease describe what is needed (which commands and packages, and how
> to use them) to produce this HTML version.

The command is:

    ./src/emacs -l admin/admin.el -f make-news-html-file
    mv etc/NEWS.XX.Y.html $EMACS_WEBROOT/news
    cd $EMACS_WEBROOT
    cvs commit -m "Update NEWS.XX.Y.html" news/NEWS.XX.Y.html

Please see `make-news-html-file' in admin/admin.el for more details.
It's still a work in progress, of course, but I'm happy to work on it.

> Also, how will this HTML version be maintained, after it is produced.

The easiest option is to just leave the old generated HTML files alone
and update them only if absolutely necessary.

But it'd be better if we could generate HTML versions of all etc/NEWS.*
files whenever we make a new release.  I didn't yet look into this, so I
don't know how realistic this is, but I have a hunch that it's doable.
Maybe we will only be able to do it for releases newer than Emacs 21 (or
something) though.

We would then also need to add some code to delete all the Emacs 28.2
stuff from "NEWS.28.1.html".  Maybe we would want to trim out the Emacs
28.1 headlines from "NEWS.28.2.html"?  I'm not sure.

The second option has the added benefit that any fixes or additions we
make in old NEWS files would eventually propagate to the website.

Re: Generate NEWS.XX.Y.html file for Emacs website

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Thu, 15 Sep 2022 01:39:44 -0700
> 
> > [P]lease describe what is needed (which commands and packages, and how
> > to use them) to produce this HTML version.
> 
> The command is:
> 
>     ./src/emacs -l admin/admin.el -f make-news-html-file
>     mv etc/NEWS.XX.Y.html $EMACS_WEBROOT/news
>     cd $EMACS_WEBROOT
>     cvs commit -m "Update NEWS.XX.Y.html" news/NEWS.XX.Y.html
> 
> Please see `make-news-html-file' in admin/admin.el for more details.

I see that you changed admin.el on master, which means Emacs 28.x will
be unable to use this?

In any case, if we want the above to be part of updating the Emacs Web
site and the on-line manuals, those commands will need to be part of
the scripts that are used for those jobs, or maybe a separate script.

> > Also, how will this HTML version be maintained, after it is produced.
> 
> The easiest option is to just leave the old generated HTML files alone
> and update them only if absolutely necessary.

I meant to ask how will we keep them under VCS (CVS in this case).

> But it'd be better if we could generate HTML versions of all etc/NEWS.*
> files whenever we make a new release.

Each time we make a new release, or just once?

> I didn't yet look into this, so I don't know how realistic this is,
> but I have a hunch that it's doable.  Maybe we will only be able to
> do it for releases newer than Emacs 21 (or something) though.

Do we have links for those older NEWS files on the Web site?  If not,
I see no point in generating NEWS in HTML format for old releases.

Re: Generate NEWS.XX.Y.html file for Emacs website

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> I see that you changed admin.el on master, which means Emacs 28.x will
> be unable to use this?

I pushed it to emacs-28 (as this is only in admin/admin.el and doesn't
touch anything else).  So we can use it on Emacs 28.2, and that's what I
did to generate the HTML file I posted.

> In any case, if we want the above to be part of updating the Emacs Web
> site and the on-line manuals, those commands will need to be part of
> the scripts that are used for those jobs, or maybe a separate script.

Indeed.  I will work on it, as well as updating the relevant docs.

>> > Also, how will this HTML version be maintained, after it is produced.
>>
>> The easiest option is to just leave the old generated HTML files alone
>> and update them only if absolutely necessary.
>
> I meant to ask how will we keep them under VCS (CVS in this case).

I might be missing some context here, because I don't think I understand
what you're asking.  Do we need to do anything else besides copying them
in place (to "news/NEWS.NN.Y.html") and running "cvs update"?

>> But it'd be better if we could generate HTML versions of all etc/NEWS.*
>> files whenever we make a new release.
>
> Each time we make a new release, or just once?

We could decide which way we want to go with it.  I imagine that the
template could change over time, so it might be nice to do it every time
we make a release (but we'd need to automate it).

On the other hand, it might save us some work if we only do it once.

>> I didn't yet look into this, so I don't know how realistic this is,
>> but I have a hunch that it's doable.  Maybe we will only be able to
>> do it for releases newer than Emacs 21 (or something) though.
>
> Do we have links for those older NEWS files on the Web site?  If not,
> I see no point in generating NEWS in HTML format for old releases.

The only benefit I see is that you could then edit the URL to visit the
pages for older releases.  I think Karl Fogel pointed out this use case,
and it's definitely something I have done in the past too.

But I'm not sure it's very important.  We could perhaps do it if it
turns out to be only a very small amount of work.

Re: Generate NEWS.XX.Y.html file for Emacs website

[Filipp Gunbin]

On 15/09/2022 01:39 -0700, Stefan Kangas wrote:

> We would then also need to add some code to delete all the Emacs 28.2
> stuff from "NEWS.28.1.html".  Maybe we would want to trim out the Emacs
> 28.1 headlines from "NEWS.28.2.html"?  I'm not sure.

Wouldn't it suffice to have one page per major release?  Other projects
do it, and it would align with how NEWS files are organized currently.

The minor release could be just an anchor in the link.

Re: Generate NEWS.XX.Y.html file for Emacs website

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Thu, 15 Sep 2022 13:45:31 -0400
> Cc: emacs-devel@gnu.org
> 
> >> > Also, how will this HTML version be maintained, after it is produced.
> >>
> >> The easiest option is to just leave the old generated HTML files alone
> >> and update them only if absolutely necessary.
> >
> > I meant to ask how will we keep them under VCS (CVS in this case).
> 
> I might be missing some context here, because I don't think I understand
> what you're asking.  Do we need to do anything else besides copying them
> in place (to "news/NEWS.NN.Y.html") and running "cvs update"?

You need to add it to CVS first and commit, right?

My question was would we maintain the HTML NEWS files in CVS together
with the rest of the Web pages, or will we do something else?

> >> I didn't yet look into this, so I don't know how realistic this is,
> >> but I have a hunch that it's doable.  Maybe we will only be able to
> >> do it for releases newer than Emacs 21 (or something) though.
> >
> > Do we have links for those older NEWS files on the Web site?  If not,
> > I see no point in generating NEWS in HTML format for old releases.
> 
> The only benefit I see is that you could then edit the URL to visit the
> pages for older releases.  I think Karl Fogel pointed out this use case,
> and it's definitely something I have done in the past too.

The Web site only has announcements going back to v27.1; the versions
before that point to an email announcement.  So I think there's no
reason to have HTML versions of NEWS for versions older than v27.1.

Re: Generate NEWS.XX.Y.html file for Emacs website

[Stefan Kangas]

Filipp Gunbin <fgunbin@fastmail.fm> writes:

> Wouldn't it suffice to have one page per major release?  Other projects
> do it, and it would align with how NEWS files are organized currently.
>
> The minor release could be just an anchor in the link.

Sounds good to me, and it also mirrors what we have in "etc".

Re: Generate NEWS.XX.Y.html file for Emacs website

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> You need to add it to CVS first and commit, right?

Indeed, I don't use CVS much these days...

> My question was would we maintain the HTML NEWS files in CVS together
> with the rest of the Web pages, or will we do something else?

Yes, I think we should put them in the "<emacs-web>/news" directory.

Re: Generate NEWS.XX.Y.html file for Emacs website

[Stefan Kangas]

Stefan Kangas <stefankangas@gmail.com> writes:

> [I'm creating a new thread for generating HTML versions of etc/NEWS for
>  the website.  This is continued from the discussion starting here:
>  https://lists.gnu.org/r/emacs-devel/2022-09/msg00759.html ]

I've now fixed any errors I could find in the HTML export, and added
this new file:

    https://www.gnu.org/software/emacs/news/NEWS.28.html

I'd suggest updating the link on

    https://www.gnu.org/software/emacs/emacs.html#Releases

to point to this new HTML file instead of the plain-text.

There is no reason not to continue distributing the plain text files,
also for new releases, to give users the choice of which to read.

Re: Generate NEWS.XX.Y.html file for Emacs website

[Stefan Kangas]

Stefan Kangas <stefankangas@gmail.com> writes:

> I've now fixed any errors I could find in the HTML export, and added
> this new file:
>
>     https://www.gnu.org/software/emacs/news/NEWS.28.html
>
> I'd suggest updating the link on
>
>     https://www.gnu.org/software/emacs/emacs.html#Releases
>
> to point to this new HTML file instead of the plain-text.

I've updated NEWS.28.html again, and IMHO it looks even better now.

I also discovered that the HTML files are readable on mobile whereas the
text files aren't.  The file was downloaded instead of viewed in-browser
on the phones I tried.  So that's nice.

If there are no objections, I will go ahead and update the link as
suggested above.  Feedback is still welcome of course, and I will do my
best to fix any issues that crop up.

---

Note that we currently have some issues with adding <code> markers
around single quoted strings.  I think it's hard to make the export
perfect given our current NEWS file format, where the markup is not
unambiguous.

Take this example from etc/NEWS.28:

> 'xref-find-definitions-other-window': if it's t or 'show', the first

Or these examples from etc/NEWS (found by Mattias Engdegård):

> across all predicates/modifiers, the list of allowed delimiters has
> been restricted to "...", '...', /.../, |...|, (...), [...], <...>,
> and {...}.  See the "(eshell) Argument Predication and Modification"

Re: Generate NEWS.XX.Y.html file for Emacs website

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Sat, 17 Sep 2022 14:50:15 -0400
> 
> Stefan Kangas <stefankangas@gmail.com> writes:
> 
> > I've now fixed any errors I could find in the HTML export, and added
> > this new file:
> >
> >     https://www.gnu.org/software/emacs/news/NEWS.28.html
> >
> > I'd suggest updating the link on
> >
> >     https://www.gnu.org/software/emacs/emacs.html#Releases
> >
> > to point to this new HTML file instead of the plain-text.
> 
> I've updated NEWS.28.html again, and IMHO it looks even better now.

It still included NEWS for Emacs 28.1.  I thought we wanted to delete
those, leaving just the news of the single release?

Also, is it possible not to number the items?  The Emacs 28.2 NEWS are
short, so it doesn't matter much, but for a major release the numbers
get to ridiculous values, IMO.  Moreover, people will start reporting
issues referencing these numbers, and we will have to reverse-engineer
them back to NEWS items.

HTML does support itemized lists without numbers, so I think I'd
prefer those instead of numbered lists.

> Note that we currently have some issues with adding <code> markers
> around single quoted strings.  I think it's hard to make the export
> perfect given our current NEWS file format, where the markup is not
> unambiguous.

Get ready for bug reports...

Thanks.

Re: Generate NEWS.XX.Y.html file for Emacs website

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> It still included NEWS for Emacs 28.1.  I thought we wanted to delete
> those, leaving just the news of the single release?

The idea came up to just have a single page per major version, which
incidentally simplified generating the files.  (I was particularly happy
about the latter.)

That's why I added the headlines for the versions, and also anchors so
we would link to:

    https://www.gnu.org/software/emacs/news/NEWS.28.html#28.1
    https://www.gnu.org/software/emacs/news/NEWS.28.html#28.2

> Also, is it possible not to number the items?  The Emacs 28.2 NEWS are
> short, so it doesn't matter much, but for a major release the numbers
> get to ridiculous values, IMO.  Moreover, people will start reporting
> issues referencing these numbers, and we will have to reverse-engineer
> them back to NEWS items.

I think we can remove the numbers.  I only kept them because it felt
easier to navigate such a large file with them, but I'm happy either way.

(The change would just be one liner to the org header.)

> Get ready for bug reports...

Of course.  No good deed goes unpunished.  ;-)

Re: Generate NEWS.XX.Y.html file for Emacs website

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Sat, 17 Sep 2022 16:08:07 -0400
> Cc: emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > It still included NEWS for Emacs 28.1.  I thought we wanted to delete
> > those, leaving just the news of the single release?
> 
> The idea came up to just have a single page per major version, which
> incidentally simplified generating the files.  (I was particularly happy
> about the latter.)

I understand, but I don't think it's TRT.  At the very least there
should be a prominent separator between the current release and the
previous ones, because it's all too easy to mistakenly go to those
older sections thinking they belong to the current release.

See, this is exactly why I said from the get-go that we should only
have NEWS for major releases -- that we will show the humongous NEWS
for minor releases (because it's easy), and people will get confused.

> That's why I added the headlines for the versions, and also anchors so
> we would link to:
> 
>     https://www.gnu.org/software/emacs/news/NEWS.28.html#28.1
>     https://www.gnu.org/software/emacs/news/NEWS.28.html#28.2

I only know that I quickly found myself crossing the line.  And I'm
supposed to know this file almost by heart.

> > Also, is it possible not to number the items?  The Emacs 28.2 NEWS are
> > short, so it doesn't matter much, but for a major release the numbers
> > get to ridiculous values, IMO.  Moreover, people will start reporting
> > issues referencing these numbers, and we will have to reverse-engineer
> > them back to NEWS items.
> 
> I think we can remove the numbers.  I only kept them because it felt
> easier to navigate such a large file with them, but I'm happy either way.
> 
> (The change would just be one liner to the org header.)

Thanks, let's do it, then.

> > Get ready for bug reports...
> 
> Of course.  No good deed goes unpunished.  ;-)

None whatsoever.