A new online publishing tool for Texinfo documents.

A new online publishing tool for Texinfo documents.

[Nic Ferrier]

Bob Chasell and I have been discussing a new way of making Texinfo
available online. I am planning to build what we have discussed over
December.

This message has been sent to all interested parties.

The aims of the new system are to make a web based info reader that
would be as good as the console based Info reader and that can be
used by people with slow Internet connections and free software web
browsers (Mozilla, Galeon, etc... as well as Emacs/W3, Lynx, etc...)


Our plan is to do this:

1 alter makeinfo --xml so that it splits the XML by Texinfo
  nodes. The --no-split switch will cause the existing output (or
  maybe add a new switch for the new XML output?)

2 write an XSLT stylesheet that transforms the chunked XML into
  specialized HTML;

  the specialized HTML will include Javascript to emulate Info
  navigation, index lookups, etc...

3 write a CGI script that will lookup a regular expression in the HTML
  files and return either a list of hits or the HTML file containing
  the Nth hit (N being an optional argument supplied to the CGI
  script)

4 write a shell script for linking all this together, and possibly to
  auto-magically install the produced files into an Apache webserver
  (other webservers to be supported as and when I have time).


The shell script(4) will be dependant on a tool called xsltproc which
comes with the GNOME libxsl library and is quite commonly available
on free software machines. xsltproc runs on all free operating
systems and also some non-free ones such as Windoze (but obviously we
don't care much about those  /8-)



I personally don't think this will deprecate the existing HTML output
from makeinfo because that has good support for ALL browsers.



There is one big problem with the current plan:

Emacs/W3 and Lynx do not support Javascript so we will have to find
another way of binding actions to keys within the HTML pages
downloaded to those browsers.

Does anyone have any bright ideas about that?


I am considering the potential of adding Mozilla's Javascript engine
to Emacs and Lynx which would solve this problem. The licence of
Mozilla's Javascript engine is compatible with the GPL. However, I
think this might be rather a big job, certainly bigger than the new
publishing system for online Texinfo.

 
On the subject of Lynx, has anyone tried this version of Links?

  http://atrey.karlin.mff.cuni.cz/~clock/twibright/links/

It apparently has Javascript support and is GPLed.


Nic

Re: A new online publishing tool for Texinfo documents.

[Richard Stallman]

    The aims of the new system are to make a web based info reader that
    would be as good as the console based Info reader and that can be
    used by people with slow Internet connections and free software web
    browsers (Mozilla, Galeon, etc... as well as Emacs/W3, Lynx, etc...)

Is this intended as a replacement for Info format, or as another
format to be used alongside it?  It can't do any harm as an additional
format.

    I am considering the potential of adding Mozilla's Javascript engine
    to Emacs and Lynx which would solve this problem.

Although the license permits adding this to Emacs, I certainly
don't want to do it.

Re: A new online publishing tool for Texinfo documents.

[Nic Ferrier]

Richard Stallman <rms@gnu.org> writes:

>     The aims of the new system are to make a web based info reader that
>     would be as good as the console based Info reader and that can be
>     used by people with slow Internet connections and free software web
>     browsers (Mozilla, Galeon, etc... as well as Emacs/W3, Lynx, etc...)
> 
> Is this intended as a replacement for Info format, or as another
> format to be used alongside it?  It can't do any harm as an additional
> format.

It's an additional format to solve the specific problem of reading
documentation that you don't happen to have on your local machine.


 
>     I am considering the potential of adding Mozilla's Javascript engine
>     to Emacs and Lynx which would solve this problem.
> 
> Although the license permits adding this to Emacs, I certainly
> don't want to do it.

That's interesting.

If it isn't done then it's likely that the W3 browser will be less
and less useful as more and more javascript looks a likely trend.


Nic

Re: A new online publishing tool for Texinfo documents.

[Kevin Rodgers]

Nic Ferrier wrote:

> It's an additional format to solve the specific problem of reading
> documentation that you don't happen to have on your local machine.

Why can't ange-ftp, tramp, or http-get be used to retrieve remote Info
files for local browsing within Emacs?

-- 
Kevin Rodgers

Re: A new online publishing tool for Texinfo documents.

[Nic Ferrier]

Kevin Rodgers <ihs_4664@yahoo.com> writes:

> Nic Ferrier wrote:
> 
> > It's an additional format to solve the specific problem of reading
> > documentation that you don't happen to have on your local machine.
> 
> Why can't ange-ftp, tramp, or http-get be used to retrieve remote Info
> files for local browsing within Emacs?

Because it would take a lot of work to alter the info reader to make
that happen.

And users would then be tied to emacs for doing it... The current
proposal offers at least as much flexibility as current Info
publishing systems:

- it will be available to browsers
- it will therefore be available to emacs
- it will also therefore be available on the console (links)


Nic

Re: A new online publishing tool for Texinfo documents.

[Kim F. Storm]

Nic Ferrier <nferrier@tapsellferrier.co.uk> writes:

> Kevin Rodgers <ihs_4664@yahoo.com> writes:
> 
> > Nic Ferrier wrote:
> > 
> > > It's an additional format to solve the specific problem of reading
> > > documentation that you don't happen to have on your local machine.
> > 
> > Why can't ange-ftp, tramp, or http-get be used to retrieve remote Info
> > files for local browsing within Emacs?
> 
> Because it would take a lot of work to alter the info reader to make
> that happen.

I doubt that it will take more work than adding XSLT and javascript
support to and otherwise enhancing emacs/w3.

> 
> And users would then be tied to emacs for doing it... 

What's wrong about that?
This is emacs-devel which is supposed to deal with emacs development!

>                                                       The current
> proposal offers at least as much flexibility as current Info
> publishing systems:
> 
> - it will be available to browsers
> - it will therefore be available to emacs
> - it will also therefore be available on the console (links)

so:

- this discussion will continue outside emacs-devel, right?

Thank you.

++kfs

Re: A new online publishing tool for Texinfo documents.

[Robert J. Chassell]

Kevin Rodgers <ihs_4664@yahoo.com> asked

   Why can't ange-ftp, tramp, or http-get be used to retrieve remote
   Info files for local browsing within Emacs?

Because all the processing is done on the local machine, so for some
actions, like navigating via search the whole file must be downloaded.

Info can have a slow connection between the display and the rendering
engine, but presumes a fast connection between the rendering engine
and the source of the Info document.

On the other hand, a Web browser usually has a fast connection between
its display and the rendering engine -- it is very hard to use a mouse
with a slow connection, and commonly used Web browsers use mice -- but
in most cases, a Web browser has a slow connection between the
rendering engine and the source of the HTML: the bandwidth is that of
a telephone dial up line.  People with wider bandwidths make up a
small fraction of the over all Internet population.

The idea for this new tool, among other goals, is to cause the
computer holding the source of the HTML to do some of the work on the
document, such as search through multiple files that make up the
document.  This requires a CGI script.

The Info system could be enhanced to do the same -- at least, I think
so.  Can anyone think why, if some of the work were done by the the
device that provides the Info document, you could not run Info
remotely over a slow connection between its rendering engine and the
source of the Info document?

The point with this proposal is that currently, HTML fails as a
documentation source in its most common situation, which is with a
dial up.  The goal is to make an HTML expression of a Texinfo manual
do as well as Info did back in the 1980s, at least.  (Also, in
commonly used Web browsers, to do index searches, to do `next',
`prev', `up', and `last' movements with default keybindings rather
than depend on mice, and maybe to enhance W3 mode or other Emacs mode,
to be a good browser.)

A different goal is to enhance Info to enable people to read documents
remotely, when `remote' means `low bandwidth connection between
renderer and Info doc source' rather than `low bandwidth connection
between display and renderer'.  I doubt this second goal is as
important as the first.

-- 
    Robert J. Chassell                         Rattlesnake Enterprises
    http://www.rattlesnake.com                  GnuPG Key ID: 004B4AC8
    http://www.teak.cc                             bob@rattlesnake.com
_______________________________________________
Texinfo home page: http://www.gnu.org/software/texinfo/
texinfo-pretest@texinfo.org
http://ff0.org/mailman/listinfo/texinfo-pretest

Re: Re: A new online publishing tool for Texinfo documents.

[Eli Zaretskii]

> Date: Tue, 25 Nov 2003 19:21:27 -0500 (EST)
> From: "Robert J. Chassell" <bob@rattlesnake.com>
> 
> The Info system could be enhanced to do the same -- at least, I think
> so.  Can anyone think why, if some of the work were done by the the
> device that provides the Info document, you could not run Info
> remotely over a slow connection between its rendering engine and the
> source of the Info document?

I don't see any problem in principle, except that the rendering engine
of the Info reader will have to be redesigned to work on output from
some external program that delivers small portions of the displayed
text, instead of loading the entire file into memory and working from
there.

We will also have to design some protocol between the local and
remote parts, or teach Info to use some existing protocol, to pass
requests to the remote.

_______________________________________________
Texinfo home page: http://www.gnu.org/software/texinfo/
texinfo-pretest@texinfo.org
http://ff0.org/mailman/listinfo/texinfo-pretest

Re: A new online publishing tool for Texinfo documents.

[Kevin Rodgers]

Robert J. Chassell wrote:

[completely missing the point]


Nic said, "It's an additional format to solve the specific problem of reading
documentation that you don't happen to have on your local machine."

First of all, I question whether an additional format is required, since that
will demand that Emacs and other Info readers be modified to support the new
format.


Second, everyone seems to agree that Info's search and navigation facilities
are superior to HTML; so shouldn't we take advantage of that?

Third, Info files can be rendered very quickly (especially compared to HTML)
because they have already been pre-processed from the Texinfo source, but
the download time for Info is no worse than HTML because it's not much bigger
than the source.  So use a network transport that's likely to be supported
by a remote server like ftp or http, and be happy you've got access to remote
documentation at all.  Searching an entire document, whether it's in Info
format or HTML and whether it's a monolithic entity or split into pieces
still requires that the data be downloaded.  So download Info files as
needed, and cache them in the /tmp filesystem.

BTW, I just did `g (/@foo:/usr/local/info/gcc)Invoking GCC' and
was able to use the `i' and `s' commands just the way I expected.

What am I missing?  It seems to me different people are trying to solve
different problems within this single discussion thread.

-- 
Kevin Rodgers

Re: A new online publishing tool for Texinfo documents.

[Robert J. Chassell]

   Nic said, "It's an additional format to solve the specific problem
   of reading documentation that you don't happen to have on your
   local machine."

Kevin Rodgers <ihs_4664@yahoo.com> replied

   First of all, I question whether an additional format is required,
   since that will demand that Emacs and other Info readers be
   modified to support the new format.

But the proposed HTML format has nothing to do with Info!  It is a
different HTML format.  Its intent is to enable standards compliant
Web browsers to read documents on the Web (and if Web servers put in
the requisit CGI, to enable Web browsers to navigate via search).

   Second, everyone seems to agree that Info's search and navigation
   facilities are superior to HTML; so shouldn't we take advantage of
   that?

Right, that is why Info should not be changed (unless someone wants
to enhance it by enabling slow connections between an Info doc server
and the Info renderer; right now the connection must be fast).

   Third, Info files can be rendered very quickly (especially compared
   to HTML) because they have already been pre-processed from the
   Texinfo source, but the download time for Info is no worse than
   HTML because it's not much bigger than the source.

I do not understand you.  It takes me 17 minutes to download the Emacs
Lisp Reference Manual.  When I do the manual in Info format, I then
later read it in Info.  I do not wait 17 minutes and then start
reading.

The goal with the HTML/CGI proposal is that if I were to browse the
manual remotely in a Web browser, I would need to download only small
parts -- nodes most likely -- quickly, so I would not have to wait 17
minutes.

   Searching an entire document, whether it's in Info format or HTML
   and whether it's a monolithic entity or split into pieces still
   requires that the data be downloaded.

No, it does not.  That is the point of a CGI script.  You do *not*
have to download a whole document if the serving computer does the
work for you.

-- 
    Robert J. Chassell                         Rattlesnake Enterprises
    http://www.rattlesnake.com                  GnuPG Key ID: 004B4AC8
    http://www.teak.cc                             bob@rattlesnake.com

Re: A new online publishing tool for Texinfo documents.

[Kevin Rodgers]

Robert J. Chassell wrote:

>    Nic said, "It's an additional format to solve the specific problem
>    of reading documentation that you don't happen to have on your
>    local machine."
> 
> Kevin Rodgers <ihs_4664@yahoo.com> replied
> 
>    First of all, I question whether an additional format is required,
>    since that will demand that Emacs and other Info readers be
>    modified to support the new format.
> 
> But the proposed HTML format has nothing to do with Info!  It is a
> different HTML format.  Its intent is to enable standards compliant
> Web browsers to read documents on the Web (and if Web servers put in
> the requisit CGI, to enable Web browsers to navigate via search).


I just wanted to point out that there are divergent goals being discussed
in this single thread.


>    Second, everyone seems to agree that Info's search and navigation
>    facilities are superior to HTML; so shouldn't we take advantage of
>    that?
> 
> Right, that is why Info should not be changed (unless someone wants
> to enhance it by enabling slow connections between an Info doc server
> and the Info renderer; right now the connection must be fast).


I don't understand why that is, i.e. why the server <-> client connection
speed for HTML is not adequate for Info as well.


>    Third, Info files can be rendered very quickly (especially compared
>    to HTML) because they have already been pre-processed from the
>    Texinfo source, but the download time for Info is no worse than
>    HTML because it's not much bigger than the source.
> 
> I do not understand you.  It takes me 17 minutes to download the Emacs
> Lisp Reference Manual.  When I do the manual in Info format, I then
> later read it in Info.  I do not wait 17 minutes and then start
> reading.
> 
> The goal with the HTML/CGI proposal is that if I were to browse the
> manual remotely in a Web browser, I would need to download only small
> parts -- nodes most likely -- quickly, so I would not have to wait 17
> minutes.


Right; but I assumed that the remote Info manual is already split into
relatively small files, and I should have said that an Info _node_ can
be displayed quickly (at least as quickly as an HTML page).  And my
little experiment showed that you can already browse a remote Info
manual, only downloading the subfiles as necessary, using ange-ftp.


>    Searching an entire document, whether it's in Info format or HTML
>    and whether it's a monolithic entity or split into pieces still
>    requires that the data be downloaded.
> 
> No, it does not.  That is the point of a CGI script.  You do *not*
> have to download a whole document if the serving computer does the
> work for you.

Indeed, I was wrong: even a remote Info file accessed via ange-ftp can

be searched incrementally.


If the goal is to make remote documentation available in current Info
browsers, they just need to be changed to recognize remote file syntax
(like ange-ftp, tramp, or http-get.el provide for Emacs).  If the goal
is to make documentation available for web browsers, makeinfo --html
already does that.  Right?

-- 
Kevin Rodgers

Re: A new online publishing tool for Texinfo documents.

[Robert J. Chassell]

   I just wanted to point out that there are divergent goals being discussed
   in this single thread.

Yes, good point.

   > No, it does not.  That is the point of a CGI script.  You do *not*
   > have to download a whole document if the serving computer does the
   > work for you.

   Indeed, I was wrong: even a remote Info file accessed via ange-ftp
   can be searched incrementally.

Are you sure it does this efficiently -- that is to say, will I be
able to navigate through a complete `Emacs Lisp Reference Manual' in
less than 17 seconds, or will it take me 17 minutes before my
`Info-search' expression has got to the last node?

My understanding is that ange-ftp/tramp downloads the file or files to
the client machine and the search is done by the client.


   If the goal is to make documentation available for web browsers,
   makeinfo --html already does that.  Right?

No, `makeinfo --html' does a poor job; you cannot move around a
document well.

-- 
    Robert J. Chassell                         Rattlesnake Enterprises
    http://www.rattlesnake.com                  GnuPG Key ID: 004B4AC8
    http://www.teak.cc                             bob@rattlesnake.com

Re: A new online publishing tool for Texinfo documents.

[Kevin Rodgers]

Robert J. Chassell wrote:

>    Indeed, I was wrong: even a remote Info file accessed via ange-ftp
>    can be searched incrementally.
> 
> Are you sure it does this efficiently -- that is to say, will I be
> able to navigate through a complete `Emacs Lisp Reference Manual' in
> less than 17 seconds, or will it take me 17 minutes before my
> `Info-search' expression has got to the last node?


17 minutes to download 711 KB?  The uncompressed Info files in the
21-2.8 Emacs Lisp Reference Manual are 2,462,047 bytes; they gzip down
to 728,336 bytes.  So I think you could download, gunzip, and search
the whole thing in a reasonable amount of time.


> My understanding is that ange-ftp/tramp downloads the file or files to
> the client machine and the search is done by the client.


Correct.


>    If the goal is to make documentation available for web browsers,
>    makeinfo --html already does that.  Right?
> 
> No, `makeinfo --html' does a poor job; you cannot move around a
> document well.

Well, that's a different problem.  First make sure it's generating the

best possible HTML, then worry about extending the markup language.

-- 
Kevin Rodgers