Have you all gone crazy? Was: On being web-friendly and why info must die

Have you all gone crazy? Was: On being web-friendly and why info must die

[Sven Axelsson]

I have watched several of the recent threads regarding changing the
documentation tool chain for no reason whatsoever from the sideline.
And I really can't understand what's going on. Texinfo looks to me as
a perfectly good format for writing structured documentation. The
markup is a bit chatty, but not unreasonably so. And none of the
suggested alternatives makes any /real/ difference for a potential
documentation writer as far as I can see. Most of what has been
discussed has /less/ functionality and /worse/ performance than what
is used currently. So, why?

If the only remaining reason for this change is to attract new, young
and hip contributors, maybe it is also time to rewrite Emacs in
Haskell and CoffeScript. That would surely bring in lots of new
contributors and breathe new life into the project. Right?

-- 
Sven Axelsson
++++++++++[>++++++++++>+++++++++++>++++++++++>++++++
>++++<<<<<-]>++++.+.++++.>+++++.>+.<<-.>>+.>++++.<<.
+++.>-.<<++.>>----.<++.>>>++++++.<<<<.>>++++.<----.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Paul Eggert]

On 12/17/2014 09:54 AM, Sven Axelsson wrote:
> Most of what has been
> discussed has/less/  functionality and/worse/  performance than what
> is used currently. So, why?

Yes, that's been a real obstacle.  I have not had time to look into the 
issue, and I assume ESR hasn't either.  So the problem festers, which is 
too bad, as Texinfo 5 is reeeallly slow.  Luckily I have not been 
updating documentation much lately, and this lessens my motivation for 
looking into the issue (though to be honest it's not clear which is 
cause and which is effect).

I've heard Asciidoctor (not Asciidoc) is fast -- does anybody who knows 
care to comment?

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Sven Axelsson <sven.axelsson@gmail.com> writes:

> If the only remaining reason for this change is to attract new, young
> and hip contributors, maybe it is also time to rewrite Emacs in
> Haskell and CoffeScript. That would surely bring in lots of new
> contributors and breathe new life into the project. Right?

Well, rewriting Emacs in Common Lisp has been done once or twice (I seem
to remember something called Hemlock for example).  The net attraction
for new developers as compared to the old ones did not really seem to
give those projects a significant boost as compared to Emacs.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stefan Monnier]

> Well, rewriting Emacs in Common Lisp has been done once or twice (I seem
> to remember something called Hemlock for example).  The net attraction
> for new developers as compared to the old ones did not really seem to
> give those projects a significant boost as compared to Emacs.

He didn't say Common Lisp, he said Haskell of CoffeeScript.
Of course, Common Lisp won't attract many new contributors, but just
think of the hordes of Haskell programmers,


        Stefan "I'd vote for Agda, instead, tho"

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Well, rewriting Emacs in Common Lisp has been done once or twice (I seem
>> to remember something called Hemlock for example).  The net attraction
>> for new developers as compared to the old ones did not really seem to
>> give those projects a significant boost as compared to Emacs.
>
> He didn't say Common Lisp, he said Haskell of CoffeeScript.
> Of course, Common Lisp won't attract many new contributors, but just
> think of the hordes of Haskell programmers,

Just what I needed.  An Emacs that will formally prove to me that
redisplay has finished.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stefan Monnier]

> Just what I needed.  An Emacs that will formally prove to me that
> redisplay has finished.

Better yet: a machine-checked proof that the redisplay will
always terminate.

Finally, an end to all those "I waited a year and redisplay still isn't
done" bug reports,


        Stefan

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Óscar Fuentes]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Better yet: a machine-checked proof that the redisplay will
> always terminate.
>
> Finally, an end to all those "I waited a year and redisplay still isn't
> done" bug reports,

You are away from your proof assistant, aren't you?

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Wed, 17 Dec 2014 17:14:05 -0500
> Cc: Sven Axelsson <sven.axelsson@gmail.com>, emacs <emacs-devel@gnu.org>
> 
> > Just what I needed.  An Emacs that will formally prove to me that
> > redisplay has finished.
> 
> Better yet: a machine-checked proof that the redisplay will
> always terminate.

Don't worry: I made sure this is unprovable.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Sven Axelsson]

Excellent responses, everybody! Just what I hoped for with this
thread. I have of course seen the arguments for changing the tool
chain. As for increased web presence, David Kastrup has shown what can
already be done with texinfo, and I have noticed several of you being
active on the texinfo bug list to raise the speed problem.

I also agree with Stefan that Agda would be an awesome choice for an
Emacs rewrite. The developer community is huge :)

-- 
Sven Axelsson
++++++++++[>++++++++++>+++++++++++>++++++++++>++++++
>++++<<<<<-]>++++.+.++++.>+++++.>+.<<-.>>+.>++++.<<.
+++.>-.<<++.>>----.<++.>>>++++++.<<<<.>>++++.<----.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Phillip Lord]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Just what I needed.  An Emacs that will formally prove to me that
>> redisplay has finished.
>
> Better yet: a machine-checked proof that the redisplay will
> always terminate.

That would be nice.

>
> Finally, an end to all those "I waited a year and redisplay still isn't
> done" bug reports,

Sadly this requires proof checkers that can distinguish between "will
terminate" and "will terminate within the known life-time of the
universe". That's a research problem I fear.

Phil

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Yuri Khan]

On Sat, Dec 20, 2014 at 2:26 AM, Phillip Lord
<phillip.lord@newcastle.ac.uk> wrote:

> Sadly this requires proof checkers that can distinguish between "will
> terminate" and "will terminate within the known life-time of the
> universe". That's a research problem I fear.

Actually “will terminate in <predefined finite time interval>” is a
decidable property while “will terminate” is not.

Although, in general, the finite-interval-termination problem is
solved by setting up a timer watchdog and running the subject program.
If it finishes first, then return t; if the watchdog barks first, then
kill the program and return nil.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Óscar Fuentes]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> On Sat, Dec 20, 2014 at 2:26 AM, Phillip Lord
> <phillip.lord@newcastle.ac.uk> wrote:
>
>> Sadly this requires proof checkers that can distinguish between "will
>> terminate" and "will terminate within the known life-time of the
>> universe". That's a research problem I fear.
>
> Actually “will terminate in <predefined finite time interval>” is a
> decidable property while “will terminate” is not.

This is true if and only if you don't live in this universe.

There is plenty of people who fits the bill, with Computer Scientists at
the front row.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stefan Monnier]

>> Finally, an end to all those "I waited a year and redisplay still isn't
>> done" bug reports,
> Sadly this requires proof checkers that can distinguish between "will
> terminate" and "will terminate within the known life-time of the
> universe". That's a research problem I fear.

That would be nice, but it's not needed, if we can simply assure the
poor user that she should just wait a bit more, because eventually it
will finish.


        Stefan

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Allen S. Rout]

On 12/17/2014 12:54 PM, Sven Axelsson wrote:
> I have watched several of the recent threads  [...] So, why?
> 

From the peanut gallery, I think the 'why' is expressed better in terms
of personal politics than anything technical.  The Git transition, I
think, has made Eric feel a great sense of ownership.  He's trying to
polish stuff up that he sees as dingy.

I don't think he's crazy.  I do think his actions look selfish and
self-aggrandizing.  Presenting a doc format change as a fait accomplis,
"I already talked to daddy, so here's what we're going to do..." is ...

Well, It's sufficiently obviously bad politics, so obviously counter to
FOSS decisionmaking patterns  (which Eric can articulate with the best
of them), that I really wonder if it was _designed_ to create the teapot
tempest which has evolved.    As a strategy to exhaust thought leaders,
a bikeshed storm works pretty well.

414 messages at the moment in my buffer on this thread.  I'm just glad
Stefan isn't bleeding much mental effort into the issue.



FWIW, IMO:  Keep info. There's nothing that's obviously better, and
arguing about it is a big waste of time.



- Allen S. Rout

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Phillip Lord]

"Allen S. Rout" <asr@ufl.edu> writes:

> On 12/17/2014 12:54 PM, Sven Axelsson wrote:
>> I have watched several of the recent threads  [...] So, why?
>> 
>
>>From the peanut gallery, I think the 'why' is expressed better in terms
> of personal politics than anything technical.  The Git transition, I
> think, has made Eric feel a great sense of ownership.  He's trying to
> polish stuff up that he sees as dingy.

The git translation was a good thing. And a tracker that allows bug
reports and pull requests to be handled and visible would be a good
thing. And finally, a documentation format that is easy to use and
familiar to the much of the world would be a good thing also.

Emacs has a difficult course to steer. That it respects its past is a
good thing, because the knowledge that I learn today will be useful in
five years time. Continually chasing the latest thing will result in a
lot of unnecessary churn. It also needs to change with the rest of the
world, or it will be left behind. Remaining relevant is not a given.

I use Gnus for email, because my email requirements haven't changed much
in 20 years (except for volume, and gnus is very good at that). But
org-mode has changed the way I manage notes and todo lists. And magit
has changed the way that I approach versioning. Stablity and change at
the same time.

If Eric is trying to stir things up a bit, that is surely not a bad
thing.

Phil

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Jay Belanger]

> The git translation was a good thing.

Yes, which is why people on the devel list had been asking to move away
from bzr for a while.  Even then, it took many pleas to the bzr
developers and for bzr to basically become defunct before we made the
switch to git.

The situation with texinfo is not similar.  While there have been
complaints about texinfo 5 being slow, I don't recall many (if any)
people asking to move away from texinfo, there has been movement in
texinfo to speed things up, and as far as I know texinfo isn't in danger
of becoming unmaintained.  It was very shocking and disappointing to
read that Richard had agreed to change formats to asciidoc.
It is even more surprising since TeXinfo is a GNU project, I believe.

Sadly, I think the best thing that can happen here is that the
discussion wastes everybody's time and we stick with texinfo.
Oh, and texinfo 5 speeds up.

Jay

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Jay Belanger <jay.p.belanger@gmail.com> writes:

>> The git translation was a good thing.
>
> Yes, which is why people on the devel list had been asking to move away
> from bzr for a while.  Even then, it took many pleas to the bzr
> developers and for bzr to basically become defunct before we made the
> switch to git.
>
> The situation with texinfo is not similar.  While there have been
> complaints about texinfo 5 being slow, I don't recall many (if any)
> people asking to move away from texinfo, there has been movement in
> texinfo to speed things up, and as far as I know texinfo isn't in danger
> of becoming unmaintained.  It was very shocking and disappointing to
> read that Richard had agreed to change formats to asciidoc.

Personally, I imagine that the agreement was, at least from one side,
intended to be about the assorted quasi-documentation text files in
.../etc, namely data-directory rather than the Info tree.  But that's
complete speculation on my part and there would have been plenty of
opportunity clearing this up had it been the case.

But at least that would have made some more sense to me as a test
balloon than aiming at all the Texinfo manuals in one fell swoop.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

phillip.lord@newcastle.ac.uk (Phillip Lord) writes:

> If Eric is trying to stir things up a bit, that is surely not a bad
> thing.

This sounds like a weird statement to make in this unqualified form.
Emacs is not in such a bad state that you can just "stir things up"
anywhere and expect an improvement to be the result.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Phillip Lord writes:

 > If Eric is trying to stir things up a bit, that is surely not a bad
 > thing.

"Double, double, toil and trouble!  Fire, burn, and cauldron, bubble!"
you mean?

Eric is stirring up nothing but trouble with his intemperate
vituperations.  Karl is a little more circumspect, but he is also
going to fail.  Of all free software philosophers, even more so than
RMS, *those two* should be well aware of the distinction between the
*software* and the *project*.  A work of free software is the world's,
'tis true, but nary a project be that be not "owned" by its
participants.

The right way to stir things up is to appeal to the choir, not to the
tourists gawking at the icons in the back of the hall.  The criterion
for appeal of a new documentation format is clear: present a proof of
concept translation of a "representative" Emacs manual[1] to the new
source format, along with built manuals in the target format(s) and
any tools needed to implement the desired navigation features.  The
cost is high, but experience shows that worthwhile moves usually have
redundant costs being paid.

For example, I've observed 6 VCS transitions closely.  In 3 cases
(including the current move of Emacs to git), the choice was based on
consensus of the involved developers, and only one conversion was done
(but note that Eric's conversion was not based on one of the existing
git mirrors, and was done a couple dozen times I guess).  In the other
3 cases, multiple repos were presented for consideration -- a lot of
redundant effort from one point of view.

In other cases (3 cases of issue tracker introduction), it was
universally agreed that "some" was better than "none".  In two cases,
projects just took the first thing that had a volunteer to implement
and run the tracker.  In the case of Emacs, however, there was a
strong demand that the existing email-centric workflow be extended,
and the only candidate with a proof-of-concept implementation that
satisfied that requirement was the current debbugs tracker.  That
despite protests that Bugzilla, Roundup, Trac, etc "can be" configured
to be controlled by email.  But no implementation was presented, and
debbugs won by default.

I suspect a careful study would substantiate such anecdotes.  For the
documentation format, the core members of the project clearly consider
the existing Texinfo manuals to be adequate (and often, excellent).
So there's no hurry to produce a proof of concept -- but I would say
one is necessary, and the cost is not exorbitant according to common
practice.


Footnotes: 
[1]  It needs to be an Emacs manual for ease of comparison.  I don't
think any of the naysayers would feel comfortable switching from
Texinfo based on the now out-of-date org-mode manual translation, for
example.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> The right way to stir things up is to appeal to the choir, not to the
> tourists gawking at the icons in the back of the hall.  The criterion
> for appeal of a new documentation format is clear: present a proof of
> concept translation of a "representative" Emacs manual[1] to the new
> source format, along with built manuals in the target format(s) and
> any tools needed to implement the desired navigation features.  The
> cost is high, but experience shows that worthwhile moves usually have
> redundant costs being paid.

There is actually another hidden hurdle that has not been mentioned: the
target format "Info" is not independent from the manual's organization
of content: content is organized into node-sized chunks, with a somewhat
hierarchical organization intended to make all non-bottom nodes fit on a
screen if feasible in order to make navigation fast.  However, this kind
of "fast" implies that not every following of a link requires
substantial time fetching and rerendering pages.  HTML (let alone http
and the Internet) is not intended for fast flipping back and forth
between independent pages, and the HTML browsers are not supposed to
deal with humongous pages comprising a whole manual either.

Finding tolerable compromises in organizing a manual into HTML-browsing
suitable form that is not in one of several ways more painful or awkward
to work with than avoidable requires a quite less hierarchical
organization than a good Info manual provides.

So it is not just a question of a flat conversion to get from a good
Info manual to a good HTML manual.

There have been people looking at the organization of the Emacs manual
in its HTML form in this thread who claimed that it is just awfully
badly organized and written.

But this impression is much more pervasive when using the manual in the
HTML renderings than in Info mode.  This kind of criticism might even
partly apply to the Info rendering, but if it does not significantly
impact working with the manual in its Info rendition, improvement
efforts tend to go somewhere else first.

The impression would likely be different if the typical HTML rendition
of a manual would closer resemble a folding editor or other non-flat but
still instantly accessible representation.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

David Kastrup writes:

 > There is actually another hidden hurdle that has not been
 > mentioned: the target format "Info" is not independent from the
 > manual's organization of content: content is organized into
 > node-sized chunks, with a somewhat hierarchical organization
 > intended to make all non-bottom nodes fit on a screen if feasible
 > in order to make navigation fast.

I don't think this is a big problem, though.  I don't see any reason
why the organization into "nodes" or "pages" (as in the original
intent of *nix "man page") would change.  It's the obvious way (at
least to me) to provide modularity in documentation to correspond to
the modularity of the program.

 > However, this kind of "fast" implies that not every following of a
 > link requires substantial time fetching and rerendering pages.
 > HTML (let alone http and the Internet) is not intended for fast
 > flipping back and forth between independent pages, and the HTML
 > browsers are not supposed to deal with humongous pages comprising a
 > whole manual either.

Sorry, David, but this is a *plus*, not a *minus*.  The Emacs manuals
will continue to be distributed with Emacs.  Users with a full Emacs
installed (OK, Debian users won't get them in the default "free"
distribution) will have local access to the manuals.  Local access is
plenty fast whether broken up into multiple files or as a single large
file, as applications like S5 prove.  I can't testify to "humongous"
files (eg, the Emacs Lisp manual), but historically those have been
divided for Info presentation, too.

So what you get with a web-friendly manual is accessibility (though
somewhat degraded) for those who *don't* have the manuals installed
locally.  For many of the advocates, it's not obvious that degraded
performance would be observed even for large manuals.  Even in Japan I
often get sub-500ms click-to-readable performance on pages that I
believe are hosted in the US (although that may be biased; the Python
manuals I consult frequently that way may be hosted on a CDN, and of
course Savannah access is often painful -- a Savannah-hosted manual
wouldn't be very useful to me).

 > Finding tolerable compromises in organizing a manual into HTML-browsing
 > suitable form that is not in one of several ways more painful or awkward
 > to work with than avoidable requires a quite less hierarchical
 > organization than a good Info manual provides.

Agreed, that is a problem that (to my knowledge, which is limited) has
not been solved.  I don't see why HTML is inherently less suited to
presenting non-hierarchical aspects of software documentation than
Info, though.  I believe that this problem is likely to be amenable to
a few Ecmascript tweaks and some CSS to get fast, agile navigation.
(Granted, I don't have proof of concept.)

I think it's quite undesirable to compromise on agile navigation.  But
I also think it's probably unnecessary.

 > There have been people looking at the organization of the Emacs
 > manual in its HTML form in this thread who claimed that it is just
 > awfully badly organized and written.

Awful is relative.  By and large software manuals suck pretty badly,
and Emacs is well to the "better" end of the continuum.  Given the
emphasis on "drive-by" contributions, I wonder if the critics are not
looking for tutorials rather than manuals.  The manuals distributed
with Emacs are not long on "how to" tutorials.

 > The impression would likely be different if the typical HTML
 > rendition of a manual would closer resemble a folding editor or
 > other non-flat but still instantly accessible representation.

It's possible achieve that.  Again I refer to "S5", for example.  But
that kind of presentation would require significant amounts of
Ecmascript.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Sat, Dec 20, 2014 at 11:22 AM, Stephen J. Turnbull
<stephen@xemacs.org> wrote:
> David Kastrup writes:
>
>  > There is actually another hidden hurdle that has not been
>  > mentioned: the target format "Info" is not independent from the
>  > manual's organization of content: content is organized into
>  > node-sized chunks, with a somewhat hierarchical organization
>  > intended to make all non-bottom nodes fit on a screen if feasible
>  > in order to make navigation fast.
>
> I don't think this is a big problem, though.  I don't see any reason
> why the organization into "nodes" or "pages" (as in the original
> intent of *nix "man page") would change.  It's the obvious way (at
> least to me) to provide modularity in documentation to correspond to
> the modularity of the program.


With HTML5 you can use "ajax" which can make it very fast to fetch
small nodes. You simply just update that information on the web page
(instead of fetching the whole web page).

http://en.wikipedia.org/wiki/Ajax_(programming)

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> On Sat, Dec 20, 2014 at 11:22 AM, Stephen J. Turnbull
> <stephen@xemacs.org> wrote:
>> David Kastrup writes:
>>
>>  > There is actually another hidden hurdle that has not been
>>  > mentioned: the target format "Info" is not independent from the
>>  > manual's organization of content: content is organized into
>>  > node-sized chunks, with a somewhat hierarchical organization
>>  > intended to make all non-bottom nodes fit on a screen if feasible
>>  > in order to make navigation fast.
>>
>> I don't think this is a big problem, though.  I don't see any reason
>> why the organization into "nodes" or "pages" (as in the original
>> intent of *nix "man page") would change.  It's the obvious way (at
>> least to me) to provide modularity in documentation to correspond to
>> the modularity of the program.
>
>
> With HTML5 you can use "ajax" which can make it very fast to fetch
> small nodes. You simply just update that information on the web page
> (instead of fetching the whole web page).
>
> http://en.wikipedia.org/wiki/Ajax_(programming)

A glimmer of possible solutions on the horizon does not seem like a
sufficient incentive to throw an established working large established
system overboard.

Emacs has enough to work on without being the loss leader for new
documentation approaches.  Emacs is where Texinfo and Info are a
best-fit solution, and 100% of Emacs users have the Emacs Info reader
available.  If one wants to experiment with alternative approaches,
something like the glibc documentation would make a better candidate as
it has much fewer inherent ties to the Texinfo/Info universe.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Sat, Dec 20, 2014 at 12:45 PM, David Kastrup <dak@gnu.org> wrote:
> Lennart Borgman <lennart.borgman@gmail.com> writes:
>
>> On Sat, Dec 20, 2014 at 11:22 AM, Stephen J. Turnbull
>> <stephen@xemacs.org> wrote:
>>> David Kastrup writes:
>>>
>>>  > There is actually another hidden hurdle that has not been
>>>  > mentioned: the target format "Info" is not independent from the
>>>  > manual's organization of content: content is organized into
>>>  > node-sized chunks, with a somewhat hierarchical organization
>>>  > intended to make all non-bottom nodes fit on a screen if feasible
>>>  > in order to make navigation fast.
>>>
>>> I don't think this is a big problem, though.  I don't see any reason
>>> why the organization into "nodes" or "pages" (as in the original
>>> intent of *nix "man page") would change.  It's the obvious way (at
>>> least to me) to provide modularity in documentation to correspond to
>>> the modularity of the program.
>>
>>
>> With HTML5 you can use "ajax" which can make it very fast to fetch
>> small nodes. You simply just update that information on the web page
>> (instead of fetching the whole web page).
>>
>> http://en.wikipedia.org/wiki/Ajax_(programming)
>
> A glimmer of possible solutions on the horizon does not seem like a
> sufficient incentive to throw an established working large established
> system overboard.

I think I have already said that this is not about throwing the
established formats overboard?!?

The solution I mentioned is not far away. It is actually quite easy to
implement today. (And it is all over the internet.)

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > The solution I mentioned is not far away. It is actually quite easy to
  > implement today. (And it is all over the internet.)

I have invited people to design an HTML-Info format that is a suitable
replacement for the existing Info format.  This is not trivial
but I think it can be done.  How about working on it?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Lennart Borgman writes:

 > With HTML5 you can use "ajax" which can make it very fast to fetch
 > small nodes. You simply just update that information on the web
 > page (instead of fetching the whole web page).

Sure.  However, doing AJAX well requires fairly sophisticated scripts,
of the size that are typically compressed into an unreadable mishmash
for web transport.  Do you have a GPL-compatible package in mind?  If
not, development would be somewhat expensive, though not terribly so.
It also would involve a local httpd to serve, or a separate format
for, local manuals.  It would also be rather expensive to add such
capability to Emacs itself.  If you want to maintain multiple back
ends, that could get rather messy if you need to keep it AJAX-
compatible.

I think all of these are likely to be resistance points without a
proof of concept implementation.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Sat, Dec 20, 2014 at 3:26 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
> Lennart Borgman writes:
>
>  > With HTML5 you can use "ajax" which can make it very fast to fetch
>  > small nodes. You simply just update that information on the web
>  > page (instead of fetching the whole web page).
>
> Sure.  However, doing AJAX well requires fairly sophisticated scripts,
> of the size that are typically compressed into an unreadable mishmash
> for web transport.  Do you have a GPL-compatible package in mind?

I think with HTML5 you do not need any package any longer. You just
write it in HTML5/EcmaScript. (I do these things very often now.)

> If you want to maintain multiple back
> ends, that could get rather messy if you need to keep it AJAX-
> compatible.

You can handle that very easily. Just write the normal HTML code and
then add EcmaScript events to the clickable links. If EcmaScript is
not turned on or not available then the normal HTML rules applies and
works. Otherwise the click event takes over and replaces the
appropriate part of the web page.

Of course you have to double the web page a bit. You have to write the
whole web page somewhere and the replaceable content somewhere else.
And you have to have code to bind these things together, but it does
not look like a very big job to me.

> I think all of these are likely to be resistance points without a
> proof of concept implementation.

Someone please give me a pointer to the code that writes the HTML
pages now. I have forgotten.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> Someone please give me a pointer to the code that writes the HTML
> pages now. I have forgotten.

That would be texi2any.  At the current point of contention at least.
And hopefully one separate file responsible for the HTML backend.

Probably what is in my installed version
/usr/share/texinfo/Texinfo/Convert/HTML.pm

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Lennart Borgman writes:

 > I think with HTML5 you do not need any package any longer. You just
 > write it in HTML5/EcmaScript. (I do these things very often now.)

If you say so, but a quick look at the current W3C recommendation for
HTML5 doesn't reveal anything like standard AJAX events, just the
now-ancient ones like onclick and so on.  I don't think HTML5 has
really changed anything in this respect: if you can do it with HTML5
you can do it with HTML4.

It's possible that adding a few lines of Ecmascript per link would do
the trick, but that sounds ugly to say the least.  And scripts, being
dynamic and possibly interacting with the network, require
error-handling to be robust.

 > On Sat, Dec 20, 2014 at 3:26 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:

 > > If you want to maintain multiple back ends, that could get rather
 > > messy if you need to keep it AJAX-compatible.
 > 
 > You can handle that very easily. Just write the normal HTML code

By "multiple backends" I mean media types like PDF and Info, not
different browsers.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

[-- Attachment #1: Type: text/plain, Size: 1003 bytes --]

On Dec 20, 2014 5:47 PM, "Stephen J. Turnbull" <stephen@xemacs.org> wrote:
>
> Lennart Borgman writes:
>
>  > I think with HTML5 you do not need any package any longer. You just
>  > write it in HTML5/EcmaScript. (I do these things very often now.)
>
> If you say so, but a quick look at the current W3C recommendation for
> HTML5 doesn't reveal anything like standard AJAX events, just the
> now-ancient ones like onclick and so on.  I don't think HTML5 has
> really changed anything in this respect: if you can do it with HTML5
> you can do it with HTML4.
>
> It's possible that adding a few lines of Ecmascript per link would do
> the trick, but that sounds ugly to say the least.  And scripts, being
> dynamic and possibly interacting with the network, require
> error-handling to be robust.

There is AddEventlistener, XMLHttpRequest, etc.

The normal way to add something like what I suggested would be to add a
class to each link and then use that to add event handlers. Very nice in my
opinion.

[-- Attachment #2: Type: text/html, Size: 1261 bytes --]

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Lennart Borgman writes:
 > On Dec 20, 2014 5:47 PM, "Stephen J. Turnbull" <stephen@xemacs.org> wrote:

 > > If you say so, but a quick look at the current W3C recommendation for
 > > HTML5 doesn't reveal anything like standard AJAX events, just the
 > > now-ancient ones like onclick and so on.  I don't think HTML5 has
 > > really changed anything in this respect: if you can do it with HTML5
 > > you can do it with HTML4.

 > There is AddEventlistener, XMLHttpRequest, etc.

Sure, but those are ancient Ecmascript and/or DOM features
(XMLHttpRequest is about 10 years old), not something added in HTML5
vs. earlier versions of HTML.[1]  As far as I can see, nothing has
changed: if we want robust reliable operation of Emacs manuals in
general purpose web browsers, we're going to want a well-tested,
maintained package that takes care of error handling, network
timeouts, and all those nitty-gritty details.

I can just imagine what the Bright. Shiny. Things. crowd will do if
Emacs publishes an HTMLized manual that tries to do AJAX and sucks at
it: laugh their asses off, and then go hack on Battle for Wesnoth.

With all due respect to your "I do this all the time" experience, I'll
believe it's "good enough" when I see a pretty big chunk of an Emacs
manual displayed in an implementation.


Footnotes: 
[1]  To be precise, HTML5 has made the DOM much more central to the
specification, but all the same things would have worked in HTML4
and/or XHTML4.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

[-- Attachment #1: Type: text/plain, Size: 707 bytes --]

On Dec 20, 2014 7:03 PM, "Stephen J. Turnbull" <stephen@xemacs.org> wrote:
>
> Lennart Borgman writes:
>  > On Dec 20, 2014 5:47 PM, "Stephen J. Turnbull" <stephen@xemacs.org>
wrote:
>
>  > There is AddEventlistener, XMLHttpRequest, etc.
>
> Sure, but those are ancient Ecmascript and/or DOM features
> (XMLHttpRequest is about 10 years old), not something added in HTML5
> vs. earlier versions of HTML.[1]  As far as I can see, nothing has
> changed:

Actually I am not sure about the changes in the standard. I notice however
things just work now. They did not really do that before (without those
famous libraries, like jQuery).

However there are still flaws. At the moment I just target Google Chrome.

[-- Attachment #2: Type: text/html, Size: 968 bytes --]

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > With HTML5 you can use "ajax" which can make it very fast to fetch
  > small nodes. You simply just update that information on the web page
  > (instead of fetching the whole web page).

Using Javascript has a downside.  We can do it, but we want to release
that Javascript code as an extension rather than have it in the page.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> David Kastrup writes:
>
>  > There is actually another hidden hurdle that has not been
>  > mentioned: the target format "Info" is not independent from the
>  > manual's organization of content: content is organized into
>  > node-sized chunks, with a somewhat hierarchical organization
>  > intended to make all non-bottom nodes fit on a screen if feasible
>  > in order to make navigation fast.
>
> I don't think this is a big problem, though.  I don't see any reason
> why the organization into "nodes" or "pages" (as in the original
> intent of *nix "man page") would change.  It's the obvious way (at
> least to me) to provide modularity in documentation to correspond to
> the modularity of the program.

Well, a lot of the complaints of people preferring man pages over info
pages significantly concern the organization of the _content_ rather
than a problem with the format.

The whole Git documentation is still available as Info manuals (thanks
to some Makefile targets and Docbook2x), but it is not really
_structurally_ Info-like material.

Perl documentation is structured into man pages, and this really strains
the concept, basically plastering chapters of a single manual across
separate man pages.

>  > However, this kind of "fast" implies that not every following of a
>  > link requires substantial time fetching and rerendering pages.
>  > HTML (let alone http and the Internet) is not intended for fast
>  > flipping back and forth between independent pages, and the HTML
>  > browsers are not supposed to deal with humongous pages comprising a
>  > whole manual either.
>
> Sorry, David, but this is a *plus*, not a *minus*.  The Emacs manuals
> will continue to be distributed with Emacs.  Users with a full Emacs
> installed (OK, Debian users won't get them in the default "free"
> distribution) will have local access to the manuals.  Local access is
> plenty fast whether broken up into multiple files or as a single large
> file, as applications like S5 prove.

For mostly text-centric stuff, maybe.  But that's what the HTML fans
loudly claim to be uncool.

Stuff like
<URL:http://lilypond.org/doc/v2.18/input/regression/collated-files.html>
is painful to scroll around even locally loaded.

> I can't testify to "humongous" files (eg, the Emacs Lisp manual), but
> historically those have been divided for Info presentation, too.

Once you work on the divided HTML form, finding stuff via plain text
search and/or index gets really painful.  And jumping around several
files refetches and rerenders them all the time.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

David Kastrup writes:

 > Well, a lot of the complaints of people preferring man pages over
 > info pages significantly concern the organization of the _content_
 > rather than a problem with the format.

I'm sure they do.  And I'm sure Emacs documentation could be vastly
improved in that respect.  But let me repeat myself: I don't know of
any software manual that really shines.  I know of a couple of great
textbooks and the occasional excellent tutorial, but that's quite
different from a reference manual.

 > The whole Git documentation is still available as Info [...].

 > Perl documentation is structured into man pages [...].

Praising with faint damns, again?

These have to be the poster children for "man page.s suck, *please* get
us a better format."  Turning them into Info make make them bearable.

 > > Sorry, David, but this is a *plus*, not a *minus*.  The Emacs manuals
 > > will continue to be distributed with Emacs.  Users with a full Emacs
 > > installed (OK, Debian users won't get them in the default "free"
 > > distribution) will have local access to the manuals.  Local access is
 > > plenty fast whether broken up into multiple files or as a single large
 > > file, as applications like S5 prove.
 > 
 > For mostly text-centric stuff, maybe.

S5 happily does the usual number of equations and graphs for a lecture
on economics (about 150 of the former and 50 of the latter, total of
200 images, for a typical 75 minute lecture with 50 slides).  There
are no noticable delays in Safari, Firefox, or Chrome.  That should be
sufficient for an Emacs documentation target format, I think.  (I
mostly still use LaTeX and PDF, though, because there's nothing like
AUCTeX for doing technical writing, thank you very much, David.)

 > But that's what the HTML fans loudly claim to be uncool.

I don't care what HTML fans claim to be uncool.  I care about what has
a chance of getting implemented in the Emacs project.  Once we get the
the HTML horse inside the walls, then the fanboys and gals can jump
out and do their damnedest to turn Emacs documentation psychedelic.
ISTM, though, that if it doesn't do text acceptably well it has no
chance of being accepted, and on the contrary, nobody who matters to
whether it is accepted (except you) will care if Lilypond docs are a
bit slow.

 > Once you work on the divided HTML form, finding stuff via plain
 > text search and/or index gets really painful.  And jumping around
 > several files refetches and rerenders them all the time.

Oh, sure.  Once again, we're agreeing violently.  The problems of
indexing and efficient node navigation need to be solved.  I'm just
saying that I believe they can be solved.  We just need someone who
cares enough about the format switch to step up to the plate and get
to first base with a proof of concept.

Unfortunately, I don't care that much.  I just want people to stop
vituperating about obsolete formats and minor inconveniences, and do
something useful if they care enough.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Stuff like
  > <URL:http://lilypond.org/doc/v2.18/input/regression/collated-files.html>
  > is painful to scroll around even locally loaded.

Could you expand on that point?  It is hard for me to look at that
myself and study the issue to understand the problem -- could you say
a few lines to explain the issue?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Stuff like
>   > <URL:http://lilypond.org/doc/v2.18/input/regression/collated-files.html>
>   > is painful to scroll around even locally loaded.
>
> Could you expand on that point?  It is hard for me to look at that
> myself and study the issue to understand the problem -- could you say
> a few lines to explain the issue?

It's a very long page, with text and images interspersed all the time.
The downloading and rendering for the images takes a while, changing the
browser's idea of the sizes all the time (maybe Texinfo should declare
the geometry of images in the HTML if it doesn't already).  As a result
of the large and potentially not yet completely rendered page (probably
corresponding to hundreds of printed pages), scrolling (via scroll bar)
is not responsive in a graphical browser.  Stuff like incremental
searches also tend to react rather sluggish.

Moving around the same stuff in Emacs Info tends to be rather quick,
even though Emacs has its own peculiarities moving across
larger-than-window images.

-- 
David Kastrup

RE: Have you all gone crazy? Was: On being web-friendly and why info must die

[Drew Adams]

> >   > <URL:http://lilypond.org/doc/v2.18/input/regression/collated-files.html>
> >   > is painful to scroll around even locally loaded.
> >
> > Could you expand on that point?
> 
> It's a very long page, with text and images interspersed all the
> time. The downloading and rendering for the images takes a while,
> changing the browser's idea of the sizes all the time (maybe Texinfo
> should declare the geometry of images in the HTML if it doesn't
> already).  As a result of the large and potentially not yet completely
> rendered page (probably corresponding to hundreds of printed pages),
> scrolling (via scroll bar) is not responsive in a graphical browser.
> Stuff like incremental searches also tend to react rather sluggish.

FWIW, I see no such sluggishness at that page, using Google Chrome.
The page loaded seemingly instantaneously, and both scroll-bar
scrolling and incremental search (C-f in Chrome) also seem
instantaneous.

Am I missing something?  Did you means something different from this?

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Drew Adams <drew.adams@oracle.com> writes:

>> >   > <URL:http://lilypond.org/doc/v2.18/input/regression/collated-files.html>
>> >   > is painful to scroll around even locally loaded.
>> >
>> > Could you expand on that point?
>> 
>> It's a very long page, with text and images interspersed all the
>> time. The downloading and rendering for the images takes a while,
>> changing the browser's idea of the sizes all the time (maybe Texinfo
>> should declare the geometry of images in the HTML if it doesn't
>> already).  As a result of the large and potentially not yet completely
>> rendered page (probably corresponding to hundreds of printed pages),
>> scrolling (via scroll bar) is not responsive in a graphical browser.
>> Stuff like incremental searches also tend to react rather sluggish.
>
> FWIW, I see no such sluggishness at that page, using Google Chrome.
> The page loaded seemingly instantaneously, and both scroll-bar
> scrolling and incremental search (C-f in Chrome) also seem
> instantaneous.
>
> Am I missing something?  Did you means something different from this?

Firefox here.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Sven Axelsson]

On 21 December 2014 at 17:47, Drew Adams <drew.adams@oracle.com> wrote:
>
> FWIW, I see no such sluggishness at that page, using Google Chrome.
> The page loaded seemingly instantaneously, and both scroll-bar
> scrolling and incremental search (C-f in Chrome) also seem
> instantaneous.
>
> Am I missing something?  Did you means something different from this?

FWIW, using Safari on OS X 10.10, the page displays after 1.5s, scrolling
and searching works immediately, and the page is fully loaded, with all
of the more than 1000 images, in about 14s.

-- 
Sven Axelsson
++++++++++[>++++++++++>+++++++++++>++++++++++>++++++
>++++<<<<<-]>++++.+.++++.>+++++.>+.<<-.>>+.>++++.<<.
+++.>-.<<++.>>----.<++.>>>++++++.<<<<.>>++++.<----.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Sun, Dec 21, 2014 at 6:33 PM, Sven Axelsson <sven.axelsson@gmail.com> wrote:
> On 21 December 2014 at 17:47, Drew Adams <drew.adams@oracle.com> wrote:
>>
>> FWIW, I see no such sluggishness at that page, using Google Chrome.
>> The page loaded seemingly instantaneously, and both scroll-bar
>> scrolling and incremental search (C-f in Chrome) also seem
>> instantaneous.
>>
>> Am I missing something?  Did you means something different from this?
>
> FWIW, using Safari on OS X 10.10, the page displays after 1.5s, scrolling
> and searching works immediately, and the page is fully loaded, with all
> of the more than 1000 images, in about 14s.


That is remarkable. However for pages with that many images I would
tell the image sizes in the HTML code and then do the actual loading
of the images in the scroll event:
https://developer.mozilla.org/en-US/docs/Web/API/window.onscroll

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> On Sun, Dec 21, 2014 at 6:33 PM, Sven Axelsson <sven.axelsson@gmail.com> wrote:
>> On 21 December 2014 at 17:47, Drew Adams <drew.adams@oracle.com> wrote:
>>>
>>> FWIW, I see no such sluggishness at that page, using Google Chrome.
>>> The page loaded seemingly instantaneously, and both scroll-bar
>>> scrolling and incremental search (C-f in Chrome) also seem
>>> instantaneous.
>>>
>>> Am I missing something?  Did you means something different from this?
>>
>> FWIW, using Safari on OS X 10.10, the page displays after 1.5s, scrolling
>> and searching works immediately, and the page is fully loaded, with all
>> of the more than 1000 images, in about 14s.
>
>
> That is remarkable. However for pages with that many images I would
> tell the image sizes in the HTML code and then do the actual loading
> of the images in the scroll event:
> https://developer.mozilla.org/en-US/docs/Web/API/window.onscroll

It is not exactly going to make moving around more pleasant when any
scrolling results in reloading/rendering.  Stuff like Google image
search does that, and while it delivers a fast first response,
scrolling/searching around gets more irritating.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Sun, Dec 21, 2014 at 7:01 PM, David Kastrup <dak@gnu.org> wrote:
> Lennart Borgman <lennart.borgman@gmail.com> writes:
>
>>
>> That is remarkable. However for pages with that many images I would
>> tell the image sizes in the HTML code and then do the actual loading
>> of the images in the scroll event:
>> https://developer.mozilla.org/en-US/docs/Web/API/window.onscroll
>
> It is not exactly going to make moving around more pleasant when any
> scrolling results in reloading/rendering.  Stuff like Google image
> search does that, and while it delivers a fast first response,
> scrolling/searching around gets more irritating.


That is maybe a slight misunderstanding. In the case we are talking
about the layout will be known to the browser since we have told the
height and width of the images. So scrolling is not affected. If the
image is not loaded there will be a gray rectangle there instead.

Of course loading the image may require some CPU cycles. I am not sure
(without testing), but in most cases I think you will not notice that
very much.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Sun, Dec 21, 2014 at 5:47 PM, Drew Adams <drew.adams@oracle.com> wrote:
>>
>> It's a very long page, with text and images interspersed all the
>> time. The downloading and rendering for the images takes a while,
>> changing the browser's idea of the sizes all the time (maybe Texinfo
>> should declare the geometry of images in the HTML if it doesn't
>> already).

Of course the size of the images should be declared in the HTML. Not
doing that create a whole lot of extra work for the web browser since
it otherwise has to recalculate the geometry all the time when a new
image gets loaded.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> On Sun, Dec 21, 2014 at 5:47 PM, Drew Adams <drew.adams@oracle.com> wrote:
>>>
>>> It's a very long page, with text and images interspersed all the
>>> time. The downloading and rendering for the images takes a while,
>>> changing the browser's idea of the sizes all the time (maybe Texinfo
>>> should declare the geometry of images in the HTML if it doesn't
>>> already).
>
> Of course the size of the images should be declared in the HTML. Not
> doing that create a whole lot of extra work for the web browser since
> it otherwise has to recalculate the geometry all the time when a new
> image gets loaded.

Well, the HTML looks like

<a href="79/lily-83620d4b.ly"><p>&lsquo;<tt>accidental-ancient.ly</tt>&rsquo;
</a>    <p>
 <a href="79/lily-83620d4b.ly">
  <img align="middle"
       border="0"
       src="79/lily-83620d4b.png"
       alt="[image of music]">
 </a>
</p></p>


so indeed the geometry appears to be absent.  Probably worth an
enhancement request on the Texinfo mailing list.  It would probably slow
down the HTML generation, though.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Yuri Khan]

On Sun, Dec 21, 2014 at 11:57 PM, David Kastrup <dak@gnu.org> wrote:

> Well, the HTML looks like
>
> <a href="79/lily-83620d4b.ly"><p>&lsquo;<tt>accidental-ancient.ly</tt>&rsquo;
> </a>    <p>
>  <a href="79/lily-83620d4b.ly">
>   <img align="middle"
>        border="0"
>        src="79/lily-83620d4b.png"
>        alt="[image of music]">
>  </a>
> </p></p>

What? That isn’t even valid HTML.

In this snippet, I count 2 instances of improper tag nesting, 1 use of
obsolete element, 2 uses of obsolete attributes and 1 unhelpful alt
text.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> On Sun, Dec 21, 2014 at 11:57 PM, David Kastrup <dak@gnu.org> wrote:
>
>> Well, the HTML looks like
>>
>> <a href="79/lily-83620d4b.ly"><p>&lsquo;<tt>accidental-ancient.ly</tt>&rsquo;
>> </a>    <p>
>>  <a href="79/lily-83620d4b.ly">
>>   <img align="middle"
>>        border="0"
>>        src="79/lily-83620d4b.png"
>>        alt="[image of music]">
>>  </a>
>> </p></p>
>
> What? That isn’t even valid HTML.
>
> In this snippet, I count 2 instances of improper tag nesting, 1 use of
> obsolete element, 2 uses of obsolete attributes and 1 unhelpful alt
> text.

Well, apart from the unhelpful alt text (which is not easy to make more
helpful, actually, given the way this is generated), that would be the
responsibility of texi2html.  Probably worth reporting to the Texinfo
list and/or proposing a fix.  Now <p> does not need to nest in HTML, and
I can't vouch definitely that the second </p> might not belong to some
starting <p> I have not cut&pasted.

But it's not really pretty and could probably be fixed by just removing
the generation of any </p>.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Yuri Khan]

On Mon, Dec 22, 2014 at 2:58 PM, David Kastrup <dak@gnu.org> wrote:

>>> <a href="79/lily-83620d4b.ly"><p>&lsquo;<tt>accidental-ancient.ly</tt>&rsquo;
>>> </a>    <p>
>>>  <a href="79/lily-83620d4b.ly">
>>>   <img align="middle"
>>>        border="0"
>>>        src="79/lily-83620d4b.png"
>>>        alt="[image of music]">
>>>  </a>
>>> </p></p>

> Well, apart from the unhelpful alt text (which is not easy to make more
> helpful, actually, given the way this is generated), that would be the
> responsibility of texi2html.

Sure. I’m just amazed that this bug exists and has likely existed
since the HTML output was introduced and no one has noticed.

> Probably worth reporting to the Texinfo
> list and/or proposing a fix.  Now <p> does not need to nest in HTML, and
> I can't vouch definitely that the second </p> might not belong to some
> starting <p> I have not cut&pasted.

<p>’s *cannot* nest. By definition, each next <p> causes the previous
unclosed <p> to auto-close.

> But it's not really pretty and could probably be fixed by just removing
> the generation of any </p>.

That would be just ugly.

I’m a firm believer in closing tags explicitly and properly.
Especially when the markup is not hand-written but generated. There’s
just no excuse for generating invalid or sloppy markup.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Mon, Dec 22, 2014 at 6:37 PM, Yuri Khan <yuri.v.khan@gmail.com> wrote:
> On Mon, Dec 22, 2014 at 2:58 PM, David Kastrup <dak@gnu.org> wrote:
>
>>>> <a href="79/lily-83620d4b.ly"><p>&lsquo;<tt>accidental-ancient.ly</tt>&rsquo;
>>>> </a>    <p>
>>>>  <a href="79/lily-83620d4b.ly">
>>>>   <img align="middle"
>>>>        border="0"
>>>>        src="79/lily-83620d4b.png"
>>>>        alt="[image of music]">
>>>>  </a>
>>>> </p></p>
>
>> Well, apart from the unhelpful alt text (which is not easy to make more
>> helpful, actually, given the way this is generated), that would be the
>> responsibility of texi2html.
>
> Sure. I’m just amazed that this bug exists and has likely existed
> since the HTML output was introduced and no one has noticed.


I took a quick look at Texinfo::Convert::HTML.

To my surprise I see that the HTML nodes are not generated by some
general function and just recursive calls.

Is there any reason for this? Doesn't it make it much harder to
maintain and change?

The reason I looked at it is that the current HTML code is a bit too
meager to enhance with JS. There ought to be classes, wrappers etc.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Steinar Bang]

>>>>> Yuri Khan <yuri.v.khan@gmail.com>:

> I’m a firm believer in closing tags explicitly and properly.
> Especially when the markup is not hand-written but generated. There’s
> just no excuse for generating invalid or sloppy markup.

+1

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

[-- Attachment #1: Type: text/plain, Size: 613 bytes --]

On Dec 23, 2014 10:28 AM, "Steinar Bang" <sb@dod.no> wrote:
>
> >>>>> Yuri Khan <yuri.v.khan@gmail.com>:
>
> > I’m a firm believer in closing tags explicitly and properly.
> > Especially when the markup is not hand-written but generated. There’s
> > just no excuse for generating invalid or sloppy markup.
>
> +1

In html5 there is also single tags (whatever they call them,I don't
remember right now).

The preferred way is to write <hr> instead of <hr />.

Btw, this is something html5 indentation code must take care of. It must
have a list of those tags to be able to do that, of course.

[-- Attachment #2: Type: text/html, Size: 871 bytes --]

[OT] HTML5

[Ivan Shmakov]

>>>>> Lennart Borgman <lennart.borgman@gmail.com> writes:
>>>>> On Dec 23, 2014 10:28 AM, "Steinar Bang" <sb@dod.no> wrote:
>>>>> Yuri Khan <yuri.v.khan@gmail.com>:

 >>> I’m a firm believer in closing tags explicitly and properly.
 >>> Especially when the markup is not hand-written but generated.
 >>> There’s just no excuse for generating invalid or sloppy markup.

 >> +1

 > In html5 there is also single tags (whatever they call them, I don't
 > remember right now).

	In HTML5, they’re called /void elements./ [1]

    […] Void elements only have a start tag; end tags must not be
    specified for void elements. […]

 > The preferred way is to write <hr> instead of <hr />.

	The HTML5 TR is mainly concerned with document /models/ (rather
	than document /markup/), and explicitly allows for /two/
	different representations (that is: different markup languages,
	essentially) of such models, – HTML and XHTML [2]:

    There are various concrete syntaxes that can be used to transmit
    resources that use this abstract language, two of which are defined
    in this specification.

	In HTML, /both/ of the above are the valid representations of a
	void element.  In XHTML, the valid representations for a void
	‘hr’ element would be <hr /> and <hr></hr>.

	The <hr /> form comes useful if, for some reason, you have to
	make a single file processable by both HTML and XML readers.
	(For the same reason, HTML5 explicitly allows the use of the
	otherwise meaningless xml:lang and xmlns attributes with the
	HTML syntax.)  There’s no other use or preference regarding
	these representations.

 > Btw, this is something html5 indentation code must take care of.  It
 > must have a list of those tags to be able to do that, of course.

	Yes.

[1] http://www.w3.org/TR/html5/syntax.html#elements-0
[2] http://www.w3.org/TR/html5/introduction.html#html-vs-xhtml

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Steinar Bang]

>>>>> Lennart Borgman <lennart.borgman@gmail.com>:

>> +1

> In html5 there is also single tags (whatever they call them,I don't
> remember right now).

My first version was a +1 to Yuri's comment followed by a rant on this
part of HTML5.

I deleted the rant, but maybe I shouldn't have...?

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Tue, Dec 23, 2014 at 1:41 PM, Steinar Bang <sb@dod.no> wrote:
>>>>>> Lennart Borgman <lennart.borgman@gmail.com>:
>
>>> +1
>
>> In html5 there is also single tags (whatever they call them,I don't
>> remember right now).
>
> My first version was a +1 to Yuri's comment followed by a rant on this
> part of HTML5.
>
> I deleted the rant, but maybe I shouldn't have...?

No idea, Steiner. I never understood what rants are good for.

Unless people are very, very upset, of course. ;-)

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Yuri Khan]

On Tue, Dec 23, 2014 at 6:41 PM, Steinar Bang <sb@dod.no> wrote:

>> In html5 there is also single tags (whatever they call them,I don't
>> remember right now).
>
> My first version was a +1 to Yuri's comment followed by a rant on this
> part of HTML5.
>
> I deleted the rant, but maybe I shouldn't have...?

Oh, I have a perpetual rant at the HTML syntax in general, with all
its optional tags, voodoo rules for recovering from bad markup, the
adoption agency algorithm, and suchlike. If something is broken, it
should be visibly broken. I’d rather HTML were obsoleted in favor of
XHTML. Sadly, W3C does not feel that way.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Tue, Dec 23, 2014 at 3:35 PM, Yuri Khan <yuri.v.khan@gmail.com> wrote:
> On Tue, Dec 23, 2014 at 6:41 PM, Steinar Bang <sb@dod.no> wrote:
>
>>> In html5 there is also single tags (whatever they call them,I don't
>>> remember right now).
>>
>> My first version was a +1 to Yuri's comment followed by a rant on this
>> part of HTML5.
>>
>> I deleted the rant, but maybe I shouldn't have...?
>
> Oh, I have a perpetual rant at the HTML syntax in general, with all
> its optional tags, voodoo rules for recovering from bad markup, the
> adoption agency algorithm, and suchlike. If something is broken, it
> should be visibly broken. I’d rather HTML were obsoleted in favor of
> XHTML. Sadly, W3C does not feel that way.

Ah, you are wasting your time! Look into the peculiarities of CSS instead ...

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Oh, I have a perpetual rant at the HTML syntax in general,

  > Ah, you are wasting your time! Look into the peculiarities of CSS instead ...

Could we avoid sending these messages, please?
They are not pertinent to the topic, but they inject a dose
of negativity into the discussion.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

texi2html output validity

[Ivan Shmakov]

[-- Attachment #1: Type: text/plain, Size: 2475 bytes --]

>>>>> David Kastrup <dak@gnu.org> writes:
>>>>> Yuri Khan <yuri.v.khan@gmail.com> writes:
>>>>> On Sun, Dec 21, 2014 at 11:57 PM, David Kastrup <dak@gnu.org> wrote:

 >>> Well, the HTML looks like

 >>> <a href="79/lily-83620d4b.ly"><p>&lsquo;<tt>accidental-ancient.ly</tt>&rsquo;
 >>> </a>    <p>
 >>>  <a href="79/lily-83620d4b.ly">
 >>>   <img align="middle"
 >>>        border="0"
 >>>        src="79/lily-83620d4b.png"
 >>>        alt="[image of music]">
 >>>  </a>
 >>> </p></p>

 >> What?  That isn’t even valid HTML.

 >> In this snippet, I count 2 instances of improper tag nesting,

	I count just a single one, but yes, that second </p> surely
	invalidates the fragment.

 >> 1 use of obsolete element, 2 uses of obsolete attributes and 1
 >> unhelpful alt text.

 > Well, apart from the unhelpful alt text (which is not easy to make
 > more helpful, actually, given the way this is generated), that would
 > be the responsibility of texi2html.  Probably worth reporting to the
 > Texinfo list

	(Hopefully they allow posts from unsubscribed addresses.)

 > and/or proposing a fix.

	I hereby suggest that:

	• the <code /> element is /always/ used instead of <tt />;

	• <img align="ALIGN" border="0" … /> is replaced with
	  <img style="text-align: ALIGN; border: 0;" … />;

	• unless there’s a really good reason to nest <p /> inside an
	  <a />, – do it in reverse: <p ><a …></a>; for one thing, this
	  makes it possible to simply omit any </p>s on output.

	FWIW, the fixed variant (MIMEd) of the example Texinfo-generated
	HTML code (above) fits my reading of the HTML5 TR, and appears
	to satisfy the W3C markup validation service [1] just fine.

 > Now <p> does not need to nest in HTML, and I can't vouch definitely
 > that the second </p> might not belong to some starting <p> I have not
 > cut&pasted.

	As <p /> elements do not nest in HTML, there should /never/ be
	such a “second” </p>.

 > But it's not really pretty and could probably be fixed by just
 > removing the generation of any </p>.

	The TR does /not/ allow one to omit the closing tag when the
	<p /> element is nested /inside/ an <a />.  Either we do it the
	other way around (as suggested above), or we still need to close
	<p /> explicitly, – in that very case, at the least.

[1] http://validator.w3.org/

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

[-- Attachment #2: Type: text/html, Size: 304 bytes --]

Re: texi2html output validity

[Lennart Borgman]

[-- Attachment #1: Type: text/plain, Size: 352 bytes --]

On Tue, Dec 23, 2014 at 11:37 AM, Ivan Shmakov <ivan@siamics.net> wrote:

>
>         I hereby suggest that:
>
>         • <img align="ALIGN" border="0" … /> is replaced with
>           <img style="text-align: ALIGN; border: 0;" … />;
>


No. ;-) Please never use inline styles. Use a class name instead (+ CSS
code for that class).

[-- Attachment #2: Type: text/html, Size: 975 bytes --]

Re: texi2html output validity

[Patrice Dumas]

On Tue, Dec 23, 2014 at 11:44:24AM +0100, Lennart Borgman wrote:
> On Tue, Dec 23, 2014 at 11:37 AM, Ivan Shmakov <ivan@siamics.net> wrote:
> 
> >
> >         I hereby suggest that:
> >
> >         • <img align="ALIGN" border="0" … /> is replaced with
> >           <img style="text-align: ALIGN; border: 0;" … />;
> >
> 
> 
> No. ;-) Please never use inline styles. Use a class name instead (+ CSS
> code for that class).

In makeinfo, we avoid CSS, though we still use some.  We use class,
except if INLINE_CSS_STYLE customization variable is set, in which case
CSS is inlined.

-- 
pat

Re: texi2html output validity

[Lennart Borgman]

On Tue, Dec 23, 2014 at 5:55 PM, Patrice Dumas <pertusus@free.fr> wrote:
> On Tue, Dec 23, 2014 at 11:44:24AM +0100, Lennart Borgman wrote:
>> On Tue, Dec 23, 2014 at 11:37 AM, Ivan Shmakov <ivan@siamics.net> wrote:
>>
>> >
>> >         I hereby suggest that:
>> >
>> >         • <img align="ALIGN" border="0" … /> is replaced with
>> >           <img style="text-align: ALIGN; border: 0;" … />;
>> >
>>
>>
>> No. ;-) Please never use inline styles. Use a class name instead (+ CSS
>> code for that class).
>
> In makeinfo, we avoid CSS, though we still use some.  We use class,
> except if INLINE_CSS_STYLE customization variable is set, in which case
> CSS is inlined.


It is nice to hear that inline style can be avoided, Patrice!

Is there a chance that you can add html attributes for height and
width on <img>-tags? I mean as read from the actual images.

(A quite different problem: I saw in Html.pm that the html code did
not seem to be written be some general function that could make sure
that tags are closed and well-formed etc. Is there any reason for
that?)

Re: texi2html output validity

[Yuri Khan]

On Tue, Dec 23, 2014 at 4:37 PM, Ivan Shmakov <ivan@siamics.net> wrote:
>  >>> <a href="79/lily-83620d4b.ly"><p>&lsquo;<tt>accidental-ancient.ly</tt>&rsquo;
>  >>> </a>    <p>
>  >>>  <a href="79/lily-83620d4b.ly">
>  >>>   <img align="middle"
>  >>>        border="0"
>  >>>        src="79/lily-83620d4b.png"
>  >>>        alt="[image of music]">
>  >>>  </a>
>  >>> </p></p>
>
>  >> In this snippet, I count 2 instances of improper tag nesting,
>
>         I count just a single one, but yes, that second </p> surely
>         invalidates the fragment.

<a><p></a> is improper* nesting in my book. All paired tags SHOULD**
be explicitly closed.

* note I did not say “invalid”
** as in RFC2119

>         I hereby suggest that:
>
>         • the <code /> element is /always/ used instead of <tt />;

Cursory reading of HTML.pm seems to indicate that <tt> is currently***
used for @key, @t, @verb, and some kinds of tables possibly related to
@example, @smallexample, @lisp and @smalllisp.

*** 5.2.0.dfsg.1-2 as packaged in Ubuntu 14.04

@key should be rendered as <kbd>, possibly with an additional class.
Yes, even when inside @kbd — HTML allows and encourages nesting <kbd>.

@t is a non-semantic command in Texinfo and should probably be
discouraged the same way <tt> has been discouraged in HTML since at
least 1997. It probably should become a <span class="t"> styled with
.t { font-family: monospace }.

@verb is syntax sugar for escaping characters which have special
meaning in Texinfo, and has a non-semantic side effect of fixed-width
rendering. It probably should become a <span class="verb">.

Code examples are a good match for <code>.

>         • <img align="ALIGN" border="0" … /> is replaced with
>           <img style="text-align: ALIGN; border: 0;" … />;

No. { border: 0 } should just be specified in CSS for all img, while
alignment should be handled by classes.

>         • unless there’s a really good reason to nest <p /> inside an
>           <a />, – do it in reverse: <p ><a …></a>; for one thing, this
>           makes it possible to simply omit any </p>s on output.

+1 for nesting <a> within <p>. -1 against omitting closing tags.


Note also that <tt> and <a>/<p> nesting order are just the tip of the
iceberg. The wider problem is that the Texinfo HTML generator
generally assumes HTML ≈3.2 even though it declares 4.01 Transitional:

* <a name> should be dropped in favor of placing an id on the parent element;
* alignment should be handled by classes;
* <table border=… cellpadding=… cellspacing=…>, <tr valign=…>, <td
align=…> should be replaced with CSS;
* tables should be generally avoided unless actually representing tabular data;
* table cells containing only non-breaking spaces indicate some
problem that should be solved, not worked around;
* a non-breaking space immediately adjacent to a normal space is nonsensical;
* more than one contiguous non-breaking space is a kludge;
* <br> are fit for poetry and postal addresses and almost nothing else;
* <font size=…> should be replaced with CSS;
* OUTPUT_ENCODING_NAME should be deprecated in favor of UTF-8;
* the encoding declaration <meta> should be the first thing in <head>;
* I might have missed something.

Re: texi2html output validity

[Lennart Borgman]

On Tue, Dec 23, 2014 at 3:29 PM, Yuri Khan <yuri.v.khan@gmail.com> wrote:
> On Tue, Dec 23, 2014 at 4:37 PM, Ivan Shmakov <ivan@siamics.net> wrote:
>>  >>> <a href="79/lily-83620d4b.ly"><p>&lsquo;<tt>accidental-ancient.ly</tt>&rsquo;
>>  >>> </a>    <p>
>>  >>>  <a href="79/lily-83620d4b.ly">
>>  >>>   <img align="middle"
>>  >>>        border="0"
>>  >>>        src="79/lily-83620d4b.png"
>>  >>>        alt="[image of music]">
>>  >>>  </a>
>>  >>> </p></p>
>>
>>  >> In this snippet, I count 2 instances of improper tag nesting,
>>
>>         I count just a single one, but yes, that second </p> surely
>>         invalidates the fragment.
>
> <a><p></a> is improper* nesting in my book. All paired tags SHOULD**
> be explicitly closed.


I think the problem that shows up there is the coding in Html.pm. It
does not keep starting and closing tags together. (The most easy way
to do that is to use functions where you add the starting tag on entry
and the closing tag on exit.)

Re: texi2html output validity

[Ivan Shmakov]

>>>>> Yuri Khan <yuri.v.khan@gmail.com> writes:
>>>>> On Tue, Dec 23, 2014 at 4:37 PM, Ivan Shmakov <ivan@siamics.net> wrote:

 >>>> In this snippet, I count 2 instances of improper tag nesting,

 >> I count just a single one, but yes, that second </p> surely
 >> invalidates the fragment.

 > <a><p></a> is improper* nesting in my book.  All paired tags SHOULD**
 > be explicitly closed.

 > * note I did not say “invalid”

	Yes; but /I/ did.  And the HTML5 TR agrees with me on that [1]:

    A p element's end tag may be omitted if the p element is immediately
    followed by an address, article, aside, blockquote, div, dl,
    fieldset, footer, form, h1, h2, h3, h4, h5, h6, header, hgroup, hr,
    main, nav, ol, p, pre, section, table, or ul, element, or if there
    is no more content in the parent element and the parent element is
    not an a element.

[1] http://www.w3.org/TR/html5/syntax.html#optional-tags

 > ** as in RFC2119

	The problem with “books” is that every other participant to this
	discussion will have his or her own one.

	My own preference is to write void elements like <hr />, and to
	spell out the opening and closing tags for all the other
	elements, – /unless/ there’s a reason not to (say, for
	educational purposes.)

	However, this is, to stress it out, – just my preference, – the
	very same thing that makes me disable global-font-lock-mode in
	Emacs, or to run Firefox with ECMAScript disabled (via NoScript)
	most of the time.  And the sole justification I have for this
	first preference is that it allows me to write valid HTML
	documents which are – at the very same time – well-formed XML.

	As for the software that I’m not the principal developer of, –
	I’d accept any output that does conform to the standards.

[…]

 > @key should be rendered as <kbd>, possibly with an additional class.
 > Yes, even when inside @kbd — HTML allows and encourages nesting
 > <kbd>.

	No objection.

 > @t is a non-semantic command in Texinfo and should probably be
 > discouraged the same way <tt> has been discouraged in HTML since at
 > least 1997.  It probably should become a <span class="t"> styled with
 > .t { font-family: monospace }.

 > @verb is syntax sugar for escaping characters which have special
 > meaning in Texinfo, and has a non-semantic side effect of fixed-width
 > rendering.  It probably should become a <span class="verb">.

	Given that either command is probably currently used for code
	fragments anyway, using <code /> (possibly with a class) sounds
	like a better solution.

[…]

 > Note also that <tt> and <a>/<p> nesting order are just the tip of the
 > iceberg. The wider problem is that the Texinfo HTML generator
 > generally assumes HTML ≈3.2 even though it declares 4.01
 > Transitional:

	… The question is: is it still necessary to offer HTML 3
	compatibility in the generated documents?

[…]

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: texi2html output validity

[Yuri Khan]

On Tue, Dec 23, 2014 at 9:37 PM, Ivan Shmakov <ivan@siamics.net> wrote:

>  >>>> In this snippet, I count 2 instances of improper tag nesting,
>
>  >> I count just a single one, but yes, that second </p> surely
>  >> invalidates the fragment.
>
>  > <a><p></a> is improper* nesting in my book.  All paired tags SHOULD**
>  > be explicitly closed.
>
>  > * note I did not say “invalid”
>
>         Yes; but /I/ did.  And the HTML5 TR agrees with me on that [1]:

So one is <a><p></a> and the other is </p></p>.

>         As for the software that I’m not the principal developer of, –
>         I’d accept any output that does conform to the standards.

Hmm. I would accept *for my own use* any output that conforms to the
standards, but I would be hesitant to distribute that output.

>  > [@t and @verb]
>
>         Given that either command is probably currently used for code
>         fragments anyway, using <code /> (possibly with a class) sounds
>         like a better solution.

No objection.

>         … The question is: is it still necessary to offer HTML 3
>         compatibility in the generated documents?

Whyfor? HTML 4.0 was blessed seventeen years ago. HTML 4.01, fifteen
years ago. HTML 5, a couple months ago.

Re: texi2html output validity

[Patrice Dumas]

Hello,

First of all it is a bit unclear where this html comes from.  In
general, both texi2html and texi2any/makeinfo, especially for makeinfo
starting at version 5 render properly nested html tags.  

On Tue, Dec 23, 2014 at 09:29:07PM +0700, Yuri Khan wrote:
> >
> >         • the <code /> element is /always/ used instead of <tt />;
> 
> Cursory reading of HTML.pm seems to indicate that <tt> is currently***
> used for @key, @t, @verb, and some kinds of tables possibly related to
> @example, @smallexample, @lisp and @smalllisp.

Use of <tt> in @example, @smallexample, @lisp and @smalllisp is for very
special case, something like a @table nested in those formats.

> *** 5.2.0.dfsg.1-2 as packaged in Ubuntu 14.04
> 
> @key should be rendered as <kbd>, possibly with an additional class.
> Yes, even when inside @kbd — HTML allows and encourages nesting <kbd>.

I am not convinced.  @key is semantically very diferent from @kbd which
is the same as <kbd>.  Indeed <kbd> is not for a keyboard key in HTML,
but for typed keyboard input.

> @t is a non-semantic command in Texinfo and should probably be
> discouraged the same way <tt> has been discouraged in HTML since at
> least 1997. It probably should become a <span class="t"> styled with
> .t { font-family: monospace }.

@t and other non-semantic commands are already discouraged in the manual.
But I see no point in not using <tt> for @t, as long as browser support
it (which is likely to be until the end of times). CSS is not supported
by every browser.

> @verb is syntax sugar for escaping characters which have special
> meaning in Texinfo, and has a non-semantic side effect of fixed-width
> rendering. It probably should become a <span class="verb">.

Once again this will only work if there is CSS support.  <tt> should
always work.  That being said, the best could be <tt class="verb">.

> Code examples are a good match for <code>.
> 
> >         • <img align="ALIGN" border="0" … /> is replaced with
> >           <img style="text-align: ALIGN; border: 0;" … />;
> 
> No. { border: 0 } should just be specified in CSS for all img, while
> alignment should be handled by classes.
> 
> >         • unless there’s a really good reason to nest <p /> inside an
> >           <a />, – do it in reverse: <p ><a …></a>; for one thing, this
> >           makes it possible to simply omit any </p>s on output.
> 
> +1 for nesting <a> within <p>. -1 against omitting closing tags.

If non valid HTML is emitted by makeinfo it is a bug, so no closing tag
omissions, no invalid nesting.  A <p> in <a> should be pretty rare, I
cannot really imagine Texinfo code that would lead to that.  But if you
have such code, don't hesitate to report it, we'll see what we can do.

> Note also that <tt> and <a>/<p> nesting order are just the tip of the
> iceberg. The wider problem is that the Texinfo HTML generator
> generally assumes HTML ≈3.2 even though it declares 4.01 Transitional:

No, there is a special code for HTML 3.2 compatibility, in
init/html32.pm, but in HTML.pm many 4.01 features are used.

I may have missed something, but in the specification of 4.01
Transitional all the element you describe as needing to be dropped
are accepted?  I see them all in
http://www.w3.org/TR/html401/sgml/loosedtd.html

Also, I think that, at least in the past, I checked the validity of
texi2html/makinfo output with some program and the HTML was valid.

We in fact choose that DTD in order to have a good rendering both in old
and new browser, with or without CSS.

> * <a name> should be dropped in favor of placing an id on the parent element;
> * alignment should be handled by classes;
> * <table border=… cellpadding=… cellspacing=…>, <tr valign=…>, <td
> align=…> should be replaced with CSS;
The previous 3 items seems to me to be ok in 4.01 Transitional.

> * tables should be generally avoided unless actually representing tabular data;
Agreed, but sometime we want to do some non-semantic formatting, for
instance for node lines, or indices.  <table> is very practical in that
case.

> * table cells containing only non-breaking spaces indicate some
> problem that should be solved, not worked around;
> * a non-breaking space immediately adjacent to a normal space is nonsensical;
> * more than one contiguous non-breaking space is a kludge;

Same as above, the use of non breaking spaces is for cases when we want
non-semantic formatting.

I agree that these are kludges, but current output is rendered ok on all
browsers we know about.  If there are proposals of other formatting that
can be used optionnally and do not involve tables and non breaking
spaces, and involve, for instance CSS, don't hesitate to propose.  But
it will be optional, the default is likely to be the kludges, as long as
they render nicely.

> * <br> are fit for poetry and postal addresses and almost nothing else;

We use it for @* and @sp as the semantics correspond.  Otherwise it is
rarely used, and always in cases when we do some non-semantic formatting
(author name, in indices formatting).

> * <font size=…> should be replaced with CSS;
> * OUTPUT_ENCODING_NAME should be deprecated in favor of UTF-8;

Default is utf-8 already, so no need to prevent setting
OUTPUT_ENCODING_NAME.  But otherwise we must stick to what the user
provides as @documentencoding.

> * the encoding declaration <meta> should be the first thing in <head>;

What would be the reason for that?

-- 
Pat

Re: texi2html output validity

[Ivan Shmakov]

>>>>> Patrice Dumas <pertusus@free.fr> writes:
>>>>> On Tue, Dec 23, 2014 at 09:29:07PM +0700, Yuri Khan wrote:

 >>> • the <code /> element is /always/ used instead of <tt />;

 >> Cursory reading of HTML.pm seems to indicate that <tt> is
 >> currently*** used for @key, @t, @verb, and some kinds of tables
 >> possibly related to @example, @smallexample, @lisp and @smalllisp.

 > Use of <tt> in @example, @smallexample, @lisp and @smalllisp is for
 > very special case, something like a @table nested in those formats.

	Could you please clarify this one?  Aren’t @example,
	etc. supposed to be used to format code examples?  If so, –
	the proper HTML element for their contents /is/ <code />.

[…]

 >> @key should be rendered as <kbd>, possibly with an additional class.
 >> Yes, even when inside @kbd — HTML allows and encourages nesting
 >> <kbd>.

 > I am not convinced.  @key is semantically very diferent from @kbd
 > which is the same as <kbd>.  Indeed <kbd> is not for a keyboard key
 > in HTML, but for typed keyboard input.

	The <kbd /> element is for both [1]:

    When the kbd element is nested inside another kbd element, it
    represents an actual key or other single unit of input as
    appropriate for the input mechanism.

    Here the kbd element is used to indicate keys to press:

<p>To make George eat an apple, press <kbd><kbd>Shift</kbd>+<kbd>F3</kbd></kbd></p>

[1] http://www.w3.org/TR/html5/text-level-semantics.html#the-kbd-element

[…]

 >> @verb is syntax sugar for escaping characters which have special
 >> meaning in Texinfo, and has a non-semantic side effect of
 >> fixed-width rendering.  It probably should become a
 >> <span class="verb">.

 > Once again this will only work if there is CSS support.  <tt> should
 > always work.  That being said, the best could be <tt class="verb">.

	I have no data on the actual usage of the @verb command, but if
	it’s used primarily for code fragments, <code class="verb" />
	would still be a better choice.

	Please note that in HTML, a “code fragment” is given a very
	broad definition [2]:

    The code element represents a fragment of computer code.  This could
    be an XML element name, a file name, a computer program, or any
    other string that a computer would recognize.

[2] http://www.w3.org/TR/html5/text-level-semantics.html#the-code-element

[…]

 > I may have missed something, but in the specification of 4.01
 > Transitional all the element you describe as needing to be dropped
 > are accepted?  I see them all in
 > http://www.w3.org/TR/html401/sgml/loosedtd.html

	The problem is that for a couple of months now, the most recent
	HTML version is HTML5, and it does /not/ specify these elements.

	Granted, there’re browsers which still support these, but I
	could easily imagine a new browser project being started that
	would just ignore all the cruft that was already scheduled to be
	removed back in the 20th century.

 > Also, I think that, at least in the past, I checked the validity of
 > texi2html/makinfo output with some program and the HTML was valid.

 > We in fact choose that DTD in order to have a good rendering both in
 > old and new browser,

	Is there a list of “old” browsers that Texinfo is bound to
	support and /why/?  Is it, say, still deemed worthwhile to
	support Arena and Mosaic?

 > with or without CSS.

	The nice thing about the (current) browsers with limited or no
	support for CSS is that they tend to lack support for fonts, and
	sometimes even for alignment, image borders (or images
	altogether), and such.  So, I expect there to be no actual loss
	in presentation after moving the formatting from HTML to CSS.

	Feel free to suggest counter-examples, though, – I by no means
	am a Web browser expert.

[…]

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: texi2html output validity

[Gavin Smith]

>         The problem is that for a couple of months now, the most recent
>         HTML version is HTML5, and it does /not/ specify these elements.
>
>         Granted, there’re browsers which still support these, but I
>         could easily imagine a new browser project being started that
>         would just ignore all the cruft that was already scheduled to be
>         removed back in the 20th century.
>
We're not bound to follow the latest HTML standard. If there become
incompatibilities between more popular, newer and more
standards-conformant browsers which are impractical to account for in
the HTML output, or if such incompatibilities exist at the moment,
then of course we should target what is the most common. At the moment
though it doesn't seem that inconvenient to provide for browsers with
limited CSS support or lacking support for newer tags.

Re: texi2html output validity

[David Kastrup]

Patrice Dumas <pertusus@free.fr> writes:

> Hello,
>
> First of all it is a bit unclear where this html comes from.  In
> general, both texi2html and texi2any/makeinfo, especially for makeinfo
> starting at version 5 render properly nested html tags.  

Ok, I've just pulled this example from the current LilyPond web sites,
and if I take a look at the header, it is

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<!-- Created on March 17, 2014 by texi2html 1.82
texi2html was written by: 
            Lionel Cons <Lionel.Cons@cern.ch> (original author)
            Karl Berry  <karl@freefriends.org>
            Olaf Bachmann <obachman@mathematik.uni-kl.de>
            and many others.
Maintained by: Many creative people.
Send bugs and suggestions to <texi2html-bug@nongnu.org>
-->

And texi2html --version indeed reports 1.82 on my own system, and I see

dak@lola:/usr/local/tmp/emacs$ which texi2html
/usr/bin/texi2html
dak@lola:/usr/local/tmp/emacs$ dpkg -S `which texi2html`
texi2html: /usr/bin/texi2html
dak@lola:/usr/local/tmp/emacs$ dpkg -l texi2html
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=================================
ii  texi2html      1.82+dfsg1-3 all          Convert Texinfo files to HTML

So this is definitely what current Ubuntu has, and
apt-get changelog texi2html

delivers

texi2html (1.82+dfsg1-3) unstable; urgency=low

  * QA upload.
  * Add dependency on libtext-unidecode-perl (Closes: #709906)

 -- Reinhard Tartler <siretart@tauware.de>  Wed, 29 May 2013 06:00:47 +0200

texi2html (1.82+dfsg1-2) unstable; urgency=low

  [ Colin Watson ]
  * QA upload.
  * Add build-indep and build-arch targets to debian/rules.
  * Adjust debian/watch to remove the +dfsg* part from the upstream version
    number when comparing with upstream.

  [ Steve Langasek ]
  * Mark texi2html Multi-Arch: foreign.

 -- Colin Watson <cjwatson@debian.org>  Wed, 22 May 2013 14:48:34 +0100


Does this make us any wiser?

-- 
David Kastrup

Re: texi2html output validity

[Yuri Khan]

On Tue, Dec 23, 2014 at 10:49 PM, Patrice Dumas <pertusus@free.fr> wrote:

> First of all it is a bit unclear where this html comes from.  In
> general, both texi2html and texi2any/makeinfo, especially for makeinfo
> starting at version 5 render properly nested html tags.

I haven’t checked. David Kastrup posted it as an example of how image
size is not declared, and I just assumed it was real Texinfo output.

>> @key should be rendered as <kbd>, possibly with an additional class.
>> Yes, even when inside @kbd — HTML allows and encourages nesting <kbd>.
>
> I am not convinced.  @key is semantically very diferent from @kbd which
> is the same as <kbd>.  Indeed <kbd> is not for a keyboard key in HTML,
> but for typed keyboard input.

http://www.w3.org/TR/html5/text-level-semantics.html#the-kbd-element says:

> When the kbd element is nested inside another kbd element, it represents an actual key or other single unit of input as appropriate for the input mechanism.


> @t and other non-semantic commands are already discouraged in the manual.
> But I see no point in not using <tt> for @t, as long as browser support
> it (which is likely to be until the end of times). CSS is not supported
> by every browser.

What browsers are there that do not support CSS *and* at the same time
have the capability of displaying proportional fonts?

Anyway this point is moot because the current HTML specification makes
CSS the only supported way of specifying presentation. Browsers that
do not yet support CSS had better keep up.

> I may have missed something, but in the specification of 4.01
> Transitional all the element you describe as needing to be dropped
> are accepted?  I see them all in
> http://www.w3.org/TR/html401/sgml/loosedtd.html

Yes, 4.01 Transitional allowed all that. That’s why it was named
Transitional — to provide backward compatibility for the transition
period.

As of this past October, that transition period is over. HTML 3.2 is
obsolete and so is 4.01 Transitional. 4.01 Strict remains an
almost-compatible subset of the current standing Recommendation, HTML
5.

>> * tables should be generally avoided unless actually representing tabular data;
> Agreed, but sometime we want to do some non-semantic formatting, for
> instance for node lines, or indices.  <table> is very practical in that
> case.

Non-semantic formatting is bad. Please do not want to do it.

I do not understand what node lines are. I thought a node is a whole
section of a document?

Indices would look much better as definition lists, possibly styled
for compact presentation on viewports wider than a certain threshold,
and possibly using multicolumn layout on even wider viewports. I
tested the Emacs manual index[1] in the Responsive Design View mode in
Firefox: it fits the 320×480 screen very poorly and under-utilizes the
1920×1200 screen.

[1]: http://www.gnu.org/software/emacs/manual/html_node/emacs/index.html

>> * the encoding declaration <meta> should be the first thing in <head>;
>
> What would be the reason for that?

Because the HTML specification says[2] the encoding declaration must
appear within the first 1024 bytes of the document, and including any
kind of document-provided content before the encoding declaration
might cause a violation of this rule.

[2]: http://www.w3.org/TR/html5/document-metadata.html#charset1024

And the reason for HTML spec saying so is that if the parser does not
know the encoding beforehand, it then has to perform a preliminary
scanning of the document in search of an encoding declaration. After
it finds one, it has to restart at the beginning. Restarting the
parsing could be very inefficient if the document is being streamed
from the network.

Re: texi2html output validity

[Stephen J. Turnbull]

Yuri Khan writes:

 > As of this past October, that transition period is over.

Just plain wrong.  The transition period *never* ends, as long as
there are any documents specifying older standards, or browsers
supporting them.  Transitional standards are a compatibility tool, not
a mandate.  And there is good, valid reason why RFCs aren't called
"standards" (despite being effectively so) and why W3C standards are
called "Recommendations", not "standards".  Publishing information to
the 'net is not like connecting to the electric power grid.

Support will be phased out, and eventually you may need to write a new
program to display HTML 3.2 that appears in some archeological dig, of
course.  But publication of a standard doesn't obsolete all the older
standards immediately.  In fact, in the case of RFCs it's in practice
the reverse: when 99% of implementations conform to an RFC, that's
when the IETF promotes it to "Internet Standard".

If you want Info-HTML to conform to HTML5, that's fine, and there are
valid arguments for it.  But please don't claim the sanction of the
W3C for that, because there isn't any (especially not in GNU projects,
but that's another story).

 > >> * the encoding declaration <meta> should be the first thing in <head>;
 > >
 > > What would be the reason for that?
 > 
 > Because the HTML specification says[2] the encoding declaration must
 > appear within the first 1024 bytes of the document, and including any
 > kind of document-provided content before the encoding declaration
 > might cause a violation of this rule.

AFAIK the encoding declaration is optional, defaulting to UTF-8.  In
that case, we can (and IMHO *should*, but I am no longer an expert on
current encoding practice) require that our software generate UTF-8
and omit the declaration.  Non-UTF-8 should be invalid in Info-HTML.

Re: texi2html output validity

[Ineiev]

On Wed, Dec 24, 2014 at 12:27:25PM +0900, Stephen J. Turnbull wrote:
> AFAIK the encoding declaration is optional, defaulting to UTF-8.  In
> that case, we can (and IMHO *should*, but I am no longer an expert on
> current encoding practice) require that our software generate UTF-8
> and omit the declaration.  Non-UTF-8 should be invalid in Info-HTML.

The fact is that some users have ASCII-incompatible default
encodings (like UTF-16). if we add the declaration, it costs little,
but the pages just work for them.

Re: texi2html output validity

[Stephen J. Turnbull]

Ineiev writes:
 > On Wed, Dec 24, 2014 at 12:27:25PM +0900, Stephen J. Turnbull wrote:
 > > AFAIK the encoding declaration is optional, defaulting to UTF-8.  In
 > > that case, we can (and IMHO *should*, but I am no longer an expert on
 > > current encoding practice) require that our software generate UTF-8
 > > and omit the declaration.  Non-UTF-8 should be invalid in Info-HTML.
 > 
 > The fact is that some users have ASCII-incompatible default
 > encodings (like UTF-16). if we add the declaration, it costs little,
 > but the pages just work for them.

AFAIK, default encodings are not a problem.  If Info-HTML is specified
to be served as XML (which has its own issues, but that's one way to
do it) then conformant browsers RFC2119-MUST assume Unicode as the
coded character set, and will automatically determine the
transformation format (UTF-8, UTF-16, UTF-16-little-endian) by
checking the first two octets.  I believe HTML5 also specifies UTF-8
as the default.

Alternatively, for such systems it's trivial to generate UTF-16 from
UTF-8.

Re: texi2html output validity

[Ineiev]

On Fri, Dec 26, 2014 at 12:58:24AM +0900, Stephen J. Turnbull wrote:
> Ineiev writes:
>  > On Wed, Dec 24, 2014 at 12:27:25PM +0900, Stephen J. Turnbull wrote:
>  > > AFAIK the encoding declaration is optional, defaulting to UTF-8.  In
>  > > that case, we can (and IMHO *should*, but I am no longer an expert on
>  > > current encoding practice) require that our software generate UTF-8
>  > > and omit the declaration.  Non-UTF-8 should be invalid in Info-HTML.
>  > 
>  > The fact is that some users have ASCII-incompatible default
>  > encodings (like UTF-16). if we add the declaration, it costs little,
>  > but the pages just work for them.
> 
> AFAIK, default encodings are not a problem.

GNU webmasters did receive reports from such visitors. I'm sure many
cases were not reported.

> If Info-HTML is specified
> to be served as XML (which has its own issues, but that's one way to
> do it) then conformant browsers RFC2119-MUST assume Unicode as the
> coded character set, and will automatically determine the
> transformation format (UTF-8, UTF-16, UTF-16-little-endian) by
> checking the first two octets.  I believe HTML5 also specifies UTF-8
> as the default.

I don't think HTML5 requirements are relevant because the browser
may not realize that it's HTML5 rather than HTML4 (and if we use <tt>,
we have few options but to produce HTML4, anyway), and for HTML4,
it obeys user's bogus settings.

Of course there may be ways to specify the encoding other than the
explicit declaration; I just believe that the explicit declaration
works reliably, and I'm not sure about other means.

> Alternatively, for such systems it's trivial to generate UTF-16 from
> UTF-8.

I think I don't understand this. do you suggest that webmasters
provide two versions of pages for the users to select them manually,
or do you say that the users should convert the pages themselves?

Re: texi2html output validity

[Stephen J. Turnbull]

Ineiev writes:

 > GNU webmasters did receive reports from such visitors. I'm sure many
 > cases were not reported.

If GNU websites are correctly configured and send the correct MIME
charset in the Content-Type in the HTTP headers, the users should not
have a problem, and if they do have such problems, a META tag probably
won't help anyway because they have a *very broken* browser.

 > > Alternatively, for such systems it's trivial to generate UTF-16 from
 > > UTF-8.
 > 
 > I think I don't understand this. do you suggest that webmasters
 > provide two versions of pages for the users to select them
 > manually,

Definitely not.  The web is not the problem (see above).  The issue
(if any) is local copies where the HTTP Content-Type header is
unavailable.

 > or do you say that the users should convert the pages themselves?

If they have a local copy and they have Emacs, this is easy enough to
script.  I really don't see why we should complicate the Info-HTML
spec to save users from self-inflicted injuries.

Re: texi2html output validity

[Ineiev]

On Fri, Dec 26, 2014 at 02:09:33AM +0900, Stephen J. Turnbull wrote:
> Ineiev writes:
> 
>  > GNU webmasters did receive reports from such visitors. I'm sure many
>  > cases were not reported.
> 
> If GNU websites are correctly configured and send the correct MIME
> charset in the Content-Type in the HTTP headers, the users should not
> have a problem,

I don't think it's easy to correctly configure www.gnu.org because
different pages written in the same language and living in the same
directory may use different encodings. they still use EUC-KR, 
iso-8859-2, gb2312, and more.

>  > I think I don't understand this. do you suggest that webmasters
>  > provide two versions of pages for the users to select them
>  > manually,
> 
> Definitely not.  The web is not the problem (see above).  The issue
> (if any) is local copies where the HTTP Content-Type header is
> unavailable.

Still it's an issue.

Re: texi2html output validity

[Stephen J. Turnbull]

Ineiev writes:

 > Still it's an issue.

OK, have it your way.  I suspect that you will not decrease the number
of complaints by much, but *shrug* it won't be my problem.

Re: texi2html output validity

[Ineiev]

Hello,

On Wed, Dec 24, 2014 at 12:59:32AM +0700, Yuri Khan wrote:
> 
> What browsers are there that do not support CSS *and* at the same time
> have the capability of displaying proportional fonts?

They may not be able to exactly display proportional fonts, but they
may highlight it in a different way (e.g. HTML reader could modify
the voice or, in a verbose mode, just say it's tt). Yes, CSS support
is much more problematic (the stylesheet isn't likely to apply to
the medium in the first place).

Re: texi2html output validity

[Gavin Smith]

On Tue, Dec 23, 2014 at 4:49 PM, Patrice Dumas <pertusus@free.fr> wrote:
> First of all it is a bit unclear where this html comes from.  In
> general, both texi2html and texi2any/makeinfo, especially for makeinfo
> starting at version 5 render properly nested html tags.
>
Could it be coming from some HTML output customization that is used
for the Lilypond manuals?

>> @t is a non-semantic command in Texinfo and should probably be
>> discouraged the same way <tt> has been discouraged in HTML since at
>> least 1997. It probably should become a <span class="t"> styled with
>> .t { font-family: monospace }.
>
> @t and other non-semantic commands are already discouraged in the manual.
> But I see no point in not using <tt> for @t, as long as browser support
> it (which is likely to be until the end of times). CSS is not supported
> by every browser.

Using a class and CSS for instead of <tt> would not make the output
file any more "semantic", it just specifies a fixed-width font in a
more verbose way. I would say if you don't like <tt> appearing because
it is deprecated, consider also the @t command deprecated. It's fine
to produce "bad" (non-semantic) output for "bad" input.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It's a very long page, with text and images interspersed all the time.

Why is it formatted as one long page?  Could it be split up?
If so, what was the reason you decided to make it one long page?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

An Info file is read into Emacs all at once, but it is subdivided
into nodes, and Info displays only one node at any time.

Would it be possible, using a Javascript extension to the browser, to
get similar behavior from a file of HTML?  That is, load a whole
manual as a single file, then display just one subdivision of it, and
change to a different subdivision in accord with user commands?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Yuri Khan]

On Mon, Dec 22, 2014 at 2:12 PM, Richard Stallman <rms@gnu.org> wrote:

> Would it be possible, using a Javascript extension to the browser, to
> get similar behavior from a file of HTML?  That is, load a whole
> manual as a single file, then display just one subdivision of it, and
> change to a different subdivision in accord with user commands?

Technically possible. Wrap each node in an <article class="node">
element, style them as article.node { display: none }, then add to a
single node a special class, e.g. <article class="active node">, and
style it as article.active.node { display: block }. The script or
extension would then only need to remove the “active” class from one
node and add it to another.

Performance-wise, this might or might not be a good idea, depending on
the file size. On the plus side, navigation between nodes of the same
file will be near-instant. On the other hand, the browser will need to
read and parse the whole file up front and keep it in process memory
at all times.

Re: HTML-Info design

[Achim Gratz]

Richard Stallman writes:
> Would it be possible, using a Javascript extension to the browser, to
> get similar behavior from a file of HTML?  That is, load a whole
> manual as a single file, then display just one subdivision of it, and
> change to a different subdivision in accord with user commands?

Yes, that's possible, both via styling (CSS) and DOM manipulation.  Not
only that, but the recent work on HTML standardization actively
encourages such implementations.  Of course, this requires a browser
that actually implements these things correctly or the use of a library
that can make the implementation holes and bugs in non-conforming
browsers disappear to some extent.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

HTML-Info design

[Stephen J. Turnbull]

Richard Stallman writes:

 > An Info file is read into Emacs all at once, but it is subdivided
 > into nodes, and Info displays only one node at any time.
 > 
 > Would it be possible, using a Javascript extension to the browser, to
 > get similar behavior from a file of HTML?  That is, load a whole
 > manual as a single file, then display just one subdivision of it, and
 > change to a different subdivision in accord with user commands?

Yes.  There are several browser+Javascript-based presentation packages
(for example, S5) that do exactly that.  It's easy to do with simple
HTML and a tiny bit of CSS,  and only a few lines of Javascript per
"primitive" navigation function (eg, "next" and "last").  Whether you
could get acceptable appearance and performance, and how much effort
that would take, I don't know.  I would guess it's not that hard.

Re: HTML-Info design

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Richard Stallman writes:
>
>  > An Info file is read into Emacs all at once, but it is subdivided
>  > into nodes, and Info displays only one node at any time.
>  > 
>  > Would it be possible, using a Javascript extension to the browser, to
>  > get similar behavior from a file of HTML?  That is, load a whole
>  > manual as a single file, then display just one subdivision of it, and
>  > change to a different subdivision in accord with user commands?
>
> Yes.  There are several browser+Javascript-based presentation packages
> (for example, S5) that do exactly that.  It's easy to do with simple
> HTML and a tiny bit of CSS,  and only a few lines of Javascript per
> "primitive" navigation function (eg, "next" and "last").  Whether you
> could get acceptable appearance and performance, and how much effort
> that would take, I don't know.  I would guess it's not that hard.

I suspect that navigation might suffer.  I'm using Firefox, and you can
use SPC to scroll down on a large page (like matches to some search on
Ebay).  Then you use middle-mouse on some link in order to get a
separate tab for a particular search result, and the next SPC in your
main list restarts much higher on the list again, presumably at the
point where the last mouse click was.  Using the mouse on the scrollbar
in order to rewind the position might or might not work for getting the
keyboard scrolling position back into shape.

So with Firefox apparently being quite unable to behave sensibly in the
context of mixed mouse/keyboard navigation in the context of one
straightforward unmodified long list (100 elements or so) on a fixed
page, I am somewhat skeptical that it will be lots better when using CSS
for magic folding and unfolding.

-- 
David Kastrup

Re: HTML-Info design

[Nic Ferrier]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Richard Stallman writes:
>
>  > An Info file is read into Emacs all at once, but it is subdivided
>  > into nodes, and Info displays only one node at any time.
>  > 
>  > Would it be possible, using a Javascript extension to the browser, to
>  > get similar behavior from a file of HTML?  That is, load a whole
>  > manual as a single file, then display just one subdivision of it, and
>  > change to a different subdivision in accord with user commands?
>
> Yes.  There are several browser+Javascript-based presentation packages
> (for example, S5) that do exactly that.  It's easy to do with simple
> HTML and a tiny bit of CSS,  and only a few lines of Javascript per
> "primitive" navigation function (eg, "next" and "last").  Whether you
> could get acceptable appearance and performance, and how much effort
> that would take, I don't know.  I would guess it's not that hard.

This is what my app does:

  http://gnudoc.ferrier.me.uk

it implements indexing (press i) and toc and all of that.

Yes, it's based on JS but the JS is free.


Nic

Re: HTML-Info design

[Jan Djärv]

[-- Attachment #1: Type: text/plain, Size: 325 bytes --]

Den 2014-12-22 11:36, Nic Ferrier skrev:
>
> This is what my app does:
>
>    http://gnudoc.ferrier.me.uk
>
> it implements indexing (press i) and toc and all of that.
>
> Yes, it's based on JS but the JS is free.

It does not work in Chrome, see attachment.  This is one of the eternal 
problems with JS/HTML.

	Jan D.






[-- Attachment #2: Skärmbild från 2014-12-22 12:53:12.png --]
[-- Type: image/png, Size: 17961 bytes --]

Re: HTML-Info design

[Lennart Borgman]

On Mon, Dec 22, 2014 at 12:56 PM, Jan Djärv <jan.h.d@swipnet.se> wrote:
> Den 2014-12-22 11:36, Nic Ferrier skrev:
>>
>>
>> This is what my app does:
>>
>>    http://gnudoc.ferrier.me.uk
>>
>> it implements indexing (press i) and toc and all of that.
>>
>> Yes, it's based on JS but the JS is free.
>
>
> It does not work in Chrome, see attachment.  This is one of the eternal
> problems with JS/HTML.


It looks more like a bug to me. Quite common in software of all kinds. ;-)

Please give us some more info, Jan, so we can track down the bug.
Exactly what do we do to trigger the bug?

(I wonder about jQuery there. Does your app use jQuery at all, Nic?)

Re: HTML-Info design

[David Kastrup]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> On Mon, Dec 22, 2014 at 12:56 PM, Jan Djärv <jan.h.d@swipnet.se> wrote:
>> Den 2014-12-22 11:36, Nic Ferrier skrev:
>>>
>>>
>>> This is what my app does:
>>>
>>>    http://gnudoc.ferrier.me.uk
>>>
>>> it implements indexing (press i) and toc and all of that.
>>>
>>> Yes, it's based on JS but the JS is free.
>>
>>
>> It does not work in Chrome, see attachment.  This is one of the eternal
>> problems with JS/HTML.
>
>
> It looks more like a bug to me. Quite common in software of all
> kinds. ;-)

Good thing that we have only one software's bugs to account for when
rendering info in Emacs.

-- 
David Kastrup

Re: HTML-Info design

[Jan Djärv]

Hi. 


> 22 dec 2014 kl. 13:49 skrev Lennart Borgman <lennart.borgman@gmail.com>:
> 
>> On Mon, Dec 22, 2014 at 12:56 PM, Jan Djärv <jan.h.d@swipnet.se> wrote:
>> Den 2014-12-22 11:36, Nic Ferrier skrev:
>>> 
>>> 
>>> This is what my app does:
>>> 
>>>   http://gnudoc.ferrier.me.uk
>>> 
>>> it implements indexing (press i) and toc and all of that.
>>> 
>>> Yes, it's based on JS but the JS is free.
>> 
>> 
>> It does not work in Chrome, see attachment.  This is one of the eternal
>> problems with JS/HTML.
> 
> 
> It looks more like a bug to me. Quite common in software of all kinds. ;-)
> 
> Please give us some more info, Jan, so we can track down the bug.
> Exactly what do we do to trigger the bug?
> 

Start Chrome, press i. 
Nothing happens but you can see the error in the console. 

       Jan D. 

> (I wonder about jQuery there. Does your app use jQuery at all, Nic?)

Re: HTML-Info design

[Lennart Borgman]

On Mon, Dec 22, 2014 at 2:21 PM, Jan Djärv <jan.h.d@swipnet.se> wrote:
> Hi.
>
>
>> 22 dec 2014 kl. 13:49 skrev Lennart Borgman <lennart.borgman@gmail.com>:
>>
>>> On Mon, Dec 22, 2014 at 12:56 PM, Jan Djärv <jan.h.d@swipnet.se> wrote:
>>> Den 2014-12-22 11:36, Nic Ferrier skrev:
>>>>
>>>>
>>>> This is what my app does:
>>>>
>>>>   http://gnudoc.ferrier.me.uk
>>>>
>>>> it implements indexing (press i) and toc and all of that.
>>>>
>>>> Yes, it's based on JS but the JS is free.
>>>
>>>
>>> It does not work in Chrome, see attachment.  This is one of the eternal
>>> problems with JS/HTML.
>>
>>
>> It looks more like a bug to me. Quite common in software of all kinds. ;-)
>>
>> Please give us some more info, Jan, so we can track down the bug.
>> Exactly what do we do to trigger the bug?
>>
>
> Start Chrome, press i.
> Nothing happens but you can see the error in the console.
>
>        Jan D.


Thanks Jan. It works for me with Chrome here (Version 39.0.2171.95 m,
Windows 7 Pro).

Since I did not see jQuery there in the source but in the picture you
posted it looks to me like it can be some add-on you have in Chrome.


>> (I wonder about jQuery there. Does your app use jQuery at all, Nic?)

Re: HTML-Info design

[Lennart Borgman]

On Mon, Dec 22, 2014 at 2:29 PM, Lennart Borgman
<lennart.borgman@gmail.com> wrote:
> On Mon, Dec 22, 2014 at 2:21 PM, Jan Djärv <jan.h.d@swipnet.se> wrote:
>> Hi.
>>
>>
>>> 22 dec 2014 kl. 13:49 skrev Lennart Borgman <lennart.borgman@gmail.com>:
>>>
>>>> On Mon, Dec 22, 2014 at 12:56 PM, Jan Djärv <jan.h.d@swipnet.se> wrote:
>>>> Den 2014-12-22 11:36, Nic Ferrier skrev:
>>>>>
>>>>>
>>>>> This is what my app does:
>>>>>
>>>>>   http://gnudoc.ferrier.me.uk
>>>>>
>>>>> it implements indexing (press i) and toc and all of that.
>>>>>
>>>>> Yes, it's based on JS but the JS is free.
>>>>
>>>>
>>>> It does not work in Chrome, see attachment.  This is one of the eternal
>>>> problems with JS/HTML.
>>>
>>>
>>> It looks more like a bug to me. Quite common in software of all kinds. ;-)
>>>
>>> Please give us some more info, Jan, so we can track down the bug.
>>> Exactly what do we do to trigger the bug?
>>>
>>
>> Start Chrome, press i.
>> Nothing happens but you can see the error in the console.
>>
>>        Jan D.
>
>
> Thanks Jan. It works for me with Chrome here (Version 39.0.2171.95 m,
> Windows 7 Pro).
>
> Since I did not see jQuery there in the source but in the picture you
> posted it looks to me like it can be some add-on you have in Chrome.


Sorry, my bad. jQuery is included. So it is something else.

(But, Nic, why use jQuery on a HTML5 page?)

Re: HTML-Info design

[David Engster]

Lennart Borgman writes:
> On Mon, Dec 22, 2014 at 2:29 PM, Lennart Borgman
> <lennart.borgman@gmail.com> wrote:
>> On Mon, Dec 22, 2014 at 2:21 PM, Jan Djärv <jan.h.d@swipnet.se> wrote:
>>> Hi.
>>>
>
>>>
>>>> 22 dec 2014 kl. 13:49 skrev Lennart Borgman <lennart.borgman@gmail.com>:
>>>>
>>>>> On Mon, Dec 22, 2014 at 12:56 PM, Jan Djärv <jan.h.d@swipnet.se> wrote:
>>>>> Den 2014-12-22 11:36, Nic Ferrier skrev:
>>>>>>
>>>>>>
>>>>>> This is what my app does:
>>>>>>
>>>>>>   http://gnudoc.ferrier.me.uk
>>>>>>
>>>>>> it implements indexing (press i) and toc and all of that.
>>>>>>
>>>>>> Yes, it's based on JS but the JS is free.
>>>>>
>>>>>
>>>>> It does not work in Chrome, see attachment.  This is one of the eternal
>>>>> problems with JS/HTML.
>>>>
>>>>
>>>> It looks more like a bug to me. Quite common in software of all kinds. ;-)
>>>>
>>>> Please give us some more info, Jan, so we can track down the bug.
>>>> Exactly what do we do to trigger the bug?
>>>>
>>>
>>> Start Chrome, press i.
>>> Nothing happens but you can see the error in the console.
>>>
>>>        Jan D.
>>
>>
>> Thanks Jan. It works for me with Chrome here (Version 39.0.2171.95 m,
>> Windows 7 Pro).
>>
>> Since I did not see jQuery there in the source but in the picture you
>> posted it looks to me like it can be some add-on you have in Chrome.
>
>
> Sorry, my bad. jQuery is included. So it is something else.

It seems to be some initialization problem. When you hit 'i', the
evt.target.value should be "" at the beginning, but for you it is
undefined. I cannot reproduce it here with Chromium or Firefox, though.

But Nic, I think this is great and I hope you keep working on it. It
would be really neat if Texinfo could create a webapp like this which
you simply throw on some server (the dependency on Node.js makes this
slightly more complicated, but maybe one can get rid of it?). Together
with a few nice CSS themes to choose from, this would make Texinfo much
more attractive for creating online documentation.

> (But, Nic, why use jQuery on a HTML5 page?)

You are very confused about HTML5 as well as jQuery.

-David

Re: HTML-Info design

[Lennart Borgman]

On Mon, Dec 22, 2014 at 3:27 PM, David Engster <deng@randomsample.de> wrote:
> Lennart Borgman writes:
>> On Mon, Dec 22, 2014 at 2:29 PM, Lennart Borgman
>
>> (But, Nic, why use jQuery on a HTML5 page?)
>
> You are very confused about HTML5 as well as jQuery.


Maybe. So please educate me. What use is jQuery to adopt to difference
between HTML5 browsers today? In a year? (This is what we are talking
about now.)

Re: HTML-Info design

[David Engster]

Lennart Borgman writes:
> On Mon, Dec 22, 2014 at 3:27 PM, David Engster <deng@randomsample.de> wrote:
>> Lennart Borgman writes:
>>> On Mon, Dec 22, 2014 at 2:29 PM, Lennart Borgman
>>
>>> (But, Nic, why use jQuery on a HTML5 page?)
>>
>> You are very confused about HTML5 as well as jQuery.
>
>
> Maybe. So please educate me.

No. This is not the place. Suffice to say that jQuery is not just some
compatibility layer. Play around with it, and you'll see.

-David

Re: HTML-Info design

[Lennart Borgman]

On Mon, Dec 22, 2014 at 3:39 PM, David Engster <deng@randomsample.de> wrote:
> Lennart Borgman writes:
>> On Mon, Dec 22, 2014 at 3:27 PM, David Engster <deng@randomsample.de> wrote:
>>> Lennart Borgman writes:
>>>> On Mon, Dec 22, 2014 at 2:29 PM, Lennart Borgman
>>>
>>>> (But, Nic, why use jQuery on a HTML5 page?)
>>>
>>> You are very confused about HTML5 as well as jQuery.
>>
>>
>> Maybe. So please educate me.
>
> No. This is not the place. Suffice to say that jQuery is not just some
> compatibility layer. Play around with it, and you'll see.


Then maybe this is not the place to pretend you know what I know about
jQuery either? We are just talking about the compatibility layer here.

I stopped using jQuery some years ago because it crashed mobile web
browsers (memory problems I guess for more complicated things). There
are still compatibility problems for simple things like we are talking
about here, but unless you are in a hurry you can just wait to see
them fixed.

Re: HTML-Info design

[David Engster]

Lennart Borgman writes:
> On Mon, Dec 22, 2014 at 3:39 PM, David Engster <deng@randomsample.de> wrote:
>> Lennart Borgman writes:
>>> On Mon, Dec 22, 2014 at 3:27 PM, David Engster <deng@randomsample.de> wrote:
>>>> Lennart Borgman writes:
>
>>>>> On Mon, Dec 22, 2014 at 2:29 PM, Lennart Borgman
>>>>
>>>>> (But, Nic, why use jQuery on a HTML5 page?)
>>>>
>>>> You are very confused about HTML5 as well as jQuery.
>>>
>>>
>>> Maybe. So please educate me.
>>
>> No. This is not the place. Suffice to say that jQuery is not just some
>> compatibility layer. Play around with it, and you'll see.
>
> Then maybe this is not the place to pretend you know what I know about
> jQuery either?

OK, let's make this official: you know more about jQuery than me. Happy
Christmas, Lennart!

> We are just talking about the compatibility layer here.

*You* are.

-David

Re: HTML-Info design

[Lennart Borgman]

On Mon, Dec 22, 2014 at 3:54 PM, David Engster <deng@randomsample.de> wrote:
> Lennart Borgman writes:
>>
>> Then maybe this is not the place to pretend you know what I know about
>> jQuery either?
>
> OK, let's make this official: you know more about jQuery than me. Happy
> Christmas, Lennart!


No need to. I do not care at all. You can have it. If you know more
about what I know about jQuery than I do it is ok. ;-)

I hope you have some use for it! Happy Christmas, David.

Re: HTML-Info design

[Nic Ferrier]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> Sorry, my bad. jQuery is included. So it is something else.
>
> (But, Nic, why use jQuery on a HTML5 page?)

There are still things that jquery makes easier than html5, for example
ajax.

And there are still lots of html5 things that absolutely aren't
standardized. And evergreen browsers are great but it actually increases
the amount of variance in the APIs.

For a "production" release of  something webapp-y I often look elsewhere
than jquery. But this was a hack it together and throw it around type
affair and the ajax alone makes it worth while.

There's almost nothing in the gnudoc app that couldn't easily be
replaced with another lib or just a small shim over what we know are
compatability issues.



Nic

Re: HTML-Info design

[Jan Djärv]

Den 2014-12-22 14:29, Lennart Borgman skrev:
> On Mon, Dec 22, 2014 at 2:21 PM, Jan Djärv <jan.h.d@swipnet.se> wrote:
>>
>> Start Chrome, press i.
>> Nothing happens but you can see the error in the console.
>>
>>         Jan D.
>
>
> Thanks Jan. It works for me with Chrome here (Version 39.0.2171.95 m,
> Windows 7 Pro).

I got Version 39.0.2171.95 (64-bit) on Mint 17.1 (GNU/Linux).
It works OK on Safari 8.0.2 though.

	Jan D.

Re: HTML-Info design

[Tom]

Jan Djärv <jan.h.d <at> swipnet.se> writes:
> 
> It does not work in Chrome, see attachment.  This is one of the eternal 
> problems with JS/HTML.
> 

It is not a problem with JS, it's a problem of browser platforms.
That's why we have JS libraries.

Like when you want to write a cross platform GUI which works under
X, Windows, etc. then you don't start hand coding GUI widgets,
you use libraries like GTK or wxWidgets instead.

So the platform differences should be handled by the library, not
the app, and if there is a problem on a platform then it is a bug
in the library which should be fixed there.

Re: HTML-Info design

[Lennart Borgman]

On Mon, Dec 22, 2014 at 2:57 PM, Tom <adatgyujto@gmail.com> wrote:
> Jan Djärv <jan.h.d <at> swipnet.se> writes:
>>
>> It does not work in Chrome, see attachment.  This is one of the eternal
>> problems with JS/HTML.
>>
>
> It is not a problem with JS, it's a problem of browser platforms.
> That's why we have JS libraries.
>
> Like when you want to write a cross platform GUI which works under
> X, Windows, etc. then you don't start hand coding GUI widgets,
> you use libraries like GTK or wxWidgets instead.
>
> So the platform differences should be handled by the library, not
> the app, and if there is a problem on a platform then it is a bug
> in the library which should be fixed there.


Sure, but I do not think we need those JS libraries for something like
this now. Just target HTML5.

Re: HTML-Info design

[Tom]

Lennart Borgman <lennart.borgman <at> gmail.com> writes:
> 
> Sure, but I do not think we need those JS libraries for something like
> this now. Just target HTML5.
> 

How does it solve the problem?

The main problem with JS development is that different browsers implement
the JS DOM differently, either by misundersanding the standard, 
implementing their own non-standard extensions or implementing things
differently which are not clearly defined by the standard.

That's why we need JS libraries to hide these implementation differences,
so one does not have to deal with them in one's app.

Targeting HTML5 still carries these problems, because it's just an other
standard which can be misunderstood, etc.

Re: HTML-Info design

[Lennart Borgman]

On Mon, Dec 22, 2014 at 3:41 PM, Tom <adatgyujto@gmail.com> wrote:
> Lennart Borgman <lennart.borgman <at> gmail.com> writes:
>>
>> Sure, but I do not think we need those JS libraries for something like
>> this now. Just target HTML5.
>>
>
> How does it solve the problem?
>
> The main problem with JS development is that different browsers implement
> the JS DOM differently

That was the case some years ago. Not today - unless people are using
older browsers, but there is no need to take care of this here IMO.

Re: HTML-Info design

[Jan Djärv]

Den 2014-12-22 14:57, Tom skrev:
> Jan Djärv <jan.h.d <at> swipnet.se> writes:
>>
>> It does not work in Chrome, see attachment.  This is one of the eternal
>> problems with JS/HTML.
>>
>
> It is not a problem with JS, it's a problem of browser platforms.

Well, JS runs on Browsers, so it is a problem with JS IMHO.

> That's why we have JS libraries.

I used JQuery extensivly for many years and know that it is not a silver 
bullet.  You can still write non-portable code quite easily with JQuery.

	Jan D.

Re: HTML-Info design

[Nic Ferrier]

jan.h.d@swipnet.se writes:

> Den 2014-12-22 11:36, Nic Ferrier skrev:
>>
>> This is what my app does:
>>
>>    http://gnudoc.ferrier.me.uk
>>
>> it implements indexing (press i) and toc and all of that.
>>
>> Yes, it's based on JS but the JS is free.
>
> It does not work in Chrome, see attachment.  This is one of the
> eternal problems with JS/HTML.

That's unfair. This was a quick hack done in less than a day with severe
constraints.

Testing webapps isn't hard now. We can do it.

I haven't because as I say, this was a POC to show that info *could* be
built in a modern web browser.



Nic

Re: HTML-Info design

[Jan D.]

Hello.

> 22 dec 2014 kl. 23:22 skrev Nic Ferrier <nferrier@ferrier.me.uk>:
> 
> jan.h.d@swipnet.se writes:
> 
>> Den 2014-12-22 11:36, Nic Ferrier skrev:
>>> 
>>> This is what my app does:
>>> 
>>>   http://gnudoc.ferrier.me.uk
>>> 
>>> it implements indexing (press i) and toc and all of that.
>>> 
>>> Yes, it's based on JS but the JS is free.
>> 
>> It does not work in Chrome, see attachment.  This is one of the
>> eternal problems with JS/HTML.
> 
> That's unfair. This was a quick hack done in less than a day with severe
> constraints.

It was meant as a datapoint for further improvements.
I used Chrome because sources (http://en.wikipedia.org/wiki/Usage_share_of_web_browsers, http://www.w3schools.com/browsers/browsers_stats.asp) says its the most used browser.

	Jan D.

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I used Chrome because sources
  > (http://en.wikipedia.org/wiki/Usage_share_of_web_browsers,
  > http://www.w3schools.com/browsers/browsers_stats.asp) says its the
  > most used browser.

Anything we do in GNU should give priority to IceCat, which is a GNU package.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Lennart Borgman]

On Tue, Dec 23, 2014 at 4:34 PM, Richard Stallman <rms@gnu.org> wrote:
>
> Anything we do in GNU should give priority to IceCat, which is a GNU package.

Yes, but maybe it is not that good for testing since it looks like it
lags Firefox. (And Firefox currently lags Chrome for some standard
parts.)

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Anything we do in GNU should give priority to IceCat, which is a GNU package.

  > Yes, but maybe it is not that good for testing since it looks like it
  > lags Firefox. (And Firefox currently lags Chrome for some standard
  > parts.)

You can test in any other browsers when you like, but do test in IceCat
because that's GNU's flagship browser.

IceCat leads both Firefox and Chromium in protecting users' privacy.

As for Chrome, that's nonfree software, thus deprecated.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Eli Zaretskii]

> From: Nic Ferrier <nferrier@ferrier.me.uk>
> Date: Mon, 22 Dec 2014 10:36:11 +0000
> Cc: rms@gnu.org, emacs-devel@gnu.org
> 
> > Yes.  There are several browser+Javascript-based presentation packages
> > (for example, S5) that do exactly that.  It's easy to do with simple
> > HTML and a tiny bit of CSS,  and only a few lines of Javascript per
> > "primitive" navigation function (eg, "next" and "last").  Whether you
> > could get acceptable appearance and performance, and how much effort
> > that would take, I don't know.  I would guess it's not that hard.
> 
> This is what my app does:
> 
>   http://gnudoc.ferrier.me.uk
> 
> it implements indexing (press i) and toc and all of that.

Thanks.  Please allow me some comments:

  . The initial display of the top-level menu is annoyingly slow.  For
    a few seconds there I thought it didn't work, then suddenly the
    menu appeared in its entirety.

  . There's no help, at least AFAICS.  Neither h nor ?  display
    anything.  The only instructions I found are in the README on the
    Github page.

  . With IE 11, typing i opens the "minibuffer" area, but doesn't show
    the "Index topic" prompt.  (Same behavior with the menu prompt.)
    Firefox does show the prompt, but there's no colon ":" and no
    space between the prompt and the cursor -- not sure if this is a
    display problem or a JS problem.

  . Typing i or m in my locale puts the cursor at the right edge of
    the window (both in Firefox v34.0 and IE 11.0.9600 on Windows 7).
    Please force the browser to use the left-to-right base paragraph
    direction, independent of the locale (that's what Emacs does with
    minibuffer prompts and input).

  . Looks like index entries are matched exactly (except for the
    letter-case), as opposed to substring matches in Info.  E.g.,
    "i display RET" takes me to http://gnudoc.ferrier.me.uk/undefined.
    Consequently, the "," key in Info has no equivalent.  This makes
    index searches much less efficient than they are in Info.

  . Typing m followed by something exhibits unexpected completion
    behavior: TAB has no effect, whereas typing enough of the menu
    item to make it unambiguous automatically completes.  Is this the
    intended behavior?  If so, why the difference from completion
    implemented for the i command?

  . Repeated i and m keypresses start with the last value I typed,
    which is not useful, and requires to always delete that.

  . A question: does i work in a manual with more than 1 index node,
    like the Emacs User manual (which has 5 index nodes)?

  . Navigation keys (n, p, u, l) are not implemented, or don't work.
    Likewise with s (regular expression search through the entire
    manual).

  . Links to footnotes don't work.  E.g., clicking on "1" on the 15th
    line in "Using 'interactive'"
    (http://gnudoc.ferrier.me.uk/info/Using-Interactive.html#Using-Interactive)
    takes me to the top-level menu instead, although the footnote is
    shown at the bottom of the "Using 'interactive'" section.

  . Info mode currently has a lot of useful features besides basic
    navigation, index-search, and cross-reference following, like
    "virtual index", info-apropos, etc.  These will have to be
    implemented or migrated to this kind of solution.

  . Summary: OK as a POC, but IMO more work is needed to make sure a
    functional equivalent of the Info reader is indeed feasible and
    practical with this technology.

Once again, thanks for working on this.

Re: HTML-Info design

[Nic Ferrier]

Eli Zaretskii <eliz@gnu.org> writes:

>> > Yes.  There are several browser+Javascript-based presentation packages
>> > (for example, S5) that do exactly that.  It's easy to do with simple
>> > HTML and a tiny bit of CSS,  and only a few lines of Javascript per
>> > "primitive" navigation function (eg, "next" and "last").  Whether you
>> > could get acceptable appearance and performance, and how much effort
>> > that would take, I don't know.  I would guess it's not that hard.
>> 
>> This is what my app does:
>> 
>>   http://gnudoc.ferrier.me.uk
>> 
>> it implements indexing (press i) and toc and all of that.
>
> Thanks.  Please allow me some comments:

Thanks for your comments. I've tried to respond. I'm conscious it could
sound a bit rude. It's not meant to be, it's just a statement or
explanation of where I am with it.


>   . The initial display of the top-level menu is annoyingly slow.  For
>     a few seconds there I thought it didn't work, then suddenly the
>     menu appeared in its entirety.

Yes, it's pulled down from the GNU HTML manuals online. So a proxy call
basically. I've been slowly working on a better way (a direct way) of
delivering the content. That's a relatively difficult logistics job. A
matter of building everyone's documentation and putting it in a place.


>   . There's no help, at least AFAICS.  Neither h nor ?  display
>     anything.  The only instructions I found are in the README on the
>     Github page.

Probably not. I did enough to show it could be done. I didn't implement
*everything* from help.


>   . With IE 11, typing i opens the "minibuffer" area, but doesn't show
>     the "Index topic" prompt.  (Same behavior with the menu prompt.)
>     Firefox does show the prompt, but there's no colon ":" and no
>     space between the prompt and the cursor -- not sure if this is a
>     display problem or a JS problem.

I never test with IE 11. So I'm sure there are a lot of problems.


>   . Typing i or m in my locale puts the cursor at the right edge of
>     the window (both in Firefox v34.0 and IE 11.0.9600 on Windows 7).
>     Please force the browser to use the left-to-right base paragraph
>     direction, independent of the locale (that's what Emacs does with
>     minibuffer prompts and input).

No. Or maybe not. The point for me was not to slavishly implement Info
but to show that the same interactive experience could be achieved. I
tried it the way Info does it and it didn't feel or look right to
me. But then I have very little feedback right now. Some of it was in
direct contravention to what you just said. So I'm not saying it's right
the way it is now. This is an advantage of webapps, you can get that
feedback.


>   . Looks like index entries are matched exactly (except for the
>     letter-case), as opposed to substring matches in Info.  E.g.,
>     "i display RET" takes me to http://gnudoc.ferrier.me.uk/undefined.
>     Consequently, the "," key in Info has no equivalent.  This makes
>     index searches much less efficient than they are in Info.

That's right. Fuzzy matching wasn't implemented yet.


>   . Typing m followed by something exhibits unexpected completion
>     behavior: TAB has no effect, whereas typing enough of the menu
>     item to make it unambiguous automatically completes.  Is this the
>     intended behavior?  If so, why the difference from completion
>     implemented for the i command?

That's right. It's clear that it could be implemented. But I didn't do
it yet. It certainly offers more completion than the existing GNU HTML
manuals.


>   . Repeated i and m keypresses start with the last value I typed,
>     which is not useful, and requires to always delete that.

Again, that's a decision I'm not prepared to listen to much definitive
thought on right now.


>   . A question: does i work in a manual with more than 1 index node,
>     like the Emacs User manual (which has 5 index nodes)?

It could do. The index nodes are understood as index nodes.


>   . Navigation keys (n, p, u, l) are not implemented, or don't work.
>     Likewise with s (regular expression search through the entire
>     manual).

The former aren't implemented and maybe I never will... the point is not
to implement info in HTML/JS but to implement everything that info can do
in HTML/JS.

The latter could be implemented. It's quite a bit more work and requires
the logistics to be done. I can't do it with a proxy connection as now.


>   . Links to footnotes don't work.  E.g., clicking on "1" on the 15th
>     line in "Using 'interactive'"
>     (http://gnudoc.ferrier.me.uk/info/Using-Interactive.html#Using-Interactive)
>     takes me to the top-level menu instead, although the footnote is
>     shown at the bottom of the "Using 'interactive'" section.

Didn't even test it.


>   . Info mode currently has a lot of useful features besides basic
>     navigation, index-search, and cross-reference following, like
>     "virtual index", info-apropos, etc.  These will have to be
>     implemented or migrated to this kind of solution.
>
>   . Summary: OK as a POC, but IMO more work is needed to make sure a
>     functional equivalent of the Info reader is indeed feasible and
>     practical with this technology.

You're wrong. This POC is proof that an info reader is feasible. Nothing
you have pointed as it substantive except search and we absolutely know
that could be done.

This has not moved only because:

1. the next obvious massively beneficial step is a difficult and time
consuming one - to generate the info for all the manuals because a proxy
is not scalable

2. no one else is helping, despite many people being excited about
it. Hey what's new?



> Once again, thanks for working on this.

I work on what I work on.


Nic

Re: HTML-Info design

[Eli Zaretskii]

> From: Nic Ferrier <nferrier@ferrier.me.uk>
> Cc: stephen@xemacs.org,  rms@gnu.org,  emacs-devel@gnu.org
> Date: Mon, 22 Dec 2014 22:06:55 +0000
> 
> >   . Summary: OK as a POC, but IMO more work is needed to make sure a
> >     functional equivalent of the Info reader is indeed feasible and
> >     practical with this technology.
> 
> You're wrong. This POC is proof that an info reader is feasible. Nothing
> you have pointed as it substantive except search and we absolutely know
> that could be done.

How can we know that without even trying to implement a substantial
subset of what Info does?  Your current implementation is 300-line
long; if the addition of at least the basic Info features will take it
to 3000, then what did we gain?  And if you cannot get rid of the
right-to-left display of the prompts for index and menu items, this
implementation is simply unacceptable, because it means we cannot
control its display.

IOW, your POC proves that it can be done, buit its not enough to
convince me that it could be done well enough.

> This has not moved only because:
> 
> 1. the next obvious massively beneficial step is a difficult and time
> consuming one - to generate the info for all the manuals because a proxy
> is not scalable
> 
> 2. no one else is helping, despite many people being excited about
> it. Hey what's new?

It will never move unless you decide to go ahead.  Nothing substantial
ever gets done in Emacs by excitement and talk.

Re: HTML-Info design

[Nic Ferrier]

Eli Zaretskii <eliz@gnu.org> writes:

>> 1. the next obvious massively beneficial step is a difficult and time
>> consuming one - to generate the info for all the manuals because a proxy
>> is not scalable
>> 
>> 2. no one else is helping, despite many people being excited about
>> it. Hey what's new?
>
> It will never move unless you decide to go ahead.  Nothing substantial
> ever gets done in Emacs by excitement and talk.

Thanks for that. It was you all who were talking and me who was building
the HTML5 front end app.


Nic

Re: HTML-Info design

[Eli Zaretskii]

> From: Nic Ferrier <nferrier@ferrier.me.uk>
> Cc: stephen@xemacs.org,  rms@gnu.org,  emacs-devel@gnu.org
> Date: Tue, 23 Dec 2014 07:58:25 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> 1. the next obvious massively beneficial step is a difficult and time
> >> consuming one - to generate the info for all the manuals because a proxy
> >> is not scalable
> >> 
> >> 2. no one else is helping, despite many people being excited about
> >> it. Hey what's new?
> >
> > It will never move unless you decide to go ahead.  Nothing substantial
> > ever gets done in Emacs by excitement and talk.
> 
> Thanks for that. It was you all who were talking and me who was building
> the HTML5 front end app.

That was meant to be an encouragement for you to continue working.

Thanks.

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > How can we know that without even trying to implement a substantial
  > subset of what Info does?  Your current implementation is 300-line
  > long; if the addition of at least the basic Info features will take it
  > to 3000, then what did we gain?

The reasons I am interested in HTML-Info are

1. It will be readable (though without full Info functionality)
in ordinary browsers.

2. It will be able to represent many things that Info can't.

If it needs a 3000-line program to work with full Info functionality,
that's not so big after all.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > How can we know that without even trying to implement a substantial
>   > subset of what Info does?  Your current implementation is 300-line
>   > long; if the addition of at least the basic Info features will take it
>   > to 3000, then what did we gain?
>
> The reasons I am interested in HTML-Info are
>
> 1. It will be readable (though without full Info functionality)
> in ordinary browsers.
>
> 2. It will be able to represent many things that Info can't.

Is there a reason we cannot extend the Info format?  It has been
extended for images already.

The basic difference I see is that Info is preformatted and has
documents organized in multi-file parts (including an index) that the
Info reader is well-informed about.

-- 
David Kastrup

Re: HTML-Info design

[Stephen J. Turnbull]

David Kastrup writes:

 > Is there a reason we cannot extend the Info format?  It has been
 > extended for images already.

Of course it's possible, but why reinvent the wheel when there are
already many implementations of the HTML standard, and a convenient
target browser (IceCat) for specifying an appropriate subset of
features for use?

Especially when you consider how poor the current implementation of
texi2any is on several fronts, maintaining multiple backends seems
like a good way to waste effort better spent on performance and
improved support for HTML (which is already supported).  On the other
hand, with a well-defined HTML-Info spec, I think it would be easy to
extend eww to handle the necessary features (eg, with a hard-coded
stylesheet -- implementing a reasonable subset of CSS would be a big
project, but simply implementing the proposed display features is much
simpler), so that Emacs could be a very competitive HTML-Info browser.

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Is there a reason we cannot extend the Info format?  It has been
  > extended for images already.

It is certainly possible to extend Info format, but it would be a lot
of duplicate effort and the result would not be as clean as HTML.
I think using stylized HTML is a superior method.

  > The basic difference I see is that Info is preformatted and has
  > documents organized in multi-file parts (including an index) that the
  > Info reader is well-informed about.

We can do that by using specific constructions in HTML that the HTML-Info
reader will recognize so that it can provide the same features.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Eli Zaretskii]

> Date: Tue, 23 Dec 2014 10:32:56 -0500
> From: Richard Stallman <rms@gnu.org>
> CC: nferrier@ferrier.me.uk, stephen@xemacs.org, emacs-devel@gnu.org
> 
>   > How can we know that without even trying to implement a substantial
>   > subset of what Info does?  Your current implementation is 300-line
>   > long; if the addition of at least the basic Info features will take it
>   > to 3000, then what did we gain?
> 
> The reasons I am interested in HTML-Info are
> 
> 1. It will be readable (though without full Info functionality)
> in ordinary browsers.
> 
> 2. It will be able to represent many things that Info can't.

No disagreement here.  The question that bothers me if it can support
the important features we have now in our Info reader.

> If it needs a 3000-line program to work with full Info functionality,
> that's not so big after all.

I didn't say it's big, just that its size won't be an advantage
compared to info.el.

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > 2. It will be able to represent many things that Info can't.

  > No disagreement here.  The question that bothers me if it can support
  > the important features we have now in our Info reader.

Yes it can.  We just have to design ways to represent, in HTML, the
necessary data for these features to operate on.  Then implement the
features (perhaps in HTML for use in IceCat, perhaps in other ways
too).

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Eli Zaretskii]

> Date: Wed, 24 Dec 2014 04:56:22 -0500
> From: Richard Stallman <rms@gnu.org>
> Cc: stephen@xemacs.org, nferrier@ferrier.me.uk, emacs-devel@gnu.org
> 
>   > > 2. It will be able to represent many things that Info can't.
> 
>   > No disagreement here.  The question that bothers me if it can support
>   > the important features we have now in our Info reader.
> 
> Yes it can.  We just have to design ways to represent, in HTML, the
> necessary data for these features to operate on.  Then implement the
> features (perhaps in HTML for use in IceCat, perhaps in other ways
> too).

I'd need to see some initial design for that, in order to agree.

I'm not an expert, but AFAIK HTML is not an extensible format, there's
no place there to represent entities that are not already defined by
whatever organizations that codify HTML.

Re: HTML-Info design

[Lennart Borgman]

On Wed, Dec 24, 2014 at 5:33 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Wed, 24 Dec 2014 04:56:22 -0500
>> From: Richard Stallman <rms@gnu.org>
>> Cc: stephen@xemacs.org, nferrier@ferrier.me.uk, emacs-devel@gnu.org
>>
>> Yes it can.  We just have to design ways to represent, in HTML, the
>> necessary data for these features to operate on.  Then implement the
>> features (perhaps in HTML for use in IceCat, perhaps in other ways
>> too).
>
> I'd need to see some initial design for that, in order to agree.
>
> I'm not an expert, but AFAIK HTML is not an extensible format, there's
> no place there to represent entities that are not already defined by
> whatever organizations that codify HTML.

You can use the class attribute in HTML for that:
<span class="mydetail">This is my detail</span>.

Though everything used in HTML is not standard, yet. This discussion
(a few years old) seems to give a good glimps on how it evolves:

http://stackoverflow.com/questions/3659778/can-i-add-microdata-from-html5-to-a-xhtml-strict-site-and-still-be-compliant

Re: HTML-Info design

[Nic Ferrier]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > How can we know that without even trying to implement a substantial
>   > subset of what Info does?  Your current implementation is 300-line
>   > long; if the addition of at least the basic Info features will take it
>   > to 3000, then what did we gain?
>
> The reasons I am interested in HTML-Info are
...

> If it needs a 3000-line program to work with full Info functionality,
> that's not so big after all.

It does not.

I've done (most) of it.


Nic

Re: HTML-Info design

[Stefan Monnier]

> IOW, your POC proves that it can be done, buit its not enough to
> convince me that it could be done well enough.

Why be so aggressive?  Nic's work is nice and I think the www.gnu.org
manuals would gain by using/developping further such a setup.

But this is largely unrelated to the "inside Emacs manual browsing",
because there's no way Emacs will be able to render this kind of
HTML+Javascript efficiently any time soon, I think.

So, Nic's work is not competing against info.el, and it's actually a work
that tends to vote in favor of keeping Texinfo as the source
documentation format.


        Stefan

Re: HTML-Info design

[Dmitry Gutov]

On 12/23/2014 07:02 PM, Stefan Monnier wrote:

> But this is largely unrelated to the "inside Emacs manual browsing",
> because there's no way Emacs will be able to render this kind of
> HTML+Javascript efficiently any time soon, I think.

As long as the JavaScript code is using some standardized metadata 
present in the document, we could reimplement it in Elisp without too 
much trouble, I think.

Re: HTML-Info design

[Nic Ferrier]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 12/23/2014 07:02 PM, Stefan Monnier wrote:
>
>> But this is largely unrelated to the "inside Emacs manual browsing",
>> because there's no way Emacs will be able to render this kind of
>> HTML+Javascript efficiently any time soon, I think.
>
> As long as the JavaScript code is using some standardized metadata
> present in the document, we could reimplement it in Elisp without too
> much trouble, I think.

I don't know why we would do that.  We don't have a single application
for viewing print documentation and online documentation.

We can have different code, surely?


Nic

Re: HTML-Info design

[Dmitry Gutov]

On 12/23/2014 09:32 PM, Nic Ferrier wrote:

>>> But this is largely unrelated to the "inside Emacs manual browsing",
>>> because there's no way Emacs will be able to render this kind of
>>> HTML+Javascript efficiently any time soon, I think.
>>
>> As long as the JavaScript code is using some standardized metadata
>> present in the document, we could reimplement it in Elisp without too
>> much trouble, I think.
>
> I don't know why we would do that.

To be able to reuse the same document output when we only have an HTML 
rendered, without JS interpreter. Like inside Emacs.

If we want to.

> We don't have a single application
> for viewing print documentation and online documentation.

Yet.

> We can have different code, surely?

I don't understand your objection. Yet, we can have different code.

Especially if the data all different versions of code work with, is the 
same.

Re: HTML-Info design

[Nic Ferrier]

Dmitry Gutov <dgutov@yandex.ru> writes:

>> We can have different code, surely?
>
> I don't understand your objection. Yet, we can have different code.
>
> Especially if the data all different versions of code work with, is
> the same.

I guess I just don't understand why we would attempt to use the same
code for different mediums.

I don't understand why we want to alter info much right now. It seems we
DO want to add the ability to use images. That seems sensible. But
any other changes? why? The only other change I can see being sensible
is that we maybe need to allow additional authorial formats other than
texinfo. But I don't see why that impinges on the info viewer at all
really. That's just conversion of whatever authorial format TO info.

We also want to view info manuals in browsers or mobile phones. We want
to support free software platforms there first. But that still means
an HTML/Javascript/CSS viewer.



Nic

Re: HTML-Info design

[David Kastrup]

[-- Attachment #1: Type: text/plain, Size: 566 bytes --]

Nic Ferrier <nferrier@ferrier.me.uk> writes:

> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>>> We can have different code, surely?
>>
>> I don't understand your objection. Yet, we can have different code.
>>
>> Especially if the data all different versions of code work with, is
>> the same.
>
> I guess I just don't understand why we would attempt to use the same
> code for different mediums.
>
> I don't understand why we want to alter info much right now. It seems we
> DO want to add the ability to use images. That seems sensible.

Huh?  Info supports images.


[-- Attachment #2: infopage.png --]
[-- Type: image/png, Size: 24883 bytes --]

[-- Attachment #3: Type: text/plain, Size: 20 bytes --]



-- 
David Kastrup

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I don't understand why we want to alter info much right now.

I have explained twice why I would like to see HTML-Info.
You may not agree, but please don't keep arguing about it.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Nic Ferrier]

Richard Stallman <rms@gnu.org> writes:

>   > I don't understand why we want to alter info much right now.
>
> I have explained twice why I would like to see HTML-Info.
> You may not agree, but please don't keep arguing about it.

That was not what I was trying to argue against. I would be happy to
make a GNU Info reader in elisp for the existing HTML output from
texinfo. It could be done (I'm not sure I have the time, but I can try).

I've already shown that an app can present the existing HTML sources in
a much more sophisticated way for a modern web browser. IceCat included.

So what I'm saying is we can build tools now for the web and Emacs.

I'm a little reluctant to just pick it all up on my own because I dont
have much time right now.


Nic

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > That was not what I was trying to argue against. I would be happy to
  > make a GNU Info reader in elisp for the existing HTML output from
  > texinfo. It could be done (I'm not sure I have the time, but I can try).

Using the existing HTML output is not the idea I have in mind.
The idea is to develop an HTML-based format which has all the data
needed to provide all the features of Info.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Steinar Bang]

>>>>> Richard Stallman <rms@gnu.org>:

> Using the existing HTML output is not the idea I have in mind.
> The idea is to develop an HTML-based format which has all the data
> needed to provide all the features of Info.

There is a format/practice/convention called RDFa (Resource Description
Format in attributes)[1] that can be used to add semantic information to
HTML pages in a machine readable form.

Perhaps that can be of help? Something that could be used by shr/eww
when navigating the pages?

Basically RDFa consists of adding the properties "property" and
sometimes also "content", to HTML elements.  If there are no elements, a
piece of text can be surrounded by a <span> elment with "property" and
"content" attributes.  The "property" attribute is a URL identifying a
property type, and "content" may be used to present a more machine
friendly form of the element content (e.g. the date in a more easily
parsed form).

[1] http://en.wikipedia.org/wiki/RDFa

Re: HTML-Info design

[Lennart Borgman]

[-- Attachment #1: Type: text/plain, Size: 697 bytes --]

On Dec 26, 2014 12:07 PM, "Steinar Bang" <sb@dod.no> wrote:
>
> >>>>> Richard Stallman <rms@gnu.org>:
>
> > Using the existing HTML output is not the idea I have in mind.
> > The idea is to develop an HTML-based format which has all the data
> > needed to provide all the features of Info.
>
> There is a format/practice/convention called RDFa (Resource Description
> Format in attributes)[1] that can be used to add semantic information to
> HTML pages in a machine readable form.
>
> Perhaps that can be of help? Something that could be used by shr/eww
> when navigating the pages?

Sure it can, but it looks to me like microdata (rather similar to RDFa) is
preferred by the search engines now.

[-- Attachment #2: Type: text/html, Size: 936 bytes --]

Re: HTML-Info design

[Steinar Bang]

>>>>> Lennart Borgman <lennart.borgman@gmail.com>:

> Sure it can, but it looks to me like microdata (rather similar to
> RDFa) is preferred by the search engines now.

Do you mean "microformat"?  I would be very surprised if google et. al
didn't look for both.  Whether they use it to improve or not, will
depend on how much they can trust that the page is what it claims to be.

But that's beside the point.  What I meant that RDFa is a good way to
add semantic information to the page, in a form that can be used by an
emacs HTML-info browser: you can define your own property types in an
RDFa compliant manner.

Re: HTML-Info design

[Lennart Borgman]

[-- Attachment #1: Type: text/plain, Size: 749 bytes --]

On Dec 26, 2014 8:19 PM, "Steinar Bang" <sb@dod.no> wrote:
>
> >>>>> Lennart Borgman <lennart.borgman@gmail.com>:
>
> > Sure it can, but it looks to me like microdata (rather similar to
> > RDFa) is preferred by the search engines now.
>
> Do you mean "microformat"?

It is called microdata, see http://schema.org

>  I would be very surprised if google et. al
> didn't look for both.

So would I be, but at the moment microdata seems to come first.

>
> But that's beside the point.  What I meant that RDFa is a good way to
> add semantic information to the page, in a form that can be used by an
> emacs HTML-info browser: you can define your own property types in an
> RDFa compliant manner.

Yes. And you can do the same with microdata I think.

[-- Attachment #2: Type: text/html, Size: 1129 bytes --]

Re: HTML-Info design

[Steinar Bang]

>>>>> Lennart Borgman <lennart.borgman@gmail.com>:

>> Do you mean "microformat"?

> It is called microdata, see http://schema.org

There is something called "microformat" (which I knew about), as well as
something called "microdata" (which I didn't know about).

They are similar, but different, and you are right that microdata seems
to be able to do the same thing as RDFa.  Both microdata and RDFa are
metaformats for creating semantic markup.

(microformat is more like a set of fixed vocabularies for certain
popular kinds of metadata)

>> I would be very surprised if google et. al didn't look for both.

> So would I be, but at the moment microdata seems to come first.

(That (in all probability) doesn't have anything to do with the formats
used, but the nature of the pages using them, and the number of "good"
pages using them)

>> But that's beside the point.  What I meant that RDFa is a good way to
>> add semantic information to the page, in a form that can be used by an
>> emacs HTML-info browser: you can define your own property types in an
>> RDFa compliant manner.

> Yes. And you can do the same with microdata I think.

Yes, it looks like you can: http://en.wikipedia.org/wiki/Microdata_(HTML)

(Personally I'm partial to RDFa, since I already know RDF and read Tim
Berners-Lee's thoughts on the semantic web way back when, and I just
heard about microdata yesterday)

RE: HTML-Info design

[Phillip Lord]

They both will work. RDFa is slightly more general, in that it maps through to RDF and can be considered
to be an alternative syntax for it.

In the case of HTML-info the level of semantics requires is pretty small. As far as I can see, the knowledge is 
rather limited to a little big of navigational, and index data and some "this is a function", "this is a var" data.




________________________________________
From: emacs-devel-bounces+phillip.lord=newcastle.ac.uk@gnu.org [emacs-devel-bounces+phillip.lord=newcastle.ac.uk@gnu.org] on behalf of Steinar Bang [sb@dod.no]
Sent: 27 December 2014 11:04
To: emacs-devel@gnu.org
Subject: Re: HTML-Info design

>>>>> Lennart Borgman <lennart.borgman@gmail.com>:

>> Do you mean "microformat"?

> It is called microdata, see http://schema.org

There is something called "microformat" (which I knew about), as well as
something called "microdata" (which I didn't know about).

They are similar, but different, and you are right that microdata seems
to be able to do the same thing as RDFa.  Both microdata and RDFa are
metaformats for creating semantic markup.

(microformat is more like a set of fixed vocabularies for certain
popular kinds of metadata)

>> I would be very surprised if google et. al didn't look for both.

> So would I be, but at the moment microdata seems to come first.

(That (in all probability) doesn't have anything to do with the formats
used, but the nature of the pages using them, and the number of "good"
pages using them)

>> But that's beside the point.  What I meant that RDFa is a good way to
>> add semantic information to the page, in a form that can be used by an
>> emacs HTML-info browser: you can define your own property types in an
>> RDFa compliant manner.

> Yes. And you can do the same with microdata I think.

Yes, it looks like you can: http://en.wikipedia.org/wiki/Microdata_(HTML)

(Personally I'm partial to RDFa, since I already know RDF and read Tim
Berners-Lee's thoughts on the semantic web way back when, and I just
heard about microdata yesterday)

Re: HTML-Info design

[Steinar Bang]

>>>>> Phillip Lord <phillip.lord@newcastle.ac.uk>:

> In the case of HTML-info the level of semantics requires is pretty
> small. As far as I can see, the knowledge is rather limited to a
> little big of navigational, and index data and some "this is a
> function", "this is a var" data.

I was mainly thinking about the navigational and index data as
candidates for RDFa markup.  But the "this is a function" and "this is a
var", with programming language information might also be useful for an
HTML-info-specific renderer (or someone else linking to the
documentation, or for search).

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > There is a format/practice/convention called RDFa (Resource Description
  > Format in attributes)[1] that can be used to add semantic information to
  > HTML pages in a machine readable form.

  > Perhaps that can be of help? Something that could be used by shr/eww
  > when navigating the pages?

In principle, it sounds useful, but someone has to try designing the
specifics to see if this approach works.

Would you like to help do that?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > As long as the JavaScript code is using some standardized metadata
  > > present in the document, we could reimplement it in Elisp without too
  > > much trouble, I think.

  > I don't know why we would do that.  We don't have a single application
  > for viewing print documentation and online documentation.

The reason I see, for doing that, is to view HTML-Info documents
inside Emacs.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Stefan Monnier]

>> But this is largely unrelated to the "inside Emacs manual browsing",
>> because there's no way Emacs will be able to render this kind of
>> HTML+Javascript efficiently any time soon, I think.
> As long as the JavaScript code is using some standardized metadata present
> in the document, we could reimplement it in Elisp without too much trouble,
> I think.

I was talking about making Emacs interpret Nic's JS code.

Whereas IIUC you're talking about making Emacs render Texinfo's HTML
output and provide an Info-mode-style UI for it.  That can be done
regardless of Nic's code.

So, it seems we agree: Nic's code is not meant as a replacement for
info.el.


        Stefan

Re: HTML-Info design

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Nic Ferrier <nferrier@ferrier.me.uk>,  stephen@xemacs.org,  rms@gnu.org,  emacs-devel@gnu.org
> Date: Tue, 23 Dec 2014 12:02:06 -0500
> 
> > IOW, your POC proves that it can be done, buit its not enough to
> > convince me that it could be done well enough.
> 
> Why be so aggressive?

I wasn't.  I posted comments and provided my opinions.  If that's not
what was expected, I'm sorry.

> So, Nic's work is not competing against info.el, and it's actually a work
> that tends to vote in favor of keeping Texinfo as the source
> documentation format.

I'm saying I'm not convince this is a viable way of providing a
replacement for info.el.  I thought this was the issue.  If I
misunderstood, I'm sorry.

Re: HTML-Info design

[Nic Ferrier]

Eli Zaretskii <eliz@gnu.org> writes:

> I'm saying I'm not convince this is a viable way of providing a
> replacement for info.el.  I thought this was the issue.  If I
> misunderstood, I'm sorry.

I'm sorry for the misunderstanding. But there is a matter of some
factual disagreement here.

You are suggesting this isn't a valid POC.

I want to make clear this:

* implements node browsing through a toc
* auto completion of toc elements
* auto completion of index elements (how they are displayed is really
  irrelevant)
* info like traversal (collecting the history)
* an info/emacs like keymap

What it does not implement:

* full text search

But I don't see why that is a difficult thing.


Nic

Re: HTML-Info design

[Eli Zaretskii]

> From: Nic Ferrier <nferrier@ferrier.me.uk>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  stephen@xemacs.org,  rms@gnu.org,  emacs-devel@gnu.org
> Date: Tue, 23 Dec 2014 19:40:54 +0000
> 
> You are suggesting this isn't a valid POC.

No, I didn't say that.  Quite the contrary, I said it does constitute
a POC.

> * implements node browsing through a toc
> * auto completion of toc elements
> * auto completion of index elements (how they are displayed is really
>   irrelevant)
> * info like traversal (collecting the history)
> * an info/emacs like keymap
> 
> What it does not implement:
> 
> * full text search
> 
> But I don't see why that is a difficult thing.

Maybe it is, maybe it isn't.  If I knew JS like you do, like I know C
and Lisp and a few other languages, I could probably have an
independent opinion without a working implementation.  But I don't, so
I must look at the implementation, see what it does and what it
doesn't, and decide if I'm convinced.

Please understand that to me, the search functionality -- all kinds of
it -- that we have in Info is the main feature.  That is what makes
Info shine in my eyes.  Not menu completion (in a browser, I'm
supposed to be using the mouse, so I don't really need completion so
much).  Not the Emacs-like keymap (I can configure my browser for that
myself, and actually did that already).  Not Info-like history -- that
is trivial with a Web browser.  All that is much less important than
the features that let me find stuff quickly and efficiently.  So text
search across the entire document is a must, and likewise substring
matching in index entries and support for several index nodes.
Without this, a documentation browser is not worth it for me.

Now, it could well be that adding those is a piece of cake, but I just
don't know that.  Until you show me that it is indeed doable without
jumping through hoops, and can work on several popular browsers, I
won't be convinced, sorry.

Likewise with the directionality of the index prompt -- it might be a
minor issue for a POC, but if it turns out that it is insoluble in
principle, it's a showstopper, I hope you agree.

And there are other valuable features in Info, please take a look at
the commands in inf*.el files.  Some of them probably won't be
possible or practical with www+js technique, like index-apropos, for
example.  Which is a pity, at least for me, but what bothers me is how
many more of such important features will have to be thrown away, how
much useful functionality we will have to give up?

We are talking about replacing an existing browser, one that is
developed for decades and is chock-full of useful features.  I use
many of them every day.  People can say it's "samizdat" all they want,
but for me it is a tremendously useful tool.  If you want to convince
me that the replacement will be worthy, you need to show that those
important features, or at least some non-trivial subset thereof, can
be had with your suggested approach.  And given my JS ignorance, the
only way of showing that is by implementing it.  Because I have no
idea what will it take to write a multi-file regexp search in JS.

I hope I made myself clear.

Once again, thanks for working on this.

Re: HTML-Info design

[Nic Ferrier]

Eli Zaretskii <eliz@gnu.org> writes:

> Now, it could well be that adding those is a piece of cake, but I just
> don't know that.  Until you show me that it is indeed doable without
> jumping through hoops, and can work on several popular browsers, I
> won't be convinced, sorry.

Ok. It's not a piece of cake. But it's totally doable. I guess you don't
have to believe me.

> Likewise with the directionality of the index prompt -- it might be a
> minor issue for a POC, but if it turns out that it is insoluble in
> principle, it's a showstopper, I hope you agree.

It's not insoluble though. It's absolutely trivial.

What's not trivial is whether this is *the right* thing to do or
not. I'm not changing it to the way you want it just because you want it
like that. The web browser is a different medium to an emacs frame and
so maybe a different presentation is necessary.


> And there are other valuable features in Info, please take a look at
> the commands in inf*.el files.  Some of them probably won't be
> possible or practical with www+js technique, like index-apropos, for
> example.  Which is a pity, at least for me, but what bothers me is how
> many more of such important features will have to be thrown away, how
> much useful functionality we will have to give up?

Everything info does is possible in a browser. I would not choose to
implement an exact copy not because it's not possible but because it's a
different medium.

Btw I've been using Emacs and Info for years as well, like, you, so I do
know how it works and commands like index-apropos.

Incidentally this is one of the reasons I've chosen to do what I'm doing
instead of working on something visible.


> We are talking about replacing an existing browser, one that is
> developed for decades and is chock-full of useful features.  I use
> many of them every day.

We aren't talking about doing that as far as I'm concerned.


> People can say it's "samizdat" all they want, but for me it is a
> tremendously useful tool.  If you want to convince me that the
> replacement will be worthy, you need to show that those important
> features, or at least some non-trivial subset thereof, can be had with
> your suggested approach.  And given my JS ignorance, the only way of
> showing that is by implementing it.  Because I have no idea what will
> it take to write a multi-file regexp search in JS.

I don't think there's any point in snapping our fingers and replacing
info. We couldn't possibly hope to do that. I think we commit to
replacing it with better tools over 10 years.

People are talking about changing the authorial format. I don't think
that is related very much to making new viewers. Any new authorial
format would necessarily have to work with the existing info app, at
least for a while.

If we don't commit to changing the authorial format, over time and
improving the viewer, over time, then I think we're guilty of the
accusation esr has levelled against us of being stick in the muds.



Nic

Re: HTML-Info design

[Eli Zaretskii]

> From: Nic Ferrier <nferrier@ferrier.me.uk>
> Cc: stephen@xemacs.org,  emacs-devel@gnu.org,  monnier@iro.umontreal.ca,  rms@gnu.org
> Date: Tue, 23 Dec 2014 21:09:50 +0000
> 
> > Now, it could well be that adding those is a piece of cake, but I just
> > don't know that.  Until you show me that it is indeed doable without
> > jumping through hoops, and can work on several popular browsers, I
> > won't be convinced, sorry.
> 
> Ok. It's not a piece of cake. But it's totally doable. I guess you don't
> have to believe me.

I need to see at least part of it to believe.

> > Likewise with the directionality of the index prompt -- it might be a
> > minor issue for a POC, but if it turns out that it is insoluble in
> > principle, it's a showstopper, I hope you agree.
> 
> It's not insoluble though. It's absolutely trivial.

Then humor me and show how trivial it is.  This is still a POC, not
the real thing, right?  We don't need to argue now whether it's
absolutely TRT, just that it's possible, right?

> What's not trivial is whether this is *the right* thing to do or
> not. I'm not changing it to the way you want it just because you want it
> like that. The web browser is a different medium to an emacs frame and
> so maybe a different presentation is necessary.

Not sure who else you will ask whether it is or isn't TRT, but at this
stage I'm only worried whether it's technically possible.

> > We are talking about replacing an existing browser, one that is
> > developed for decades and is chock-full of useful features.  I use
> > many of them every day.
> 
> We aren't talking about doing that as far as I'm concerned.

Then I don't know what are we talking about.

> I don't think there's any point in snapping our fingers and replacing
> info. We couldn't possibly hope to do that. I think we commit to
> replacing it with better tools over 10 years.

Based on a 20-year experience in working on Emacs development, I don't
believe in such long-term plans.  There's no stable organization here
to carry out such plans.  Only some single motivated individual could
do it, assuming we have on board someone who'd dedicate the next 10
years (or whatever it takes) to this job.

> If we don't commit to changing the authorial format, over time and
> improving the viewer, over time, then I think we're guilty of the
> accusation esr has levelled against us of being stick in the muds.

Again, based on my experience here, there's no way of "us committing"
to something of this scale.  It doesn't work like that in Emacs.
Significant new features are developed by very small groups, mostly
single individuals, then dumped on the community one bright day, and
we spend the next 5 or 10 years improving those features, fixing bugs,
and learning to live with them.

Thanks.

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I don't think anyone needs to prove that HTML-Info can be implemented
with Javascript code.  I think it can be.

Would someone like to help do it?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Nic Ferrier]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> I don't think anyone needs to prove that HTML-Info can be implemented
> with Javascript code.  I think it can be.
>
> Would someone like to help do it?


Well, since I already started... I guess I should volunteer.


Nic

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Well, since I already started... I guess I should volunteer.

Would you like to organize the project?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Nic Ferrier]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Well, since I already started... I guess I should volunteer.
>
> Would you like to organize the project?

I honestly don't know. I'm not sure we think the same things about it.

What would be success?


Nic

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > What would be success?

My idea of the goal is

1. A spec for HTML-Info format, saying how the Info data such as
menus, indices and node structure navigation are represented
in HTML.

2. Something to generate HTML-Info from Texinfo input.

3. An extension for Firefox to implement Info-style commands
using that data.

4. Emacs Lisp code to browse HTML-Info files.

#1 is conceptually prior to the others, but I would
develop it and #3 together.

Any comments on this plan?

Are you interested in working on it?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Lennart Borgman]

On Sat, Dec 27, 2014 at 11:54 PM, Richard Stallman <rms@gnu.org> wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > What would be success?
>
> My idea of the goal is
>
> 3. An extension for Firefox to implement Info-style commands
> using that data.


If you mean a specific extension for Firefox I think that might be a
waste of time. To me it looks much better to use just JavaScript and a
web server with a search engine like http://opensearchserver.com/.
(The software, not the host, though.)

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > If you mean a specific extension for Firefox I think that might be a
  > waste of time. To me it looks much better to use just JavaScript and a
  > web server with a search engine like http://opensearchserver.com/.

I've already stated why I have rejected this approach.  Please don't
keep arguing about a question that has been decided.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Nic Ferrier]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > What would be success?
>
> My idea of the goal is
>
> 1. A spec for HTML-Info format, saying how the Info data such as
> menus, indices and node structure navigation are represented
> in HTML.
>
> 2. Something to generate HTML-Info from Texinfo input.
>
> 3. An extension for Firefox to implement Info-style commands
> using that data.
>
> 4. Emacs Lisp code to browse HTML-Info files.
>
> #1 is conceptually prior to the others, but I would
> develop it and #3 together.
>
> Any comments on this plan?
>
> Are you interested in working on it?

What you are asking for is consistent with what I have been working on.

I would say I have made a first pass at 1. and 2. and 3. (presuming 3 is
ok as just a "normal" web application).

I could start to do 4, as I've said.

It is a BIG job. Getting really good Info like behaviour into a browser
isn't trivial. It's also a thankless task (as I believe has been proved
on this thread).

Replacing Info in Emacs with an HTML based Info will also, I suspect, be
thankless.

I cannot commit to deliver this in the first quarter or anything like
that.


But I can promise to keep working on what I've got and making it better.

What else would I need to do?


Nic

Re: HTML-Info design

[David Kastrup]

Nic Ferrier <nferrier@ferrier.me.uk> writes:

> It is a BIG job. Getting really good Info like behaviour into a browser
> isn't trivial. It's also a thankless task (as I believe has been proved
> on this thread).

Replacing an existing solution with existing userbase with something
that is bound to have worse performance and/or less integration with
Emacs is going to be thankless, obviously.  The people most expected to
profit from it are not even using preinstalled documentation yet.

So any thanks will take years to arrive.  The most thanks you can expect
from existing Info users is that it works hardly worse than before.

And it will be quite a bit of work to let the current "I websearch for
everything" target clientele lean towards changing their ways as well.

So yes: this will very likely require a lot of self-motivation before
satisfactory external appreciation is going to come in.

> Replacing Info in Emacs with an HTML based Info will also, I suspect,
> be thankless.
>
> I cannot commit to deliver this in the first quarter or anything like
> that.
>
>
> But I can promise to keep working on what I've got and making it
> better.
>
> What else would I need to do?

I think it might make sense trying to work out a roadmap that draws in
some helpers eager to prove themselves.  Problem with that is that those
might want to work on more "cutting-edge" solutions rather than what
simple browsers support.

-- 
David Kastrup

Re: HTML-Info design

[Nic Ferrier]

David Kastrup <dak@gnu.org> writes:

>> But I can promise to keep working on what I've got and making it
>> better.
>>
>> What else would I need to do?
>
> I think it might make sense trying to work out a roadmap that draws in
> some helpers eager to prove themselves.  Problem with that is that those
> might want to work on more "cutting-edge" solutions rather than what
> simple browsers support.

I never said I was supporting simple browsers.

Emacs and HTML5 is what I am supporting. I can't see how I could support
simple browsers, by which I mean Lynx, links, e-links, w3 and even eww.

Of those eww has the most chance of someone fixing it to support Info
like behaviour. But I don't think I'll be doing it.

For clarification, one path I am considering for Emacs HTML/Info would
be to use shr.


Nic

Re: HTML-Info design

[Steinar Bang]

>>>>> Nic Ferrier <nferrier@ferrier.me.uk>:

> Emacs and HTML5 is what I am supporting. I can't see how I could support
> simple browsers, by which I mean Lynx, links, e-links, w3 and even eww.

> Of those eww has the most chance of someone fixing it to support Info
> like behaviour. But I don't think I'll be doing it.

> For clarification, one path I am considering for Emacs HTML/Info would
> be to use shr.

I thought eww and shr were two sides of the same coin?

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Emacs and HTML5 is what I am supporting. I can't see how I could support
  > simple browsers, by which I mean Lynx, links, e-links, w3 and even eww.

The HTML-Info files will display fine in these browsers, since they
will contain proper standard HTML.  These browsers will not provide
the special Info navigation, menu and index commands, but they will
work.

  > For clarification, one path I am considering for Emacs HTML/Info would
  > be to use shr.

What is shr?  What are the implications of this?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Nic Ferrier]

Richard Stallman <rms@gnu.org> writes:

> What is shr?  What are the implications of this?

shr is part of Emacs. It's part of the eww browser and gnus.


Nic

Re: HTML-Info design

[Filipp Gunbin]

On 28/12/2014 09:59 +0000, Nic Ferrier wrote:

> David Kastrup <dak@gnu.org> writes:
>
>>> But I can promise to keep working on what I've got and making it
>>> better.
>>>
>>> What else would I need to do?
>>
>> I think it might make sense trying to work out a roadmap that draws in
>> some helpers eager to prove themselves.  Problem with that is that those
>> might want to work on more "cutting-edge" solutions rather than what
>> simple browsers support.
>
> I never said I was supporting simple browsers.
>
> Emacs and HTML5 is what I am supporting. I can't see how I could support
> simple browsers, by which I mean Lynx, links, e-links, w3 and even
> eww.

emacs-w3m is also worth mentioning.  That's an enigma for me, why people
generally ignore it.  Is it flawed in some way? (like licensing or
whatever).  The approach it takes looks very natural to me: delegate
html & css processing to external program (w3m) and build a good Emacs
interface (emacs-w3m).  Its rendering of mail messages looks more
natural to me than eww's (this is not against the eww).  It deals with
tables very well.  It allows to disable tabs to make every opened page
be a usual Emacs buffer.  Not to mention the ability to separately
improve w3m (and get those improvements in console usage, too) and
interface (improvements visible only in Emacs, of course).

Generally I like the idea that tedious work in specific areas should be
delegated to some existing and supported library (like xml handling to
libxml2).

The good example is soap-client.  While it's great that some SOAP
support exists in emacs, I found it unusable for JAX-WS (Java web
services framework) SOAP services.  I wasn't able to fix it in a
reasonable amount of time (there were mostly namespace prefix - related
problems AFAIR).  Rather, it makes more sense to use some
well-maintained library to do the core job (though, I don't know whether
some exists for SOAP).

This is off-topic here, I know, but this problem concerns me for a long
time, and I couldn't resist, sorry.

Filipp

soap-client (Was: HTML-Info design)

[Michael Albinus]

Filipp Gunbin <fgunbin@fastmail.fm> writes:

> The good example is soap-client.  While it's great that some SOAP
> support exists in emacs, I found it unusable for JAX-WS (Java web
> services framework) SOAP services.  I wasn't able to fix it in a
> reasonable amount of time (there were mostly namespace prefix - related
> problems AFAIR).  Rather, it makes more sense to use some
> well-maintained library to do the core job (though, I don't know whether
> some exists for SOAP).
>
> This is off-topic here, I know, but this problem concerns me for a long
> time, and I couldn't resist, sorry.

You might file a bug report. People are working to extend soap-client.el
in order to support more complex SOAP services (for example an Exchange
Server). Your problem could get attention there.

> Filipp

Best regards, Michael.

Re: HTML-Info design

[David Kastrup]

David Kastrup <dak@gnu.org> writes:

> Nic Ferrier <nferrier@ferrier.me.uk> writes:
>
>> It is a BIG job. Getting really good Info like behaviour into a browser
>> isn't trivial. It's also a thankless task (as I believe has been proved
>> on this thread).
>
> Replacing an existing solution with existing userbase with something
> that is bound to have worse performance and/or less integration with
> Emacs is going to be thankless, obviously.  The people most expected to
> profit from it are not even using preinstalled documentation yet.
>
> So any thanks will take years to arrive.  The most thanks you can expect
> from existing Info users is that it works hardly worse than before.
>
> And it will be quite a bit of work to let the current "I websearch for
> everything" target clientele lean towards changing their ways as well.
>
> So yes: this will very likely require a lot of self-motivation before
> satisfactory external appreciation is going to come in.

As another data point: quite a few people here claimed that Texinfo 5's
drop of compilation speed was a deal breaker for them.

Compilation speed.  And you are working on a replacement for an
interactive application that is supposed to tie into one's editing
workflow.  And it is supposed to tie into Emacs workflows by engaging an
Emacs-based HTML browser.

That's a long long uphill battle.  So one should very much focus on
thinking about game-changing workflow features implementable in the HTML
backend that are not accessible from Info.

-- 
David Kastrup

Re: HTML-Info design

[Nic Ferrier]

David Kastrup <dak@gnu.org> writes:

> As another data point: quite a few people here claimed that Texinfo 5's
> drop of compilation speed was a deal breaker for them.
>
> Compilation speed.  And you are working on a replacement for an
> interactive application that is supposed to tie into one's editing
> workflow.  And it is supposed to tie into Emacs workflows by engaging an
> Emacs-based HTML browser.
>
> That's a long long uphill battle.  So one should very much focus on
> thinking about game-changing workflow features implementable in the HTML
> backend that are not accessible from Info.

I don't really get this. We're not replacing texinfo. We're replacing
Info.

We're replacing Info so you can generate documentation in multiple
authorial formats, easily write converters for your authorial format of
choice into one Info and have it displayed the same.

So I don't think compilation speed is much of a problem.

It won't change as a result of what I do.


I would hope to alter the EmacsLisp texinfo generator as an initial POC
for rms' objective #2 - "Something to generate HTML-Info from Texinfo
input"


Nic

Re: HTML-Info design

[David Kastrup]

Nic Ferrier <nferrier@ferrier.me.uk> writes:

> David Kastrup <dak@gnu.org> writes:
>
>> As another data point: quite a few people here claimed that Texinfo 5's
>> drop of compilation speed was a deal breaker for them.
>>
>> Compilation speed.  And you are working on a replacement for an
>> interactive application that is supposed to tie into one's editing
>> workflow.  And it is supposed to tie into Emacs workflows by engaging an
>> Emacs-based HTML browser.
>>
>> That's a long long uphill battle.  So one should very much focus on
>> thinking about game-changing workflow features implementable in the HTML
>> backend that are not accessible from Info.
>
> I don't really get this. We're not replacing texinfo. We're replacing
> Info.
>
> We're replacing Info so you can generate documentation in multiple
> authorial formats, easily write converters for your authorial format of
> choice into one Info and have it displayed the same.
>
> So I don't think compilation speed is much of a problem.

Exactly.  People were annoyed enough at compilation speed to call it a
game changer, but it is really of very little relevance when compared
with raw interactive response.  And raw interactive response is where
you have to compete with Info.

With a crowd that already calls a change in *compilation* speed from
Texinfo 4 to Texinfo 5 inacceptable, getting InfoHTML to acceptable
interactive performance when compared with Info is going to be quite the
hard pitch.

-- 
David Kastrup

Re: HTML-Info design

[Nic Ferrier]

David Kastrup <dak@gnu.org> writes:

> With a crowd that already calls a change in *compilation* speed from
> Texinfo 4 to Texinfo 5 inacceptable, getting InfoHTML to acceptable
> interactive performance when compared with Info is going to be quite the
> hard pitch.

Now I understand your point. I agree.

It just makes me grumpier, doesn't make me stop.


Nic

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I would say I have made a first pass at 1. and 2. and 3.

That's a good way to start, because it will include designing the
format and verifying it can really server the purpose.  That's the
crucial part of the whole thing.

							     (presuming 3 is
  > ok as just a "normal" web application).

We don't want to put it into service as a "web application", but it is
ok to develop it that way at first.  We can then adapt the code to
be loaded as a browser extension.

  > But I can promise to keep working on what I've got and making it better.

Thank you.

  > What else would I need to do?

I think it would be good to set up a project and give others a way to
join in.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Nic Ferrier]

Richard Stallman <rms@gnu.org> writes:

>   > (presuming 3 is
>   > ok as just a "normal" web application).
>
> We don't want to put it into service as a "web application", but it is
> ok to develop it that way at first.  We can then adapt the code to
> be loaded as a browser extension.

I disagree with you there. I don't think a browser extension is very
sensible (for various reasons which I'm happy to go into later). I think
a web application that does not require a network connection is a much
better idea. This is possible.

But, of course, you are right that we don't have to have this discussion
now. We can revisit it later once there is something that works.


>   > What else would I need to do?
>
> I think it would be good to set up a project and give others a way to
> join in.

I can promise to do that. I'll set something up on savannah and keep the
Emacs client, the HTML based one and the texinfo->HTML generator there.


Nic

Re: HTML-Info design

[Stefan Monnier]

> 4. Emacs Lisp code to browse HTML-Info files.

I think this will be most difficult part, so better focus on this part
than on the #3 which should be a piece of cake once we have an Elisp
solution for the rendering.


        Stefan

Re: HTML-Info design

[Nic Ferrier]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> 4. Emacs Lisp code to browse HTML-Info files.
>
> I think this will be most difficult part, so better focus on this part
> than on the #3 which should be a piece of cake once we have an Elisp
> solution for the rendering.

That seems a reasonable request.

I believe I can make some progress on the logistical problems that have
stalled my existing HTML 5 solution as well as making some progress on
an Emacs HTML/Info solution.


Nic

Re: HTML-Info design

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> 4. Emacs Lisp code to browse HTML-Info files.
>
> I think this will be most difficult part, so better focus on this part
> than on the #3 which should be a piece of cake once we have an Elisp
> solution for the rendering.

I don't see why this would be difficult.

`M-x eww RET
http://www.gnu.org/software/emacs/manual/html_node/emacs/index.html RET'

and you're off.  The `n'/`p'/`u' commands work as in Info already, so
the only thing that's missing is the index (which is trivial to add).

And a search engine, which sounds like a fun weekend project for
somebody who finds that sort of thing amusing.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Nic Ferrier]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> 4. Emacs Lisp code to browse HTML-Info files.
>>
>> I think this will be most difficult part, so better focus on this part
>> than on the #3 which should be a piece of cake once we have an Elisp
>> solution for the rendering.
>
> I don't see why this would be difficult.
>
> `M-x eww RET
> http://www.gnu.org/software/emacs/manual/html_node/emacs/index.html RET'
>
> and you're off.  The `n'/`p'/`u' commands work as in Info already, so
> the only thing that's missing is the index (which is trivial to add).
>
> And a search engine, which sounds like a fun weekend project for
> somebody who finds that sort of thing amusing.

Do that then.

But I think the problems with that will be the problems with eww. Poor
caching (your solution would immediately invalidate what Stefan said was
his #1), difficult to make multiple buffers, etc...

Also, I think it would be unacceptable to most people. A separate Info
viewer is desired.


Nic

Re: HTML-Info design

[David Kastrup]

Nic Ferrier <nferrier@ferrier.me.uk> writes:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>
>>>> 4. Emacs Lisp code to browse HTML-Info files.
>>>
>>> I think this will be most difficult part, so better focus on this part
>>> than on the #3 which should be a piece of cake once we have an Elisp
>>> solution for the rendering.
>>
>> I don't see why this would be difficult.
>>
>> `M-x eww RET
>> http://www.gnu.org/software/emacs/manual/html_node/emacs/index.html RET'
>>
>> and you're off.  The `n'/`p'/`u' commands work as in Info already, so
>> the only thing that's missing is the index (which is trivial to add).
>>
>> And a search engine, which sounds like a fun weekend project for
>> somebody who finds that sort of thing amusing.
>
> Do that then.
>
> But I think the problems with that will be the problems with eww. Poor
> caching (your solution would immediately invalidate what Stefan said was
> his #1), difficult to make multiple buffers, etc...
>
> Also, I think it would be unacceptable to most people. A separate Info
> viewer is desired.

It's probably more realistic to arrive at acceptable semantics that way,
but it would likely still be sensible to share transports, general HTML
parsing and a number of other things (like most of the rendering) with
eww.

-- 
David Kastrup

Re: HTML-Info design

[Lars Ingebrigtsen]

David Kastrup <dak@gnu.org> writes:

> It's probably more realistic to arrive at acceptable semantics that way,
> but it would likely still be sensible to share transports, general HTML
> parsing and a number of other things (like most of the rendering) with
> eww.

An HTML-based manual viewer would just be a very thin minor mode on top
of eww.

Honestly, I have no idea what all y'all are talking about in this
thread, what with the "semantic markup" (why?) and stuff.  Texinfo
generates perfectly fine HTML that all web browsers render perfectly.
What's missing is easy access to the index (which is easy enough to
Javascript together) and a search engine (which is the fun bit, but has
nothing (or little) to do with the format or the tools).

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Nic Ferrier]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> David Kastrup <dak@gnu.org> writes:
>
>> It's probably more realistic to arrive at acceptable semantics that way,
>> but it would likely still be sensible to share transports, general HTML
>> parsing and a number of other things (like most of the rendering) with
>> eww.
>
> An HTML-based manual viewer would just be a very thin minor mode on top
> of eww.
>
> Honestly, I have no idea what all y'all are talking about in this
> thread, what with the "semantic markup" (why?) and stuff.  Texinfo
> generates perfectly fine HTML that all web browsers render perfectly.
> What's missing is easy access to the index (which is easy enough to
> Javascript together) and a search engine (which is the fun bit, but has
> nothing (or little) to do with the format or the tools).

Semantic markup matters to define the meaning of HTML-Info better such
that other tools might generate it.

I don't agree that we could "just use eww" for an HTML-Info viewer but
shr is an obvious starting point.


Nic

Re: HTML-Info design

[Ivan Shmakov]

>>>>> Nic Ferrier <nferrier@ferrier.me.uk> writes:

[…]

 > I don't agree that we could "just use eww" for an HTML-Info viewer
 > but shr is an obvious starting point.

	I tend to disagree.  SHR does the rendering and little else; on
	the contrary, EWW has provides for at least some navigation
	(including Info-style eww-up-url, eww-next-url, etc.)

	Having developed my own little code that uses Emacs
	HTML-rendering facilities, I’m pretty sure that EWW is the way
	to go.  Unless you need to render HTML in a “non-browsing”
	buffer, that is.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Search engines (was: HTML-Info design)

[Lars Ingebrigtsen]

An interesting problem is Emacs and searching.  The various Gnus mail
backends have various searching interfaces, but it would be nice to have
a more general Emacs searching mechanism, where you can search parts of
directory trees, etc.

Has anybody looked into hooking into the various OS-level search
interfaces?  Spotlight in OS X, the Explorer thingie in Windows and
<something> in the various GNU/Linux distributions?

A basic search engine is easy to implement (I've done it a couple of
times), but the main work is in understanding the various file formats
and keeping the reverse index updated when files change.  If there were
OS-level search stuff we could just hook into, that would obviously be
better.

And provide searching in the Emacs manuals.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Search engines (was: HTML-Info design)

[Lennart Borgman]

On Sun, Dec 28, 2014 at 7:00 PM, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> An interesting problem is Emacs and searching.  The various Gnus mail
> backends have various searching interfaces, but it would be nice to have
> a more general Emacs searching mechanism, where you can search parts of
> directory trees, etc.
>
> Has anybody looked into hooking into the various OS-level search
> interfaces?  Spotlight in OS X, the Explorer thingie in Windows and
> <something> in the various GNU/Linux distributions?
>
> A basic search engine is easy to implement (I've done it a couple of
> times), but the main work is in understanding the various file formats
> and keeping the reverse index updated when files change.  If there were
> OS-level search stuff we could just hook into, that would obviously be
> better.
>
> And provide searching in the Emacs manuals.


I did that some years ago and posted about it here.

I was mostly looking for a faster grep in the files that I did not
change, i e Emacs internals (C/Elisp). It worked pretty well then. The
interface is very similar to grep.

The search enginges I tried was MS Windows built in and Lucene (not
sure there, but I think it was Lucene).

I put the code in nXhtml, but have not touched it since then.

Looking at http://opensearchserver.com/ I think it would be much
easier to implement it now. The API there looks pretty good.

Re: Search engines (was: HTML-Info design)

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sun, 28 Dec 2014 19:00:16 +0100
> 
> An interesting problem is Emacs and searching.  The various Gnus mail
> backends have various searching interfaces, but it would be nice to have
> a more general Emacs searching mechanism, where you can search parts of
> directory trees, etc.

Search for what, exactly?  Only for text inside files (like Mairix
does for mail archives), or something more general?

With the exception of text files, searching anything else for text
needs libraries or programs that are capable of accessing text within
those files, and also some way of providing an easily-followable
reference to the found text.  Do we really have Free Software
solutions for these problems?

> the main work is in understanding the various file formats
> and keeping the reverse index updated when files change.

The second part is a cron job, nothing else.  Or am I missing
something?

> If there were OS-level search stuff we could just hook into, that
> would obviously be better.

Is there?

> And provide searching in the Emacs manuals.

We already have "M-x info-apropos", which comes close.

Re: Search engines

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> With the exception of text files, searching anything else for text
> needs libraries or programs that are capable of accessing text within
> those files, and also some way of providing an easily-followable
> reference to the found text.  Do we really have Free Software
> solutions for these problems?

No, that's why I said:

>> the main work is in understanding the various file formats

:-)

>> and keeping the reverse index updated when files change.
>
> The second part is a cron job, nothing else.  Or am I missing
> something?

It's really nice if the index is updated in real time.  If I understand
correctly, (some of) the OS-level search indexes allow that.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Search engines

[Lennart Borgman]

On Mon, Dec 29, 2014 at 5:37 PM, Lars Ingebrigtsen <larsi@gnus.org> wrote:
>
> It's really nice if the index is updated in real time.  If I understand
> correctly, (some of) the OS-level search indexes allow that.

You can do that with http://opensearchserver.com/ too (but it is not
builtin, you will have to add your own notification service - which is
rather easy).

Re: HTML-Info design

[Lars Ingebrigtsen]

Nic Ferrier <nferrier@ferrier.me.uk> writes:

> But I think the problems with that will be the problems with eww. Poor
> caching (your solution would immediately invalidate what Stefan said was
> his #1), difficult to make multiple buffers, etc...

Huh?  Caching?  The (HTML) Info files are in the Emacs distribution, so
eww would just load them from disk.

And it's not more difficult to make multiple buffers in eww than it is
in Info.

Etc.

> Also, I think it would be unacceptable to most people. A separate Info
> viewer is desired.

That's indeed your claim, yes.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Nic Ferrier]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Nic Ferrier <nferrier@ferrier.me.uk> writes:
>
>> But I think the problems with that will be the problems with eww. Poor
>> caching (your solution would immediately invalidate what Stefan said was
>> his #1), difficult to make multiple buffers, etc...
>
> Huh?  Caching?  The (HTML) Info files are in the Emacs distribution, so
> eww would just load them from disk.

Right.

So it's not just:

`M-x eww RET
http://www.gnu.org/software/emacs/manual/html_node/emacs/index.html RET'

as you claimed.

Instead it would be:

M-x eww file:///home/nicferrier/emacs-24.4/share/info...

Oh wait. The HTML files are in the distribution and not the
installation... so we could alter the installation... but then where
would we get them from? I guess users will want a wrapper command around
eww....


There are lots of things to sort out. I'm not sure eww is the right
starting point.

But if you think it is then go to it! Seriously, I am not that
enthusiastic about taking on another job. I'll get it done but you're
already sniping about how I'm talking about doing it so I'd be really
happy to cheer you on instead of listening to you complain about how I'm
suggesting approaching the problem.


> And it's not more difficult to make multiple buffers in eww than it is
> in Info.

Specifically you can easily create a new buffer link from an existing
Info link (`info-menu'). I don't think you can do that in eww yet. Or am
I wrong?


Nic

Re: HTML-Info design

[Lars Ingebrigtsen]

Nic Ferrier <nferrier@ferrier.me.uk> writes:

> Oh wait. The HTML files are in the distribution and not the
> installation... so we could alter the installation... but then where
> would we get them from? I guess users will want a wrapper command around
> eww....

If we were to use a web browser internally in Emacs to look at the
documentation, there would be a command for doing that.  Like, say,
`C-h i'.

> There are lots of things to sort out. I'm not sure eww is the right
> starting point.

So you keep claiming, but you haven't said why.

> But if you think it is then go to it!

Like I said, I no clear idea what all y'all are after here.  I think
Info works just fine inside of Emacs, and I see no particular reason to
move to anything else.

Moving to HTML might offer more flexible rendering, though, and I can
sort of see some slight advantages to having the same files in Emacs as
on the web, but, like, meh.

> Specifically you can easily create a new buffer link from an existing
> Info link (`info-menu'). I don't think you can do that in eww yet. Or am
> I wrong?

(defun eww-browse-url (url &optional new-window)
  ...)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Steinar Bang]

>>>>> Lars Ingebrigtsen <larsi@gnus.org>:

> Like I said, I no clear idea what all y'all are after here.  I think
> Info works just fine inside of Emacs, and I see no particular reason to
> move to anything else.

Me neither, and if there was, that sxml stuff mentioned earlier, was
kind of cool (I wish I could find a use for it, but libxml2 parsing in
emacs kind of takes away the advantage of the lisp reader... except
maybe for big files...?).

> Moving to HTML might offer more flexible rendering, though, and I can
> sort of see some slight advantages to having the same files in Emacs as
> on the web, but, like, meh.

Well, the web rendering could do with some improvements: properly
balanced tags, and Nic's JavaScript.

Maybe a bit of CSS as well..?

Re: HTML-Info design

[Lars Ingebrigtsen]

Steinar Bang <sb@dod.no> writes:

> Well, the web rendering could do with some improvements: properly
> balanced tags, and Nic's JavaScript.

Redundant end tags is not a requirement for proper HTML.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Steinar Bang]

>>>>> Lars Ingebrigtsen <larsi@gnus.org>:

> Steinar Bang <sb@dod.no> writes:
>> Well, the web rendering could do with some improvements: properly
>> balanced tags, and Nic's JavaScript.

> Redundant end tags is not a requirement for proper HTML.

Not if HTML5 constitutes "proper HTML", no.

<rant removed>

[OT] HTML5

[Ivan Shmakov]

>>>>> Steinar Bang <sb@dod.no> writes:
>>>>> Lars Ingebrigtsen <larsi@gnus.org>:
>>>>> Steinar Bang <sb@dod.no> writes:

 >>> Well, the web rendering could do with some improvements: properly
 >>> balanced tags, and Nic's JavaScript.

 >> Redundant end tags is not a requirement for proper HTML.

 > Not if HTML5 constitutes "proper HTML", no.

	HTML5 explicitly allows for (and gives more or less equal
	standing to) /both/ XML and “classic” HTML markup [1]:

    This specification defines an abstract language for describing
    documents and applications, […]

    There are various concrete syntaxes that can be used to transmit
    resources that use this abstract language, two of which are defined in
    this specification.

    The first such concrete syntax is the HTML syntax.  […]

    The second concrete syntax is the XHTML syntax, which is an
    application of XML.  […]

	Moreover, the HTML5 TR makes certain provisions for the HTML
	syntax, which make it possible to represent a sheer class of
	documents in a form that’d be both syntactically-valid HTML
	/and/ well-formed XML at the same time.  Namely:

	• <element /> may be used to represent any of the void elements;

	• the xml:lang attribute is allowed;

	• and so is xmlns.

[1] http://www.w3.org/TR/html5/introduction.html#html-vs-xhtml

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: HTML-Info design

[Yuri Khan]

On Mon, Dec 29, 2014 at 2:00 AM, Lars Ingebrigtsen <larsi@gnus.org> wrote:

>> Well, the web rendering could do with some improvements: properly
>> balanced tags, and Nic's JavaScript.
>
> Redundant end tags is not a requirement for proper HTML.

Right, but they are helpful for debugging, verification and maintenance.

With explicit closing tags, it is immediately visible where the author
(or their tool) intended the element to end. Modifying the HTML
generation logic only involves ensuring that nesting is not broken.

With implicit tags, browsers can and will infer tag nesting on their
own, and have an intricate system of rules to do so. Modifying the
logic involves carefully working out where browsers would infer the
missing tags, and then work with that knowledge to ensure that nesting
won’t break.

Re: HTML-Info design

[Lars Ingebrigtsen]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> With explicit closing tags, it is immediately visible where the author
> (or their tool) intended the element to end. Modifying the HTML
> generation logic only involves ensuring that nesting is not broken.
>
> With implicit tags, browsers can and will infer tag nesting on their
> own, and have an intricate system of rules to do so. Modifying the
> logic involves carefully working out where browsers would infer the
> missing tags, and then work with that knowledge to ensure that nesting
> won’t break.

Yeah, that's why all Python code looks like

 for x in range(10): # THE LINE ENDED THERE
     squares.append(x**2)  # THE FOR ENDS HERE I PROMISE!!!

It's then immediately visible where the author intended the lines to
end.

XHTML was history over five years ago.  It's time you XML fanatics
accept that HTML is a different language with different rules and stop
this incessant kvetching. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Nic Ferrier]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Yuri Khan <yuri.v.khan@gmail.com> writes:
>
>> With explicit closing tags, it is immediately visible where the author
>> (or their tool) intended the element to end. Modifying the HTML
>> generation logic only involves ensuring that nesting is not broken.
>>
>> With implicit tags, browsers can and will infer tag nesting on their
>> own, and have an intricate system of rules to do so. Modifying the
>> logic involves carefully working out where browsers would infer the
>> missing tags, and then work with that knowledge to ensure that nesting
>> won’t break.
>
> Yeah, that's why all Python code looks like
>
>  for x in range(10): # THE LINE ENDED THERE
>      squares.append(x**2)  # THE FOR ENDS HERE I PROMISE!!!
>
> It's then immediately visible where the author intended the lines to
> end.
>
> XHTML was history over five years ago.  It's time you XML fanatics
> accept that HTML is a different language with different rules and stop
> this incessant kvetching. 

That's not very helpful either.

It's certainly the case that definite ending is easier to process.


Nic

Re: HTML-Info design

[David Kastrup]

Nic Ferrier <nferrier@ferrier.me.uk> writes:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> Yuri Khan <yuri.v.khan@gmail.com> writes:
>>
>>> With explicit closing tags, it is immediately visible where the author
>>> (or their tool) intended the element to end. Modifying the HTML
>>> generation logic only involves ensuring that nesting is not broken.
>>>
>>> With implicit tags, browsers can and will infer tag nesting on their
>>> own, and have an intricate system of rules to do so. Modifying the
>>> logic involves carefully working out where browsers would infer the
>>> missing tags, and then work with that knowledge to ensure that nesting
>>> won’t break.
>>
>> Yeah, that's why all Python code looks like
>>
>>  for x in range(10): # THE LINE ENDED THERE
>>      squares.append(x**2)  # THE FOR ENDS HERE I PROMISE!!!
>>
>> It's then immediately visible where the author intended the lines to
>> end.
>>
>> XHTML was history over five years ago.  It's time you XML fanatics
>> accept that HTML is a different language with different rules and stop
>> this incessant kvetching. 
>
> That's not very helpful either.
>
> It's certainly the case that definite ending is easier to process.

And Lisp code is even easier to process.  But HTML is HTML and XML is
XML and XHTML is XHTML and SGML is SGML.  Whether or not XHTML is easier
to process, web browsers are talking HTML.

-- 
David Kastrup

Re: HTML-Info design

[Steinar Bang]

>>>>> David Kastrup <dak@gnu.org>:

> And Lisp code is even easier to process.  But HTML is HTML and XML is
> XML and XHTML is XHTML and SGML is SGML.  Whether or not XHTML is
> easier to process, web browsers are talking HTML.

I really feel the urge to go in to teaspoon mode, bring up some old
90-ies history, talk about tagsoup and quirks mode and the like, not to
mention IE6 workarounds.

But I will desist.

I _will_ say that XML-formed is legal HTML5, and if your texinfo-to-HTML
tool produces well formed XML (which there is no real reason why it
can't) then all browser will get the same internal DOM representation
and there will be no rendering surprises in modern browsers (and few in
old browsers).

Re: HTML-Info design

[Lars Ingebrigtsen]

Nic Ferrier <nferrier@ferrier.me.uk> writes:

> It's certainly the case that definite ending is easier to process.

I don't really know what to say.  "HTML parsing is a solved problem"?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Ivan Shmakov]

[-- Attachment #1: Type: text/plain, Size: 1745 bytes --]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:
>>>>> Nic Ferrier <nferrier@ferrier.me.uk> writes:

 >> It's certainly the case that definite ending is easier to process.

 > I don't really know what to say.  "HTML parsing is a solved problem"?

	Granted, my Libxml2 installation may be out of date, but for the
	HTML5 document MIMEd (valid per http://validator.w3.org/check),
	libxml-parse-html-region (surprisingly) produces the following:

(html
 ((lang . "en") (dir . "ltr"))
 (head nil (title nil "HTML parsing"))
 (body nil (dl nil
               (dt nil "This\n")
               (dd nil "is\n"
                   (dd nil "a\n"
                       (dd nil "perfectly\n"
                           (dd nil "valid\n"
                               (dd nil "HTML5\n"
                                   (dd nil "document.\n")))))))))

	Naturally, SHR rendition of the document would be just as
	unreasonable as is the tree above.

	On the contrary, using Lynx to render the very same document
	results in:

$ lynx --dump --stdin --force-html < example.html 
   This
          is
          a
          perfectly
          valid
          HTML5
          document.
$ 

	The relevant part of the specification [1] is as follows.

    A dt element’s end tag may be omitted if the dt element is
    immediately followed by another dt element or a dd element.

    A dd element’s end tag may be omitted if the dd element is
    immediately followed by another dd element or a dt element, or if
    there is no more content in the parent element.

[1] http://www.w3.org/TR/html5/syntax.html#optional-tags

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

[-- Attachment #2: Type: text/html, Size: 160 bytes --]

Re: HTML-Info design

[Steinar Bang]

>>>>> Lars Ingebrigtsen <larsi@gnus.org>:

> Nic Ferrier <nferrier@ferrier.me.uk> writes:

>> It's certainly the case that definite ending is easier to process.

> I don't really know what to say.  "HTML parsing is a solved problem"?

No, it isn't.

Clue: it's not just about the end tags.

Re: HTML-Info design

[Yuri Khan]

On Mon, Dec 29, 2014 at 5:40 PM, Lars Ingebrigtsen <larsi@gnus.org> wrote:

>> With implicit tags, browsers can and will infer tag nesting on their
>> own, and have an intricate system of rules to do so.
>
> Yeah, that's why all Python code looks like
>
>  for x in range(10): # THE LINE ENDED THERE
>      squares.append(x**2)  # THE FOR ENDS HERE I PROMISE!!!

Interesting that you take for an example Python, a language with very
clear rules on where indented blocks end.

Rather, let’s take C.

In C, indentation is insignificant. For that matter, all whitespace is
insignificant (except in string literals). But we don’t write our
programs all in one line, and we indent consistently, and we choose
either spaces or tabs but not both, even though the compiler doesn’t
give a damn either way.

Also in C, braces are optional if there is only a single statement
inside. But most coding style guides at least STRONGLY RECOMMEND
spelling out all braces explicitly, and often REQUIRE that.

Parentheses are often redundant thanks to the operator precedence
rules, but the GNU Coding Standard RECOMMENDs spelling out parentheses
where doing so helps readability.

Optional tags in HTML are closest to optional parentheses, except that
the precedence of HTML elements is much more intricate than the
precedence of C operators, and so explicit closing tags help
readability almost always, except for the simplest of cases.

Re: HTML-Info design

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 2821 bytes --]

() Steinar Bang <sb@dod.no>
() Sun, 28 Dec 2014 20:43:55 +0100

   Me neither, and if there was, that sxml stuff mentioned
   earlier, was kind of cool (I wish I could find a use for it,
   but libxml2 parsing in emacs kind of takes away the advantage
   of the lisp reader... except maybe for big files...?).

(S)XML is just text; you must use ‘read’ (or ‘string-to-number’)
to get numerical information, and ‘intern’ to get symbols.  In
contrast, IXIN is predigested; you get text, numbers and symbols
(binary (e.g., image) data is base64-encoded, i.e., text), in
nicely nested paren-delimited lists.  What's not to love?  :-D

This lessened grunt-work for downstream programs means they can
concentrate on the fun stuff -- more tree wrangling, rendering
(and re-rendering per user whim and whimsy), and so on.

Probably there exists some Javascript equivalent of librep, and
if not, librep itself is there.  Really, it's time for GNU to
lead the way out of the angle-bracket ghetto.  Life's too short.

Since last spew, i have added HTML output of The IXIN Chronicles
to the IXIN homepage:

 http://www.gnuvola.org/software/ixin/

The link is "HTML", unsurprisingly.  In lieu of a public repo,
here is a list of changes that will go into 1.9:

 $ glog 1.8..
 035e151 2014-12-28 [maint] Update years in copyright notice; nfc.
 51688d1 2014-12-28 [spit] Abandon hope for shr.el rebasing.
 d28f6d2 2014-12-28 [int] Drop abstraction: open-input-file-if-exists
 566834c 2014-12-28 [int] Factor and extend DTD filename search.
 7fd9ab7 2014-12-26 a1-nf3-guile2: Reorder attributes, heuristically.
 129584d 2014-12-26 a1-nf3-guile2: Omit ‘*TOP*’, etc.
 98949b7 2014-12-25 [dist] Distribute HTML spec, too.
 d65fe5e 2014-12-23 [maint] Add AUTHORS file; nfc.
 3a0d9e6 2014-12-23 a1-nf3-guile2: Support entity-substitution.
 3bb53ef 2014-12-22 Fix bug: Make a1-nf3-from-nf1 handle childless elements.
 9b4f638 2014-12-18 [maint] Mention Emacs hackers interest in README; nfc.
 b70b103 2014-12-17 [maint] Mention p-o-c rendering program domain in README; nfc.
 6c75fc1 2013-07-24 [maint] Reformat NEWS; nfc.
 01e430d 2013-02-18 No longer distribute {epsf,texinfo}.tex.
 9ba97fe 2013-02-18 [spec int] Fix typo: Include "(news)" heading.

People have asked where to discuss IXIN.  Good question.
Guile-specific questions should probably go to guile-user,
likewise for Emacs (help-gnu-emacs) and Texinfo (help-texinfo).
The latter is good for overall design hashing, too.

If you include "IXIN" in the subject line, i'd appreciate it.

-- 
Thien-Thi Nguyen
   GPG key: 4C807502
   (if you're human and you know it)
      read my lisp: (responsep (questions 'technical)
                               (not (via 'mailing-list)))
                     => nil

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: HTML-Info design

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 1411 bytes --]

() Thien-Thi Nguyen <ttn@gnu.org>
() Mon, 29 Dec 2014 11:59:13 +0100

   Since last spew, i have added HTML output of The IXIN
   Chronicles to the IXIN homepage:

    http://www.gnuvola.org/software/ixin/

   The link is "HTML", unsurprisingly.  In lieu of a public repo,
   here is a list of changes that will go into 1.9 [...]

Thanks to the savannah admins, there is now a savannah-hosted
(but non-gnu) project, and public repo, for IXIN:

 http://savannah.nongnu.org/projects/ixin
 http://git.savannah.gnu.org/cgit/ixin.git?h=p

Use the source, Luke!  NB: No "master" branch!  (See HACKING.)

   People have asked where to discuss IXIN.  Good question.

There is now a mailing list as well (see project page above).  I
added it somewhat reluctantly, knowing my distaste (and thus
willful negligence) for this kind of admin scutwork, but let's
see what kind of heading ttn v2015 takes...  (Of course, all
this fretting is theoretical; no one seems to want to discuss
IXIN on any other existin mailing list, anyway.  :-D)

   If you include "IXIN" in the subject line, i'd appreciate it.

Not required; the mailing list software DTRT automatically.

-- 
Thien-Thi Nguyen
   GPG key: 4C807502
   (if you're human and you know it)
      read my lisp: (responsep (questions 'technical)
                               (not (via 'mailing-list)))
                     => nil

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: HTML-Info design

[Phillip Lord]

On the HTML-Info front, I've been playing with documentation from
org-mode recently. I didn't know this at the time but there is actually
some Javascript for giving a somewhat Info like experience to HTML
exported from HTML.

http://orgmode.org/worg/code/org-info-js/

I've been using it for my own package lentic. The HTML output looks like
so:

http://homepages.cs.ncl.ac.uk/phillip.lord/lentic/lenticular.html

In this case, the documentation is generated directly from the
emacs-lisp source (I'm not totally happy with this process yet, and
there needs to be some more work done, for instance getting rid of the
multiple badly formatted copies of the GPL header). It's also gives
reasonable output in EWW. Finally, because it's all Emacs based, I can
just ship the source code and generate the HTML on-the-fly, on demand.

All a bit experimental, but I thought some here might be interested.

Phil

Re: HTML-Info design

[Yuri Khan]

On Mon, Mar 2, 2015 at 11:20 PM, Phillip Lord
<phillip.lord@newcastle.ac.uk> wrote:
>
> On the HTML-Info front, I've been playing with documentation from
> org-mode recently. I didn't know this at the time but there is actually
> some Javascript for giving a somewhat Info like experience to HTML
> exported from HTML.
>
> http://orgmode.org/worg/code/org-info-js/
>
> I've been using it for my own package lentic. The HTML output looks like
> so:
>
> http://homepages.cs.ncl.ac.uk/phillip.lord/lentic/lenticular.html

I tried it out. The keyboard shortcut handling code misbehaves in
browsers which support “find as you type” — in my case, Firefox.

Keyboard event handlers need to invoke .preventDefault() on the event
object if they handle it.

Re: HTML-Info design

[Phillip Lord]

Yuri Khan <yuri.v.khan@gmail.com> writes:
>> On the HTML-Info front, I've been playing with documentation from
>> org-mode recently. I didn't know this at the time but there is actually
>> some Javascript for giving a somewhat Info like experience to HTML
>> exported from HTML.
>>
>> http://orgmode.org/worg/code/org-info-js/
>>
>> I've been using it for my own package lentic. The HTML output looks like
>> so:
>>
>> http://homepages.cs.ncl.ac.uk/phillip.lord/lentic/lenticular.html
>
> I tried it out. The keyboard shortcut handling code misbehaves in
> browsers which support “find as you type” — in my case, Firefox.
>
> Keyboard event handlers need to invoke .preventDefault() on the event
> object if they handle it.

It works on my firefox to be honest. The JS is I think developed on
org-mode.org.

I wonder if it could be made to work with different source formats? And
whether several different manuals could be linked into a larger whole?

Phil

Re: HTML-Info design

[Yuri Khan]

On Mon, Mar 2, 2015 at 11:58 PM, Phillip Lord
<phillip.lord@newcastle.ac.uk> wrote:
> Yuri Khan <yuri.v.khan@gmail.com> writes:
>> I tried it out. The keyboard shortcut handling code misbehaves in
>> browsers which support “find as you type” — in my case, Firefox.
>>
>> Keyboard event handlers need to invoke .preventDefault() on the event
>> object if they handle it.
>
> It works on my firefox to be honest.

That is probably because find-as-you-type is disabled by default and
only invoked explicitly by pressing “/” (for any text) or “'” (for
links only). However, if you check Preferences | Advanced | General |
[x] Search for text when I start typing, typing any character
(provided that the page does not preventDefault it) starts search for
the entered character. Any subsequently typed characters append to the
search string and are not noticed by the page.

> The JS is I think developed on org-mode.org.

(I was hoping that you, as a direct user of org-mode, might forward
the bug report.)

Re: HTML-Info design

[Phillip Lord]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> On Mon, Mar 2, 2015 at 11:58 PM, Phillip Lord
> <phillip.lord@newcastle.ac.uk> wrote:
>> Yuri Khan <yuri.v.khan@gmail.com> writes:
>>> I tried it out. The keyboard shortcut handling code misbehaves in
>>> browsers which support “find as you type” — in my case, Firefox.
>>>
>>> Keyboard event handlers need to invoke .preventDefault() on the event
>>> object if they handle it.
>>
>> It works on my firefox to be honest.
>
> That is probably because find-as-you-type is disabled by default and
> only invoked explicitly by pressing “/” (for any text) or “'” (for
> links only). However, if you check Preferences | Advanced | General |
> [x] Search for text when I start typing, typing any character
> (provided that the page does not preventDefault it) starts search for
> the entered character. Any subsequently typed characters append to the
> search string and are not noticed by the page.


Okay. Not seen that before. org-info uses single keypresses for
next/prev, to enable search or occur functionality. What are you seeing,
and what you be expecting it to do? I'm not sure how the org-info
functionality could NOT conflict with this sort of find-as-you-type
functionality.



>> The JS is I think developed on org-mode.org.
>
> (I was hoping that you, as a direct user of org-mode, might forward
> the bug report.)


I might do so, but I need to understand the bug. I don't at the moment.

Phil

Re: HTML-Info design

[Yuri Khan]

On Wed, Mar 4, 2015 at 6:19 PM, Phillip Lord
<phillip.lord@newcastle.ac.uk> wrote:
> Okay. Not seen that before. org-info uses single keypresses for
> next/prev, to enable search or occur functionality. What are you seeing,
> and what you be expecting it to do? I'm not sure how the org-info
> functionality could NOT conflict with this sort of find-as-you-type
> functionality.

OK, here’s a formal bug report.

The conflict can be avoided if the event handler notifies the browser
that it has handled the event, as detailed below.

===
Versions:

* Firefox 35.0.1 on Ubuntu 14.04 x86_64
* http://orgmode.org/org-info.js as retrieved from the origin server
on 2015-03-04

To reproduce:

0. In the Firefox Preferences dialog, Advanced | General tab,
    Accessibility section, check the checkbox [x] Search for text when
    I start typing.
1. Open the page
http://homepages.cs.ncl.ac.uk/phillip.lord/lentic/lenticular.html
    * Observed behavior: The node(?) “1 Introduction” is displayed.
2. Press “n”.
    * Expected behavior:
        1. The original node is hidden.
        2. The next node is displayed.
        3. No other activity happens.
    * Observed behavior:
        1. The original node is hidden, as expected.
        2. The next node (1.1 Caveat) is displayed, as expected.
        3. Find-as-you-type is activated with the search text set to
            the single character “n”, finding the character “n” in
            “Le[n]ticular Text For Emacs”.
3. Wait until the search bar times out, or press ESC to dismiss it.
4. Press “p”.
    * Expected behavior:
        1. The current node is hidden.
        2. The previous node (1 Introduction) is displayed.
        3. No other activity happens.
    * Observed behavior:
        1. The node (1.1 Caveat) is hidden, as expected.
        2. The next node (1 Introduction) is displayed, as expected.
        3. Find-as-you-type is activated with the search text set to
            the single character “p”, finding the character “P” in
            “HELP”.

Cause:

The event handler function, OrgHtmlManagerKeyEvent, receives an event
object as its argument b. It analyzes this object and performs any
actions associated with the pressed key (or does nothing if the key is
not recognized). The browser then also analyzes the object, notices
that default handling has not been prevented, and applies the default
handling, which is to start a search within the page.

To prevent the default handling, the handler function is supposed to
invoke the preventDefault() method on the event object, or cause that
method to be invoked by other functions involved in the handling
(namely, org_html_manager.getKey()), when and only when the event
represents a key known to the handler.

References:

* https://developer.mozilla.org/en-US/docs/Web/API/Event/preventDefault
* http://www.w3.org/TR/DOM-Level-2-Events/events.html#Events-Event-preventDefault
===

Re: HTML-Info design

[Stefan Monnier]

> `M-x eww RET
> http://www.gnu.org/software/emacs/manual/html_node/emacs/index.html RET'

> as you claimed.

> Instead it would be:

> M-x eww file:///home/nicferrier/emacs-24.4/share/info...

No, it's obvious to me that we want to use a "remote URL" and then have
Emacs internally redirect this to a local version, when available.
The purpose is to get rid of the "local only" references (such as
(info "(emacs)Top") or file:///blabla) and only use globally-valid URLs,
so as to improve the answers given by search engines like DuckDuckGo.


        Stefan

Re: HTML-Info design

[Achim Gratz]

Am 29.12.2014 um 01:14 schrieb Stefan Monnier:
>> `M-x eww RET
>> http://www.gnu.org/software/emacs/manual/html_node/emacs/index.html RET'
>
>> as you claimed.
>
>> Instead it would be:
>
>> M-x eww file:///home/nicferrier/emacs-24.4/share/info...
>
> No, it's obvious to me that we want to use a "remote URL" and then have
> Emacs internally redirect this to a local version, when available.

Yes, but most certainly not from this particular URL as there is no way 
of knowing which version of the manual was meant by it.  So at the 
minimum you'd have to do a HEAD request and check if you've cached that 
exact file earlier.  And you'd need to read that file in full the first 
time to compare it with the local version.  Since you want to skip a 
proper URI scheme you need to have a canonical URL that provides the 
same information and allows an off-line info reader to skip going to the 
network altogether.


-- 
Achim.

(on the road :-)

Re: HTML-Info design

[Stefan Monnier]

>>> http://www.gnu.org/software/emacs/manual/html_node/emacs/index.html RET'
[...]
> Yes, but most certainly not from this particular URL as there is no way of
> knowing which version of the manual was meant by it.

So what?  "(emacs)Top" doesn't contain any version information either.
I don't very much like the above URL (I'd like something shorter), but
lack of version info is a feature rather than a bug.


        Stefan

Re: HTML-Info design

[Stefan Monnier]

> it with the local version.  Since you want to skip a proper URI scheme you
> need to have a canonical URL that provides the same information and allows
> an off-line info reader to skip going to the network altogether.

I did say that it would be redirected to the local version (when
available).  And by "local" I don't mean "cached": the redirection would
happen without any network packet being sent.


        Stefan

Re: HTML-Info design

[Stefan Monnier]

> earlier.  And you'd need to read that file in full the first time to compare
> it with the local version.

No, the local version might be in Info format while the remote version
might be in HTML format.  So we neither need nor want to look at the
remote file.


        Stefan

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > No, it's obvious to me that we want to use a "remote URL" and then have
  > Emacs internally redirect this to a local version, when available.

This would cause grave problems with ordinary browsers: they would
look at the net, not at the local info files.

I think that means we MUST use relative file names.  They will work
properly in any browser for local files and for web pages.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Ivan Shmakov]

>>>>> Nic Ferrier <nferrier@ferrier.me.uk> writes:

[…]

 > But I think the problems with that will be the problems with eww.
 > Poor caching (your solution would immediately invalidate what Stefan
 > said was his #1), difficult to make multiple buffers, etc...

	FWIW, what I use is like C-x b ⟨new buffer name⟩, M-x eww-mode,
	G https://example.org/.  Not exactly all that a hard thing.

[…]

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: HTML-Info design

[Stefan Monnier]

> I don't see why this would be difficult.

> `M-x eww RET
> http://www.gnu.org/software/emacs/manual/html_node/emacs/index.html RET'

> and you're off.  The `n'/`p'/`u' commands work as in Info already, so
> the only thing that's missing is the index (which is trivial to add).

AFAIK, the main reason why Info-mode is not sexy is its plain
visual appearance.  The above page in `eww' looks like Emacs-20's
Info-mode.

E.g. we need titles to be bigger and the main text to use
proportional font.  Otherwise, I don't see the advantage.


        Stefan

Re: HTML-Info design

[David Kastrup]

[-- Attachment #1: Type: text/plain, Size: 655 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I don't see why this would be difficult.
>
>> `M-x eww RET
>> http://www.gnu.org/software/emacs/manual/html_node/emacs/index.html RET'
>
>> and you're off.  The `n'/`p'/`u' commands work as in Info already, so
>> the only thing that's missing is the index (which is trivial to add).
>
> AFAIK, the main reason why Info-mode is not sexy is its plain
> visual appearance.  The above page in `eww' looks like Emacs-20's
> Info-mode.
>
> E.g. we need titles to be bigger and the main text to use
> proportional font.  Otherwise, I don't see the advantage.

That's not an advantage.  It is breaking even.


[-- Attachment #2: prop.png --]
[-- Type: image/png, Size: 18233 bytes --]

[-- Attachment #3: Type: text/plain, Size: 19 bytes --]


-- 
David Kastrup

Saving default font? (was: HTML-Info design)

[David Kastrup]

David Kastrup <dak@gnu.org> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> I don't see why this would be difficult.
>>
>>> `M-x eww RET
>>> http://www.gnu.org/software/emacs/manual/html_node/emacs/index.html
>>> RET'
>>
>>> and you're off.  The `n'/`p'/`u' commands work as in Info already,
>>> so
>>> the only thing that's missing is the index (which is trivial to
>>> add).
>>
>> AFAIK, the main reason why Info-mode is not sexy is its plain
>> visual appearance.  The above page in `eww' looks like Emacs-20's
>> Info-mode.
>>
>> E.g. we need titles to be bigger and the main text to use
>> proportional font.  Otherwise, I don't see the advantage.
>
> That's not an advantage.  It is breaking even.

Ok, I am _totally_ pissed right now.  For making the screen shot for the
above posting, I used the "Options / Change Default Font" menu.  Then I
exited and restarted Emacs in order to get my carefully customized
settings back.

Except that somebody thought it a smart idea to just overwrite my
default settings without asking.  What do we have the "Options/Save"
menu for in the first place if Options will be permasaved nilly-willy?
Whoever thought that a good idea or interface?

Now I have to crank out my backups to get my settings back and hope that
they are stored in .emacs and not somewhere else.

How is one supposed to make temporary changes to the font?  How is one
supposed to get the original settings back when one does not relish the
change?

What kind of goofy decision is it to make all in-session changes
permanent and undoable?

-- 
David Kastrup

Re: Saving default font?

[Lars Ingebrigtsen]

David Kastrup <dak@gnu.org> writes:

> What kind of goofy decision is it to make all in-session changes
> permanent and undoable?

Perhaps this has something to do with bug#19328?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> E.g. we need titles to be bigger and the main text to use
> proportional font.  Otherwise, I don't see the advantage.

Then somebody will have to get cracking on making the Emacs display
engine more expressive so that that's possible, I guess?

(Yes, Emacs can display proportional fonts and fonts of different sizes,
but until you can fold (etc) proportional text (and text with a mixture
of font sizes) in a pretty manner, that's more of a toy than anything
else.)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Lars Ingebrigtsen]

[-- Attachment #1: Type: text/plain, Size: 329 bytes --]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> (Yes, Emacs can display proportional fonts and fonts of different sizes,
> but until you can fold (etc) proportional text (and text with a mixture
> of font sizes) in a pretty manner, that's more of a toy than anything
> else.)

This is what an Info buffer looks like on Fedora 19:


[-- Attachment #2: so-pretty-info.png --]
[-- Type: image/png, Size: 112014 bytes --]

[-- Attachment #3: Type: text/plain, Size: 103 bytes --]


-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 1264 bytes --]

On 12/28/2014 03:08 PM, Lars Ingebrigtsen wrote:
> This is what an Info buffer looks like on Fedora 19:

I see more than that on Fedora 21 with "emacs -Q".  Please see the first 
attachment.  There are two ways to go back (a "Previous" icon, and a 
"Prev: Buffer List" text, and two ways to go forward (a "Next" icon, and 
a "Next: Killing Buffers" text that is confusingly in the opposite order 
of the buttons), and two ways to go up (an unlabeled up icon, and an 
"Up: Buffers" text).  Plus, the text does not rearrange itself to fit 
the window size, and since my window is not the default size the text is 
pretty hard to read.  It's a confusing UI, for someone not already used 
to it.

In contrast, the same documentation on Icecat (see 2nd attachment) is 
easier to read and for a newbie to navigate through.  There's only one 
set of buttons, the text fits the window, the font (though still pretty 
large) is not the enormous size that Emacs uses, and the text is nicely 
reformatted to fit the window. (One other nicety: some unnecessary and 
distracting quotes are omitted.)  Overall there's about 50% more useful 
information in the same screen space, and it's more readable despite the 
smaller font size.  This is a significantly better experience.

[-- Attachment #2: Emacs.png --]
[-- Type: image/png, Size: 113817 bytes --]

[-- Attachment #3: Icecat.png --]
[-- Type: image/png, Size: 124506 bytes --]

Re: HTML-Info design

[Nic Ferrier]

eggert@cs.ucla.edu writes:

> In contrast, the same documentation on Icecat (see 2nd attachment) is
> easier to read and for a newbie to navigate through.  There's only one
> set of buttons, the text fits the window, the font (though still
> pretty large) is not the enormous size that Emacs uses, and the text
> is nicely reformatted to fit the window. (One other nicety: some
> unnecessary and distracting quotes are omitted.)  Overall there's
> about 50% more useful information in the same screen space, and it's
> more readable despite the smaller font size.  This is a significantly
> better experience.

The eww version looks much less nice.

Particularly neither eww nor info re-flow the text when the window size
changes. Or even if a new frame is created for the buffer.

One very nice thing about browsers is that they conventionally let you
open a link in a new window (in Emacs parlance, a frame). That is very
useful when you are reading documentation.

Emacs buffer/windows are probably as good but I'd like the text to
re-flow when that happened. Perhaps that can be easily hacked together.


Nic

Re: HTML-Info design

[Lars Ingebrigtsen]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Plus, the text does not rearrange itself to fit the window size, and
> since my window is not the default size the text is pretty hard to
> read.

eww would render the info to the width of the window.  It won't
re-render without the user hitting `g', but if somebody thinks that a
deal killer, then that can be done automatically on frame resizing...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: eliz@gnu.org,  stephen@xemacs.org,  Richard Stallman <rms@gnu.org>,  Nic Ferrier <nferrier@ferrier.me.uk>,  emacs-devel@gnu.org
> Date: Mon, 29 Dec 2014 00:04:38 +0100
> 
> (Yes, Emacs can display proportional fonts and fonts of different sizes,
> but until you can fold (etc) proportional text (and text with a mixture
> of font sizes) in a pretty manner, that's more of a toy than anything
> else.)

What's non-pretty with how we do this now?  What features are missing?

Re: HTML-Info design

[Steinar Bang]

>>>>> Eli Zaretskii <eliz@gnu.org>:

> What's non-pretty with how we do this now?  What features are missing?

Auto-rewrap on emacs resize seems to be missing...? (I got this
impression from other messages in this thread)

I'm less sure about the rest.  The screen shot Lars sent seemed to have
some very bad font choices, but I don't know if emacs is to blame for
that?

Re: HTML-Info design

[Lars Ingebrigtsen]

Steinar Bang <sb@dod.no> writes:

> I'm less sure about the rest.  The screen shot Lars sent seemed to have
> some very bad font choices, but I don't know if emacs is to blame for
> that?

We know that Emacs often makes unfortunate choices, so we know that
using a mixture of fonts in buffers usually makes the buffers look
pretty unreadable.  So it's our fault.

We should either stop doing stuff like that, or we should start
recommending font sets that will give us better-looking buffers.  I went
with the former in eww.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Eli Zaretskii]

> From: Steinar Bang <sb@dod.no>
> Date: Mon, 29 Dec 2014 08:28:35 +0100
> 
> > What's non-pretty with how we do this now?  What features are missing?
> 
> Auto-rewrap on emacs resize seems to be missing...? (I got this
> impression from other messages in this thread)

Info doesn't currently turn on word wrap, so it cannot possibly cause
any auto-rewrap on resizing.

AFAIU, this is not what Lars had in mind.

bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Ivan Shmakov]

[-- Attachment #1: Type: text/plain, Size: 1583 bytes --]

Package:  emacs
Severity: wishlist
X-Debbugs-Cc: emacs-devel@gnu.org

>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>>> From: Lars Ingebrigtsen  Date: Mon, 29 Dec 2014 00:04:38 +0100

 >> (Yes, Emacs can display proportional fonts and fonts of different
 >> sizes, but until you can fold (etc) proportional text (and text with
 >> a mixture of font sizes) in a pretty manner, that's more of a toy
 >> than anything else.)

 > What's non-pretty with how we do this now?  What features are
 > missing?

	The only feature that I’m aware to be missing is the actual
	support for Emacs native text wrapping (as in: the word-wrap
	variable and wrap-prefix text property) in SHR.

	Please thus consider the patch MIMEd.

	* lisp/net/shr.el (shr-force-fill): New variable to disable this
	feature if needed.
	(shr-internal-width): Defer initialization until...
	(shr-insert-document): ... here; set to nil if neither
	shr-force-fill nor shr-width are non-nil.
	(shr-fold-text, shr-tag-table-1): Likewise.
	(shr-insert): Use insert-and-inherit; do not fill if
	shr-internal-width is nil.
	(shr-setup-wrap): New function.
	(shr-indent, shr-tag-blockquote, shr-tag-dd, shr-tag-li):
	Call shr-setup-wrap.
	(shr-tag-hr): Use a constant if shr-internal-width is nil.

	A test case is also MIMEd.  The buffer it produces shows the
	text being dynamically filled as the window width changes
	(as in: C-x 3, for instance.)

	The table rendering is not changed in any way.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

[-- Attachment #2: Type: text/diff, Size: 5260 bytes --]

diff --git a/lisp/net/shr.el b/lisp/net/shr.el
index 26bb292..e634a5a 100644
--- a/lisp/net/shr.el
+++ b/lisp/net/shr.el
@@ -128,13 +128,16 @@
 (defvar shr-inhibit-images nil
   "If non-nil, inhibit loading images.")
 
+(defvar shr-force-fill nil
+  "If non-nil, fill text even in the cases Emacs can wrap it by itself.")
+
 ;;; Internal variables.
 
 (defvar shr-folding-mode nil)
 (defvar shr-state nil)
 (defvar shr-start nil)
 (defvar shr-indentation 0)
-(defvar shr-internal-width (or shr-width (1- (window-width))))
+(defvar shr-internal-width nil)		; set in shr-insert-document
 (defvar shr-list-mode nil)
 (defvar shr-content-cache nil)
 (defvar shr-kinsoku-shorten nil)
@@ -206,7 +209,8 @@ defun shr-insert-document (dom)
 	(shr-base nil)
 	(shr-depth 0)
 	(shr-warning nil)
-	(shr-internal-width (or shr-width (1- (window-width)))))
+	(shr-internal-width
+	 (or shr-width (and shr-force-fill (1- (window-width))))))
     (shr-descend dom)
     (shr-remove-trailing-whitespace start (point))
     (when shr-warning
@@ -420,7 +424,8 @@ defun shr-fold-text (text)
       (let ((shr-indentation 0)
 	    (shr-state nil)
 	    (shr-start nil)
-	    (shr-internal-width (window-width)))
+	    (shr-internal-width (and shr-force-fill
+				     (1- (window-width)))))
 	(shr-insert text)
 	(buffer-string)))))
 
@@ -454,12 +459,14 @@ defun shr-insert (text)
     (setq shr-state nil))
   (cond
    ((eq shr-folding-mode 'none)
-    (insert text))
+    (insert-and-inherit text))
    (t
+    ;; We generally use insert-and-inherit below so to inherit the
+    ;; wrap-prefix property, if any.  See shr-setup-wrap.
     (when (and (string-match "\\`[ \t\n ]" text)
 	       (not (bolp))
 	       (not (eq (char-after (1- (point))) ? )))
-      (insert " "))
+      (insert-and-inherit " "))
     (dolist (elem (split-string text "[ \f\t\n\r\v ]+" t))
       (when (and (bolp)
 		 (> shr-indentation 0))
@@ -482,17 +489,18 @@ defun shr-insert (text)
       ;; starts.
       (unless shr-start
 	(setq shr-start (point)))
-      (insert elem)
+      (insert-and-inherit elem)
       (setq shr-state nil)
       (let (found)
-	(while (and (> (current-column) shr-internal-width)
+	(while (and shr-internal-width	; Use Emacs native wrapping if nil.
+		    (> (current-column) shr-internal-width)
 		    (> shr-internal-width 0)
 		    (progn
 		      (setq found (shr-find-fill-point))
 		      (not (eolp))))
 	  (when (eq (preceding-char) ? )
 	    (delete-char -1))
-	  (insert "\n")
+	  (insert-and-inherit "\n")
 	  (unless found
 	    ;; No space is needed at the beginning of a line.
 	    (when (eq (following-char) ? )
@@ -500,11 +508,12 @@ defun shr-insert (text)
 	  (when (> shr-indentation 0)
 	    (shr-indent))
 	  (end-of-line))
-	(if (<= (current-column) shr-internal-width)
-	    (insert " ")
+	(if (or (not shr-internal-width)
+		(<= (current-column) shr-internal-width))
+	    (insert-and-inherit " ")
 	  ;; In case we couldn't get a valid break point (because of a
 	  ;; word that's longer than `shr-internal-width'), just break anyway.
-	  (insert "\n")
+	  (insert-and-inherit "\n")
 	  (when (> shr-indentation 0)
 	    (shr-indent)))))
     (unless (string-match "[ \t\r\n ]\\'" text)
@@ -663,7 +672,17 @@
 
 (defun shr-indent ()
   (when (> shr-indentation 0)
-    (insert (make-string shr-indentation ? ))))
+    (insert (make-string shr-indentation ? ))
+    (shr-setup-wrap)))
+
+(defun shr-setup-wrap ()
+  (when (> shr-indentation 0)
+    ;; The wrap-prefix property is sticky; abuse that here.  We use
+    ;; this after at least shr-indent (or within it), so we may safely
+    ;; assume that there is at least one character before the point.
+    (put-text-property (+ -1 (point)) (point)
+		       'wrap-prefix
+		       `(space :align-to ,shr-indentation))))
 
 (defun shr-fontize-dom (dom &rest types)
   (let (shr-start)
@@ -1309,6 +1334,7 @@ defun shr-tag-blockquote (dom)
   (shr-ensure-paragraph)
   (shr-indent)
   (let ((shr-indentation (+ shr-indentation 4)))
+    (shr-setup-wrap)
     (shr-generic dom))
   (shr-ensure-paragraph))
 
@@ -1325,6 +1351,7 @@
 (defun shr-tag-dd (dom)
   (shr-ensure-newline)
   (let ((shr-indentation (+ shr-indentation 4)))
+    (shr-setup-wrap)
     (shr-generic dom)))
 
 (defun shr-tag-ul (dom)
@@ -1350,6 +1377,7 @@ defun shr-tag-li (dom)
 	    shr-bullet))
 	 (shr-indentation (+ shr-indentation (length bullet))))
     (insert bullet)
+    (shr-setup-wrap)
     (shr-generic dom)))
 
 (defun shr-tag-br (dom)
@@ -1386,7 +1414,8 @@
 
 (defun shr-tag-hr (_dom)
   (shr-ensure-newline)
-  (insert (make-string shr-internal-width shr-hr-line) "\n"))
+  (insert (make-string (or shr-internal-width 31) ; FIXME: magic
+		       shr-hr-line) "\n"))
 
 (defun shr-tag-title (dom)
   (shr-heading dom 'bold 'underline))
@@ -1414,6 +1443,7 @@
 (defun shr-tag-table-1 (dom)
   (setq dom (or (dom-child-by-tag dom 'tbody) dom))
   (let* ((shr-inhibit-images t)
+	 (shr-internal-width (or shr-internal-width (1- (window-width))))
 	 (shr-table-depth (1+ shr-table-depth))
 	 (shr-kinsoku-shorten t)
 	 ;; Find all suggested widths.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: Type: text/emacs-lisp, Size: 961 bytes --]

(with-current-buffer (generate-new-buffer "*shr*")
  (setq-local shr-width nil)
  (setq-local word-wrap t)
  (setq-local truncate-partial-width-windows nil)
  (shr-insert-document
   '(base
     ((href . "https://example.com/"))
     (html
      nil
      (head nil (title nil "Lorem ipsum"))
      (body
       nil
       (hr nil)
       (ol nil
	   (li ((lang . "la"))
	       "Lorem ipsum dolor sit amet, consectetur adipisicing"
	       " elit, sed do eiusmod tempor incididunt ut labore et"
	       " dolore magna aliqua.  Ut enim ad minim veniam, quis"
	       " nostrud exercitation ullamco laboris nisi ut"
	       " aliquip ex ea commodo consequat.  Duis aute irure"
	       " dolor in reprehenderit in voluptate velit esse"
	       " cillum dolore eu fugiat nulla pariatur.  Excepteur"
	       " sint occaecat cupidatat non proident, sunt in culpa"
	       " qui officia deserunt mollit anim id est laborum."))))))
  (pop-to-buffer (current-buffer)))

bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Ivan Shmakov]

[-- Attachment #1: Type: text/plain, Size: 1583 bytes --]

Package:  emacs
Severity: wishlist
X-Debbugs-Cc: emacs-devel@gnu.org

>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>>> From: Lars Ingebrigtsen  Date: Mon, 29 Dec 2014 00:04:38 +0100

 >> (Yes, Emacs can display proportional fonts and fonts of different
 >> sizes, but until you can fold (etc) proportional text (and text with
 >> a mixture of font sizes) in a pretty manner, that's more of a toy
 >> than anything else.)

 > What's non-pretty with how we do this now?  What features are
 > missing?

	The only feature that I’m aware to be missing is the actual
	support for Emacs native text wrapping (as in: the word-wrap
	variable and wrap-prefix text property) in SHR.

	Please thus consider the patch MIMEd.

	* lisp/net/shr.el (shr-force-fill): New variable to disable this
	feature if needed.
	(shr-internal-width): Defer initialization until...
	(shr-insert-document): ... here; set to nil if neither
	shr-force-fill nor shr-width are non-nil.
	(shr-fold-text, shr-tag-table-1): Likewise.
	(shr-insert): Use insert-and-inherit; do not fill if
	shr-internal-width is nil.
	(shr-setup-wrap): New function.
	(shr-indent, shr-tag-blockquote, shr-tag-dd, shr-tag-li):
	Call shr-setup-wrap.
	(shr-tag-hr): Use a constant if shr-internal-width is nil.

	A test case is also MIMEd.  The buffer it produces shows the
	text being dynamically filled as the window width changes
	(as in: C-x 3, for instance.)

	The table rendering is not changed in any way.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

[-- Attachment #2: Type: text/diff, Size: 5260 bytes --]

diff --git a/lisp/net/shr.el b/lisp/net/shr.el
index 26bb292..e634a5a 100644
--- a/lisp/net/shr.el
+++ b/lisp/net/shr.el
@@ -128,13 +128,16 @@
 (defvar shr-inhibit-images nil
   "If non-nil, inhibit loading images.")
 
+(defvar shr-force-fill nil
+  "If non-nil, fill text even in the cases Emacs can wrap it by itself.")
+
 ;;; Internal variables.
 
 (defvar shr-folding-mode nil)
 (defvar shr-state nil)
 (defvar shr-start nil)
 (defvar shr-indentation 0)
-(defvar shr-internal-width (or shr-width (1- (window-width))))
+(defvar shr-internal-width nil)		; set in shr-insert-document
 (defvar shr-list-mode nil)
 (defvar shr-content-cache nil)
 (defvar shr-kinsoku-shorten nil)
@@ -206,7 +209,8 @@ defun shr-insert-document (dom)
 	(shr-base nil)
 	(shr-depth 0)
 	(shr-warning nil)
-	(shr-internal-width (or shr-width (1- (window-width)))))
+	(shr-internal-width
+	 (or shr-width (and shr-force-fill (1- (window-width))))))
     (shr-descend dom)
     (shr-remove-trailing-whitespace start (point))
     (when shr-warning
@@ -420,7 +424,8 @@ defun shr-fold-text (text)
       (let ((shr-indentation 0)
 	    (shr-state nil)
 	    (shr-start nil)
-	    (shr-internal-width (window-width)))
+	    (shr-internal-width (and shr-force-fill
+				     (1- (window-width)))))
 	(shr-insert text)
 	(buffer-string)))))
 
@@ -454,12 +459,14 @@ defun shr-insert (text)
     (setq shr-state nil))
   (cond
    ((eq shr-folding-mode 'none)
-    (insert text))
+    (insert-and-inherit text))
    (t
+    ;; We generally use insert-and-inherit below so to inherit the
+    ;; wrap-prefix property, if any.  See shr-setup-wrap.
     (when (and (string-match "\\`[ \t\n ]" text)
 	       (not (bolp))
 	       (not (eq (char-after (1- (point))) ? )))
-      (insert " "))
+      (insert-and-inherit " "))
     (dolist (elem (split-string text "[ \f\t\n\r\v ]+" t))
       (when (and (bolp)
 		 (> shr-indentation 0))
@@ -482,17 +489,18 @@ defun shr-insert (text)
       ;; starts.
       (unless shr-start
 	(setq shr-start (point)))
-      (insert elem)
+      (insert-and-inherit elem)
       (setq shr-state nil)
       (let (found)
-	(while (and (> (current-column) shr-internal-width)
+	(while (and shr-internal-width	; Use Emacs native wrapping if nil.
+		    (> (current-column) shr-internal-width)
 		    (> shr-internal-width 0)
 		    (progn
 		      (setq found (shr-find-fill-point))
 		      (not (eolp))))
 	  (when (eq (preceding-char) ? )
 	    (delete-char -1))
-	  (insert "\n")
+	  (insert-and-inherit "\n")
 	  (unless found
 	    ;; No space is needed at the beginning of a line.
 	    (when (eq (following-char) ? )
@@ -500,11 +508,12 @@ defun shr-insert (text)
 	  (when (> shr-indentation 0)
 	    (shr-indent))
 	  (end-of-line))
-	(if (<= (current-column) shr-internal-width)
-	    (insert " ")
+	(if (or (not shr-internal-width)
+		(<= (current-column) shr-internal-width))
+	    (insert-and-inherit " ")
 	  ;; In case we couldn't get a valid break point (because of a
 	  ;; word that's longer than `shr-internal-width'), just break anyway.
-	  (insert "\n")
+	  (insert-and-inherit "\n")
 	  (when (> shr-indentation 0)
 	    (shr-indent)))))
     (unless (string-match "[ \t\r\n ]\\'" text)
@@ -663,7 +672,17 @@
 
 (defun shr-indent ()
   (when (> shr-indentation 0)
-    (insert (make-string shr-indentation ? ))))
+    (insert (make-string shr-indentation ? ))
+    (shr-setup-wrap)))
+
+(defun shr-setup-wrap ()
+  (when (> shr-indentation 0)
+    ;; The wrap-prefix property is sticky; abuse that here.  We use
+    ;; this after at least shr-indent (or within it), so we may safely
+    ;; assume that there is at least one character before the point.
+    (put-text-property (+ -1 (point)) (point)
+		       'wrap-prefix
+		       `(space :align-to ,shr-indentation))))
 
 (defun shr-fontize-dom (dom &rest types)
   (let (shr-start)
@@ -1309,6 +1334,7 @@ defun shr-tag-blockquote (dom)
   (shr-ensure-paragraph)
   (shr-indent)
   (let ((shr-indentation (+ shr-indentation 4)))
+    (shr-setup-wrap)
     (shr-generic dom))
   (shr-ensure-paragraph))
 
@@ -1325,6 +1351,7 @@
 (defun shr-tag-dd (dom)
   (shr-ensure-newline)
   (let ((shr-indentation (+ shr-indentation 4)))
+    (shr-setup-wrap)
     (shr-generic dom)))
 
 (defun shr-tag-ul (dom)
@@ -1350,6 +1377,7 @@ defun shr-tag-li (dom)
 	    shr-bullet))
 	 (shr-indentation (+ shr-indentation (length bullet))))
     (insert bullet)
+    (shr-setup-wrap)
     (shr-generic dom)))
 
 (defun shr-tag-br (dom)
@@ -1386,7 +1414,8 @@
 
 (defun shr-tag-hr (_dom)
   (shr-ensure-newline)
-  (insert (make-string shr-internal-width shr-hr-line) "\n"))
+  (insert (make-string (or shr-internal-width 31) ; FIXME: magic
+		       shr-hr-line) "\n"))
 
 (defun shr-tag-title (dom)
   (shr-heading dom 'bold 'underline))
@@ -1414,6 +1443,7 @@
 (defun shr-tag-table-1 (dom)
   (setq dom (or (dom-child-by-tag dom 'tbody) dom))
   (let* ((shr-inhibit-images t)
+	 (shr-internal-width (or shr-internal-width (1- (window-width))))
 	 (shr-table-depth (1+ shr-table-depth))
 	 (shr-kinsoku-shorten t)
 	 ;; Find all suggested widths.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: Type: text/emacs-lisp, Size: 961 bytes --]

(with-current-buffer (generate-new-buffer "*shr*")
  (setq-local shr-width nil)
  (setq-local word-wrap t)
  (setq-local truncate-partial-width-windows nil)
  (shr-insert-document
   '(base
     ((href . "https://example.com/"))
     (html
      nil
      (head nil (title nil "Lorem ipsum"))
      (body
       nil
       (hr nil)
       (ol nil
	   (li ((lang . "la"))
	       "Lorem ipsum dolor sit amet, consectetur adipisicing"
	       " elit, sed do eiusmod tempor incididunt ut labore et"
	       " dolore magna aliqua.  Ut enim ad minim veniam, quis"
	       " nostrud exercitation ullamco laboris nisi ut"
	       " aliquip ex ea commodo consequat.  Duis aute irure"
	       " dolor in reprehenderit in voluptate velit esse"
	       " cillum dolore eu fugiat nulla pariatur.  Excepteur"
	       " sint occaecat cupidatat non proident, sunt in culpa"
	       " qui officia deserunt mollit anim id est laborum."))))))
  (pop-to-buffer (current-buffer)))

Re: bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Ivan Shmakov]

[-- Attachment #1: Type: text/plain, Size: 1364 bytes --]

	As it seems, the initial version of the patch didn’t play well
	with other essential shr.el features (as in: hyperlinks.)

	Please thus consider the revised patch MIMEd.

	* lisp/net/shr.el (shr-force-fill): New variable to disable this
	feature if needed.
	(shr-internal-width): Defer initialization until...
	(shr-insert-document): ... here; set to nil if neither
	shr-force-fill nor shr-width are non-nil.
	(shr-fold-text, shr-tag-table-1): Likewise.
	(shr-insert): Do not fill if shr-internal-width is nil.
	(shr-setup-wrap-1, shr-setup-wrap): New function.
	(shr-tag-blockquote, shr-tag-dd, shr-tag-li):
	Call shr-setup-wrap.
	(shr-tag-hr): Use a constant if shr-internal-width is nil.

	The other so far unresolved issue with this approach is that the
	tables and <pre /> elements may actually require truncate-lines.
	Unfortunately, I know of no way to allow for word-wrapped and
	truncated lines to exist in the same buffer; I guess we may need
	either a truncate-lines or word-wrap property (or both) to
	override the buffer-local variables in this case.

	Similarly to wrap-prefix, we may also use line-prefix in place
	of shr-indent.  But that may not be a good idea if quoting
	text/html messages in text/plain replies, for instance.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/diff, Size: 4441 bytes --]

--- a/lisp/net/shr.el
+++ b/lisp/net/shr.el
@@ -128,13 +128,16 @@
 (defvar shr-inhibit-images nil
   "If non-nil, inhibit loading images.")
 
+(defvar shr-force-fill nil
+  "If non-nil, fill text even in the cases Emacs can wrap it by itself.")
+
 ;;; Internal variables.
 
 (defvar shr-folding-mode nil)
 (defvar shr-state nil)
 (defvar shr-start nil)
 (defvar shr-indentation 0)
-(defvar shr-internal-width (or shr-width (1- (window-width))))
+(defvar shr-internal-width nil)		; set in shr-insert-document
 (defvar shr-list-mode nil)
 (defvar shr-content-cache nil)
 (defvar shr-kinsoku-shorten nil)
@@ -206,7 +209,8 @@ defun shr-insert-document (dom)
 	(shr-base nil)
 	(shr-depth 0)
 	(shr-warning nil)
-	(shr-internal-width (or shr-width (1- (window-width)))))
+	(shr-internal-width
+	 (or shr-width (and shr-force-fill (1- (window-width))))))
     (shr-descend dom)
     (shr-remove-trailing-whitespace start (point))
     (when shr-warning
@@ -420,7 +424,8 @@ defun shr-fold-text (text)
       (let ((shr-indentation 0)
 	    (shr-state nil)
 	    (shr-start nil)
-	    (shr-internal-width (window-width)))
+	    (shr-internal-width (and shr-force-fill
+				     (1- (window-width)))))
 	(shr-insert text)
 	(buffer-string)))))
 
@@ -485,7 +490,8 @@ defun shr-insert (text)
       (insert elem)
       (setq shr-state nil)
       (let (found)
-	(while (and (> (current-column) shr-internal-width)
+	(while (and shr-internal-width	; Use Emacs native wrapping if nil.
+		    (> (current-column) shr-internal-width)
 		    (> shr-internal-width 0)
 		    (progn
 		      (setq found (shr-find-fill-point))
@@ -500,7 +506,8 @@ defun shr-insert (text)
 	  (when (> shr-indentation 0)
 	    (shr-indent))
 	  (end-of-line))
-	(if (<= (current-column) shr-internal-width)
+	(if (or (not shr-internal-width)
+		(<= (current-column) shr-internal-width))
 	    (insert " ")
 	  ;; In case we couldn't get a valid break point (because of a
 	  ;; word that's longer than `shr-internal-width'), just break anyway.
@@ -665,6 +672,23 @@
   (when (> shr-indentation 0)
     (insert (make-string shr-indentation ? ))))
 
+(defun shr-setup-wrap-1 (from to pval)
+  (put-text-property from to 'wrap-prefix pval))
+
+(defun shr-setup-wrap (from to)
+  (let ((prev from)
+	(pos  (next-property-change from nil to))
+	(pval (and (> shr-indentation 0)
+		   `(space :align-to ,shr-indentation))))
+    (while (and pos (> pos prev))
+      (unless (get-text-property prev 'wrap-prefix)
+	(shr-setup-wrap-1 prev pos pval))
+      (setq prev pos
+	    pos  (next-property-change pos nil to)))
+    (unless (or (<= to prev)
+		(get-text-property prev 'wrap-prefix))
+	(shr-setup-wrap-1 prev to pval))))
+
 (defun shr-fontize-dom (dom &rest types)
   (let (shr-start)
     (shr-generic dom)
@@ -1308,8 +1338,10 @@
 (defun shr-tag-blockquote (dom)
   (shr-ensure-paragraph)
   (shr-indent)
-  (let ((shr-indentation (+ shr-indentation 4)))
-    (shr-generic dom))
+  (let ((from (point))
+	(shr-indentation (+ shr-indentation 4)))
+    (shr-generic dom)
+    (shr-setup-wrap from (point)))
   (shr-ensure-paragraph))
 
 (defun shr-tag-dl (dom)
@@ -1324,8 +1356,10 @@
 
 (defun shr-tag-dd (dom)
   (shr-ensure-newline)
-  (let ((shr-indentation (+ shr-indentation 4)))
-    (shr-generic dom)))
+  (let ((from (point))
+	(shr-indentation (+ shr-indentation 4)))
+    (shr-generic dom)
+    (shr-setup-wrap from (point))))
 
 (defun shr-tag-ul (dom)
   (shr-ensure-paragraph)
@@ -1348,9 +1382,11 @@ defun shr-tag-li (dom)
 		  (format "%d " shr-list-mode)
 		(setq shr-list-mode (1+ shr-list-mode)))
 	    shr-bullet))
+	 (from (point))
 	 (shr-indentation (+ shr-indentation (length bullet))))
     (insert bullet)
-    (shr-generic dom)))
+    (shr-generic dom)
+    (shr-setup-wrap from (point))))
 
 (defun shr-tag-br (dom)
   (when (and (not (bobp))
@@ -1386,7 +1422,8 @@
 
 (defun shr-tag-hr (_dom)
   (shr-ensure-newline)
-  (insert (make-string shr-internal-width shr-hr-line) "\n"))
+  (insert (make-string (or shr-internal-width 31) ; FIXME: magic
+		       shr-hr-line) "\n"))
 
 (defun shr-tag-title (dom)
   (shr-heading dom 'bold 'underline))
@@ -1414,6 +1451,7 @@
 (defun shr-tag-table-1 (dom)
   (setq dom (or (dom-child-by-tag dom 'tbody) dom))
   (let* ((shr-inhibit-images t)
+	 (shr-internal-width (or shr-internal-width (1- (window-width))))
 	 (shr-table-depth (1+ shr-table-depth))
 	 (shr-kinsoku-shorten t)
 	 ;; Find all suggested widths.

bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Ivan Shmakov]

[-- Attachment #1: Type: text/plain, Size: 1364 bytes --]

	As it seems, the initial version of the patch didn’t play well
	with other essential shr.el features (as in: hyperlinks.)

	Please thus consider the revised patch MIMEd.

	* lisp/net/shr.el (shr-force-fill): New variable to disable this
	feature if needed.
	(shr-internal-width): Defer initialization until...
	(shr-insert-document): ... here; set to nil if neither
	shr-force-fill nor shr-width are non-nil.
	(shr-fold-text, shr-tag-table-1): Likewise.
	(shr-insert): Do not fill if shr-internal-width is nil.
	(shr-setup-wrap-1, shr-setup-wrap): New function.
	(shr-tag-blockquote, shr-tag-dd, shr-tag-li):
	Call shr-setup-wrap.
	(shr-tag-hr): Use a constant if shr-internal-width is nil.

	The other so far unresolved issue with this approach is that the
	tables and <pre /> elements may actually require truncate-lines.
	Unfortunately, I know of no way to allow for word-wrapped and
	truncated lines to exist in the same buffer; I guess we may need
	either a truncate-lines or word-wrap property (or both) to
	override the buffer-local variables in this case.

	Similarly to wrap-prefix, we may also use line-prefix in place
	of shr-indent.  But that may not be a good idea if quoting
	text/html messages in text/plain replies, for instance.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/diff, Size: 4441 bytes --]

--- a/lisp/net/shr.el
+++ b/lisp/net/shr.el
@@ -128,13 +128,16 @@
 (defvar shr-inhibit-images nil
   "If non-nil, inhibit loading images.")
 
+(defvar shr-force-fill nil
+  "If non-nil, fill text even in the cases Emacs can wrap it by itself.")
+
 ;;; Internal variables.
 
 (defvar shr-folding-mode nil)
 (defvar shr-state nil)
 (defvar shr-start nil)
 (defvar shr-indentation 0)
-(defvar shr-internal-width (or shr-width (1- (window-width))))
+(defvar shr-internal-width nil)		; set in shr-insert-document
 (defvar shr-list-mode nil)
 (defvar shr-content-cache nil)
 (defvar shr-kinsoku-shorten nil)
@@ -206,7 +209,8 @@ defun shr-insert-document (dom)
 	(shr-base nil)
 	(shr-depth 0)
 	(shr-warning nil)
-	(shr-internal-width (or shr-width (1- (window-width)))))
+	(shr-internal-width
+	 (or shr-width (and shr-force-fill (1- (window-width))))))
     (shr-descend dom)
     (shr-remove-trailing-whitespace start (point))
     (when shr-warning
@@ -420,7 +424,8 @@ defun shr-fold-text (text)
       (let ((shr-indentation 0)
 	    (shr-state nil)
 	    (shr-start nil)
-	    (shr-internal-width (window-width)))
+	    (shr-internal-width (and shr-force-fill
+				     (1- (window-width)))))
 	(shr-insert text)
 	(buffer-string)))))
 
@@ -485,7 +490,8 @@ defun shr-insert (text)
       (insert elem)
       (setq shr-state nil)
       (let (found)
-	(while (and (> (current-column) shr-internal-width)
+	(while (and shr-internal-width	; Use Emacs native wrapping if nil.
+		    (> (current-column) shr-internal-width)
 		    (> shr-internal-width 0)
 		    (progn
 		      (setq found (shr-find-fill-point))
@@ -500,7 +506,8 @@ defun shr-insert (text)
 	  (when (> shr-indentation 0)
 	    (shr-indent))
 	  (end-of-line))
-	(if (<= (current-column) shr-internal-width)
+	(if (or (not shr-internal-width)
+		(<= (current-column) shr-internal-width))
 	    (insert " ")
 	  ;; In case we couldn't get a valid break point (because of a
 	  ;; word that's longer than `shr-internal-width'), just break anyway.
@@ -665,6 +672,23 @@
   (when (> shr-indentation 0)
     (insert (make-string shr-indentation ? ))))
 
+(defun shr-setup-wrap-1 (from to pval)
+  (put-text-property from to 'wrap-prefix pval))
+
+(defun shr-setup-wrap (from to)
+  (let ((prev from)
+	(pos  (next-property-change from nil to))
+	(pval (and (> shr-indentation 0)
+		   `(space :align-to ,shr-indentation))))
+    (while (and pos (> pos prev))
+      (unless (get-text-property prev 'wrap-prefix)
+	(shr-setup-wrap-1 prev pos pval))
+      (setq prev pos
+	    pos  (next-property-change pos nil to)))
+    (unless (or (<= to prev)
+		(get-text-property prev 'wrap-prefix))
+	(shr-setup-wrap-1 prev to pval))))
+
 (defun shr-fontize-dom (dom &rest types)
   (let (shr-start)
     (shr-generic dom)
@@ -1308,8 +1338,10 @@
 (defun shr-tag-blockquote (dom)
   (shr-ensure-paragraph)
   (shr-indent)
-  (let ((shr-indentation (+ shr-indentation 4)))
-    (shr-generic dom))
+  (let ((from (point))
+	(shr-indentation (+ shr-indentation 4)))
+    (shr-generic dom)
+    (shr-setup-wrap from (point)))
   (shr-ensure-paragraph))
 
 (defun shr-tag-dl (dom)
@@ -1324,8 +1356,10 @@
 
 (defun shr-tag-dd (dom)
   (shr-ensure-newline)
-  (let ((shr-indentation (+ shr-indentation 4)))
-    (shr-generic dom)))
+  (let ((from (point))
+	(shr-indentation (+ shr-indentation 4)))
+    (shr-generic dom)
+    (shr-setup-wrap from (point))))
 
 (defun shr-tag-ul (dom)
   (shr-ensure-paragraph)
@@ -1348,9 +1382,11 @@ defun shr-tag-li (dom)
 		  (format "%d " shr-list-mode)
 		(setq shr-list-mode (1+ shr-list-mode)))
 	    shr-bullet))
+	 (from (point))
 	 (shr-indentation (+ shr-indentation (length bullet))))
     (insert bullet)
-    (shr-generic dom)))
+    (shr-generic dom)
+    (shr-setup-wrap from (point))))
 
 (defun shr-tag-br (dom)
   (when (and (not (bobp))
@@ -1386,7 +1422,8 @@
 
 (defun shr-tag-hr (_dom)
   (shr-ensure-newline)
-  (insert (make-string shr-internal-width shr-hr-line) "\n"))
+  (insert (make-string (or shr-internal-width 31) ; FIXME: magic
+		       shr-hr-line) "\n"))
 
 (defun shr-tag-title (dom)
   (shr-heading dom 'bold 'underline))
@@ -1414,6 +1451,7 @@
 (defun shr-tag-table-1 (dom)
   (setq dom (or (dom-child-by-tag dom 'tbody) dom))
   (let* ((shr-inhibit-images t)
+	 (shr-internal-width (or shr-internal-width (1- (window-width))))
 	 (shr-table-depth (1+ shr-table-depth))
 	 (shr-kinsoku-shorten t)
 	 ;; Find all suggested widths.

word-wrap and wrapping before window-width (was: bug#19462: shr: use wrap-prefix when possible, instead of filling the text)

[Stefan Monnier]

> 	The only feature that I’m aware to be missing is the actual
> 	support for Emacs native text wrapping (as in: the word-wrap
> 	variable and wrap-prefix text property) in SHR.

Reminds me that while I opposed adding a `wrap-column' option for those
people who want to do word-wrap without using the whole window width,
I would actually welcome a new text-property which lets the wrap-column
be set locally to a different value (ideally, this should also allow
specifying that some part of the text should be word-wrapped while
others should be truncated).

Re: word-wrap and wrapping before window-width

[Ivan Shmakov]

>>>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:

 >> The only feature that I’m aware to be missing is the actual support
 >> for Emacs native text wrapping (as in: the word-wrap variable and
 >> wrap-prefix text property) in SHR.

 > Reminds me that while I opposed adding a ‘wrap-column’ option for
 > those people who want to do word-wrap without using the whole window
 > width, I would actually welcome a new text-property which lets the
 > wrap-column be set locally to a different value (ideally, this should
 > also allow specifying that some part of the text should be
 > word-wrapped while others should be truncated).

	There’s an minor issue of how to display word-wrapped lines
	while the window is scrolled horizontally.  Currently,
	horizontal scrolling simply inhibits word-wrap.

	Otherwise, I have very little knowledge of the Emacs display
	engine implementation to actually code support for such a
	property myself.  However, should it be implemented, I’d happily
	update my patch for #19462 to use that so that non-flowable
	content gets marked as such.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: word-wrap and wrapping before window-width

[Eli Zaretskii]

> From: Ivan Shmakov <ivan@siamics.net>
> Date: Mon, 29 Dec 2014 15:01:47 +0000
> 
> 	There’s an minor issue of how to display word-wrapped lines
> 	while the window is scrolled horizontally.  Currently,
> 	horizontal scrolling simply inhibits word-wrap.

What other GUI application does something different in this case?
Word wrap and horizontal scrolling are 2 different ways of coping with
the same problem, so does it really make sense to have them both at
the same time?

Re: word-wrap and wrapping before window-width

[Stefan Monnier]

>> There’s an minor issue of how to display word-wrapped lines
>> while the window is scrolled horizontally.  Currently,
>> horizontal scrolling simply inhibits word-wrap.
> What other GUI application does something different in this case?

If the word-wrapped lines are wrapped at a text-specified column rather
than "at the end of the window", then it would make a lot of sense, to
just keep the lines "wrapped as before" and just visually shift them
according to the horizontal scrolling.

It might also make sense to consider that a "wrap-column" which is set past
the window-width would cause those lines to be displayed as "truncated
and wrapped".  E.g. display it as:

   +---------+
    here is t|he example|
    wrapped t|ext       |
             ^          |
       window border    |
                        |
                        ^
                   wrap-column

IOW, the wrapping would be window (and hscrolling) independent.

And a "truncated line" could then "simply" be a line with an "infinite"
wrap-column.


        Stefan

Re: word-wrap and wrapping before window-width

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Ivan Shmakov <ivan@siamics.net>,  emacs-devel@gnu.org
> Date: Mon, 29 Dec 2014 22:12:03 -0500
> 
> If the word-wrapped lines are wrapped at a text-specified column rather
> than "at the end of the window", then it would make a lot of sense, to
> just keep the lines "wrapped as before" and just visually shift them
> according to the horizontal scrolling.
> 
> It might also make sense to consider that a "wrap-column" which is set past
> the window-width would cause those lines to be displayed as "truncated
> and wrapped".  E.g. display it as:
> 
>    +---------+
>     here is t|he example|
>     wrapped t|ext       |
>              ^          |
>        window border    |
>                         |
>                         ^
>                    wrap-column
> 
> IOW, the wrapping would be window (and hscrolling) independent.
> 
> And a "truncated line" could then "simply" be a line with an "infinite"
> wrap-column.

I simply fail to see any practical use cases for this kind of
display.  What are we trying to support with this?

It's hard to reason about this without having some use cases in mind.

Re: word-wrap and wrapping before window-width

[Ivan Shmakov]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

[…]

 >> IOW, the wrapping would be window (and hscrolling) independent.

	We may still want some value to mean “wrap at window width,
	however it is currently set.”

 >> And a "truncated line" could then "simply" be a line with an
 >> "infinite" wrap-column.

 > I simply fail to see any practical use cases for this kind of
 > display.  What are we trying to support with this?

	That’d make it easy to display format=flowed (RFC 3676) MIME
	parts, as well as enriched-mode documents, MediaWiki pages, and
	pretty much any other kind of text which allows for /both/
	wrappable and preformatted parts at the same time.

 > It's hard to reason about this without having some use cases in mind.

	For one thing, I edit MediaWiki pages on almost daily basis, and
	using word-wrap (and wrap-prefix) is more or less a no-brainer
	here.  Occasionally, however, it may be sensible to mark some
	parts of the buffer (as in: <pre />, <source /> and
	“leading blank” parts) to use truncation instead of wrapping.

	Now, to repeat myself, I know very little of the current Emacs
	display implementation.  However, it seems to me that
	wrap-column makes us one property closer to native multicolumn
	display.  Consider, e. g.:

This is an example sentence with wrap-column set to 23.

This is yet another example sentence with line-prefix and wrap-prefix
both set to (space :align-to 25), – or something like that.

	From there, we may display it as follows:

This is an example
sentence with
wrap-column set to 23.

                         This is yet another example sentence with line-prefix
                         and wrap-prefix both set to (space :align-to 25), –
                         or something like that.

	Yet, provided that some other property is switched on, the Emacs
	display engine may decide to show it like this instead:

This is an example       This is yet another example sentence with line-prefix
sentence with            and wrap-prefix both set to (space :align-to 25), –
wrap-column set to 23.   or something like that.

	As already imagined in this thread, forward- and backward-char
	commands would then still follow the logical order of text in the
	buffer (that is: the “23” sentence, then the “25” one), while
	left-char, etc. would (under visual-order-cursor-movement) follow the
	visual order.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

bug#19661: wrapping before window-width (new wrap-column text property?)

[Ivan Shmakov]

Package:  emacs
Severity: wishlist

	[As suggested in news:jwviogtdei4.fsf-monnier+emacs@gnu.org,
	news:87387w8r2j.fsf@violet.siamics.net, etc.; parts of the
	second message are reiterated below.]

	Please provide support for window-width-independent wrapping in
	the Emacs display engine; possibly by the means of a new
	wrap-column text property (and still perhaps complemented by a
	eponymous buffer-local variable), treated similarly to the likes
	of wrap-prefix.

	This feature could be used to display format=flowed (RFC 3676)
	MIME parts, as well as enriched-mode documents, documents using
	MediaWiki markup, SHR-rendered HTML documents, and pretty much
	any other kind of text which allows for /both/ wrappable and
	preformatted parts at the same time.

	It is already possible to influence the wrap width somewhat by
	setting the margin width variables (right-margin-width,
	left-margin-width) appropriately (see bug#4172, for instance.)
	The suggested wrap-column text property should probably have no
	effect on the window marginal areas, however.

	I admit that I know very little of the current Emacs display
	implementation.  However, it seems to me that wrap-column makes
	us one property closer to native multicolumn display (which’d
	warrant a separate wishlist bug report, though.)

	Consider, e. g.:

This is an example sentence with wrap-column set to 23.

This is yet another example sentence with line-prefix and wrap-prefix
both set to (space :align-to 25), – or something like that.

	From there, we may display it as follows:

This is an example
sentence with
wrap-column set to 23.

                         This is yet another example sentence with line-prefix
                         and wrap-prefix both set to (space :align-to 25), –
                         or something like that.

	Yet, provided that some other property is switched on, the Emacs
	display engine may decide to show it like this instead:

This is an example       This is yet another example sentence with line-prefix
sentence with            and wrap-prefix both set to (space :align-to 25), –
wrap-column set to 23.   or something like that.

	As already imagined in the preceding discussion, forward- and
	backward-char commands would then still follow the logical order
	of text in the buffer (that is: the “23” sentence, then the “25”
	one), while left-char, etc. would follow the visual order
	(assuming visual-order-cursor-movement.)

-- 
FSF associate member #7257  np. Mi memoras — Kajto      … 3013 B6A0 230E 334A

bug#19661: wrapping before window-width (new wrap-column text property?)

[Eli Zaretskii]

> From: Ivan Shmakov <ivan@siamics.net>
> Date: Fri, 23 Jan 2015 13:17:08 +0000
> 
> 	Please provide support for window-width-independent wrapping in
> 	the Emacs display engine; possibly by the means of a new
> 	wrap-column text property (and still perhaps complemented by a
> 	eponymous buffer-local variable), treated similarly to the likes
> 	of wrap-prefix.
> 
> 	This feature could be used to display format=flowed (RFC 3676)
> 	MIME parts, as well as enriched-mode documents, documents using
> 	MediaWiki markup, SHR-rendered HTML documents, and pretty much
> 	any other kind of text which allows for /both/ wrappable and
> 	preformatted parts at the same time.

format=flowed etc. is already supported by word-wrap, isn't it?

> 	I admit that I know very little of the current Emacs display
> 	implementation.

How about biting the bullet and trying to do this yourself?  I can
provide guidance and advice, if needed.

> 	Yet, provided that some other property is switched on, the Emacs
> 	display engine may decide to show it like this instead:
> 
> This is an example       This is yet another example sentence with line-prefix
> sentence with            and wrap-prefix both set to (space :align-to 25), –
> wrap-column set to 23.   or something like that.

This is a much harder nut to crack, and having wrap-column doesn't
help with that.

The fundamental problem here is that the Emacs display engine is based
on an "iterator" object that basically walks a buffer and generates
"glyph" objects that are then given to the display back-end for actual
display.  The iterator object has only a very myopic view of the text
it walks through.  Before Emacs 24, that view was one-character long
-- we only looked at the next character in the logical order.  With
Emacs 24's bidirectional display, the field of view became slightly
wider, but it is still limited to a single physical line, and most of
the display doesn't even know about that, the single exception being
bidi.c.

Now, the current display engine's workhorse is display_line, which
produces glyph objects for a single screen line.  What it does is call
a function to find the next "display element" (character, image,
composition, etc.) to display, produces glyphs for it, and goes to the
next display element in the visual order.

With your suggestion, once the width of the laid out glyphs reaches
some pixel value, the next display element will need to come from a
different part of the buffer.  But how to know where in the buffer is
that?  You cannot know that until you are done with layout of the
entire visible portion of the left-side pane, the one that in the
above example ends with "set to 23."

So either we need a deep surgery of display_line, so that it acquires
the ability to produce layout of each screen line in several parts, or
we write some tricky code that would perform all the necessary
calculations to find the buffer position of "This yet another example"
when we are done producing "This is an example" and want to continue
with the same screen line.

The former alternative means significant changes all over the display
engine, the latter means redisplay will be slower (not sure by how
much).  So both are highly non-trivial.

> 	As already imagined in the preceding discussion, forward- and
> 	backward-char commands would then still follow the logical order
> 	of text in the buffer (that is: the “23” sentence, then the “25”
> 	one), while left-char, etc. would follow the visual order
> 	(assuming visual-order-cursor-movement.)

That's the least of our trouble: the function that finds the place to
put the cursor (set_cursor_from_row) already thoroughly analyzes the
window display, and in Emacs 24 was rewritten to make it independent
of many assumptions that were broken by bidirectional display.

Perhaps you think that Emacs moves cursor visually, in which case it
would have had problems when the logical flow of text is broken like
that.  But that's not what Emacs does to move cursor: it moves point,
updates the display, and then figures out where in the new display to
put the cursor.

bug#19661: wrapping before window-width (new wrap-column text property?)

[martin rudalics]

 >> This is an example       This is yet another example sentence with line-prefix
 >> sentence with            and wrap-prefix both set to (space :align-to 25), –
 >> wrap-column set to 23.   or something like that.
 >
 > This is a much harder nut to crack, and having wrap-column doesn't
 > help with that.

This could be done with two side-by-side windows, `follow-mode' (Anders
Lindgren would certainly help with that) and some non-trivial changes in
window layout.  You'd probably want a zero width window to display a
common vertical scroll bar, a zero height window to display a horizontal
scroll bar and two zero height windows to display common mode and header
lines.  And obviously some meta mode that turns on multicolumn display
for parts of the text and manages the window layout appropriately.

In any case you can easily try a prototype with a number of side-by-side
windows turning off all decorations but the scroll bar of the rightmost
one and enabling `follow-mode'.  And you could insert (barely visible)
window dividers and use the mouse to change the widths of the columns.

martin

bug#19661: wrapping before window-width (new wrap-column text property?)

[Ivan Shmakov]

>>>>> martin rudalics <rudalics@gmx.at> writes:

 >>> This is an example       This is yet another example sentence with
 >>> sentence with            line-prefix and wrap-prefix both set to
 >>> wrap-column set to 23.   (space :align-to 25), – or something like that.

 >> This is a much harder nut to crack, and having wrap-column doesn't
 >> help with that.

 > This could be done with two side-by-side windows, `follow-mode'
 > (Anders Lindgren would certainly help with that) and some non-trivial
 > changes in window layout.

	Yes, I was thinking about something like that.  However, the
	ultimate goal is for Emacs to set foot on that wordprocessing
	land, so to say; and there, such window layouts may easily
	become unmanageable.

[…]

 > In any case you can easily try a prototype with a number of
 > side-by-side windows turning off all decorations but the scroll bar
 > of the rightmost one and enabling `follow-mode'.  And you could
 > insert (barely visible) window dividers and use the mouse to change
 > the widths of the columns.

	I doubt I really could experiment much with scrollbars with the
	tty-only Emacs builds I use exclusively for over a year now.

-- 
FSF associate member #7257  np. Innocent Exile — Iron Maiden … B6A0 230E 334A

bug#19661: wrapping before window-width (new wrap-column text property?)

[martin rudalics]

 > 	Yes, I was thinking about something like that.  However, the
 > 	ultimate goal is for Emacs to set foot on that wordprocessing
 > 	land, so to say; and there, such window layouts may easily
 > 	become unmanageable.

And you think the display engine can manage such layouts?

 > 	I doubt I really could experiment much with scrollbars with the
 > 	tty-only Emacs builds I use exclusively for over a year now.

So you are in the lucky position where you can omit scrollbars from such
experiments.

martin

bug#19661: wrapping before window-width (new wrap-column text property?)

[Eli Zaretskii]

> Date: Fri, 23 Jan 2015 17:55:53 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: 19661@debbugs.gnu.org
> 
>  >> This is an example       This is yet another example sentence with line-prefix
>  >> sentence with            and wrap-prefix both set to (space :align-to 25), –
>  >> wrap-column set to 23.   or something like that.
>  >
>  > This is a much harder nut to crack, and having wrap-column doesn't
>  > help with that.
> 
> This could be done with two side-by-side windows, `follow-mode' (Anders
> Lindgren would certainly help with that) and some non-trivial changes in
> window layout.

That'd be a kludge-around.  I thought we were talking about teaching
Emacs new layout tricks, not overloading existing ones with features
they weren't designed to support.  We all know the subtle problems in
follow-mode, right?

bug#19661: wrapping before window-width (new wrap-column text property?)

[martin rudalics]

 > That'd be a kludge-around.  I thought we were talking about teaching
 > Emacs new layout tricks, not overloading existing ones with features
 > they weren't designed to support.

Layouts should be handled at the Elisp level.  Where exactly and how is
a matter of taste.  Currently, windows can provide a tiled layout only.
For example, having (multicolumn) text flow around an image is tedious.
So this would be just an incentive to provide different window layouts
(or layers).

 > We all know the subtle problems in
 > follow-mode, right?

Which we would have to solve anyway.  I wouldn't reinvent the wheel.

martin

bug#19661: wrapping before window-width (new wrap-column text property?)

[Eli Zaretskii]

> Date: Sat, 24 Jan 2015 10:08:33 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: 19661@debbugs.gnu.org
> 
>  > That'd be a kludge-around.  I thought we were talking about teaching
>  > Emacs new layout tricks, not overloading existing ones with features
>  > they weren't designed to support.
> 
> Layouts should be handled at the Elisp level.

This is impossible with the current Emacs design, and you know it.
The design is that Lisp programs _specify_ the layout, by setting up
text properties, overlays, and local variables.  The actual _handling_
of the layout is done by the display engine, which is not exposed to
Lisp.

So if a particular kind of layout is not supported by the display
engine, you cannot specify it in Lisp.

> For example, having (multicolumn) text flow around an image is tedious.
> So this would be just an incentive to provide different window layouts
> (or layers).

I agree, but I don't think this can or should be done in Lisp.  Over
the years, I've seen many features that attempted to produce fancy
display traits not supported by the engine, and they all look kludgey
to me.  They also break very easily.

>  > We all know the subtle problems in follow-mode, right?
> 
> Which we would have to solve anyway.

The solutions are on the C level, not in Lisp.

> I wouldn't reinvent the wheel.

The wheel should be round, then it's a wheel.

bug#19661: wrapping before window-width (new wrap-column text property?)

[martin rudalics]

 >> Layouts should be handled at the Elisp level.
 >
 > This is impossible with the current Emacs design, and you know it.
 > The design is that Lisp programs _specify_ the layout, by setting up
 > text properties, overlays, and local variables.  The actual _handling_
 > of the layout is done by the display engine, which is not exposed to
 > Lisp.
 >
 > So if a particular kind of layout is not supported by the display
 > engine, you cannot specify it in Lisp.

The windows code does provide the display engine with a clipping
rectangle and two buffer positions where to start displaying text in
that rectangle and where to display the cursor (the latter may be
overridden by the display engine).  Together, these determine the basic
layout of buffer portions on screen and can be used by Lisp programs.

 > I agree, but I don't think this can or should be done in Lisp.  Over
 > the years, I've seen many features that attempted to produce fancy
 > display traits not supported by the engine, and they all look kludgey
 > to me.  They also break very easily.

With multiple columns we have to provide an API.  For example, to decide
whether the first character of a buffer's line is also the the first
character of a line in the rectangle displaying that line.  Otherwise,
we cannot provide the navigation facilities Ivan asked for.  If each
column is displayed in a separate rectangle, the first character of a
line is always the first character of the rectangle displaying that line
and you can handle this distinction, and thus provide the API, on the
Lisp level.

 > The wheel should be round, then it's a wheel.

And it should spin freely.

martin

bug#19661: wrapping before window-width (new wrap-column text property?)

[Eli Zaretskii]

> Date: Sun, 25 Jan 2015 11:38:42 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: 19661@debbugs.gnu.org
> 
>  >> Layouts should be handled at the Elisp level.
>  >
>  > This is impossible with the current Emacs design, and you know it.
>  > The design is that Lisp programs _specify_ the layout, by setting up
>  > text properties, overlays, and local variables.  The actual _handling_
>  > of the layout is done by the display engine, which is not exposed to
>  > Lisp.
>  >
>  > So if a particular kind of layout is not supported by the display
>  > engine, you cannot specify it in Lisp.
> 
> The windows code does provide the display engine with a clipping
> rectangle and two buffer positions where to start displaying text in
> that rectangle and where to display the cursor (the latter may be
> overridden by the display engine).  Together, these determine the basic
> layout of buffer portions on screen and can be used by Lisp programs.

Sorry for being dense, this being just the first weekday for me, but
what "windows code" does that, please?

In any case, telling the display engine where to start the display is
a far cry from telling it how to lay out the screen from that point
onwards.

>  > I agree, but I don't think this can or should be done in Lisp.  Over
>  > the years, I've seen many features that attempted to produce fancy
>  > display traits not supported by the engine, and they all look kludgey
>  > to me.  They also break very easily.
> 
> With multiple columns we have to provide an API.  For example, to decide
> whether the first character of a buffer's line is also the the first
> character of a line in the rectangle displaying that line.  Otherwise,
> we cannot provide the navigation facilities Ivan asked for.  If each
> column is displayed in a separate rectangle, the first character of a
> line is always the first character of the rectangle displaying that line
> and you can handle this distinction, and thus provide the API, on the
> Lisp level.

Providing an API is not equivalent to implementing it.  In order for
us to be able to provide such an API, the display engine should
implement the support such an API will need.  And that's the hard part
of the job -- how to perform this layout.  Once we figure that out,
providing an API for controlling it will be easy.

bug#19661: wrapping before window-width (new wrap-column text property?)

[martin rudalics]

 > Sorry for being dense, this being just the first weekday for me, but
 > what "windows code" does that, please?

The one that calculates the text size of windows.  Or did you read
"Windows code"?

 > In any case, telling the display engine where to start the display is
 > a far cry from telling it how to lay out the screen from that point
 > onwards.

My point was to _not_ change the code of the display engine.

 > Providing an API is not equivalent to implementing it.  In order for
 > us to be able to provide such an API, the display engine should
 > implement the support such an API will need.  And that's the hard part
 > of the job -- how to perform this layout.  Once we figure that out,
 > providing an API for controlling it will be easy.

It essentially would have to subdivide a window into rectangles.  And I
would do that on the window(s) level to avoid the hard part of the job.

martin

bug#19661: wrapping before window-width (new wrap-column text property?)

[Eli Zaretskii]

> Date: Sun, 25 Jan 2015 18:46:34 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: 19661@debbugs.gnu.org
> 
>  > Sorry for being dense, this being just the first weekday for me, but
>  > what "windows code" does that, please?
> 
> The one that calculates the text size of windows.

OK, but I fail to see how this, or anything similar, can provide the
features being discussed.

>  > In any case, telling the display engine where to start the display is
>  > a far cry from telling it how to lay out the screen from that point
>  > onwards.
> 
> My point was to _not_ change the code of the display engine.

Why not?

>  > Providing an API is not equivalent to implementing it.  In order for
>  > us to be able to provide such an API, the display engine should
>  > implement the support such an API will need.  And that's the hard part
>  > of the job -- how to perform this layout.  Once we figure that out,
>  > providing an API for controlling it will be easy.
> 
> It essentially would have to subdivide a window into rectangles.  And I
> would do that on the window(s) level to avoid the hard part of the job.

Well, like I said, I consider such (ab)using of windows a kludge.

bug#19661: wrapping before window-width (new wrap-column text property?)

[Ivan Shmakov]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

 >> From: Ivan Shmakov  Date: Fri, 23 Jan 2015 13:17:08 +0000

 >> Please provide support for window-width-independent wrapping in the
 >> Emacs display engine; possibly by the means of a new wrap-column
 >> text property (and still perhaps complemented by a eponymous
 >> buffer-local variable), treated similarly to the likes of
 >> wrap-prefix.

	… Or that could rather be wrap-width, given that Emacs supports
	pixelwise resize now (not to mention variable-width fonts, etc…)

 >> This feature could be used to display format=flowed (RFC 3676) MIME
 >> parts, as well as enriched-mode documents, documents using MediaWiki
 >> markup, SHR-rendered HTML documents, and pretty much any other kind
 >> of text which allows for /both/ wrappable and preformatted parts at
 >> the same time.

 > format=flowed etc. is already supported by word-wrap, isn't it?

	Not in its entirety; consider, e. g. (section 4.1 of RFC 3676):

   If the line ends in a space, the line is flowed.  Otherwise it is
   fixed.  The exception to this rule is a signature separator line,
   described in Section 4.3.  Such lines end in a space but are neither
   flowed nor fixed.

	I know of no way to make the Emacs display engine wrap one line
	but not the other in the very same buffer.

 >> I admit that I know very little of the current Emacs display
 >> implementation.

 > How about biting the bullet and trying to do this yourself?  I can
 > provide guidance and advice, if needed.

	I guess I can, but most probably /not/ in the next few months.

	(The biggest problem for me would be the change of the tools and
	the workflow.  Say, nothing like eval-defun is going to be
	available while working on the C code.  Also, I’m not really
	keen when it comes to non-tty Emacs frames, – never felt those
	as of being of much value for my tasks.)

[…]

 > The fundamental problem here is that the Emacs display engine is
 > based on an "iterator" object that basically walks a buffer and
 > generates "glyph" objects that are then given to the display back-end
 > for actual display.  The iterator object has only a very myopic view
 > of the text it walks through.  Before Emacs 24, that view was
 > one-character long -- we only looked at the next character in the
 > logical order.  With Emacs 24's bidirectional display, the field of
 > view became slightly wider, but it is still limited to a single
 > physical line, and most of the display doesn't even know about that,
 > the single exception being bidi.c.

	“Physical” as in “display” or “buffer”?

 > With your suggestion, once the width of the laid out glyphs reaches
 > some pixel value, the next display element will need to come from a
 > different part of the buffer.  But how to know where in the buffer is
 > that?  You cannot know that until you are done with layout of the
 > entire visible portion of the left-side pane, the one that in the
 > above example ends with "set to 23."

 > So either we need a deep surgery of display_line, so that it acquires
 > the ability to produce layout of each screen line in several parts,
 > or we write some tricky code that would perform all the necessary
 > calculations to find the buffer position of "This yet another
 > example" when we are done producing "This is an example" and want to
 > continue with the same screen line.

	Basically, yes, I thought about the display engine keeping track
	of the latest “wasted” rectangular chunk of screen space, and
	allowing for it to be occupied by the text coming later in the
	“buffer” order.  (Or perhaps up to two such chunks: one to the
	right and one to the left of the text being drawn.)

	IIUC, display_line would potentially have to be called several
	times to draw a single display line.

	And all this behavior only triggered when the text being drawn
	has some specific property; once the property value changes, —
	the state is reset.

 > The former alternative means significant changes all over the display
 > engine, the latter means redisplay will be slower (not sure by how
 > much).  So both are highly non-trivial.

	ACK; thanks for the detailed explanation.

 >> As already imagined in the preceding discussion, forward- and
 >> backward-char commands would then still follow the logical order of
 >> text in the buffer (that is: the “23” sentence, then the “25” one),
 >> while left-char, etc. would follow the visual order (assuming
 >> visual-order-cursor-movement.)

 > That's the least of our trouble: the function that finds the place to
 > put the cursor (set_cursor_from_row) already thoroughly analyzes the
 > window display, and in Emacs 24 was rewritten to make it independent
 > of many assumptions that were broken by bidirectional display.

 > Perhaps you think that Emacs moves cursor visually, in which case it
 > would have had problems when the logical flow of text is broken like
 > that.  But that's not what Emacs does to move cursor: it moves point,
 > updates the display, and then figures out where in the new display to
 > put the cursor.

	That was a pure UI consideration.  (And not even one I’ve
	personally thought of.)

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

bug#19661: wrapping before window-width (new wrap-column text property?)

[Eli Zaretskii]

> From: Ivan Shmakov <ivan@siamics.net>
> Date: Fri, 23 Jan 2015 19:45:31 +0000
> 
>  > format=flowed etc. is already supported by word-wrap, isn't it?
> 
> 	Not in its entirety; consider, e. g. (section 4.1 of RFC 3676):
> 
>    If the line ends in a space, the line is flowed.  Otherwise it is
>    fixed.  The exception to this rule is a signature separator line,
>    described in Section 4.3.  Such lines end in a space but are neither
>    flowed nor fixed.
> 
> 	I know of no way to make the Emacs display engine wrap one line
> 	but not the other in the very same buffer.

But then the feature you suggest will have the same problem, since it
will be build on the same infrastructure as word-wrap.

>  >> I admit that I know very little of the current Emacs display
>  >> implementation.
> 
>  > How about biting the bullet and trying to do this yourself?  I can
>  > provide guidance and advice, if needed.
> 
> 	I guess I can, but most probably /not/ in the next few months.

There's no rush.

> 
> 	(The biggest problem for me would be the change of the tools and
> 	the workflow.  Say, nothing like eval-defun is going to be
> 	available while working on the C code.  Also, I’m not really
> 	keen when it comes to non-tty Emacs frames, – never felt those
> 	as of being of much value for my tasks.)

Something to learn, I guess.

>  > The fundamental problem here is that the Emacs display engine is
>  > based on an "iterator" object that basically walks a buffer and
>  > generates "glyph" objects that are then given to the display back-end
>  > for actual display.  The iterator object has only a very myopic view
>  > of the text it walks through.  Before Emacs 24, that view was
>  > one-character long -- we only looked at the next character in the
>  > logical order.  With Emacs 24's bidirectional display, the field of
>  > view became slightly wider, but it is still limited to a single
>  > physical line, and most of the display doesn't even know about that,
>  > the single exception being bidi.c.
> 
> 	“Physical” as in “display” or “buffer”?

In the buffer, i.e. up to the next newline.

>  > So either we need a deep surgery of display_line, so that it acquires
>  > the ability to produce layout of each screen line in several parts,
>  > or we write some tricky code that would perform all the necessary
>  > calculations to find the buffer position of "This yet another
>  > example" when we are done producing "This is an example" and want to
>  > continue with the same screen line.
> 
> 	Basically, yes, I thought about the display engine keeping track
> 	of the latest “wasted” rectangular chunk of screen space, and
> 	allowing for it to be occupied by the text coming later in the
> 	“buffer” order.  (Or perhaps up to two such chunks: one to the
> 	right and one to the left of the text being drawn.)

That's the whole problem: the current display engine doesn't manage
any rectangular space.  It proceeds one line at a time.

> 	IIUC, display_line would potentially have to be called several
> 	times to draw a single display line.

Several calls is not the problem.  The problem is how to know with
which buffer position to call it.

> 	And all this behavior only triggered when the text being drawn
> 	has some specific property; once the property value changes, —
> 	the state is reset.

Redisplay will take care of that, so again, this isn't the problem.

bug#19661: wrapping before window-width (new wrap-column text property?)

[Ivan Shmakov]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>>> From: Ivan Shmakov  Date: Fri, 23 Jan 2015 19:45:31 +0000

[…]

 >> I know of no way to make the Emacs display engine wrap one line but
 >> not the other in the very same buffer.

 > But then the feature you suggest will have the same problem, since it
 > will be build on the same infrastructure as word-wrap.

	When I set wrap-width for a specific line to a value greater
	than that line’s own width, I expect the line to no longer be
	wrapped.

	Also, as Stefan suggests, there can be a distinct “infinity”
	value for the property in question, to inhibit wrapping
	unconditionally for the line(s) covered.

[…]

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: word-wrap and wrapping before window-width

[Stefan Monnier]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

>> From: Ivan Shmakov <ivan@siamics.net>
>> Date: Mon, 29 Dec 2014 15:01:47 +0000
>> 
>> There’s an minor issue of how to display word-wrapped lines
>> while the window is scrolled horizontally.  Currently,
>> horizontal scrolling simply inhibits word-wrap.

> What other GUI application does something different in this case?
> Word wrap and horizontal scrolling are 2 different ways of coping with
> the same problem, so does it really make sense to have them both at
> the same time?

Re: word-wrap and wrapping before window-width

[Ivan Shmakov]

[-- Attachment #1: Type: text/plain, Size: 1024 bytes --]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>>> From: Ivan Shmakov  Date: Mon, 29 Dec 2014 15:01:47 +0000

 >> There’s an minor issue of how to display word-wrapped lines while
 >> the window is scrolled horizontally.  Currently, horizontal
 >> scrolling simply inhibits word-wrap.

 > What other GUI application does something different in this case?

	Firefox.  Provided that the properties are set that way, of
	course.  For one thing, <pre /> elements’ content gets truncated
	unless fits into its respective place (or gets adorned with a
	scrollbar, or otherwise spills over its neighbor) by default,
	while all the rest get folded.

	Please consider the HTML document MIMEd, for instance, – it
	should show three of these cases.

 > Word wrap and horizontal scrolling are 2 different ways of coping
 > with the same problem, so does it really make sense to have them both
 > at the same time?

	Yes.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

[-- Attachment #2: Type: text/html, Size: 608 bytes --]

Re: word-wrap and wrapping before window-width

[Eli Zaretskii]

> From: Ivan Shmakov <ivan@siamics.net>
> Date: Wed, 31 Dec 2014 03:17:14 +0000
> 
>  >> There’s an minor issue of how to display word-wrapped lines while
>  >> the window is scrolled horizontally.  Currently, horizontal
>  >> scrolling simply inhibits word-wrap.
> 
>  > What other GUI application does something different in this case?
> 
> 	Firefox.

Showing what URL?

Re: word-wrap and wrapping before window-width

[Ivan Shmakov]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>>> From: Ivan Shmakov  Date: Wed, 31 Dec 2014 03:17:14 +0000

 >>>> There’s an minor issue of how to display word-wrapped lines while
 >>>> the window is scrolled horizontally.  Currently, horizontal
 >>>> scrolling simply inhibits word-wrap.

 >>> What other GUI application does something different in this case?

 >> Firefox.

 > Showing what URL?

	This one:

https://www.gnu.org/software/emacs/manual/html_node/elisp/Box-Diagrams.html

	I can find you dozens more if you need, – just tell me and I’ll
	do it right away.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: word-wrap and wrapping before window-width

[Eli Zaretskii]

> From: Ivan Shmakov <ivan@siamics.net>
> Date: Wed, 31 Dec 2014 06:15:10 +0000
> 
> >>>>> Eli Zaretskii <eliz@gnu.org> writes:
> >>>>> From: Ivan Shmakov  Date: Wed, 31 Dec 2014 03:17:14 +0000
> 
>  >>>> There’s an minor issue of how to display word-wrapped lines while
>  >>>> the window is scrolled horizontally.  Currently, horizontal
>  >>>> scrolling simply inhibits word-wrap.
> 
>  >>> What other GUI application does something different in this case?
> 
>  >> Firefox.
> 
>  > Showing what URL?
> 
> 	This one:
> 
> https://www.gnu.org/software/emacs/manual/html_node/elisp/Box-Diagrams.html

What I see there is not what was described in this discussion.  There
are blocks of text there that are exempt from word wrap, that's all.
But as soon as you make the window narrow enough to have the
horizontal scroll bar appear, and you scroll horizontally using that
scroll bar, the entire text is scrolled as one rigid body, exactly as
Emacs does with horizontal scrolling.

> 	I can find you dozens more if you need, – just tell me and I’ll
> 	do it right away.

If they do something different from this one, please do, and thanks.

Re: word-wrap and wrapping before window-width

[Ivan Shmakov]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>>> From: Ivan Shmakov  Date: Wed, 31 Dec 2014 06:15:10 +0000

[…]

 >> https://www.gnu.org/software/emacs/manual/html_node/elisp/Box-Diagrams.html

 > What I see there is not what was described in this discussion.  There
 > are blocks of text there that are exempt from word wrap, that's all.

	That’s also all what I’ve initially requested: to be able to
	mark portions of text as exempt from word wrap.  (Or, better
	still, – to force truncation for such lines.)

 > But as soon as you make the window narrow enough to have the
 > horizontal scroll bar appear, and you scroll horizontally using that
 > scroll bar, the entire text is scrolled as one rigid body,

	With lines being wrapped /past/ the window’s own borders, right?
	So, you’d see exactly the picture Stefan has provided earlier:

 SM> +---------+
 SM>  here is t|he example|
 SM>  wrapped t|ext       |
 SM>           ^          |
 SM>     window border    |
 SM>                      |
 SM>                      ^
 SM>                 wrap-column

	Or perhaps even:

      |← window borders →|
    He|e is the more or l|ss
    th| same example text| ↖
      |                  |  wrap-column

	Such display is clearly possible with Firefox, while the Emacs
	display engine so far doesn’t support it.

 > exactly as Emacs does with horizontal scrolling.

	It isn’t the behavior I observe, – when I scroll a Emacs window
	horizontally, all the lines that were wrapped are magically
	unwrapped.

[…]

-- 
FSF associate member #7257  np. Balance — David Modica  … 3013 B6A0 230E 334A

Re: word-wrap and wrapping before window-width

[Eli Zaretskii]

> From: Ivan Shmakov <ivan@siamics.net>
> Date: Wed, 31 Dec 2014 16:37:41 +0000
> 
> >>>>> Eli Zaretskii <eliz@gnu.org> writes:
> >>>>> From: Ivan Shmakov  Date: Wed, 31 Dec 2014 06:15:10 +0000
> 
> […]
> 
>  >> https://www.gnu.org/software/emacs/manual/html_node/elisp/Box-Diagrams.html
> 
>  > What I see there is not what was described in this discussion.  There
>  > are blocks of text there that are exempt from word wrap, that's all.
> 
> 	That’s also all what I’ve initially requested: to be able to
> 	mark portions of text as exempt from word wrap.  (Or, better
> 	still, – to force truncation for such lines.)

No, you said:

	There’s an minor issue of how to display word-wrapped lines
	while the window is scrolled horizontally.     Currently,
	horizontal scrolling simply inhibits word-wrap.^^^^^^^^^
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

And Firefox does that too: it inhibits word wrap when horizontal
scrolling is in effect.  It just doesn't unwrap what was already
wrapped, that's all the difference.


>  > But as soon as you make the window narrow enough to have the
>  > horizontal scroll bar appear, and you scroll horizontally using that
>  > scroll bar, the entire text is scrolled as one rigid body,
> 
> 	With lines being wrapped /past/ the window’s own borders, right?

No, dynamic wrapping is disabled there.  Firefox simply keeps the
result of the last wrapping.

> 	So, you’d see exactly the picture Stefan has provided earlier:
> 
>  SM> +---------+
>  SM>  here is t|he example|
>  SM>  wrapped t|ext       |
>  SM>           ^          |
>  SM>     window border    |
>  SM>                      |
>  SM>                      ^
>  SM>                 wrap-column
> 
> 	Or perhaps even:
> 
>       |← window borders →|
>     He|e is the more or l|ss
>     th| same example text| ↖
>       |                  |  wrap-column
> 
> 	Such display is clearly possible with Firefox, while the Emacs
> 	display engine so far doesn’t support it.

Yes, but Emacs has a harder job to do: the above model is problematic
with bidirectional text when a single buffer has paragraphs of
different directionality (which Firefox doesn't seem to support).

>  > exactly as Emacs does with horizontal scrolling.
> 
> 	It isn’t the behavior I observe, – when I scroll a Emacs window
> 	horizontally, all the lines that were wrapped are magically
> 	unwrapped.

I meant the scrolling part -- everything as a rigid body.  The
unwrapping part is a clear evidence that some feature is missing in
Emacs, no argument there.

Re: word-wrap and wrapping before window-width

[Ivan Shmakov]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>>> From: Ivan Shmakov  Date: Wed, 31 Dec 2014 16:37:41 +0000

[…]

 >>> What I see there is not what was described in this discussion.
 >>> There are blocks of text there that are exempt from word wrap,
 >>> that's all.

 >> That’s also all what I’ve initially requested: to be able to mark
 >> portions of text as exempt from word wrap.  (Or, better still, – to
 >> force truncation for such lines.)

 > No, you said:

 IS> There’s an minor issue of how to display word-wrapped lines while
 IS> the window is scrolled horizontally.  Currently, horizontal
 IS> scrolling simply inhibits word-wrap.             ^^^^^^^^^^
 IS> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

	I’ve also said in this same discussion (though not, strictly
	speaking, at an ancestor node to this message) [1]:

 IS> The other so far unresolved issue with this approach is that the
 IS> tables and <pre /> elements may actually require truncate-lines.
 IS> Unfortunately, I know of no way to allow for word-wrapped and
 IS> truncated lines to exist in the same buffer; I guess we may need
 IS> either a truncate-lines or word-wrap property (or both) to override
 IS> the buffer-local variables in this case.

	Please consider /that/ a request.  TIA.  And sorry for the
	confusion.

[1] news:8761cubx18.fsf@violet.siamics.net
    http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19462#10

 > And Firefox does that too: it inhibits word wrap when horizontal
 > scrolling is in effect.  It just doesn't unwrap what was already
 > wrapped, that's all the difference.

	Frankly, I’m not so sure of this interpretation.  Anyway, from
	the user’s perspective, – is there a difference between
	“not unwrapping” and “wrapping at an arbitrary column”?

	Note that there’s the ‘width’ CSS property, – which can easily
	be set on per-paragraph basis, to the possible effect of making
	these paragraphs all wrap at different columns.

[…]

 >> Such display is clearly possible with Firefox, while the Emacs
 >> display engine so far doesn’t support it.

 > Yes, but Emacs has a harder job to do: the above model is problematic
 > with bidirectional text when a single buffer has paragraphs of
 > different directionality (which Firefox doesn't seem to support).

	You mean, your install Firefox install doesn’t cope with, say,
	the simplistic example document shown at [2]?

[2] https://ru.wikibooks.org/wiki/HTML_в_профилях/Базовый_профиль#multilingual.html

[…]

-- 
FSF associate member #7257  np. The Light (Part II) — Justin Bianco 230E 334A

Re: word-wrap and wrapping before window-width

[Eli Zaretskii]

> From: Ivan Shmakov <ivan@siamics.net>
> Date: Wed, 31 Dec 2014 17:55:33 +0000
> 
>  > And Firefox does that too: it inhibits word wrap when horizontal
>  > scrolling is in effect.  It just doesn't unwrap what was already
>  > wrapped, that's all the difference.
> 
> 	Frankly, I’m not so sure of this interpretation.  Anyway, from
> 	the user’s perspective, – is there a difference between
> 	“not unwrapping” and “wrapping at an arbitrary column”?

This is emacs-devel, not help-gnu-emacs ;-)  So our perspective is
different.

>  > Yes, but Emacs has a harder job to do: the above model is problematic
>  > with bidirectional text when a single buffer has paragraphs of
>  > different directionality (which Firefox doesn't seem to support).
> 
> 	You mean, your install Firefox install doesn’t cope with, say,
> 	the simplistic example document shown at [2]?
> 
> [2] https://ru.wikibooks.org/wiki/HTML_в_профилях/Базовый_профиль#multilingual.html

No, I meant it cannot determine the base paragraph direction
automatically, as Unicode requires.

Also, its horizontal scrolling of mixed-directional paragraphs makes
it hard to read the stuff, because scrolling to the right makes RTL
text at the beginning of a line disappear.  Emacs does a much better
job in that respect.

Re: word-wrap and wrapping before window-width

[Ivan Shmakov]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>>> From: Ivan Shmakov  Date: Wed, 31 Dec 2014 17:55:33 +0000

 >>> And Firefox does that too: it inhibits word wrap when horizontal
 >>> scrolling is in effect.  It just doesn't unwrap what was already
 >>> wrapped, that's all the difference.

 >> Frankly, I’m not so sure of this interpretation.  Anyway, from the
 >> user’s perspective, – is there a difference between “not unwrapping”
 >> and “wrapping at an arbitrary column”?

 > This is emacs-devel, not help-gnu-emacs ;-)  So our perspective is
 > different.

	My understanding of how CSS support is implemented is that
	window geometry changes are one but by no means the only way
	that may lead Firefox to recompute the container widths.  Once
	the width is computed, – the text is wrapped at that exact point
	(if at all.)

	In the case at hand, the width of the container was constrained
	to be no less of the contained object (<pre /> in this case.)
	Which gave the behavior observed.  This is a sure possibility,
	but I know of nothing in the specifications that’d suggest
	that’s a necessity.

 >>> Yes, but Emacs has a harder job to do: the above model is
 >>> problematic with bidirectional text when a single buffer has
 >>> paragraphs of different directionality (which Firefox doesn't seem
 >>> to support).

 >> You mean, your install Firefox install doesn’t cope with, say, the
 >> simplistic example document shown at [2]?

 >> [2] https://ru.wikibooks.org/wiki/HTML_в_профилях/Базовый_профиль#multilingual.html

 > No, I meant it cannot determine the base paragraph direction
 > automatically, as Unicode requires.

	I presume that Unicode paragraphs are not meant to be exactly
	the same thing as HTML5 paragraphs?

	Anyway, the HTML5 TR clearly specifies at which point the
	paragraph direction is reconsidered; see [1].  For one thing,
	the directionality of the <pre /> contents is (AIUI) determined
	on per line basis.

[1] http://www.w3.org/TR/html5/dom.html#the-dir-attribute

 > Also, its horizontal scrolling of mixed-directional paragraphs makes
 > it hard to read the stuff, because scrolling to the right makes RTL
 > text at the beginning of a line disappear.  Emacs does a much better
 > job in that respect.

	I do not think I understand.  Care to provide examples?

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: word-wrap and wrapping before window-width

[Eli Zaretskii]

> From: Ivan Shmakov <ivan@siamics.net>
> Date: Wed, 31 Dec 2014 18:57:53 +0000
> 
>  > Also, its horizontal scrolling of mixed-directional paragraphs makes
>  > it hard to read the stuff, because scrolling to the right makes RTL
>  > text at the beginning of a line disappear.  Emacs does a much better
>  > job in that respect.
> 
> 	I do not think I understand.  Care to provide examples?

Make an HTML file with 2 <pre> paragraphs of very long text, one
paragraph with dir="ltr", the other with dir="rtl", then make the
window narrower than the text, and scroll with the horizontal scroll
bar to see what I meant.  Compare with what Emacs does, e.g., in
TUTORIAL.he (there's an LTR paragraph at the end of the document).

Re: word-wrap and wrapping before window-width (was: bug#19462: shr: use wrap-prefix when possible, instead of filling the text)

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 29 Dec 2014 09:17:55 -0500
> Cc: emacs-devel@gnu.org
> 
> I would actually welcome a new text-property which lets the wrap-column
> be set locally to a different value (ideally, this should also allow
> specifying that some part of the text should be word-wrapped while
> others should be truncated).

This is too vague.  On what text will this property be put?  E.g., it
cannot be on the part that could be after the wrap point, so it will
probably have to be on the first character of a line.

Next, what is the extent of text for which this takes effect?  IOW,
where does this setting end?

There are also issues with cursor placement and continuation
glyphs/bitmaps.

This  all needs to be decided before it can be implemented.

And of course, it doesn't fix Lars's use case, because he needed to
fit several "columns" of text inside a window-width.  Wrapping at some
column is not enough for that.

Re: word-wrap and wrapping before window-width

[Stefan Monnier]

>> I would actually welcome a new text-property which lets the wrap-column
>> be set locally to a different value (ideally, this should also allow
>> specifying that some part of the text should be word-wrapped while
>> others should be truncated).
> This is too vague.

Unsurprisingly, yes.

> On what text will this property be put?  E.g., it cannot be on the
> part that could be after the wrap point, so it will probably have to
> be on the first character of a line.

I was thinking of placing on the whole wrappable chunk of text.  But if
that's inconvenient, it can be placed elsewhere.  The first char of
the line sounds usable as well.

> Next, what is the extent of text for which this takes effect?  IOW,
> where does this setting end?

Either at the end of the line, or as soon as the text-property is not
present any more.

> There are also issues with cursor placement and continuation
> glyphs/bitmaps.

I'd expect the continuation glyphs to be placed in the fringe as usual.
As for cursor placement, I'm not sure what issues that entails (I can't
think of any situation where it's not clear where the cursor should be
drawn, ideally).

> This all needs to be decided before it can be implemented.

To some extent the implementation can constrain the design.


        Stefan

bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Lars Ingebrigtsen]

Ivan Shmakov <ivan@siamics.net> writes:

>  >> (Yes, Emacs can display proportional fonts and fonts of different
>  >> sizes, but until you can fold (etc) proportional text (and text with
>  >> a mixture of font sizes) in a pretty manner, that's more of a toy
>  >> than anything else.)
>
>  > What's non-pretty with how we do this now?  What features are
>  > missing?
>
> 	The only feature that I’m aware to be missing is the actual
> 	support for Emacs native text wrapping (as in: the word-wrap
> 	variable and wrap-prefix text property) in SHR.
>
> 	Please thus consider the patch MIMEd.

I think this was superseded by the shr proportional font rewrite, so I'm
closing the bug.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Ivan Shmakov]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:
>>>>> Ivan Shmakov <ivan@siamics.net> writes:

[…]

 >> The only feature that I’m aware to be missing is the actual support
 >> for Emacs native text wrapping (as in: the word-wrap variable and
 >> wrap-prefix text property) in SHR.

 >> Please thus consider the patch MIMEd.

 > I think this was superseded by the shr proportional font rewrite, so
 > I'm closing the bug.

	I’ve stopped using the SHR version that comes with Emacs this
	May, so I don’t think I care much about that any longer.

	I have several items in my wishlist regarding rendering HTML in
	Emacs buffers, but I guess they will warrant writing the code
	anew, presumably to be released under a different name, so that
	it can be installed alongside SHR.

	(Not that I currently have much spare time to dedicate to that.)

-- 
FSF associate member #7257  http://am-1.org/~ivan/      … 3013 B6A0 230E 334A

Re: bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Ivan Shmakov]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:
>>>>> Ivan Shmakov <ivan@siamics.net> writes:

[…]

 >> The only feature that I’m aware to be missing is the actual support
 >> for Emacs native text wrapping (as in: the word-wrap variable and
 >> wrap-prefix text property) in SHR.

 >> Please thus consider the patch MIMEd.

 > I think this was superseded by the shr proportional font rewrite, so
 > I'm closing the bug.

	I’ve stopped using the SHR version that comes with Emacs this
	May, so I don’t think I care much about that any longer.

	I have several items in my wishlist regarding rendering HTML in
	Emacs buffers, but I guess they will warrant writing the code
	anew, presumably to be released under a different name, so that
	it can be installed alongside SHR.

	(Not that I currently have much spare time to dedicate to that.)

-- 
FSF associate member #7257  http://am-1.org/~ivan/      … 3013 B6A0 230E 334A

Re: bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Lars Ingebrigtsen]

Ivan Shmakov <ivan@siamics.net> writes:

>  >> (Yes, Emacs can display proportional fonts and fonts of different
>  >> sizes, but until you can fold (etc) proportional text (and text with
>  >> a mixture of font sizes) in a pretty manner, that's more of a toy
>  >> than anything else.)
>
>  > What's non-pretty with how we do this now?  What features are
>  > missing?
>
> 	The only feature that I’m aware to be missing is the actual
> 	support for Emacs native text wrapping (as in: the word-wrap
> 	variable and wrap-prefix text property) in SHR.
>
> 	Please thus consider the patch MIMEd.

I think this was superseded by the shr proportional font rewrite, so I'm
closing the bug.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Clément Pit--Claudel]

[-- Attachment #1: Type: text/plain, Size: 1680 bytes --]

On 12/25/2015 06:34 PM, Lars Ingebrigtsen wrote:
> Ivan Shmakov <ivan@siamics.net> writes:
> 
>>  >> (Yes, Emacs can display proportional fonts and fonts of different
>>  >> sizes, but until you can fold (etc) proportional text (and text with
>>  >> a mixture of font sizes) in a pretty manner, that's more of a toy
>>  >> than anything else.)
>>
>>  > What's non-pretty with how we do this now?  What features are
>>  > missing?
>>
>> 	The only feature that I’m aware to be missing is the actual
>> 	support for Emacs native text wrapping (as in: the word-wrap
>> 	variable and wrap-prefix text property) in SHR.
>>
>> 	Please thus consider the patch MIMEd.
> 
> I think this was superseded by the shr proportional font rewrite, so I'm
> closing the bug.

I'd love a tiny bit more of information about this :) 

In one of my packages I do some post-processing of a buffer rendered by shr from an html page to increase the font size of a particular title (this is combined with company's support for documentation buffers for completion). 

For this to work properly, I need to ensure that shr does not insert hard newlines to wrap the text; otherwise, the larger sized textx wraps in awkward ways. To do this in Emacs < 25 I set `shr-width' to `most-positive-fixnum' and preprocess the html to remove <table>s and <hr>s (which otherwise cause an out of memory exception due to the `shr-width' hack), and in Emacs >= 25 I set it to 0, which shr seems to recognize there thanks to this test in `shr-fill-lines':
  (if (<= shr-internal-width 0)
      nil

Is there another solution to use visual-line-mode with shr that I have overlooked?

Clément.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Lars Ingebrigtsen]

Clément Pit--Claudel <clement.pit@gmail.com> writes:

> In one of my packages I do some post-processing of a buffer rendered
> by shr from an html page to increase the font size of a particular
> title (this is combined with company's support for documentation
> buffers for completion).

Why not just make shr increase the size of the particular headings
before rendering?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Clément Pit--Claudel]

[-- Attachment #1: Type: text/plain, Size: 661 bytes --]

On 12/25/2015 07:55 PM, Lars Ingebrigtsen wrote:
> Clément Pit--Claudel <clement.pit@gmail.com> writes:
> 
>> In one of my packages I do some post-processing of a buffer rendered
>> by shr from an html page to increase the font size of a particular
>> title (this is combined with company's support for documentation
>> buffers for completion).
> 
> Why not just make shr increase the size of the particular headings
> before rendering?

I do not know how to do this, but it sounds like a convenient solution! How does one tell shr to apply a particular font to its target (all that I know of is `shr-target-id', which seems to insert a '*')?




[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Lars Ingebrigtsen]

Clément Pit--Claudel <clement.pit@gmail.com> writes:

> I do not know how to do this, but it sounds like a convenient
> solution! How does one tell shr to apply a particular font to its
> target (all that I know of is `shr-target-id', which seems to insert a
> '*')?

Look at `shr-tag-h1', for instance, and create whatever similar version
of that you want by overriding with `shr-external-rendering-functions'.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Clément Pit--Claudel]

[-- Attachment #1: Type: text/plain, Size: 679 bytes --]

On 12/25/2015 11:51 PM, Lars Ingebrigtsen wrote:
> Clément Pit--Claudel <clement.pit@gmail.com> writes:
> 
>> I do not know how to do this, but it sounds like a convenient
>> solution! How does one tell shr to apply a particular font to its
>> target (all that I know of is `shr-target-id', which seems to insert a
>> '*')?
> 
> Look at `shr-tag-h1', for instance, and create whatever similar version
> of that you want by overriding with `shr-external-rendering-functions'.

That sounds like a great idea, actually. And I guess I can get the id of the current element to check it against the target id using (equal (dom-attr dom 'id) TARGET-ID).

I'll try that!


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Clément Pit--Claudel]

[-- Attachment #1: Type: text/plain, Size: 1078 bytes --]

On 12/26/2015 05:53 PM, Clément Pit--Claudel wrote:
> On 12/25/2015 11:51 PM, Lars Ingebrigtsen wrote:
>> Clément Pit--Claudel <clement.pit@gmail.com> writes:
>>
>>> I do not know how to do this, but it sounds like a convenient
>>> solution! How does one tell shr to apply a particular font to its
>>> target (all that I know of is `shr-target-id', which seems to insert a
>>> '*')?
>>
>> Look at `shr-tag-h1', for instance, and create whatever similar version
>> of that you want by overriding with `shr-external-rendering-functions'.
> 
> That sounds like a great idea, actually. And I guess I can get the id of the current element to check it against the target id using (equal (dom-attr dom 'id) TARGET-ID).
> 
> I'll try that!

I just tried this (capturing the id that I was interested in in a closure instead of putting it in shr-target-id), but shr-descend only works with globally defined functions; is there a reason for that? It's due to the following bit: 

	(if (fboundp function)
	    (funcall function dom)
	  (shr-generic dom))

Clément.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Clément Pit--Claudel]

[-- Attachment #1: Type: text/plain, Size: 1137 bytes --]

On 12/27/2015 04:36 AM, Clément Pit--Claudel wrote:
> On 12/26/2015 05:53 PM, Clément Pit--Claudel wrote:
>> On 12/25/2015 11:51 PM, Lars Ingebrigtsen wrote:
>>> Clément Pit--Claudel <clement.pit@gmail.com> writes:
>>>
>>>> I do not know how to do this, but it sounds like a convenient
>>>> solution! How does one tell shr to apply a particular font to its
>>>> target (all that I know of is `shr-target-id', which seems to insert a
>>>> '*')?
>>>
>>> Look at `shr-tag-h1', for instance, and create whatever similar version
>>> of that you want by overriding with `shr-external-rendering-functions'.
>>
>> That sounds like a great idea, actually. And I guess I can get the id of the current element to check it against the target id using (equal (dom-attr dom 'id) TARGET-ID).
>>
>> I'll try that!

I looked into this more, but I don't think it will work. I want to highlight the context of the element that contains the target id, so if lines are not broken up then highlighting the full line works well. If they are, on the other hand, then I can't tell how much surrounding text to highlight.

Cheers,
Clément.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Lars Ingebrigtsen]

Clément Pit--Claudel <clement.pit@gmail.com> writes:

> I looked into this more, but I don't think it will work. I want to
> highlight the context of the element that contains the target id, so
> if lines are not broken up then highlighting the full line works
> well. If they are, on the other hand, then I can't tell how much
> surrounding text to highlight.

I'm not sure what you mean by "context".  But you can look "down" into
the DOM and see if your id is in an element there.  Or you can transform
the HTML, or transform the DOM, to wrap the elements you want to
highlight.

For instance, you can loop over all elements that have ids that match
with `dom-by-id', and you can alter the DOM to insert, say, h1 elements
around those that you want to have highlit.

Etc.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Clément Pit--Claudel]

[-- Attachment #1: Type: text/plain, Size: 1521 bytes --]

On 12/27/2015 07:22 AM, Lars Ingebrigtsen wrote:
> Clément Pit--Claudel <clement.pit@gmail.com> writes:
> 
>> I looked into this more, but I don't think it will work. I want to
>> highlight the context of the element that contains the target id, so
>> if lines are not broken up then highlighting the full line works
>> well. If they are, on the other hand, then I can't tell how much
>> surrounding text to highlight.
> 
> I'm not sure what you mean by "context".  But you can look "down" into
> the DOM and see if your id is in an element there.  Or you can transform
> the HTML, or transform the DOM, to wrap the elements you want to
> highlight.
> 
> For instance, you can loop over all elements that have ids that match
> with `dom-by-id', and you can alter the DOM to insert, say, h1 elements
> around those that you want to have highlit.

Thanks for the suggestions. It's not always obvious what element I want to highlight, though, beyond "everything that ends up on the same line. It may be that the thing I want to draw the attention of the user to is in a paragraph; in that case, I highlight the full paragrahp. Or it may be in a title; in that case, I highlight the full title. Or it may be part of a list; in that case I only highlight the item that contains it.

When the text that shr inserts isn't wrapped, I just need to highlight the line that contains that text after rendering is complete. When it is wrapped, however, it becomes unclear what exactly to highlight.

Clément.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

bug#19462: shr: use wrap-prefix when possible, instead of filling the text

[Lars Ingebrigtsen]

Clément Pit--Claudel <clement.pit@gmail.com> writes:

> I just tried this (capturing the id that I was interested in in a
> closure instead of putting it in shr-target-id), but shr-descend only
> works with globally defined functions; is there a reason for that?
> It's due to the following bit:
>
> 	(if (fboundp function)
> 	    (funcall function dom)
> 	  (shr-generic dom))

I've now fixed this in the Emacs trunk.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: HTML-Info design

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> What's non-pretty with how we do this now?  What features are missing?

We don't know (before redisplay) how wide a piece of text is, so we
can't fill the text.  This makes it impossible to use proportional fonts
in common layouts like

first column with         second column with
some flowing text here    some other text
                          here

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Ivan Shmakov]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:
>>>>> Eli Zaretskii <eliz@gnu.org> writes:

 >> What's non-pretty with how we do this now?  What features are
 >> missing?

 > We don't know (before redisplay) how wide a piece of text is, so we
 > can't fill the text.  This makes it impossible to use proportional
 > fonts in common layouts like

 > first column with         second column with
 > some flowing text here    some other text
 >                           here

	As per the current shr.el version, this only affects HTML
	documents which use tables to implement such layouts, – and
	HTML5 already discourages that [1]:

    Tables should not be used as layout aids.  Historically, many Web
    authors have tables in HTML as a way to control their page layout
    making it difficult to extract tabular data from such documents.
    […]

    There are a variety of alternatives to using HTML tables for layout,
    primarily using CSS positioning and the CSS table model.

	When the layout is implemented with CSS, the Emacs own display
	model, combined with the incomplete CSS support in SHR, ensure
	that the above is instead rendered as, say:

This is the page content proper.  Emacs may apply its usual facilities to flow it as necessary, without any trouble whatsoever.  For one thing, many MediaWiki instances use exactly this layout.

This is the sidebar, which is placed to the left of the “payload” content in the browsers implementing (a larger subset of) CSS.  Unless being tweaked by the user to his or her own taste, that is.

	Personally, I think that that’s even better.

[1] http://www.w3.org/TR/html5/tabular-data.html#the-table-element

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: HTML-Info design

[Lars Ingebrigtsen]

Ivan Shmakov <ivan@siamics.net> writes:

> 	As per the current shr.el version, this only affects HTML
> 	documents which use tables to implement such layouts, – and
> 	HTML5 already discourages that [1]:

The reality is that these layouts are very common.  I don't see any need
to discuss that point further.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Ivan Shmakov]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:
>>>>> Ivan Shmakov <ivan@siamics.net> writes:

 >> As per the current shr.el version, this only affects HTML documents
 >> which use tables to implement such layouts, – and HTML5 already
 >> discourages that [1]:

 > The reality is that these layouts are very common.  I don't see any
 > need to discuss that point further.

	Good.  Still, such layouts are a nuisance to deal with in EWW
	(or with any other code that relies on SHR, for that matter.)
	Consider, for instance, the following:

    This is the sidebar, which          This is the page content proper.
    is placed to the left of the        Emacs may apply its usual
    “payload” content in the            facilities to flow it as
    browsers implementing (a            necessary, without any trouble
    larger subset of) CSS.              whatsoever.  For one thing, many
    Unless being tweaked by the         MediaWiki instances use exactly
    user to his or her own              this layout.
    taste, that is.

	Now, what’s the easy way to put the second sentence of the right
	column into the kill ring?

	My idea is that EWW should provide a way for the user to ignore
	the “layout” tables, – either applying some heuristics (there’re
	some ideas on that in [1], BTW), or via an explicit user command
	(not dissimilar to eww-readable, I guess, – I do not seem to
	understand what the latter is intended to do.)  Or both.

	Sure, we may instead try to improve the Emacs display model to
	support such layouts natively, but, well, – I won’t believe
	/that/ until I see actual patches to that end.

 >> [1] http://www.w3.org/TR/html5/tabular-data.html#the-table-element

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: HTML-Info design

[Lars Ingebrigtsen]

Ivan Shmakov <ivan@siamics.net> writes:

> 	Now, what’s the easy way to put the second sentence of the right
> 	column into the kill ring?

That is, indeed, a problem.  I usually kill the rectangle, paste it into
a different buffer, and extract the sentence there.  Which kinda sucks.

However, it would be easy enough to fix this for `M-w' and friends with
some text properties that say where the continuation of a line is.  The
greater problem would be the user interface -- sometimes somebody wants
to yank what's displayed, and sometimes the "logical flow".

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Lennart Borgman]

On Mon, Dec 29, 2014 at 1:36 PM, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> Ivan Shmakov <ivan@siamics.net> writes:
>
>>       Now, what’s the easy way to put the second sentence of the right
>>       column into the kill ring?
>
> That is, indeed, a problem.  I usually kill the rectangle, paste it into
> a different buffer, and extract the sentence there.  Which kinda sucks.
>
> However, it would be easy enough to fix this for `M-w' and friends with
> some text properties that say where the continuation of a line is.  The
> greater problem would be the user interface -- sometimes somebody wants
> to yank what's displayed,

Rectangle copy?

> and sometimes the "logical flow".

Normal copy.

Re: HTML-Info design

[Ivan Shmakov]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:
>>>>> Ivan Shmakov <ivan@siamics.net> writes:

 >> Now, what’s the easy way to put the second sentence of the right
 >> column into the kill ring?

 > That is, indeed, a problem.  I usually kill the rectangle, paste it
 > into a different buffer, and extract the sentence there.  Which kinda
 > sucks.

 > However, it would be easy enough to fix this for ‘M-w’ and friends
 > with some text properties that say where the continuation of a line
 > is.

	Either that or we could use a text property that’d point back to
	the “relevant” DOM element (as in: subtree.)

	Given that SHR only does columns inside <table />, – using the
	closest <td /> or <th /> ancestor for the purpose would solve
	the issue.

 > The greater problem would be the user interface -- sometimes somebody
 > wants to yank what's displayed, and sometimes the "logical flow".

	With a property, we could just provide a command to display the
	relevant element in a separate buffer.

	However, given that using tables for layout tends to create
	accessibility issues out of nothing (did I hear someone saying
	Emacspeak in this discussion?), why exactly can’t we provide the
	user a way to get rid of all the “layout” tables at once?

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: HTML-Info design

[Lars Ingebrigtsen]

Ivan Shmakov <ivan@siamics.net> writes:

> 	Given that SHR only does columns inside <table />, – using the
> 	closest <td /> or <th /> ancestor for the purpose would solve
> 	the issue.

At some point, eww will also support CSS layouts.

> 	However, given that using tables for layout tends to create
> 	accessibility issues out of nothing (did I hear someone saying
> 	Emacspeak in this discussion?), why exactly can’t we provide
> the
> 	user a way to get rid of all the “layout” tables at once?

The layouts sometimes have semantic meaning, and there's no easy way to
separate a "layout table" from a "non-layout table".  I just got a
ticket confirmation email, for instance, that had a lot of "bla bla",
but would have been completely incomprehensible if the things that
needed to be lined up hadn't been lined up.

People use layouts because they use layouts, and eww should strive,
within what's practical given Emacs' layout engine, to replicate those
layouts.  Having a "readability" command to make the layout go away can
also be nice in some circumstances, but that should remain a user
command.  As it is now.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Lennart Borgman]

On Mon, Dec 29, 2014 at 2:56 PM, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> Ivan Shmakov <ivan@siamics.net> writes:
>
>>       Given that SHR only does columns inside <table />, – using the
>>       closest <td /> or <th /> ancestor for the purpose would solve
>>       the issue.
>
> At some point, eww will also support CSS layouts.

CSS layout is a moving target. The flexbox model and the column layout
are part of the newer thinking and IMO better than the old <table>
layout (for text, of course, not for tables... ;-) ).

Re: HTML-Info design

[Lars Ingebrigtsen]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> CSS layout is a moving target.

The world is ever changing.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Ivan Shmakov]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:
>>>>> Ivan Shmakov <ivan@siamics.net> writes:

 >> Given that SHR only does columns inside <table />, – using the
 >> closest <td /> or <th /> ancestor for the purpose would solve the
 >> issue.

 > At some point, eww will also support CSS layouts.

	If it will also fully support CSS cascading at the same time, –
	I’m all in favor of that.  I already employ some local CSS
	overrides with Iceweasel, and I’d be just happy to offer my .css
	files for Emacs to use.

[…]

 > Having a "readability" command to make the layout go away can also be
 > nice in some circumstances, but that should remain a user command.
 > As it is now.

	My point is: it didn’t work for me.

	I guess I should investigate it further, but at the first
	glance, it uses some ad-hoc scoring model, /and/ it doesn’t
	provide an easy way for the user to influence it in any way.

	… Well, that’s enough to make me highly suspicious of if it
	could work at all outside of the cases it was specifically
	tested on.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: HTML-Info design

[Eli Zaretskii]

> From: Ivan Shmakov <ivan@siamics.net>
> Date: Mon, 29 Dec 2014 13:45:55 +0000
> 
> >>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:
> >>>>> Ivan Shmakov <ivan@siamics.net> writes:
> 
>  >> Now, what’s the easy way to put the second sentence of the right
>  >> column into the kill ring?
> 
>  > That is, indeed, a problem.  I usually kill the rectangle, paste it
>  > into a different buffer, and extract the sentence there.  Which kinda
>  > sucks.
> 
>  > However, it would be easy enough to fix this for ‘M-w’ and friends
>  > with some text properties that say where the continuation of a line
>  > is.
> 
> 	Either that or we could use a text property that’d point back to
> 	the “relevant” DOM element (as in: subtree.)

I think such a command should work independently of shr or eww or DOM
or anything else.  Or at least that's the ideal we should strive for.

Re: HTML-Info design

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> I think such a command should work independently of shr or eww or DOM
> or anything else.  Or at least that's the ideal we should strive for.

Yeah, totally.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Stefan Monnier]

>> >> Now, what’s the easy way to put the second sentence of the right
>> >> column into the kill ring?

I think this is beyond the scope of HTML-Info's design.


        Stefan

Re: HTML-Info design

[Ivan Shmakov]

>>>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:

 >>>>> Now, what’s the easy way to put the second sentence of the right
 >>>>> column into the kill ring?

 > I think this is beyond the scope of HTML-Info's design.

	My point here is that /if/ HTML-Info is /not/ supposed to be
	rich in tables (and in particular, – rich on tables used for
	purely presentation purposes, rather than for holding tabular
	data proper), – my patch for #19462 may happen to be a nice
	starting point for experiments with EWW-Info support for
	proportional fonts.

	However, I always build my Emacs with a tty-only interface, and
	thus cannot really test proportional fonts with it.  (But on a
	tty, – word-wrap paragraphs work just perfectly, IMO.)

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: HTML-Info design

[Stefan Monnier]

> 	data proper), – my patch for #19462 may happen to be a nice
> 	starting point for experiments with EWW-Info support for
> 	proportional fonts.

Yes, that's a completely separate issue from the issue of handling
sentences in multi-column tables.


        Stefan

Re: HTML-Info design

[Ivan Shmakov]

>>>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:

 >> my patch for #19462 may happen to be a nice starting point for
 >> experiments with EWW-Info support for proportional fonts.

 > Yes, that's a completely separate issue from the issue of handling
 > sentences in multi-column tables.

	Not quite.  Multi-column tables in SHR generally require line
	truncation, a priorly decided rendering width, and (as of the
	current shr.el implementation) a fixed-width font.

	On the contrary, using proportional fonts – at least the way I
	suggest it (that is: via the word-wrap and wrap-prefix
	facilities) – do not allow (assuming the current Emacs display
	implementation) for line truncation at the least.  (Yet it does
	allow for a dynamic window width, which I believe a nice feature
	to have anyway.)

	Naturally, it still should just work when all the tables fit the
	window.  (Though, of course, the tables still need a fixed-width
	font, while the rest of the SHR-rendered contents may use a
	proportional one.)

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: HTML-Info design

[Eli Zaretskii]

> From: Ivan Shmakov <ivan@siamics.net>
> Date: Mon, 29 Dec 2014 12:20:26 +0000
> 
>     This is the sidebar, which          This is the page content proper.
>     is placed to the left of the        Emacs may apply its usual
>     “payload” content in the            facilities to flow it as
>     browsers implementing (a            necessary, without any trouble
>     larger subset of) CSS.              whatsoever.  For one thing, many
>     Unless being tweaked by the         MediaWiki instances use exactly
>     user to his or her own              this layout.
>     taste, that is.
> 
> 	Now, what’s the easy way to put the second sentence of the right
> 	column into the kill ring?

Would you like to code such a command?

shr: tables

[Ivan Shmakov]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

 >> From: Ivan Shmakov  Date: Mon, 29 Dec 2014 12:20:26 +0000

 >> This is the sidebar, which          This is the page content proper.
 >> is placed to the left of the        Emacs may apply its usual
 >> “payload” content in the            facilities to flow it as
 >> browsers implementing (a            necessary, without any trouble
 >> larger subset of) CSS.              whatsoever.  For one thing, many
 >> Unless being tweaked by the         MediaWiki instances use exactly
 >> user to his or her own              this layout.
 >> taste, that is.

 >> Now, what’s the easy way to put the second sentence of the right
 >> column into the kill ring?

 > Would you like to code such a command?

	I have a somewhat different view on this issue.  I believe that
	there should be a command to render the above as two paragraphs
	instead, one under the other:

This is the page content proper.  Emacs may apply its usual facilities
to flow it as necessary, without any trouble whatsoever.  For one thing,
many MediaWiki instances use exactly this layout.

This is the sidebar, which is placed to the left of the “payload”
content in the browsers implementing (a larger subset of) CSS.  Unless
being tweaked by the user to his or her own taste, that is.

	After such a command is applied, – the issue above simply does
	not arise, and there’s no need for a separate command to put
	whatever portion of either of these paragraphs into a kill ring.

	The details for such a command are yet to be worked out; I have
	no solid idea on which cases it should handle and how.  (Should
	it, for instance, affect all tables, some tables, just the table
	at point, or tables matching some entirely different criteria.)

	(And just in case, – there /is/ a way to do just that with the
	“big” browsers, – even though by no means necessary,
	clipboard-wise.  Well, with Iceweasel at the least.)

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: HTML-Info design

[Yuri Khan]

On Mon, Dec 29, 2014 at 6:20 PM, Ivan Shmakov <ivan@siamics.net> wrote:

>     This is the sidebar, which          This is the page content proper.
>     is placed to the left of the        Emacs may apply its usual
>     “payload” content in the            facilities to flow it as
>     browsers implementing (a            necessary, without any trouble
>     larger subset of) CSS.              whatsoever.  For one thing, many
>     Unless being tweaked by the         MediaWiki instances use exactly
>     user to his or her own              this layout.
>     taste, that is.

>         Now, what’s the easy way to put the second sentence of the right
>         column into the kill ring?

Depends.

Currently, Emacs supports right-to-left scripts. It provides a set of
commands that move the point in logical order
({forward|backward}-{char|word|symbol|line|sentence|paragraph|page})
and a separate set of commands that move in visual order
({left|right}-{char|word}, {previous|next}-line).

Let’s assume Emacs will advance to be able to display multicolumn
layouts while keeping these concepts of logical and visual order
separate.

Assuming best current practices for HTML, the main column precedes the
sidebar in the document order. Therefore, M-< moves to the T in “This
is the page content”. Then, “forward-sentence” moves the point after
the period. After that, you set the mark and use “forward-sentence” to
move point after “whatsoever.”. Now “copy-region-as-kill” puts the
sentence on the kill ring.

At the same time, moving with arrow keys would give the user intuitive
visual-order movement, where pressing <left> while the point is before
E in “Emacs may apply” moves the point all the way forward to after
“left of the”.

Re: HTML-Info design

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: stephen@xemacs.org,  emacs-devel@gnu.org,  monnier@iro.umontreal.ca,  nferrier@ferrier.me.uk,  rms@gnu.org
> Date: Mon, 29 Dec 2014 11:41:59 +0100
> 
> > What's non-pretty with how we do this now?  What features are missing?
> 
> We don't know (before redisplay) how wide a piece of text is, so we
> can't fill the text.  This makes it impossible to use proportional fonts
> in common layouts like
> 
> first column with         second column with
> some flowing text here    some other text
>                           here

Not sure why you say "impossible".  First, to align the right-hand
pane of the text you could use '(space :align-to HPOS)' display spec
after each line of the text on the left-hand pane.

Granted, that might waste some screen real estate, since you would
need to be conservative about the amount of text you put in the
left-hand pane, to avoid overrunning the right-hand pane.  So a
capability to know the display width in advance is indeed desirable.

Fortunately, such a capability already exists, I think: see the
function 'font-get-glyphs'.  Does it solve your problem?  If not, what
API would you like to have?

In any case, I don't think Info manuals use such a layout, at least
not yet, so it's not a show-stopper.

Re: HTML-Info design

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Fortunately, such a capability already exists, I think: see the
> function 'font-get-glyphs'.  Does it solve your problem?  If not, what
> API would you like to have?

I've just looked at the doc string of that function briefly, and I'm not
sure how I would use that to do filling.  I need to know the width a
text will take in the buffer, so that I know when to break the line and
start a new one.  Is it now possible to write a function like
`pixel-region-width' that would say how much space the text will occupy?

Given that the font used for that text is variable-width (and the region
possibly uses many fonts).  If that's possible, I'm all for moving eww
to use a proportional font by default.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Yuri Khan]

On Mon, Dec 29, 2014 at 10:32 PM, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
>
>> Fortunately, such a capability already exists, I think: see the
>> function 'font-get-glyphs'.  Does it solve your problem?  If not, what
>> API would you like to have?
>
> I've just looked at the doc string of that function briefly, and I'm not
> sure how I would use that to do filling.  I need to know the width a
> text will take in the buffer, so that I know when to break the line and
> start a new one.  Is it now possible to write a function like
> `pixel-region-width' that would say how much space the text will occupy?

Naively, one might keep track of the remaining part of the paragraph
being filled (initially the whole paragraph) and the remaining width
in the current line (initially the whole column width).

1: Ask for glyphs for the remaining part of paragraph.
2: Find the last possible line breaking point such that the sum of
glyph widths up to it is less than or equal to the remaining width.
3: Revalidate it by asking for glyphs for this part of paragraph. (In
complex scripts, breaking the line might switch to a different
presentation form.) If the sum of glyph widths is now greater than the
remaining width, backtrack to the preceding possible line breaking
point if it exists.
4: Layout the line, update the remaining part of paragraph, reset the
remaining width of line, and repeat from step 1 if the remaining part
of paragraph is not empty.

(This intentionally omits almost all complexities related to RTL
languages, and assumes that “font-get-glyphs” is smart enough to
return context-sensitive widths in complex scripts and to handle
combining diacritics. It also assumes the existence of a predicate
that tells if it a line break is possible after a certain position in
a string. For many Western languages, that returns t after whitespace
characters except for the non-breaking space. For CJK languages, it
involves some context-specific logic. The German language might demand
hyphenation logic roughly at the same time.)

(Also, the Pango text rendering library implements this and probably much more.)

Re: HTML-Info design

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Mon, 29 Dec 2014 17:32:52 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Fortunately, such a capability already exists, I think: see the
> > function 'font-get-glyphs'.  Does it solve your problem?  If not, what
> > API would you like to have?
> 
> I've just looked at the doc string of that function briefly, and I'm not
> sure how I would use that to do filling.  I need to know the width a
> text will take in the buffer, so that I know when to break the line and
> start a new one.  Is it now possible to write a function like
> `pixel-region-width' that would say how much space the text will occupy?

Either make a string of the text you want to display, or insert that
text in a temporary buffer, then use this function to find the width
of that text by summing the widths of all the glyphs.  Find the
largest substring whose glyphs' summary width fits into the portion of
the window width you allotted to the left pane, and insert that
substring.  Loop around.

Should work, no?

> Given that the font used for that text is variable-width (and the region
> possibly uses many fonts).

The function returns a vector of glyphs, where each glyph is described
using its attributes, including its pixel width.

You have font-at function to tell you which font is used to display
what buffer/string position.

OK?

Re: HTML-Info design

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> I've just looked at the doc string of that function briefly, and I'm not
>> sure how I would use that to do filling.  I need to know the width a
>> text will take in the buffer, so that I know when to break the line and
>> start a new one.  Is it now possible to write a function like
>> `pixel-region-width' that would say how much space the text will occupy?
>
> Either make a string of the text you want to display, or insert that
> text in a temporary buffer, then use this function to find the width
> of that text by summing the widths of all the glyphs.

I haven't played much with fonts before, so I'm afraid I don't know how
to do this.  I've played around with buffer-setting font code for half
an hour, and I'm still not able to get to a font object from text in a
buffer.

So, starting with (say)

(with-temp-buffer
  (buffer-face-set '(:family "Symbola" :height 150 :width semi-condensed))
  (insert "This is a text"))

how do I get the font object for each character there so that I can feed
it to `font-get-glyphs'?

`buffer-face-set' is probably the wrong thing here, though, since we
would have many fonts in a buffer, but it's a start, perhaps...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Sun, 25 Jan 2015 11:01:43 +1100
> 
> >> I've just looked at the doc string of that function briefly, and I'm not
> >> sure how I would use that to do filling.  I need to know the width a
> >> text will take in the buffer, so that I know when to break the line and
> >> start a new one.  Is it now possible to write a function like
> >> `pixel-region-width' that would say how much space the text will occupy?
> >
> > Either make a string of the text you want to display, or insert that
> > text in a temporary buffer, then use this function to find the width
> > of that text by summing the widths of all the glyphs.
> 
> I haven't played much with fonts before, so I'm afraid I don't know how
> to do this.  I've played around with buffer-setting font code for half
> an hour, and I'm still not able to get to a font object from text in a
> buffer.
> 
> So, starting with (say)
> 
> (with-temp-buffer
>   (buffer-face-set '(:family "Symbola" :height 150 :width semi-condensed))
>   (insert "This is a text"))
> 
> how do I get the font object for each character there so that I can feed
> it to `font-get-glyphs'?

The function 'font-at' will give you the font object you can feed to
'font-get-glyphs'.

Does this solve your problem?

Re: HTML-Info design

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> The function 'font-at' will give you the font object you can feed to
> 'font-get-glyphs'.

Thanks.  I hacked up this function:

(defun pixel-region-width (start end)
  (let ((width 0))
    (while (< start end)
      (let ((glyphs (font-get-glyphs (font-at start) start start)))
	(setq width (+ width (aref (car glyphs) 4))))
      (setq start (1+ start)))
    width))

But it fails because `font-get-glyphs' always returns nil when I try it.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Mon, 26 Jan 2015 13:16:43 +1100
> 
> (defun pixel-region-width (start end)
>   (let ((width 0))
>     (while (< start end)
>       (let ((glyphs (font-get-glyphs (font-at start) start start)))
> 	(setq width (+ width (aref (car glyphs) 4))))
>       (setq start (1+ start)))
>     width))
> 
> But it fails because `font-get-glyphs' always returns nil when I try it.

Try this instead:

(defun pixel-region-width (start end)
  (let ((width 0))
    (while (< start end)
      (let ((glyphs (font-get-glyphs (font-at start) start (1+ start))))
	(setq width (+ width (aref (car glyphs) 4)))) ;;   ^^^^^^^^^^
      (setq start (1+ start)))
    width))

Re: HTML-Info design

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Try this instead:
>
> (defun pixel-region-width (start end)
>   (let ((width 0))
>     (while (< start end)
>       (let ((glyphs (font-get-glyphs (font-at start) start (1+ start))))
> 	(setq width (+ width (aref (car glyphs) 4)))) ;;   ^^^^^^^^^^

D'oh.

Thanks for the help.  :-) I'll start experimenting with making shr using
proportional fonts for the text and see how far I get.  A new branch
will probably appear in the Emacs repo tomorrowish...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Lars Ingebrigtsen]

`font-at' doesn't seem to like to be called in buffers that aren't
displayed in a window:

  (with-temp-buffer
    (insert (propertize "foo" 'face 'variable-pitch))
    (font-at (point-min)))

=>

Debugger entered--Lisp error: (error "Specified window is not displaying the current buffer")
  font-at(1)
  (progn (insert "foo") (font-at (point-min)))
  (unwind-protect (progn (insert "foo") (font-at (point-min))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))
  (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert "foo") (font-at (point-min))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))

shr uses temporary buffers to format various bits, so this doesn't seem
to be a practical approach when determining the pixel widths of text.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Tue, 27 Jan 2015 12:26:48 +1100
> 
> `font-at' doesn't seem to like to be called in buffers that aren't
> displayed in a window:
> 
>   (with-temp-buffer
>     (insert (propertize "foo" 'face 'variable-pitch))
>     (font-at (point-min)))
> 
> =>
> 
> Debugger entered--Lisp error: (error "Specified window is not displaying the current buffer")

Yes, you need to arrange for the buffer to be current and displayed in
the window you pass to the function.  See the function's doc string.

> shr uses temporary buffers to format various bits, so this doesn't seem
> to be a practical approach when determining the pixel widths of text.

You can invoke font-at on a string instead of buffer.

Re: HTML-Info design

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> Debugger entered--Lisp error: (error "Specified window is not displaying the current buffer")
>
> Yes, you need to arrange for the buffer to be current and displayed in
> the window you pass to the function.  See the function's doc string.
>
>> shr uses temporary buffers to format various bits, so this doesn't seem
>> to be a practical approach when determining the pixel widths of text.
>
> You can invoke font-at on a string instead of buffer.

Thanks.  Does that make sense, though?  I'm guessing that using
`font-at' on a string, it implicitly uses the current window for
something.  `font-at' on a buffer could use the same logic, couldn't it?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Wed, 28 Jan 2015 13:27:32 +1100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Debugger entered--Lisp error: (error "Specified window is not displaying the current buffer")
> >
> > Yes, you need to arrange for the buffer to be current and displayed in
> > the window you pass to the function.  See the function's doc string.
> >
> >> shr uses temporary buffers to format various bits, so this doesn't seem
> >> to be a practical approach when determining the pixel widths of text.
> >
> > You can invoke font-at on a string instead of buffer.
> 
> Thanks.  Does that make sense, though?  I'm guessing that using
> `font-at' on a string, it implicitly uses the current window for
> something.  `font-at' on a buffer could use the same logic, couldn't it?

No, it couldn't use the same logic.  But it could use something
similar, if Someone(TM) cared to write code to do that.

As things stand, the function uses existing infrastructure, which
imposes this limitation.

Pixel-based display functions (was: HTML-Info design)

[Lars Ingebrigtsen]

I've now made a proof-of-concept implementation of eww using
proportional fonts.  Here are some screen shots:

http://lars.ingebrigtsen.no/2015/01/28/eww-can-haz-font/

But it's so slow that it's not practical to use.

shr needs three functions to be kind-of fast to work:

1) A pixel equivalent of `current-column'

2) A pixel equivalent of `move-to-column' (which will be imprecise, of
course, but that's OK)

3) A pixel equivalent of `string-width'

If somebody could whip these up, then Emacs would have a web browser
that kinda almost looks like a real one.  :-)

I've pushed the shr changes to the "shr-fontified" branch, and it would
perhaps be practical if whoever volunteers to implement this also puts
these new functions there, and we can merge them all to trunk once
things work satisfactorily.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 460 bytes --]

() Lars Ingebrigtsen <larsi@gnus.org>
() Wed, 28 Jan 2015 14:27:17 +1100

   1) A pixel equivalent of `current-column'

See also mid-2002 thread "column->int float diff".
Gist: rms suggests using (C language) ‘iterator’.

-- 
Thien-Thi Nguyen
   GPG key: 4C807502
   (if you're human and you know it)
      read my lisp: (responsep (questions 'technical)
                               (not (via 'mailing-list)))
                     => nil

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Thien-Thi Nguyen <ttn@gnu.org>
> Date: Wed, 28 Jan 2015 08:16:46 +0100
> 
>    1) A pixel equivalent of `current-column'

As I wrote elsewhere, these already exist.

> See also mid-2002 thread "column->int float diff".
> Gist: rms suggests using (C language) ‘iterator’.

Indeed.  So I think the "float column" approach is not TRT, and we
should instead work in pixels.  That is also how the 'iterator' object
works, so this will naturally plug into the existing infrastructure.
Using a float column solves only half of the problem.  The other half,
how to convert such a column into the actual screen position, is still
there.

The indentation code should use the '(space . (PIXELS))' value of
display property to align text on pixel-granular boundary, instead of
inserting TABs and spaces.  This already works, so does not need any
changes on the C level.

We do need to teach the display engine how to fill and/or justify text
with pixel granularity.  But that's a separate isue.

I asked for someone to show a list of tasks included in this, and no
one did.  I think different people have different ideas about what
needs to be done and how, and also about what's already available.  So
preparing such an agreed-upon list is IMO necessary for any
constructive discussions and advances.

Re: Pixel-based display functions

[Ivan Shmakov]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:

[…]

 > 1) A pixel equivalent of `current-column'

 > 2) A pixel equivalent of `move-to-column' (which will be imprecise,
 > of course, but that's OK)

	JFTR, – for the indentation purposes, the line-prefix property
	alone should already suffice; say:

   (put-text-property from to 'line-prefix
                      `(space :align-to (,indentation-in-pixels)))

	(Relying on the Emacs’ own display engine for wrapping /and/
	indentation in SHR is the essence of my earlier patch [1].)

[1] news:8761cubx18.fsf@violet.siamics.net
    http://debbugs.gnu.org/19462#10

[…]

-- 
FSF associate member #7257  np. Cloud of Unknowing — Steffen Nicolaisen

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Ivan Shmakov <ivan@siamics.net> writes:

> 	JFTR, – for the indentation purposes, the line-prefix property
> 	alone should already suffice; say:

Well, this has nothing to do with indentation, so I'm not sure how
that's relevant.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Stefan Monnier]

> 	(Relying on the Emacs’ own display engine for wrapping /and/
> 	indentation in SHR is the essence of my earlier patch [1].)

I tend to think that this is a more promising direction, at least for
the short term.  Computing the pixel sizes from Elisp (instead of during
redisplay) sounds like a recipe for slowness (not only because Elisp is
slow but also because it's harder to make it lazy (i.e. not bother
doing the work as long as it's not displayed)).

Of course, the current display engine can't support filling text in
multiple columns currently, and extending it in this direction seems far
from trivial.
But I'm not sure how important this is at this stage.

OTOH, it'd be good to provide better support (in the redisplay code) for
column display as used in things like tabulated-list-mode
(e.g. truncating the text of one column when it would overflow into the
next, for example, or providing right alignment of text in a column).


        Stefan

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Wed, 28 Jan 2015 14:20:44 -0500
> Cc: emacs-devel@gnu.org
> 
> Of course, the current display engine can't support filling text in
> multiple columns currently, and extending it in this direction seems far
> from trivial.
> But I'm not sure how important this is at this stage.

It's important.  I get quite a few HTML emails that use 2 and
sometimes even 3 columns (and shr lays them out beautifully with
fixed-pitch fonts, thank you very much).

> OTOH, it'd be good to provide better support (in the redisplay code) for
> column display as used in things like tabulated-list-mode
> (e.g. truncating the text of one column when it would overflow into the
> next, for example, or providing right alignment of text in a column).

I agree.  We just need a volunteer.

Re: Pixel-based display functions

[Stefan Monnier]

> It's important.  I get quite a few HTML emails that use 2 and
> sometimes even 3 columns (and shr lays them out beautifully with
> fixed-pitch fonts, thank you very much).

Right, and I think we can keep living with fixed-pitch fonts for such
HTML pages.  I do want variable-pitched fonts when I browse the manual,
but this one does not require multiple filled columns.


        Stefan

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> It's important.  I get quite a few HTML emails that use 2 and
>> sometimes even 3 columns (and shr lays them out beautifully with
>> fixed-pitch fonts, thank you very much).
>
> Right, and I think we can keep living with fixed-pitch fonts for such
> HTML pages.  I do want variable-pitched fonts when I browse the manual,
> but this one does not require multiple filled columns.

True.

However, having the three functions I described (that work "fast
enough"; I've already written versions of them that aren't fast enough)
is necessary for multi-column layout.  And in addition, it would also
fix the "variable pitch in the manual" problem.

So two birds with one mechanism, instead of having to invent one
mechanism for single-column layouts, and then use a totally different
one for multi-column layouts.

(And in a web context, mixing the two mechanisms sounds somewhat
awkward.  And *a lot* of web pages use multi-column layouts.)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Stefan Monnier]

> However, having the three functions I described (that work "fast
> enough"; I've already written versions of them that aren't fast enough)
> is necessary for multi-column layout.  And in addition, it would also
> fix the "variable pitch in the manual" problem.

Coding it in C can make it faster, but you still have the problem of
laziness (which is a general problem for shr, by the way): how to only
render the displayed part of the HTML document, which is crucial to get
reasonable speed on non-small documents.

> So two birds with one mechanism, instead of having to invent one
> mechanism for single-column layouts,

That's the only I really care about, and I think there's much more of
a chance that we can actually make it work reliably with good speed.

If we do it using the "general" approach in Elisp, I'm afraid we won't
make it work well enough within the foreseeable future (e.g. remember:
one of the features of current info-mode is that it's very quick, so
a replacement that introduces a 1s rendering delay whenever we switch to
another page won't be good enough).


        Stefan

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> Coding it in C can make it faster, but you still have the problem of
> laziness (which is a general problem for shr, by the way): how to only
> render the displayed part of the HTML document, which is crucial to get
> reasonable speed on non-small documents.

Well, large pages with simple layouts (i.e., single columns) render with
linear speed and aren't generally a major problem.

> That's the only I really care about, and I think there's much more of
> a chance that we can actually make it work reliably with good speed.
>
> If we do it using the "general" approach in Elisp, I'm afraid we won't
> make it work well enough within the foreseeable future (e.g. remember:
> one of the features of current info-mode is that it's very quick, so
> a replacement that introduces a 1s rendering delay whenever we switch to
> another page won't be good enough).

I see no reason why rendering a typical info node should take anything
approaching one second.  I mean, Emacs can display text fast, and shr
can lay out stuff fast.  If we had access to the pixel widths in advance
of redisplay.

(eww uses 0.04 seconds to display a typical Emacs manual node with a
fixed-width layout on this laptop, according to `benchmark-run'.)

I know nothing about how redisplay works, but would (conceptually)
saying to Emacs "hey, render this line" (generally very fast) "but don't
display it" (should be instantaneous) "and tell me how long that was"
(ditto) make sense?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Lars Ingebrigtsen]

An alternate approach to doing the folding just occurred to me using
`font-get-glyphs' just occurred to me that may be fast enough for shr,
and I won't need the three functions mentioned.

I'll do some exploratory hacking and see how that goes...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I'll do some exploratory hacking and see how that goes...

I've gotten rendering a typical variable-pitch manual page down to
0.04s, but rendering multi-column layouts is still too slow to be used
on realistic pages.

`font-get-glyphs' on a large region is quite fast, but since the font
may vary, I'm calling it on a char-by-char basis, and that's s-l-o-w.
Hm...  `font-at' is fast, too, so perhaps I can get something working by
collating regions or something.

`font-get-glyphs' returns an awful lot of data, and virtually all of it
isn't used for doing this sort of filling, so I have a suspicion that
I'm going to end up writing a new C-level function that just returns
glyph widths fast.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Fri, 30 Jan 2015 17:37:00 +1100
> 
> `font-get-glyphs' on a large region is quite fast, but since the font
> may vary, I'm calling it on a char-by-char basis, and that's s-l-o-w.

That's your problem: you should call it on a run of characters that
have the same face.  Use next-single-char-property-change to find
where such runs begin and end.

> `font-get-glyphs' returns an awful lot of data, and virtually all of it
> isn't used for doing this sort of filling, so I have a suspicion that
> I'm going to end up writing a new C-level function that just returns
> glyph widths fast.

That'd be a waste of effort, IMO.  Just do the above, and I think the
speed will be acceptable.

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Lars Ingebrigtsen <larsi@gnus.org>
>> Date: Fri, 30 Jan 2015 17:37:00 +1100
>> 
>> `font-get-glyphs' on a large region is quite fast, but since the font
>> may vary, I'm calling it on a char-by-char basis, and that's s-l-o-w.
>
> That's your problem: you should call it on a run of characters that
> have the same face.  Use next-single-char-property-change to find
> where such runs begin and end.

The face doesn't say what font is going to be used.  If I put 日本語
here, the entire preceding line has the same face, but two different
fonts.

I've done further experimentation, and it turns out my previous analysis
was kinda wrong.  `font-get-glyphs' isn't really the problem --
`font-at' is.  Calling it once per character is just way too slow.

If you have a handy function that can segment lines according to font
(even though it can't say what the font is (yet)), that would be useful.
If I could (on the first line there) just call `font-at' twice, I think
that would allow me to write an algorithm that should be fast enough.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Fri, 30 Jan 2015 18:27:23 +1100
> 
> >> `font-get-glyphs' on a large region is quite fast, but since the font
> >> may vary, I'm calling it on a char-by-char basis, and that's s-l-o-w.
> >
> > That's your problem: you should call it on a run of characters that
> > have the same face.  Use next-single-char-property-change to find
> > where such runs begin and end.
> 
> The face doesn't say what font is going to be used.  If I put 日本語
> here, the entire preceding line has the same face, but two different
> fonts.

Then check the 'charset' text property as well.

> If you have a handy function that can segment lines according to font
> (even though it can't say what the font is (yet)), that would be useful.
> If I could (on the first line there) just call `font-at' twice, I think
> that would allow me to write an algorithm that should be fast enough.

I think face and charset are what you want.

Re: Pixel-based display functions

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> The face doesn't say what font is going to be used.  If I put 日本語
>> here, the entire preceding line has the same face, but two different
>> fonts.
>
> Then check the 'charset' text property as well.

That isn't guaranteed to exist.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: Lars Ingebrigtsen <larsi@gnus.org>,  emacs-devel@gnu.org
> Date: Fri, 30 Jan 2015 11:20:24 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> The face doesn't say what font is going to be used.  If I put 日本語
> >> here, the entire preceding line has the same face, but two different
> >> fonts.
> >
> > Then check the 'charset' text property as well.
> 
> That isn't guaranteed to exist.

Yes, I know.  But it should be a stopgap at least, and will allow Lars
to develop the code, even if it will not be perfect for now.

I will look in the display engine for how this is done, perhaps
there's some trick that Lars could adopt.  Or maybe a new primitive is
in order.

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> The face doesn't say what font is going to be used.  If I put 日本語
>> here, the entire preceding line has the same face, but two different
>> fonts.
>
> Then check the 'charset' text property as well.

(get-text-property (point) 'charset)
=> nil

on that entire line.  So I'm not sure what you're referring to.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Fri, 30 Jan 2015 22:28:40 +1100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> The face doesn't say what font is going to be used.  If I put 日本語
> >> here, the entire preceding line has the same face, but two different
> >> fonts.
> >
> > Then check the 'charset' text property as well.
> 
> (get-text-property (point) 'charset)
> => nil
> 
> on that entire line.

Not here, it isn't:

  (get-text-property (point) 'charset) => japanese-jisx0208

on the Kanji characters, and

  (get-text-property (point) 'charset) => nil

on ASCII.

Maybe it depends on how your MUA decodes non-ASCII text.  (I use
Rmail, as you know.)

But I know that the 'face' text property is not enough.  I just
suggest that you use it for now to see if you get enough of speedup in
your code.  We can later find a way of detecting character sets, and
providing new primitives for that, if needed.  (In the display engine,
there's a thing called the "face ID" that is computed as part of
displaying text, which takes the characters into consideration, which
is why I suggested 'face' in the first place.  I will try to see how
to do the same in Lisp.)

Re: Pixel-based display functions

[Eli Zaretskii]

> Date: Fri, 30 Jan 2015 13:56:21 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> (In the display engine, there's a thing called the "face ID" that is
> computed as part of displaying text, which takes the characters into
> consideration, which is why I suggested 'face' in the first place.
> I will try to see how to do the same in Lisp.)

Not really the same, but how about using

  (aref char-script-table (char-after POS))

to find where in the text you have characters from a different script?
If this is fast enough, it should give you a conservative estimation
of where to call font-at again.  (It's conservative because the same
font can, and usually does, cover more than a single script.)

One more issue that I think needs to be handled are characters for
which there's no font installed on the user's system.  We display them
as defined by glyphless-char-display-control.  Or maybe you should
allow the text to overflow in this case: after all, it's not really
legible, is it?

Re: Pixel-based display functions

[Stefan Monnier]

Why are we re-implementing the display code in Elisp, again?


        Stefan


>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Fri, 30 Jan 2015 13:56:21 +0200
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: emacs-devel@gnu.org
>> 
>> (In the display engine, there's a thing called the "face ID" that is
>> computed as part of displaying text, which takes the characters into
>> consideration, which is why I suggested 'face' in the first place.
>> I will try to see how to do the same in Lisp.)

> Not really the same, but how about using

>   (aref char-script-table (char-after POS))

> to find where in the text you have characters from a different script?
> If this is fast enough, it should give you a conservative estimation
> of where to call font-at again.  (It's conservative because the same
> font can, and usually does, cover more than a single script.)

> One more issue that I think needs to be handled are characters for
> which there's no font installed on the user's system.  We display them
> as defined by glyphless-char-display-control.  Or maybe you should
> allow the text to overflow in this case: after all, it's not really
> legible, is it?

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Fri, 30 Jan 2015 13:02:22 -0500
> Cc: larsi@gnus.org, emacs-devel@gnu.org
> 
> Why are we re-implementing the display code in Elisp, again?

No, we aren't.  This is about something the display code cannot do
right now.

Re: Pixel-based display functions

[Stefan Monnier]

> No, we aren't.  This is about something the display code cannot do
> right now.

But all those computations of pixel sizes are really what the
redisplay does.


        Stefan

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> But all those computations of pixel sizes are really what the
> redisplay does.

When redisplay happens it's kinda too late to make layout decisions.
Unless you extend the display engine to allow constraint-based layouts.
That'd be cool, but I didn't think that was in the offing?

If that's what you're intending, then remember things like that the
document language may, for instance, describe a layout like this

+---------------+----+----------------+-----------------------+
|R1C1           |R1C2|RA1C2 With Space|R1C2                   |
+---------------+----+----------------+-----------------------+
|R2C1 and R2C2 in one|RC4-with-a-really-long-unbreakable-thing|
|simple box          |                                        |
+--------------------+----------------------------------------+

where each of the two main columns are said to be 50% of the width, but
where one of the elements can't be filled, so you have to extend that
column and compress the other columns to make things fit.

And you may have columns that themselves contain further column
specification, where the same issues apply, so you have to do a descent
into each layout cell to try to make things fit.

And so on.

If this is not what you're planning, then shr needs access to pixel
sizes so that it can do these things, and the display engine can just do
what it does now.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> Date: Sat, 31 Jan 2015 17:59:31 +1100
> 
> When redisplay happens it's kinda too late to make layout decisions.
> Unless you extend the display engine to allow constraint-based layouts.
> That'd be cool, but I didn't think that was in the offing?

AFAIR, no one has ever presented a set of requirements for supporting
such layouts.  If you or someone else can do that, it might be good to
have that in TODO.

> +---------------+----+----------------+-----------------------+
> |R1C1           |R1C2|RA1C2 With Space|R1C2                   |
> +---------------+----+----------------+-----------------------+
> |R2C1 and R2C2 in one|RC4-with-a-really-long-unbreakable-thing|
> |simple box          |                                        |
> +--------------------+----------------------------------------+

How to decipher the RnCm notation here? "row n, column m"?  If so, why
are there two instances of R1C2, and what does RA1C2 mean?

Also what is the order of the texts in a buffer, i.e. is any of the
RnCm a continuation of text in some other cell, or are they all
independent strings that don't exits as single contiguous text in any
buffer?

> where each of the two main columns are said to be 50% of the width, but
> where one of the elements can't be filled, so you have to extend that
> column and compress the other columns to make things fit.

I'm not sure I understand how the layout is specified on the "document
language" (whatever that is) level.  IOW, what is known about the
layout when this text is about to be rendered?

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> +---------------+----+----------------+-----------------------+
>> |R1C1           |R1C2|RA1C2 With Space|R1C2                   |
>> +---------------+----+----------------+-----------------------+
>> |R2C1 and R2C2 in one|RC4-with-a-really-long-unbreakable-thing|
>> |simple box          |                                        |
>> +--------------------+----------------------------------------+
>
> How to decipher the RnCm notation here? "row n, column m"?  If so, why
> are there two instances of R1C2, and what does RA1C2 mean?

Sorry; the text doesn't really mean anything here.  It was just a test
case I had handy...

> Also what is the order of the texts in a buffer, i.e. is any of the
> RnCm a continuation of text in some other cell, or are they all
> independent strings that don't exits as single contiguous text in any
> buffer?

They are all independent texts.

>> where each of the two main columns are said to be 50% of the width, but
>> where one of the elements can't be filled, so you have to extend that
>> column and compress the other columns to make things fit.
>
> I'm not sure I understand how the layout is specified on the "document
> language" (whatever that is) level.  IOW, what is known about the
> layout when this text is about to be rendered?

The layout for web pages is specified in HTML in either a <table> (which
is equivalent, more or less, to a TCL-ish "gridbaglayout", if you've
played with that back in the 90s) or CSS3, which has a different type of
layout specification.

Anyway, I've now implemented the face/script segmentation version, and
it does help speed-wise.  A typical Gnus info node now takes 0.02s
instead of 0.04s.  However, it's still too slow to be usable for complex
layouts where we need to compute the widths of things a lot (like a
Wikipedia page with a huge genealogy table).

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: larsi@gnus.org, emacs-devel@gnu.org
> Date: Sat, 31 Jan 2015 01:25:30 -0500
> 
> > No, we aren't.  This is about something the display code cannot do
> > right now.
> 
> But all those computations of pixel sizes are really what the
> redisplay does.

Lars doesn't want to compute pixel sizes to repeat what the display
engine does.  He wants to do that to implement stuff that the display
engine cannot, at least currently (if it ever should).

Re: Pixel-based display functions

[Stefan Monnier]

> Lars doesn't want to compute pixel sizes to repeat what the display
> engine does.  He wants to do that to implement stuff that the display
> engine cannot, at least currently (if it ever should).

AFAICT he wants to compute the pixel length of a chunk of text.
I don't see why we can't export to Elisp a C function that does that
re-using the same machinery used by the redisplay (and by vertical-motion).


        Stefan

Re: Pixel-based display functions

[martin rudalics]

 > AFAICT he wants to compute the pixel length of a chunk of text.
 > I don't see why we can't export to Elisp a C function that does that
 > re-using the same machinery used by the redisplay (and by vertical-motion).

`window-text-pixel-size' is one attempt to do that.

martin

Re: Pixel-based display functions

[Lars Ingebrigtsen]

martin rudalics <rudalics@gmx.at> writes:

> `window-text-pixel-size' is one attempt to do that.

Can that function be called before redisplay has happened?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> martin rudalics <rudalics@gmx.at> writes:
>
>> `window-text-pixel-size' is one attempt to do that.
>
> Can that function be called before redisplay has happened?

I guess so, because it calls

  start_display (&it, w, startp);

And it seems pretty fast.  The main problem with it is that it has to
work in a window, and much of the computation in shr takes place in
temporary buffers.  But perhaps that can be worked around somehow.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[martin rudalics]

 > The main problem with it is that it has to
 > work in a window, and much of the computation in shr takes place in
 > temporary buffers.  But perhaps that can be worked around somehow.

No.  Display can be affected by the window the text is displayed in, for
example, if you use something like an overlay with a window property.
In addition, you have to account for the default face of the window's
frame.  So you have to, at least temporarily, associate the text with
the window where you eventually want to show it to get these applied.

martin

Re: Pixel-based display functions

[Lars Ingebrigtsen]

martin rudalics <rudalics@gmx.at> writes:

>> The main problem with it is that it has to
>> work in a window, and much of the computation in shr takes place in
>> temporary buffers.  But perhaps that can be worked around somehow.
>
> No.  Display can be affected by the window the text is displayed in, for
> example, if you use something like an overlay with a window property.
> In addition, you have to account for the default face of the window's
> frame.  So you have to, at least temporarily, associate the text with
> the window where you eventually want to show it to get these applied.

Yes, that's what I meant by "worked around somehow".  :-)  If the eww
buffer is visible in a window (and it usually should be), I can insert
text I want to know the width of there, call the function, and then
delete it again.

However, if the eww buffer isn't visible, that approach won't be
possible...

But the other things you mention here (default window faces and overlays
and stuff) doesn't apply to eww at all, so it kinda sounds like I could
just do all this in a buffer that's even more likely to be associated
with a window, like the echo area.

Or is that too hacky to contemplate?  I think it might be...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[martin rudalics]

 > Yes, that's what I meant by "worked around somehow".  :-)  If the eww
 > buffer is visible in a window (and it usually should be), I can insert
 > text I want to know the width of there, call the function, and then
 > delete it again.
 >
 > However, if the eww buffer isn't visible, that approach won't be
 > possible...
 >
 > But the other things you mention here (default window faces and overlays
 > and stuff) doesn't apply to eww at all,

But a general function would have to take care of these.

 > so it kinda sounds like I could
 > just do all this in a buffer that's even more likely to be associated
 > with a window, like the echo area.
 >
 > Or is that too hacky to contemplate?  I think it might be...

Simply do a `set-window-buffer' for some window on your frame and
restore the previously shown buffer afterwards.  A macro along these
lines:

(let ((old-buffer (window-buffer (frame-selected-window)))
       (old-start (window-start (frame-selected-window)))
       (old-point (window-point (frame-selected-window))))
   (set-window-buffer (frame-selected-window) your-buffer)
   ... do your calculations ....
   (set-window-buffer (frame-selected-window) old-buffer)
   (set-window-point (frame-selected-window) old-point)
   (set-window-start (frame-selected-window) old-start))

Or yet simpler: Save the entire window configuration around your
calculations.  If redisplay doesn't run for some miraculous reason in
between, nobody will notice.

And if this is still too complicated for you, simply give
`window-text-pixel-size' an additional argument which suppresses the

   buf = w->contents;
   CHECK_BUFFER (buf);
   b = XBUFFER (buf);

   if (b != current_buffer)
     {
       old_buffer = current_buffer;
       set_buffer_internal (b);
     }

   ...

   if (old_buffer)
     set_buffer_internal (old_buffer);

sections.  In that case it would be sufficient to make your temporary
buffer current around the `window-text-pixel-size' call.

martin

Re: Pixel-based display functions

[Eli Zaretskii]

> Date: Sun, 01 Feb 2015 13:52:23 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: Eli Zaretskii <eliz@gnu.org>, 
>  Stefan Monnier <monnier@iro.umontreal.ca>,
>  emacs-devel@gnu.org
> 
> And if this is still too complicated for you, simply give
> `window-text-pixel-size' an additional argument which suppresses the
> 
>    buf = w->contents;
>    CHECK_BUFFER (buf);
>    b = XBUFFER (buf);
> 
>    if (b != current_buffer)
>      {
>        old_buffer = current_buffer;
>        set_buffer_internal (b);
>      }
> 
>    ...
> 
>    if (old_buffer)
>      set_buffer_internal (old_buffer);
> 
> sections.  In that case it would be sufficient to make your temporary
> buffer current around the `window-text-pixel-size' call.

Maybe I misunderstand what Lars wants to do, but I think
window-text-pixel-size is the wrong tool for the job: it returns the
pixel size of text between 2 given positions, whereas Lars needs the
opposite: where is the position that produces a given pixel size.

Also, there's the visual vs logical order issue I mentioned in my
other mail.

Re: Pixel-based display functions

[martin rudalics]

 > Maybe I misunderstand what Lars wants to do, but I think
 > window-text-pixel-size is the wrong tool for the job: it returns the
 > pixel size of text between 2 given positions, whereas Lars needs the
 > opposite: where is the position that produces a given pixel size.

That's possible.  But `window-text-pixel-size' would give him a way to
precalculate the sizes of chunks of his text and allow him to manually
insert linebreaks for each column.  After that he would insert spaces as
column separators and go on.  Tedious, but doing the same thing
(calculating line breaks with variable width font) in the display engine
sounds no fun either .

 > Also, there's the visual vs logical order issue I mentioned in my
 > other mail.

If `window-text-pixel-size' is kaput in that sense I'll have to fix it
anyway.

martin

Re: Pixel-based display functions

[Eli Zaretskii]

> Date: Sun, 01 Feb 2015 17:30:27 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: larsi@gnus.org, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> 
>  > Maybe I misunderstand what Lars wants to do, but I think
>  > window-text-pixel-size is the wrong tool for the job: it returns the
>  > pixel size of text between 2 given positions, whereas Lars needs the
>  > opposite: where is the position that produces a given pixel size.
> 
> That's possible.  But `window-text-pixel-size' would give him a way to
> precalculate the sizes of chunks of his text and allow him to manually
> insert linebreaks for each column.

The problem, as I understand it, is to find the place where to insert
these breaks, when column text is wider than the column.  And
window-text-pixel-size cannot tell you that, it can only tell whether
the chunk of text fits or not.

>  > Also, there's the visual vs logical order issue I mentioned in my
>  > other mail.
> 
> If `window-text-pixel-size' is kaput in that sense I'll have to fix it
> anyway.

It's not kaput.  The problem happens only if its results are used for
inserting a newline in order to break the line in two.  In all other
use cases, it does exactly TRT.

Re: Pixel-based display functions

[martin rudalics]

 > The problem, as I understand it, is to find the place where to insert
 > these breaks, when column text is wider than the column.  And
 > window-text-pixel-size cannot tell you that, it can only tell whether
 > the chunk of text fits or not.

IIUC he would have to check each word whether it still fits into his
column including a leading space.  Maybe he could start with some n
words and remove those that don't fit.

Modulo the R2L problem.  But I have no idea how Emacs deals with
line-breaking embedded R2L text anyway.

 > It's not kaput.  The problem happens only if its results are used for
 > inserting a newline in order to break the line in two.  In all other
 > use cases, it does exactly TRT.

I haven't looked into this yet but am afraid that if FROM or TO are
somewhere within R2L text it won't necessarily report the maximum length
of a line.

martin

Re: Pixel-based display functions

[Eli Zaretskii]

> Date: Sun, 01 Feb 2015 19:09:22 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: larsi@gnus.org, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> 
>  > The problem, as I understand it, is to find the place where to insert
>  > these breaks, when column text is wider than the column.  And
>  > window-text-pixel-size cannot tell you that, it can only tell whether
>  > the chunk of text fits or not.
> 
> IIUC he would have to check each word whether it still fits into his
> column including a leading space.  Maybe he could start with some n
> words and remove those that don't fit.

Yes, that's possible.  But font-get-glyphs allows to do this directly,
without iterations.

> I have no idea how Emacs deals with line-breaking embedded R2L text
> anyway.

The same as it does with L2R text.  Line breaking in Emacs works in
visual order.

>  > It's not kaput.  The problem happens only if its results are used for
>  > inserting a newline in order to break the line in two.  In all other
>  > use cases, it does exactly TRT.
> 
> I haven't looked into this yet but am afraid that if FROM or TO are
> somewhere within R2L text it won't necessarily report the maximum length
> of a line.

I don't see why it wouldn't report the maximum length.  It actually
reports the maximum X coordinate, so bidirectional reordering cannot
break that, since pixel coordinates are still measured left to right.

Unless, that is, you mean R2L lines (and not R2L text within a L2R
line), in which case you indeed need to offset the X coordinate by the
window's text-box width.  This needs to be done in move_it_to.  See
near the end of pos_visible_p for how this is done.

Re: Pixel-based display functions

[Eli Zaretskii]

> Date: Sun, 01 Feb 2015 20:36:19 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: larsi@gnus.org, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> 
> Unless, that is, you mean R2L lines (and not R2L text within a L2R
> line), in which case you indeed need to offset the X coordinate by the
> window's text-box width.  This needs to be done in move_it_to.  See
> near the end of pos_visible_p for how this is done.

Sorry, that was a thinko: there's no need to change anything, since
move_it_to works in left-to-right geometry even when it traverses R2L
lines.  So it will measure the pixel length of each line correctly.

Interpretation of the result, when the region FROM..TO includes both
L2R and R2L lines is another matter.  But that's not your problem,
it's the problem of whoever calls the function ;-)

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Ok, back in Sydney again and trying to make this work.

The function for moving to a char near a specific pixel was suggested as 

      (vertical-motion (cons (/ PIXELS (frame-char-width)) 0))

right?  That won't work, since `frame-char-width' apparently does its
calculation based on "On a graphical screen, the width is the standard
width of the default font.", and we have lots of different fonts in the
buffer.

So is there some other way of (quickly) going to a pixel position that
I've missed?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Stefan Monnier]

>       (vertical-motion (cons (/ PIXELS (frame-char-width)) 0))

> right?  That won't work, since `frame-char-width' apparently does its
> calculation based on "On a graphical screen, the width is the standard
> width of the default font.", and we have lots of different fonts in the
> buffer.

IIRC this "standard width of the default font" matches the unit assumed
by `vertical-motion', so it should indeed work (IOW, vertical-motion
will do the equivalent of multiplying its arg by (frame-char-width)
internally to get a pixel size).


        Stefan

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> IIRC this "standard width of the default font" matches the unit assumed
> by `vertical-motion', so it should indeed work (IOW, vertical-motion
> will do the equivalent of multiplying its arg by (frame-char-width)
> internally to get a pixel size).

Oh, I see.  Thanks.

I've been doing some benchmarking to get a feel for the speed of
`vertical-motion'.  

shr-tag-body                         100         2.2069731249  0.0220697312
shr-fold-lines                       100         1.2538841410  0.0125388414
shr-fold-line                        1500        1.2457802040  0.0008305201
shr-goto-pixel-column                5000        1.1738643779  0.0002347728

`shr-goto-pixel-column' is just a call to `vertical-motion' separated
out to see how much time it takes.

So, basically, folding 1500 lines takes 1.24s, of which 1.17s is spent
in `vertical-motion' (plus function call overhead).  (It's called a lot
of times because the lines are very long and need to be filled more than
once.)

(benchmark-run 5000 (vertical-motion (cons (/ 700 (frame-char-width)) 0)))
=> (0.942894006 2 0.05716983599999992)

But it's all kinda moot since `window-text-pixel-size' doesn't work on
non-displayed buffers (yet).  But if that could be made to work somehow
(Eli seemed to have an idea here?), I wonder whether a faster interface
would be to have a version of `window-text-pixel-size' that returns a
vector of glyph sizes.  Then that vector could be used both for going to
a column (by just adding until we get to the required pixel count) as
well as for computing the displayed width of each line.

And this would be only one call to this new function per each (long)
line instead of repeatedly calling `vertical-motion' after each line
break has been inserted, so it may be faster.

The glyph width vector would also allow caching of the data, which would
speed up multi-column computation, where each text line is usually
folded more than once to reach an optimal layout.

Of course, computing the vector may be unreasonably slow.  Or not.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[martin rudalics]

 > I wonder whether a faster interface
 > would be to have a version of `window-text-pixel-size' that returns a
 > vector of glyph sizes.  Then that vector could be used both for going to
 > a column (by just adding until we get to the required pixel count) as
 > well as for computing the displayed width of each line.

Isn't what you want a function that stops at some whitespace preceding
non-whitespace that won't fit within some specified limit?

martin

Re: Pixel-based display functions

[Lars Ingebrigtsen]

martin rudalics <rudalics@gmx.at> writes:

> Isn't what you want a function that stops at some whitespace preceding
> non-whitespace that won't fit within some specified limit?

No, filling text isn't just about whitespace.  In some scripts (eastern
Asian, for instance), there's not necessarily any whitespace at all, but
the text still has to be filled, and that's only valid between certain
characters.  Search for the "kinsuko" in shr.el.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Fri, 06 Feb 2015 20:39:25 +1100
> Cc: Eli Zaretskii <eliz@gnu.org>, Stefan Monnier <monnier@IRO.UMontreal.CA>,
> 	emacs-devel@gnu.org
> 
> martin rudalics <rudalics@gmx.at> writes:
> 
> > Isn't what you want a function that stops at some whitespace preceding
> > non-whitespace that won't fit within some specified limit?
> 
> No, filling text isn't just about whitespace.  In some scripts (eastern
> Asian, for instance), there's not necessarily any whitespace at all, but
> the text still has to be filled, and that's only valid between certain
> characters.  Search for the "kinsuko" in shr.el.

You mean, "kinsoku".

Btw, why doesn't shr.el use kinsoku.el functions for that?

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> You mean, "kinsoku".

Yeah.  :-)

> Btw, why doesn't shr.el use kinsoku.el functions for that?

I don't know -- it was implemented by Katsumi.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rudalics@gmx.at, monnier@IRO.UMontreal.CA, emacs-devel@gnu.org
> Date: Fri, 06 Feb 2015 21:02:20 +1100
> 
> > Btw, why doesn't shr.el use kinsoku.el functions for that?
> 
> I don't know -- it was implemented by Katsumi.

Well, then maybe Katsumi could explain?

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org, Eli Zaretskii <eliz@gnu.org>,
>         martin rudalics <rudalics@gmx.at>
> Date: Fri, 06 Feb 2015 12:17:43 +1100
> 
> I've been doing some benchmarking to get a feel for the speed of
> `vertical-motion'.  
> 
> shr-tag-body                         100         2.2069731249  0.0220697312
> shr-fold-lines                       100         1.2538841410  0.0125388414
> shr-fold-line                        1500        1.2457802040  0.0008305201
> shr-goto-pixel-column                5000        1.1738643779  0.0002347728
> 
> `shr-goto-pixel-column' is just a call to `vertical-motion' separated
> out to see how much time it takes.
> 
> So, basically, folding 1500 lines takes 1.24s, of which 1.17s is spent
> in `vertical-motion' (plus function call overhead).  (It's called a lot
> of times because the lines are very long and need to be filled more than
> once.)

Can you explain why you need to call vertical-motion so many times?

> But it's all kinda moot since `window-text-pixel-size' doesn't work on
> non-displayed buffers (yet).

window-text-pixel-size is equivalent to vertical-motion, so I don't
understand why you need both.  Can you explain?

More generally, perhaps you could post an outline of your
algorithm(s), so that their overall design could be kept in mind.  It
just could be that the speedups you are looking for are on the
algorithm level, not on the level of primitives.

> I wonder whether a faster interface would be to have a version of
> `window-text-pixel-size' that returns a vector of glyph sizes.

??? Isn't that font-get-glyphs that you already tried?  If not, why
not?  What API would you like to have for that hypothetical function?

> Of course, computing the vector may be unreasonably slow.

It is again equivalent to vertical-motion and font-get-glyphs, so it's
not slow.  But I don't yet see the issue clearly enough to tell what
could be done for you, so please post more information about what you
are trying to do.

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> So, basically, folding 1500 lines takes 1.24s, of which 1.17s is spent
>> in `vertical-motion' (plus function call overhead).  (It's called a lot
>> of times because the lines are very long and need to be filled more than
>> once.)
>
> Can you explain why you need to call vertical-motion so many times?

shr inserts text as long lines into the buffer, and I then use
`vertical-motion' to go to the desired width, and then backtracking
finding a fill point using the kinsuko algorithm.  Then a newline is
inserted, and we repeat until the line is completely filled.

>> But it's all kinda moot since `window-text-pixel-size' doesn't work on
>> non-displayed buffers (yet).
>
> window-text-pixel-size is equivalent to vertical-motion, so I don't
> understand why you need both.  Can you explain?

`window-text-pixel-size' tells me what width I ended up with, while
`vertical-motion' goes to an approximate fill point.  

>> I wonder whether a faster interface would be to have a version of
>> `window-text-pixel-size' that returns a vector of glyph sizes.
>
> ??? Isn't that font-get-glyphs that you already tried?  If not, why
> not?  What API would you like to have for that hypothetical function?

I though we established pretty thoroughly that for `font-get-glyphs' to
be a solution, we'd have to reimplement too much of the redisplay
algorithm in Emacs Lisp (what with segmenting on scripts and faces, and
then having to see whether a script uses multiple fonts, etc).  And it
was too slow during my testing.

>> Of course, computing the vector may be unreasonably slow.
>
> It is again equivalent to vertical-motion and font-get-glyphs, so it's
> not slow.  But I don't yet see the issue clearly enough to tell what
> could be done for you, so please post more information about what you
> are trying to do.

It would be equivalent to a `window-text-pixel-size' that advances one
character at a time (since we'd avoid the entire `font-at' problem), and
returns an vector of pixel points (instead of calling move_it_to just
once).  I haven't played with `window-text-pixel-size' at that level,
since Martin seems to feel that it's not possible to get it to work on
undisplayed buffers?  But I may be misunderstanding what Martin was
saying...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rudalics@gmx.at, monnier@IRO.UMontreal.CA, emacs-devel@gnu.org
> Date: Fri, 06 Feb 2015 20:47:40 +1100
> 
> shr inserts text as long lines into the buffer, and I then use
> `vertical-motion' to go to the desired width, and then backtracking
> finding a fill point using the kinsuko algorithm.  Then a newline is
> inserted, and we repeat until the line is completely filled.

When you "repeat", do you start from the part of the line that's left,
without the part that is before the just-inserted newline, or from the
original long one?  The time it takes vertical-motion to do its job
will be smaller if the line is shorter.

Also, try invoking vertical-motion from a position that is not
immediately after a newline, I think this might speed it up a little
more.

> >> But it's all kinda moot since `window-text-pixel-size' doesn't work on
> >> non-displayed buffers (yet).
> >
> > window-text-pixel-size is equivalent to vertical-motion, so I don't
> > understand why you need both.  Can you explain?
> 
> `window-text-pixel-size' tells me what width I ended up with, while
> `vertical-motion' goes to an approximate fill point.  

Why do you need to know what width you ended up with?  Or maybe I
don't understand "ended up with" when -- before or after
vertical-motion did its job?  IOW, do you need window-text-pixel-size
for measuring the long unfilled line, or the lines after filling?

> >> I wonder whether a faster interface would be to have a version of
> >> `window-text-pixel-size' that returns a vector of glyph sizes.
> >
> > ??? Isn't that font-get-glyphs that you already tried?  If not, why
> > not?  What API would you like to have for that hypothetical function?
> 
> I though we established pretty thoroughly that for `font-get-glyphs' to
> be a solution, we'd have to reimplement too much of the redisplay
> algorithm in Emacs Lisp (what with segmenting on scripts and faces, and
> then having to see whether a script uses multiple fonts, etc).  And it
> was too slow during my testing.

So what kind of API would you like to have?

> >> Of course, computing the vector may be unreasonably slow.
> >
> > It is again equivalent to vertical-motion and font-get-glyphs, so it's
> > not slow.  But I don't yet see the issue clearly enough to tell what
> > could be done for you, so please post more information about what you
> > are trying to do.
> 
> It would be equivalent to a `window-text-pixel-size' that advances one
> character at a time (since we'd avoid the entire `font-at' problem), and
> returns an vector of pixel points (instead of calling move_it_to just
> once).

You are describing an implementation.  Please describe the API and the
contract instead.

Also, as long as you use these functions, there's still a problem with
visual vs logical order that needs to be solved.

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> When you "repeat", do you start from the part of the line that's left,
> without the part that is before the just-inserted newline, or from the
> original long one?  The time it takes vertical-motion to do its job
> will be smaller if the line is shorter.

I start from the part of the line that's left.

> Also, try invoking vertical-motion from a position that is not
> immediately after a newline, I think this might speed it up a little
> more.

Ah, right.

> Why do you need to know what width you ended up with?  Or maybe I
> don't understand "ended up with" when -- before or after
> vertical-motion did its job?  IOW, do you need window-text-pixel-size
> for measuring the long unfilled line, or the lines after filling?

I need to measure the lines after filling so that I know how wide the
column ended up being.  (This is to better use the available horizontal
space when rendering tables, which is quite important.)

>> It would be equivalent to a `window-text-pixel-size' that advances one
>> character at a time (since we'd avoid the entire `font-at' problem), and
>> returns an vector of pixel points (instead of calling move_it_to just
>> once).
>
> You are describing an implementation.  Please describe the API and the
> contract instead.

Well, it depends on what's realistic to implement, really.  If calling
`font-at' had been fast enough, I would have been happy sticking with
that, but it turned out not to be.

So the API kinda depends on what's feasible...

I think the ideal interface for this would be a function that returns
a vector of glyph widths in the region, possibly with one vector per
line.

So (glyph-pixel-widths FROM TO) =>
([4 5 10 12] [4 14 1] ...)

That would allow caching the computations so that they would only be
called once per (unfilled) buffer.

> Also, as long as you use these functions, there's still a problem with
> visual vs logical order that needs to be solved.

Yes.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rudalics@gmx.at, monnier@IRO.UMontreal.CA, emacs-devel@gnu.org
> Date: Sat, 07 Feb 2015 02:22:26 +1100
> 
> I need to measure the lines after filling so that I know how wide the
> column ended up being.  (This is to better use the available horizontal
> space when rendering tables, which is quite important.)

But then that's an optimization, isn't it?  IOW, you can do without
it.

> I think the ideal interface for this would be a function that returns
> a vector of glyph widths in the region, possibly with one vector per
> line.

I'll see what I can do.

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> I need to measure the lines after filling so that I know how wide the
>> column ended up being.  (This is to better use the available horizontal
>> space when rendering tables, which is quite important.)
>
> But then that's an optimization, isn't it?  IOW, you can do without
> it.

Well, web pages look kinda ugly if you don't do this.  

>> I think the ideal interface for this would be a function that returns
>> a vector of glyph widths in the region, possibly with one vector per
>> line.
>
> I'll see what I can do.

Great!

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Lars Ingebrigtsen]

I've now reworked the table layout algorithm to behave more nicely when
dealing with nested tables.

As a test case, I'm using the "cat" Wikipedia genealogy table, which has
tables in tables galore, and lots of little text snippets to try to
fold.  Here's the relevant data:

shr-insert-document                  1           5.11859327    5.11859327
shr-pixel-buffer-width               1940        1.8345742170  0.0009456568
shr-fold-lines                       1579        1.7859513240  0.0011310648
shr-vertical-motion                  2625        1.7517520130  0.0006673341

`shr-pixel-buffer-width' and `shr-vertical-motion' are just wrappers
over `window-text-pixel-size' and `vertical-motion' respectively (for
easier profiling).

So out of the five seconds it takes to render the page now (which is, of
course, way too slow to be usable), (+ 1.8 1.7) => 3.5s is spent moving
to pixels and computing pixel widths.

There may still be algorithmic fixes that can be done to get the number
of calls down, but having the primitives be faster (i.e., the glyph
width vector thing) may also help a lot...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Lars Ingebrigtsen]

I cached some computations, and shaved off 40% more.  So now 80%-ish of
the time is spent doing pixel computations:

shr-insert-document                  1           3.16461479    3.16461479
shr-vertical-motion                  854         1.4641583980  0.0017144711
shr-pixel-buffer-width               988         0.9732391799  0.0009850598

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sat, 07 Feb 2015 16:22:01 +1100
> 
> I cached some computations, and shaved off 40% more.  So now 80%-ish of
> the time is spent doing pixel computations:
> 
> shr-insert-document                  1           3.16461479    3.16461479
> shr-vertical-motion                  854         1.4641583980  0.0017144711
> shr-pixel-buffer-width               988         0.9732391799  0.0009850598

How many lines are there?  And where's that page to begin with?

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> shr-insert-document                  1           3.16461479    3.16461479
>> shr-vertical-motion                  854         1.4641583980  0.0017144711
>> shr-pixel-buffer-width               988         0.9732391799  0.0009850598
>
> How many lines are there? 

`shr-fold-line' is called 723 times in that page, which lines up pretty
well with the `shr-vertical-motion' calls.  `shr-pixel-buffer-width' is
called (at least) once per <td> in the table.

> And where's that page to begin with?

http://en.wikipedia.org/wiki/Ocelot

The torture test part of the page isn't shown by default.  It's the bit
at the end that says "Extant Carnivora species [Show]".

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Sat, 07 Feb 2015 20:10:23 +1100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> shr-insert-document                  1           3.16461479    3.16461479
> >> shr-vertical-motion                  854         1.4641583980  0.0017144711
> >> shr-pixel-buffer-width               988         0.9732391799  0.0009850598
> >
> > How many lines are there? 
> 
> `shr-fold-line' is called 723 times in that page, which lines up pretty
> well with the `shr-vertical-motion' calls.  `shr-pixel-buffer-width' is
> called (at least) once per <td> in the table.

It looks like your previous timings, viz.

> shr-insert-document                  1           5.11859327    5.11859327
> shr-pixel-buffer-width               1940        1.8345742170  0.0009456568
> shr-fold-lines                       1579        1.7859513240  0.0011310648
> shr-vertical-motion                  2625        1.7517520130  0.0006673341

and

> (benchmark-run 5000 (vertical-motion (cons (/ 700 (frame-char-width)) 0)))
> => (0.942894006 2 0.05716983599999992)

were better, at least as far as vertical-motion was concerned.  What
happened that caused almost twofold degradation in speed of
shr-vertical-motion?  And why does a single call to vertical-motion
take ~0.2 msec, while a single call to shr-vertical-motion takes 1.7
msec, more than 8 times more?

> > And where's that page to begin with?
> 
> http://en.wikipedia.org/wiki/Ocelot
> 
> The torture test part of the page isn't shown by default.  It's the bit
> at the end that says "Extant Carnivora species [Show]".

I don't see "Extant Carnivora species [Show]", I see "Extant Carnivora
species" with "Carnivora" a link to
http://en.wikipedia.org/wiki/Carnivora.  Is that what you meant?  (In
what browser do you see what you described?  I tried eww and Forefox,
and both show what I described.)

Anyway, which part of http://en.wikipedia.org/wiki/Carnivora is the
torture part? the "Phylogenetic tree" part?

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> It looks like your previous timings, viz.
>
>> shr-insert-document                  1           5.11859327    5.11859327
>> shr-pixel-buffer-width               1940        1.8345742170  0.0009456568
>> shr-fold-lines                       1579        1.7859513240  0.0011310648
>> shr-vertical-motion                  2625        1.7517520130  0.0006673341
>
> and
>
>> (benchmark-run 5000 (vertical-motion (cons (/ 700 (frame-char-width)) 0)))
>> => (0.942894006 2 0.05716983599999992)
>
> were better, at least as far as vertical-motion was concerned.  What
> happened that caused almost twofold degradation in speed of
> shr-vertical-motion?  And why does a single call to vertical-motion
> take ~0.2 msec, while a single call to shr-vertical-motion takes 1.7
> msec, more than 8 times more?

My guess would be the extra function call overhead, along with the ELP
instrumentation overhead.  ELP results are never all that precise, in my
opinion, but they're useful to give you a general overview of where time
is spent, since it adds about the same overhead to all functions.

>> http://en.wikipedia.org/wiki/Ocelot
>> 
>> The torture test part of the page isn't shown by default.  It's the bit
>> at the end that says "Extant Carnivora species [Show]".
>
> I don't see "Extant Carnivora species [Show]", I see "Extant Carnivora
> species" with "Carnivora" a link to
> http://en.wikipedia.org/wiki/Carnivora.  Is that what you meant?  (In
> what browser do you see what you described?  I tried eww and Forefox,
> and both show what I described.)

In my Firefox, it's shown as "Extant Carnivora species [Show]".  Do you
have Javascript switched on or something?  It's a JS table toggle or
something. 

> Anyway, which part of http://en.wikipedia.org/wiki/Carnivora is the
> torture part? the "Phylogenetic tree" part?

It's not that page, it on the Ocelot page.  In eww, it's the large table
that starts with "Extant Carnivora species" in the heading.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Sat, 07 Feb 2015 20:57:05 +1100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > It looks like your previous timings, viz.
> >
> >> shr-insert-document                  1           5.11859327    5.11859327
> >> shr-pixel-buffer-width               1940        1.8345742170  0.0009456568
> >> shr-fold-lines                       1579        1.7859513240  0.0011310648
> >> shr-vertical-motion                  2625        1.7517520130  0.0006673341
> >
> > and
> >
> >> (benchmark-run 5000 (vertical-motion (cons (/ 700 (frame-char-width)) 0)))
> >> => (0.942894006 2 0.05716983599999992)
> >
> > were better, at least as far as vertical-motion was concerned.  What
> > happened that caused almost twofold degradation in speed of
> > shr-vertical-motion?  And why does a single call to vertical-motion
> > take ~0.2 msec, while a single call to shr-vertical-motion takes 1.7
> > msec, more than 8 times more?
> 
> My guess would be the extra function call overhead, along with the ELP
> instrumentation overhead.

That could explain the shr-vertical-motion vs vertical-motion alone,
but what about the twofold degradation of speed, from 1.75 sec for
2625 calls to 1.46 sec for 854 calls?

> In my Firefox, it's shown as "Extant Carnivora species [Show]".  Do you
> have Javascript switched on or something?  It's a JS table toggle or
> something. 

I have no idea, this is the default Firefox configuration.

Anyway, I see the "[show]" part now -- it'ss at the extreme right of
that line, so it was outside of my FOV.

But it is merely 500 lines, whereas your previous message said 1500, I
think, and showed 1500 calls to shr-fold-line to prove it.  What am I
missing?

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> That could explain the shr-vertical-motion vs vertical-motion alone,
> but what about the twofold degradation of speed, from 1.75 sec for
> 2625 calls to 1.46 sec for 854 calls?

I don't have an explanation for that.  ELP weirdness?

> But it is merely 500 lines, whereas your previous message said 1500, I
> think, and showed 1500 calls to shr-fold-line to prove it.  What am I
> missing?

The <td>s may be rendered more than once according to different
constraints while computing an optimal table.  I added more caching of
data to avoid re-rendering stuff that wasn't needed, so the number of
folded lines went down from 1500 to 753.

shr-insert-document                  1           3.367278072   3.367278072
shr-fold-lines                       627         1.462917592   0.0023332019
shr-fold-line                        753         1.4551749930  0.0019325033
shr-vertical-motion                  854         1.450865314   0.0016989055
shr-pixel-buffer-width               988         0.9033882480  0.0009143605

<td>s are still rendered more than once, though, but it's not as
excessive as before.  And other table nestings with more "difficult"
data than that table (for instance, with wide <pre> texts that makes
getting at a pretty table more difficult) may require more renderings.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Stefan Monnier]

>> That could explain the shr-vertical-motion vs vertical-motion alone,
>> but what about the twofold degradation of speed, from 1.75 sec for
>> 2625 calls to 1.46 sec for 854 calls?
> I don't have an explanation for that.  ELP weirdness?

You can also try M-x profiler-start/report which, being sample-based
rather than rewriting code, should have less side-effects.  Of course,
it won't tell you how many times a function was called.


        Stefan

Display of images in eww (was: Pixel-based display functions)

[Eli Zaretskii]

In this page:

  http://en.wikipedia.org/wiki/Carnivora

there's a large tree-like display of carnivore families starting about
1/3d of the page down.  Firefox displays one or more small images at
the end of each line in the tree, but eww displays them all as a
single long line after the tree.  Why cannot eww show each image where
it belongs?

Re: Display of images in eww

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> In this page:
>
>   http://en.wikipedia.org/wiki/Carnivora
>
> there's a large tree-like display of carnivore families starting about
> 1/3d of the page down.  Firefox displays one or more small images at
> the end of each line in the tree, but eww displays them all as a
> single long line after the tree.  Why cannot eww show each image where
> it belongs?

eww doesn't display images inside tables because of alignment issues.
Now that the shr branch uses `align-to' to do table layouts, that
decision can be revisited.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> The torture test part of the page isn't shown by default.  It's the bit
> at the end that says "Extant Carnivora species [Show]".

I've now adapted the versions of the algorithms in the shr branch to
also be able to render fixed-width text, using `move-to-column' and
friends.

A version using that takes 0.66s to render the Ocelot page, while the
pixel version takes 2.86s.

As a comparison, the version of shr in the trunk uses about 2.5s to
render the same page.  So I think merging to the trunk in a few days may
be a good idea, but defaulting `shr-use-fonts' to nil.

(Well, I won't merge.  I'll just make a big patch and apply it to trunk.)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[martin rudalics]

 > The torture test part of the page isn't shown by default.  It's the bit
 > at the end that says "Extant Carnivora species [Show]".

So IIUC you want to lay out text in cells like this

Nandiniidae   Nandinia   African palm civet (N. binotata)

               Atilax     Marsh mongoose (A. paludinosus)

               Bdeogale   Bushy-tailed mongoose (B. crassicauda) Jackson's mongoose (B. jacksoni)
                          Black-footed mongoose (B. nigripes)

Then we could proceed as follows:

(1) Turn on word wrap.

(2) For each cell put the text in a buffer and show the buffer in a
     window.  Arrange that the text width of the window is as wide as the
     desired with of the the cell (you can split the window or adjust its
     fringes for that).

(3) Run `window-text-pixel-size' or `vertical-motion' over it to get the
     maximum height of the text in the cell.

Proceed with the next cell in the same row.

"All" that's missing is one thing: The word wrap algorithm would have to
inform you where each visual line in step (3) ends, either by inserting
something like a CR or a text property into the buffer or by returning a
list of all buffer positions where it broke the line.  This should be
manageable with God's and Eli's help.

Using these indications you could add the necessary inter-cell
whitespaces.  Then make each cell as high as the maximum value returned
by the (3) steps in one "row" and continue with the next row.

And before I forget: We don't support different line heights with this
approach.  Actually this means that in

               Bdeogale   Bushy-tailed mongoose (B. crassicauda) Jackson's mongoose (B. jacksoni)
                          Black-footed mongoose (B. nigripes)

Bdeogale would have to appear on the same of either of the two lines on
the right.  Firefox can do better - it centers the line here.  I suppose
Linné wouldn't complain, though ...

martin

Re: Pixel-based display functions

[Lars Ingebrigtsen]

martin rudalics <rudalics@gmx.at> writes:

> Then we could proceed as follows:
>
> (1) Turn on word wrap.

Does word wrap do kinsoku?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> Date: Sat, 07 Feb 2015 23:01:51 +1100
> 
> martin rudalics <rudalics@gmx.at> writes:
> 
> > Then we could proceed as follows:
> >
> > (1) Turn on word wrap.
> 
> Does word wrap do kinsoku?

It doesn't wrap such text at all.  It only wraps on whitespace
characters.

Re: Pixel-based display functions

[martin rudalics]

 >> (1) Turn on word wrap.
 >
 > Does word wrap do kinsoku?

I haven't seen any kinsoku on the ocelot page.  But if you need it you
can either insert breakble spaces befor passing them to the iterator or
we agree on some text property that the iterator will consider as
breakable.  In any case this would have to be done before step (3) of my
scenario.

martin

Re: Pixel-based display functions

[Eli Zaretskii]

> Date: Sat, 07 Feb 2015 12:45:27 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: emacs-devel@gnu.org
> 
> (2) For each cell put the text in a buffer and show the buffer in a
>      window.

I thought Lars didn't want to flash display of such windows.

> "All" that's missing is one thing: The word wrap algorithm would have to
> inform you where each visual line in step (3) ends, either by inserting
> something like a CR or a text property into the buffer or by returning a
> list of all buffer positions where it broke the line.  This should be
> manageable with God's and Eli's help.

Doesn't posn-at-point and end-of-visual-line already provide the means
to get this information?

Re: Pixel-based display functions

[martin rudalics]

 >> (2) For each cell put the text in a buffer and show the buffer in a
 >>       window.
 >
 > I thought Lars didn't want to flash display of such windows.

How comes we'd flash them?  He would obviously restore the window
configuration before the next redisplay.

 > Doesn't posn-at-point and end-of-visual-line already provide the means
 > to get this information?

The latter calls `vertical-motion' so this should work indeed.  But does
`vertical-motion' have to go back to the beginning of a logical line
when it's broken into several visual lines?  This would mean that we'd
always have to rescan from the beginning of the logical line for each
visual line break and the contents of each cell might often consist of
one long logical line only.

martin

Re: Pixel-based display functions

[Eli Zaretskii]

> Date: Sat, 07 Feb 2015 13:28:18 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: larsi@gnus.org, emacs-devel@gnu.org
> 
>  >> (2) For each cell put the text in a buffer and show the buffer in a
>  >>       window.
>  >
>  > I thought Lars didn't want to flash display of such windows.
> 
> How comes we'd flash them?  He would obviously restore the window
> configuration before the next redisplay.

Then how to interpret "show the buffer" in your text above?

>  > Doesn't posn-at-point and end-of-visual-line already provide the means
>  > to get this information?
> 
> The latter calls `vertical-motion' so this should work indeed.  But does
> `vertical-motion' have to go back to the beginning of a logical line
> when it's broken into several visual lines?

Yes, it does.  There's no other place where the display engine can
_know_ that the X coordinate is zero.

> This would mean that we'd always have to rescan from the beginning
> of the logical line for each visual line break

Yes.

Re: Pixel-based display functions

[martin rudalics]

 > Then how to interpret "show the buffer" in your text above?

As `set-window-buffer'.  Sorry for being ambiguous.

 >> The latter calls `vertical-motion' so this should work indeed.  But does
 >> `vertical-motion' have to go back to the beginning of a logical line
 >> when it's broken into several visual lines?
 >
 > Yes, it does.  There's no other place where the display engine can
 > _know_ that the X coordinate is zero.
 >
 >> This would mean that we'd always have to rescan from the beginning
 >> of the logical line for each visual line break
 >
 > Yes.

This would get us quadratic time.  We should stay linear.

martin

Re: Pixel-based display functions

[Eli Zaretskii]

> Date: Sat, 07 Feb 2015 14:51:10 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: larsi@gnus.org, emacs-devel@gnu.org
> 
>  >> The latter calls `vertical-motion' so this should work indeed.  But does
>  >> `vertical-motion' have to go back to the beginning of a logical line
>  >> when it's broken into several visual lines?
>  >
>  > Yes, it does.  There's no other place where the display engine can
>  > _know_ that the X coordinate is zero.
>  >
>  >> This would mean that we'd always have to rescan from the beginning
>  >> of the logical line for each visual line break
>  >
>  > Yes.
> 
> This would get us quadratic time.  We should stay linear.

The only way to stay linear is to display the entire physical line in
one go.

Re: Pixel-based display functions

[martin rudalics]

 > The only way to stay linear is to display the entire physical line in
 > one go.

That was the idea of my initial proposal.

martin

Re: Pixel-based display functions

[Eli Zaretskii]

> Date: Fri, 06 Feb 2015 17:42:40 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: rudalics@gmx.at, monnier@IRO.UMontreal.CA, emacs-devel@gnu.org
> 
> > I think the ideal interface for this would be a function that returns
> > a vector of glyph widths in the region, possibly with one vector per
> > line.
> 
> I'll see what I can do.

A few preliminary questions about this:

 . Is it good enough to handle only a single physical line, starting
   from a given POSITION argument and ending at the first newline that
   follows POSITION?  (Handling of additional lines will then have to
   be done in Lisp.)

 . What to assume/do with the various display features, like overlay
   and display strings, images, align-to space specs, line-prefix,
   etc., that can be pertinent to the portion of text being processed?
   The easiest alternative is to handle them "as usual" in redisplay,
   i.e. the corresponding glyphs will be produced and included in the
   return value.  Is that OK?  If not, what else is needed for these
   use cases?  (Note that if a display or overlay string includes
   newlines, this means the result could span multiple screen lines --
   will that be a problem?.)

 . How will the caller specify the information about the display
   defaults, like the faces?  The usual way is that some window is
   specified, either explicitly as an argument or implicitly as the
   currently selected window, and all the defaults are taken from that
   window and its frame.  Is that good enough for your purposes?

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> A few preliminary questions about this:
>
>  . Is it good enough to handle only a single physical line, starting
>    from a given POSITION argument and ending at the first newline that
>    follows POSITION?  (Handling of additional lines will then have to
>    be done in Lisp.)

If this can be done as fast as collecting the line data in C, that would
be fine.  But I would assume that there's quite a lot set-up overhead
before getting started.  For instance, calling `window-text-pixel-size'
ten times over separate lines instead of once on a region is way slower.

But if there's no setup overhead, then doing it per line basis would be
fine. 

>  . What to assume/do with the various display features, like overlay
>    and display strings, images, align-to space specs, line-prefix,
>    etc., that can be pertinent to the portion of text being processed?
>    The easiest alternative is to handle them "as usual" in redisplay,
>    i.e. the corresponding glyphs will be produced and included in the
>    return value.  Is that OK?

Yes, including all that stuff is necessary to do the line filling.

>    If not, what else is needed for these
>    use cases?  (Note that if a display or overlay string includes
>    newlines, this means the result could span multiple screen lines --
>    will that be a problem?.)

Hm...  I hadn't though of that.  Well, in the shr case, there are no
overlays at all, and the only display strings are `align-to' one, I
think...

>  . How will the caller specify the information about the display
>    defaults, like the faces?  The usual way is that some window is
>    specified, either explicitly as an argument or implicitly as the
>    currently selected window, and all the defaults are taken from that
>    window and its frame.  Is that good enough for your purposes?

Passing in an optional window, or if that's nil, the current window,
would be fine for shr's use case.  

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rudalics@gmx.at, monnier@IRO.UMontreal.CA, emacs-devel@gnu.org
> Date: Sat, 07 Feb 2015 22:59:52 +1100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > A few preliminary questions about this:
> >
> >  . Is it good enough to handle only a single physical line, starting
> >    from a given POSITION argument and ending at the first newline that
> >    follows POSITION?  (Handling of additional lines will then have to
> >    be done in Lisp.)
> 
> If this can be done as fast as collecting the line data in C, that would
> be fine.

What do you mean by "collecting the line data"?

>          But I would assume that there's quite a lot set-up overhead
> before getting started.  For instance, calling `window-text-pixel-size'
> ten times over separate lines instead of once on a region is way slower.
> 
> But if there's no setup overhead, then doing it per line basis would be
> fine. 

There's always setup overhead when using the display engine
subroutines.  Whether it's significant in this case, remains to be
seen.

Since handling more than one line needs the one-line case as its
workhorse anyway, I think I will go with that first, and then extend
it if it's not fast enough for many lines.

> >  . What to assume/do with the various display features, like overlay
> >    and display strings, images, align-to space specs, line-prefix,
> >    etc., that can be pertinent to the portion of text being processed?
> >    The easiest alternative is to handle them "as usual" in redisplay,
> >    i.e. the corresponding glyphs will be produced and included in the
> >    return value.  Is that OK?
> 
> Yes, including all that stuff is necessary to do the line filling.

Do you need to know the "type" of each glyph as well, or are
dimensions enough?

> >    If not, what else is needed for these
> >    use cases?  (Note that if a display or overlay string includes
> >    newlines, this means the result could span multiple screen lines --
> >    will that be a problem?.)
> 
> Hm...  I hadn't though of that.  Well, in the shr case, there are no
> overlays at all, and the only display strings are `align-to' one, I
> think...

Well, align-to will produce a stretch glyph that has no underlying
character -- do you care about that?

And then what about images?

Re: Pixel-based display functions

[Eli Zaretskii]

> Date: Sat, 07 Feb 2015 13:37:09 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: rudalics@gmx.at, monnier@IRO.UMontreal.CA, emacs-devel@gnu.org
> 
>    Note that if a display or overlay string includes newlines, this
>    means the result could span multiple screen lines -- will that be
>    a problem?

Actually, if simplicity of the implementation is the main guideline,
then the function will stop at the first newline, whether from buffer
or from some display/overlay string.  Is that OK?

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Actually, if simplicity of the implementation is the main guideline,
> then the function will stop at the first newline, whether from buffer
> or from some display/overlay string.  Is that OK?

Given low setup costs, that would be OK.

But before you proceed with this, this interface function may not
actually be what's needed at all.  Sorry for the confusion.  What was
attractive about it was that it would provide cacheable information when
strings have to be filled in several different ways.

Now that I've made shr cache more rendering information, this may not be
such a major win, anyway.  Let me do some more benchmarking and think
about this yet another time before you start implementing.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> But before you proceed with this, this interface function may not
> actually be what's needed at all.  Sorry for the confusion.  What was
> attractive about it was that it would provide cacheable information when
> strings have to be filled in several different ways.

I've got one question about `vertical-motion': Why is it apparently
slower than `window-text-pixel-size'?

shr-vertical-motion                  854         2.0539825089  0.0024051317
shr-pixel-buffer-width               988         1.0268266620  0.0010392982

Would it be possible to get a "go to pixel position X on the current
line" function that's faster than the current `vertical-motion' function
if there are some ... assumptions we could make?  Or something?  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rudalics@gmx.at, emacs-devel@gnu.org
> Date: Sat, 07 Feb 2015 23:42:12 +1100
> 
> I've got one question about `vertical-motion': Why is it apparently
> slower than `window-text-pixel-size'?

Because it solves a harder problem.  It is easier to get to a given
buffer position than it is to get to a given pixel X coordinate: in
the latter case you need to start from a position that has a known X
coordinate, so you need first to get there.

But I suggest to time these 2 functions without their shr-* wrappers,
as I'm not sure the difference is so large.

Also, did you invoke vertical-motion from the beginning of a physical
line, or from some place in the middle of the line?

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Also, did you invoke vertical-motion from the beginning of a physical
> line, or from some place in the middle of the line?

D'oh!  Thank you for reminding me to test that.

This is from the beginning of the line:

shr-vertical-motion                  854         2.0654068049  0.0024185091
shr-pixel-buffer-width               988         0.7436217859  0.0007526536

This is from the first character on the line:

shr-pixel-buffer-width               988         0.7434988130  0.0007525291
shr-vertical-motion                  854         0.2932210100  0.0003433501

So that's an 7-fold improvement in speed in `vertical-motion'.  :-)

Running the Ocelot test in an Emacs without any ELP-instrumented
functions now renders that page in 1.6s with fonts, which is faster than
the version in trunk does on fixed-width layouts, so we may just declare
success.  :-)

(A typical manual page renders in 0.015s, if anybody's counting.)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sat, 07 Feb 2015 23:42:12 +1100
> Cc: rudalics@gmx.at, emacs-devel@gnu.org
> 
> Would it be possible to get a "go to pixel position X on the current
> line" function that's faster than the current `vertical-motion' function
> if there are some ... assumptions we could make?  Or something?  :-)

I think I can make it faster if we add an additional optional argument
that tells vertical-motion the X coordinate at point.  vertical-motion
invests a lot of cycles in finding a place whose X coordinate is
known, so providing that for its starting point will avoid all that
effort.  Will this help you, i.e. can you provide such a value with
each call?

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> I think I can make it faster if we add an additional optional argument
> that tells vertical-motion the X coordinate at point.  vertical-motion
> invests a lot of cycles in finding a place whose X coordinate is
> known, so providing that for its starting point will avoid all that
> effort.  Will this help you, i.e. can you provide such a value with
> each call?

Sure, I would normally call `vertical-motion' from `beginning-of-line'
(that is, I'm currently calling it from the first character on the line,
because that's faster), so I could pass in 0 as the X coordinate.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rudalics@gmx.at, emacs-devel@gnu.org
> Date: Mon, 09 Feb 2015 15:12:19 +1100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I think I can make it faster if we add an additional optional argument
> > that tells vertical-motion the X coordinate at point.  vertical-motion
> > invests a lot of cycles in finding a place whose X coordinate is
> > known, so providing that for its starting point will avoid all that
> > effort.  Will this help you, i.e. can you provide such a value with
> > each call?
> 
> Sure, I would normally call `vertical-motion' from `beginning-of-line'
> (that is, I'm currently calling it from the first character on the line,
> because that's faster), so I could pass in 0 as the X coordinate.

Please try the latest master, where I implemented that.

Contrary to what I wrote above, I decided to interpret the additional
argument in the same units as the COLS element of the cons cell that
is the first argument.  I think this is better for consistency and
also more convenient, since both values are in the same units and both
can be floats, so no accuracy is lost.

I'd be interested to know if this produces a noticeable speed-up.

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Please try the latest master, where I implemented that.

Great!

> Contrary to what I wrote above, I decided to interpret the additional
> argument in the same units as the COLS element of the cons cell that
> is the first argument.  I think this is better for consistency and
> also more convenient, since both values are in the same units and both
> can be floats, so no accuracy is lost.
>
> I'd be interested to know if this produces a noticeable speed-up.

I did the measurements with this (over a variable-width text buffer):

(benchmark-run 5000 (let ((start (point))) (vertical-motion (cons (/ 500 (frame-char-width)) 0) nil 0) (goto-char start)))

Without the column parameter, and starting from beginning-of-line, it
takes 1s.  Starting from the first characters, it takes 0.6s.

With the column parameter, and starting from beginning-of-line, it takes
0.2s.  So improvement makes it about 3x faster than the previous
best-case scenario, and 5x faster than the worst-case scenario (in this
simple test).  :-)

The major single component that takes time when figuring out
multi-column layouts now is the calls to

	(save-window-excursion
	  (set-window-buffer nil (current-buffer))
          (window-text-pixel-size ...))

If something could be done to make it faster to find out the (maximum)
pixel width of a (possibly undisplayed) buffer, that would be a great
win, I think.

I'll apply the shr branch stuff to the trunk later today and post some
new ELP traces with the new `vertical-motion' speedups.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I'll apply the shr branch stuff to the trunk later today and post some
> new ELP traces with the new `vertical-motion' speedups.

Here's the output when not giving the col number to `vertical-motion',
and starting from the second character on the line:

shr-insert-document                  1           2.545489884   2.545489884
shr-pixel-buffer-width               1018        0.7405463030  0.0007274521
shr-vertical-motion                  1993        0.2620109859  0.0001314656

Here's when giving a 0 argument, and starting from `beginning-of-line':

shr-insert-document                  1           4.449059093   4.449059093
shr-vertical-motion                  1993        2.1624376150  0.0010850163
shr-pixel-buffer-width               1018        0.7389822159  0.0007259157

So this is apparently 10x slower now?

Uhm...

Could there be an issue with the new code being slower when the buffer
in question isn't being displayed?

These are the two versions of the function being used, with the first
one being 10x slower than the second.

(defun shr-vertical-motion (column)
  (if (not shr-use-fonts)
      (move-to-column column)
    (beginning-of-line)
    (vertical-motion (cons (/ column (frame-char-width)) 0) nil 0)
    (unless (eolp)
      (forward-char 1))))

(defun shr-vertical-motion (column)
  (if (not shr-use-fonts)
      (move-to-column column)
    (beginning-of-line)
    (unless (eolp)
      (forward-char 1))
    (vertical-motion (cons (/ column (frame-char-width)) 0))
    (unless (eolp)
      (forward-char 1))))



-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Could there be an issue with the new code being slower when the buffer
> in question isn't being displayed?

Apparently not.  I wrapped it all in

    (save-window-excursion
      (set-window-buffer nil (current-buffer))

and I'm getting the same results.  Which doesn't tally up with my
benchmark-runs at all.  Very strange.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Anyway, it's all now in the trunk and should...  probably work:

http://lars.ingebrigtsen.no/2015/02/10/eww-now-with-fonts/

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rudalics@gmx.at, emacs-devel@gnu.org
> Date: Tue, 10 Feb 2015 17:07:33 +1100
> 
> Here's the output when not giving the col number to `vertical-motion',
> and starting from the second character on the line:
> 
> shr-insert-document                  1           2.545489884   2.545489884
> shr-pixel-buffer-width               1018        0.7405463030  0.0007274521
> shr-vertical-motion                  1993        0.2620109859  0.0001314656
> 
> Here's when giving a 0 argument, and starting from `beginning-of-line':
> 
> shr-insert-document                  1           4.449059093   4.449059093
> shr-vertical-motion                  1993        2.1624376150  0.0010850163
> shr-pixel-buffer-width               1018        0.7389822159  0.0007259157
> 
> So this is apparently 10x slower now?

That's simply impossible, since the new code _bypasses_ several calls
to display-engine functions, when the 3rd argument is provided, and
doesn't do anything in addition except trivial things like
extract_float and a couple of variable assignments.  All it does is
the last part of the function: move to a certain X coordinate.  A
partial job cannot possibly take longer than a full job.

So I tend not to believe to these timings.

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> That's simply impossible, since the new code _bypasses_ several calls
> to display-engine functions, when the 3rd argument is provided, and
> doesn't do anything in addition except trivial things like
> extract_float and a couple of variable assignments.  All it does is
> the last part of the function: move to a certain X coordinate.  A
> partial job cannot possibly take longer than a full job.
>
> So I tend not to believe to these timings.

Yeah, I don't understand it either, but I don't see what I'm doing
wrong.  Hm...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rudalics@gmx.at, emacs-devel@gnu.org
> Date: Tue, 10 Feb 2015 15:48:15 +1100
> 
> > I'd be interested to know if this produces a noticeable speed-up.
> 
> I did the measurements with this (over a variable-width text buffer):
> 
> (benchmark-run 5000 (let ((start (point))) (vertical-motion (cons (/ 500 (frame-char-width)) 0) nil 0) (goto-char start)))
> 
> Without the column parameter, and starting from beginning-of-line, it
> takes 1s.  Starting from the first characters, it takes 0.6s.
> 
> With the column parameter, and starting from beginning-of-line, it takes
> 0.2s.  So improvement makes it about 3x faster than the previous
> best-case scenario, and 5x faster than the worst-case scenario (in this
> simple test).  :-)

Thanks, these speed-ups match my expectations, given the amount of
code I bypass when this new argument is provided.

> The major single component that takes time when figuring out
> multi-column layouts now is the calls to
> 
> 	(save-window-excursion
> 	  (set-window-buffer nil (current-buffer))
>           (window-text-pixel-size ...))
> 
> If something could be done to make it faster to find out the (maximum)
> pixel width of a (possibly undisplayed) buffer, that would be a great
> win, I think.

Well, you insert those lines into the buffer yourself, right?  Can you
find the longest line while you are at it, and insert only it?
AFAICS, that's the only way of making window-text-pixel-size do its
job faster: by making the region FROM..TO it needs to traverse
smaller.

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Well, you insert those lines into the buffer yourself, right?  Can you
> find the longest line while you are at it, and insert only it?
> AFAICS, that's the only way of making window-text-pixel-size do its
> job faster: by making the region FROM..TO it needs to traverse
> smaller.

Well, I don't know how long the line is (in pixels) in the first
place.  Only calling `window-text-pixel-size' will tell me, right?

There's once special case I do know the width of the <td> element, and
that's when the only element in the <td> is another <table>.  I already
know the width of that table, to I wouldn't really have to recompute the
width again.

It's not an unusual case, but I don't think there would be much of a
speed up in total, though.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rudalics@gmx.at, emacs-devel@gnu.org
> Date: Wed, 11 Feb 2015 16:31:11 +1100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Well, you insert those lines into the buffer yourself, right?  Can you
> > find the longest line while you are at it, and insert only it?
> > AFAICS, that's the only way of making window-text-pixel-size do its
> > job faster: by making the region FROM..TO it needs to traverse
> > smaller.
> 
> Well, I don't know how long the line is (in pixels) in the first
> place.  Only calling `window-text-pixel-size' will tell me, right?

No, I meant the longest in characters.

> It's not an unusual case, but I don't think there would be much of a
> speed up in total, though.

It depends on how many lines at a time, on the average, you are
inserting in a temp buffer.

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> > Well, you insert those lines into the buffer yourself, right?  Can you
>> > find the longest line while you are at it, and insert only it?
>> > AFAICS, that's the only way of making window-text-pixel-size do its
>> > job faster: by making the region FROM..TO it needs to traverse
>> > smaller.
>> 
>> Well, I don't know how long the line is (in pixels) in the first
>> place.  Only calling `window-text-pixel-size' will tell me, right?
>
> No, I meant the longest in characters.

The line longest in characters doesn't really say much about what line
is longest in pixels.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: rudalics@gmx.at,  emacs-devel@gnu.org
> Date: Wed, 18 Feb 2015 12:17:57 +1100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> > Well, you insert those lines into the buffer yourself, right?  Can you
> >> > find the longest line while you are at it, and insert only it?
> >> > AFAICS, that's the only way of making window-text-pixel-size do its
> >> > job faster: by making the region FROM..TO it needs to traverse
> >> > smaller.
> >> 
> >> Well, I don't know how long the line is (in pixels) in the first
> >> place.  Only calling `window-text-pixel-size' will tell me, right?
> >
> > No, I meant the longest in characters.
> 
> The line longest in characters doesn't really say much about what line
> is longest in pixels.

I beg to differ.

Re: Pixel-based display functions

[Stefan Monnier]

>> The line longest in characters doesn't really say much about what line
>> is longest in pixels.
> I beg to differ.

They are correlated but I agree with Lars that the correlation is not
strong enough that one can simply take the longest-line-in-chars,
measure it, and expect the result to be the longer in pixel than all
other lines.

A long line of "llllllll" can be significantly shorter in pixels than
a slightly-less-long line of "mmmmmmm" if you use something
like Helvetica.

And a long line of "mmmmm" can be significantly shorter than a much
shorter line of "lllll" if the line of "lllll" uses a much larger face
or has an embedded after-string, or ...


        Stefan

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Lars Ingebrigtsen <larsi@gnus.org>,  rudalics@gmx.at,  emacs-devel@gnu.org
> Date: Wed, 18 Feb 2015 00:53:15 -0500
> 
> >> The line longest in characters doesn't really say much about what line
> >> is longest in pixels.
> > I beg to differ.
> 
> They are correlated but I agree with Lars that the correlation is not
> strong enough that one can simply take the longest-line-in-chars,
> measure it, and expect the result to be the longer in pixel than all
> other lines.
> 
> A long line of "llllllll" can be significantly shorter in pixels than
> a slightly-less-long line of "mmmmmmm" if you use something
> like Helvetica.
> 
> And a long line of "mmmmm" can be significantly shorter than a much
> shorter line of "lllll" if the line of "lllll" uses a much larger face
> or has an embedded after-string, or ...

Granted, I knew that.

I never said this was simple, or that the longest line in character
units is the only candidate.  I suggested to use the character-unit
length of the lines inserted into the temporary buffer as a criterion
for finding a subset of lines that is large enough to estimate the
pixel width of the whole chunk of text, and yet small enough to slash
the time needed for window-text-pixel-size to do its job.

IOW, I suggested to find a heuristic that would allow to invoke
window-text-pixel-size on a shorter chunk of text.

Maybe that's unworkable (I didn't try to develop the idea far enough
to see if it is), but that's the only idea I had, take it or leave it.

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> IOW, I suggested to find a heuristic that would allow to invoke
> window-text-pixel-size on a shorter chunk of text.
>
> Maybe that's unworkable (I didn't try to develop the idea far enough
> to see if it is), but that's the only idea I had, take it or leave it.

Well, it does seem unworkable, which is what I said.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Stefan Monnier]

> shr inserts text as long lines into the buffer, and I then use
> `vertical-motion' to go to the desired width, and then backtracking
> finding a fill point using the kinsuko algorithm.  Then a newline is
> inserted, and we repeat until the line is completely filled.

That sounds OK.  It should end up considering an amount of text O(N),
with very little wasted work (the only wasted work is on the text before
fill-column but after the chosen fill point).

I'm not sure if vertical-column might waste some extra time on the
vertical movement (which you don't need), but other than that, it should
be about as fast as you can get.

IOW, if it's not fast enough, your next best chance is lazyness ;-)


        Stefan

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: Eli Zaretskii <eliz@gnu.org>, rudalics@gmx.at, emacs-devel@gnu.org
> Date: Fri, 06 Feb 2015 10:18:00 -0500
> 
> I'm not sure if vertical-column might waste some extra time on the
> vertical movement

It doesn't, when called with LINES equal to zero.

Re: Pixel-based display functions

[martin rudalics]

 > And if this is still too complicated for you, simply give
 > `window-text-pixel-size' an additional argument ...

`window-text-pixel-size' now _has_ an optional BUFFER argument on
trunk/master.

martin

Re: Pixel-based display functions

[Lars Ingebrigtsen]

martin rudalics <rudalics@gmx.at> writes:

> `window-text-pixel-size' now _has_ an optional BUFFER argument on
> trunk/master.

Great; I'll try it out when I'm back in Sydney in a couple of days.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> martin rudalics <rudalics@gmx.at> writes:
>
>> `window-text-pixel-size' now _has_ an optional BUFFER argument on
>> trunk/master.
>
> Great; I'll try it out when I'm back in Sydney in a couple of days.

Evaling the following:

(progn
  (switch-to-buffer "empty")
  (with-temp-buffer
    (dotimes (i 100) (insert "hello"))
    (window-text-pixel-size nil 1 100 nil nil nil (current-buffer))))

Gives me this backtrace:

Debugger entered--Lisp error: (args-out-of-range 81)
  window-text-pixel-size(nil 1 100 nil nil nil #<buffer  *temp*>)
  (progn (let ((--dotimes-limit-- 100) (i 0)) (while (< i --dotimes-limit--) (insert "hello") (setq i (1+ i)))) (window-text-pixel-size nil 1 100 nil nil nil (current-buffer)))
  (unwind-protect (progn (let ((--dotimes-limit-- 100) (i 0)) (while (< i --dotimes-limit--) (insert "hello") (setq i (1+ i)))) (window-text-pixel-size nil 1 100 nil nil nil (current-buffer))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))
  (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (let ((--dotimes-limit-- 100) (i 0)) (while (< i --dotimes-limit--) (insert "hello") (setq i (1+ i)))) (window-text-pixel-size nil 1 100 nil nil nil (current-buffer))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))
  (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (let ((--dotimes-limit-- 100) (i 0)) (while (< i --dotimes-limit--) (insert "hello") (setq i (1+ i)))) (window-text-pixel-size nil 1 100 nil nil nil (current-buffer))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))
  (progn (switch-to-buffer "empty") (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (let ((--dotimes-limit-- 100) (i 0)) (while (< i --dotimes-limit--) (insert "hello") (setq i ...))) (window-text-pixel-size nil 1 100 nil nil nil (current-buffer))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))))
  eval((progn (switch-to-buffer "empty") (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (let ((--dotimes-limit-- 100) (i 0)) (while (< i --dotimes-limit--) (insert "hello") (setq i ...))) (window-text-pixel-size nil 1 100 nil nil nil (current-buffer))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))) nil)
  elisp--eval-last-sexp(nil)
  eval-last-sexp(nil)
  funcall-interactively(eval-last-sexp nil)
  call-interactively(eval-last-sexp nil nil)
  command-execute(eval-last-sexp)


-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[martin rudalics]

 > Evaling the following:
 >
 > (progn
 >    (switch-to-buffer "empty")
 >    (with-temp-buffer
 >      (dotimes (i 100) (insert "hello"))
 >      (window-text-pixel-size nil 1 100 nil nil nil (current-buffer))))
 >
 > Gives me this backtrace:
 >
 > Debugger entered--Lisp error: (args-out-of-range 81)
 >    window-text-pixel-size(nil 1 100 nil nil nil #<buffer  *temp*>)

Indeed.  I was too optimistic.  The window's buffer is apparently
hard-coded into the iterator in so many ways that this can't be fixed
reasonably.  Just look for all the it->object = it->w->contents
assignments.  It's sheer luck that this didn't crash more seriously.

I have to back out my change, sorry.  And if you want to
use `window-text-pixel-size' you'll have to do the saving stuff I
proposed earlier.

Thanks for catching this, martin

Re: Pixel-based display functions

[Eli Zaretskii]

> Date: Thu, 05 Feb 2015 10:38:52 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: Eli Zaretskii <eliz@gnu.org>, 
>  Stefan Monnier <monnier@iro.umontreal.ca>,
>  emacs-devel@gnu.org
> 
>  > Evaling the following:
>  >
>  > (progn
>  >    (switch-to-buffer "empty")
>  >    (with-temp-buffer
>  >      (dotimes (i 100) (insert "hello"))
>  >      (window-text-pixel-size nil 1 100 nil nil nil (current-buffer))))
>  >
>  > Gives me this backtrace:
>  >
>  > Debugger entered--Lisp error: (args-out-of-range 81)
>  >    window-text-pixel-size(nil 1 100 nil nil nil #<buffer  *temp*>)
> 
> Indeed.  I was too optimistic.  The window's buffer is apparently
> hard-coded into the iterator in so many ways that this can't be fixed
> reasonably.  Just look for all the it->object = it->w->contents
> assignments.  It's sheer luck that this didn't crash more seriously.

Perhaps you could use the technique similar to what vertical-motion
does near its beginning.

Re: Pixel-based display functions

[martin rudalics]

 > Perhaps you could use the technique similar to what vertical-motion
 > does near its beginning.

We can do that.  But it's hardly cheaper than

(save-window-excursion
   (set-window-buffer nil ...)
   (window-text-pixel-size ...)
   ...)

especially when packing multiple `window-text-pixel-size' calls into
the same excursion.

martin

Re: Pixel-based display functions

[Lars Ingebrigtsen]

martin rudalics <rudalics@gmx.at> writes:

> We can do that.  But it's hardly cheaper than
>
> (save-window-excursion
>   (set-window-buffer nil ...)
>   (window-text-pixel-size ...)
>   ...)
>
> especially when packing multiple `window-text-pixel-size' calls into
> the same excursion.

Ok; I can try doing that and see what the results are.

Another possible interface function just occurred to me, though.  :-)
(Many ways to skin a ... fill...)

I haven't looked closely at the code, but I think in all (or most)
instances where I need to know the pixel width I ended up with in a
column, what I really need to know is the width of the longest line.

So a `window-text-pixel-width' function that loops over all the lines in
the specified region, and return the maximum width, might be what I
really need here.  Hm...  Yeah, I think that might be a fast and handy
function?

It doesn't really solve the problem of `vertical-motion' being too slow,
really, when finding fill points in large HTML documents.

Hm...  so many approaches...

Hm.  A `window-text-glyphs-pixel-widths' that returns all the pixel
widths for all the glyphs in the region, segmented by line?  That would
avoid calling `vertical-motion' at all, but may not be any faster than
calling `vertical-motion'.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> We can do that.  But it's hardly cheaper than
>>
>> (save-window-excursion
>>   (set-window-buffer nil ...)
>>   (window-text-pixel-size ...)
>>   ...)
>>
>> especially when packing multiple `window-text-pixel-size' calls into
>> the same excursion.
>
> Ok; I can try doing that and see what the results are.

With complex table-based layouts, this was unbearably slow.  I'll try to
hack up a version of the function that returns the max-width in a region
and see whether that gives acceptable results.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, monnier@iro.umontreal.ca,
>         emacs-devel@gnu.org
> Date: Fri, 06 Feb 2015 20:58:23 +1100
> 
> So a `window-text-pixel-width' function that loops over all the lines in
> the specified region, and return the maximum width, might be what I
> really need here.

Isn't that what window-text-pixel-width already does, given FROM..TO
that span multiple lines?

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Isn't that what window-text-pixel-width already does, given FROM..TO
> that span multiple lines?

Er.  So it does; thanks.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Matthew Carter]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> Isn't that what window-text-pixel-width already does, given FROM..TO
>> that span multiple lines?
>
> Er.  So it does; thanks.

Hi Lars,

A little off-topic but I notice you have both 'w' and 'u' mapped to
'shr-copy-url inside of the shr-map in shr.el.

This interferes with evil-mode's 'w' binding (to move forward a word at
a time) as the shr-map seems to be dynamically applied to page links and
can't be overwritten or bound over (unless I'm just doing it wrong, but
I've tried many mode map changes).

Is it necessary to keep the 'w' binding in this shr-map when 'u' is
available?

Or has anyone successfully kept their own key bind when shr tries to
remap the keys when the cursor is on top of a link in eww-mode?

-- 
Matthew Carter (m@ahungry.com)
http://ahungry.com

Rebinding local keys

[Alan Schmitt]

[-- Attachment #1: Type: text/plain, Size: 600 bytes --]

Hello Matthew,

On 2015-02-06 10:29, Matthew Carter <m@ahungry.com> writes:

> Or has anyone successfully kept their own key bind when shr tries to
> remap the keys when the cursor is on top of a link in eww-mode?

I've had a similar issues with gnus, and after a long discussion (see
https://emacs.stackexchange.com/questions/653/how-can-i-find-out-in-which-keymap-a-key-is-bound)
I found the keymap where the key was bound and disabled it there. There
is some code there that helps in finding where the key is bound.

Hope this helps,

Alan

-- 
OpenPGP Key ID : 040D0A3B4ED2E5C7

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 494 bytes --]

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Matthew Carter <m@ahungry.com> writes:

> A little off-topic but I notice you have both 'w' and 'u' mapped to
> 'shr-copy-url inside of the shr-map in shr.el.
>
> This interferes with evil-mode's 'w' binding (to move forward a word at
> a time) as the shr-map seems to be dynamically applied to page links and
> can't be overwritten or bound over (unless I'm just doing it wrong, but
> I've tried many mode map changes).
>
> Is it necessary to keep the 'w' binding in this shr-map when 'u' is
> available?

The `u' binding is a shr/Gnus legacy binding.  The `w' binding is used
throughout Emacs to copy the URL/thing under point to the kill ring.

The `u' binding will go away eventually.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: larsi@gnus.org,  emacs-devel@gnu.org
> Date: Sat, 31 Jan 2015 15:33:11 -0500
> 
> > Lars doesn't want to compute pixel sizes to repeat what the display
> > engine does.  He wants to do that to implement stuff that the display
> > engine cannot, at least currently (if it ever should).
> 
> AFAICT he wants to compute the pixel length of a chunk of text.
> I don't see why we can't export to Elisp a C function that does that
> re-using the same machinery used by the redisplay (and by vertical-motion).

Those functions are already exposed, and that's exactly what Lars is
doing, AFAIU.

Or maybe I don't understand which functions you want to expose, so
please elaborate.

Re: Pixel-based display functions

[Stefan Monnier]

> Or maybe I don't understand which functions you want to expose, so
> please elaborate.

I just don't want Elisp code to compute pixel sizes based on
glyph info.  It's what the C code already does.  So whenever Elisp code
tries to do that, it's a mistake (not so much because we should do it
in C, but because we end up having to reproduce in new code what
existing code already does, which is exactly what's going on with this
whole discussion about guessing which chunks of text will use the same
font and how we should handle chars which aren't covered by the font
etc...).

IIUC in the case at hand, the function which Lars needs is slightly
different from what we already expose.  Instead of
pixel-size-of-chunk-of-text, we want to have position-of-pixel-in-text
(i.e. you pass a chunk of text, along with a pixel position, and it
returns the position of that pixel in the text).  I.e. something more
like vertical-motion (which receives a pixel horizontal coordinate and
figures out where that is in the text).


        Stefan

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: larsi@gnus.org,  emacs-devel@gnu.org
> Date: Sun, 01 Feb 2015 01:28:16 -0500
> 
> > Or maybe I don't understand which functions you want to expose, so
> > please elaborate.
> 
> I just don't want Elisp code to compute pixel sizes based on
> glyph info.  It's what the C code already does.  So whenever Elisp code
> tries to do that, it's a mistake (not so much because we should do it
> in C, but because we end up having to reproduce in new code what
> existing code already does, which is exactly what's going on with this
> whole discussion about guessing which chunks of text will use the same
> font and how we should handle chars which aren't covered by the font
> etc...).

I don't think Lars's Elisp does what the display engine does.  It's
close, but different enough to warrant a different approach.  See
below.

> IIUC in the case at hand, the function which Lars needs is slightly
> different from what we already expose.  Instead of
> pixel-size-of-chunk-of-text, we want to have position-of-pixel-in-text
> (i.e. you pass a chunk of text, along with a pixel position, and it
> returns the position of that pixel in the text).  I.e. something more
> like vertical-motion (which receives a pixel horizontal coordinate and
> figures out where that is in the text).

I explicitly mentioned vertical-motion in one of my previous messages,
and explained there how to use it to find buffer position that
corresponds to a given pixel coordinate.  Lars said it wasn't what he
needed, I don't know why.  I assume he will be able to explain that.
If he can use vertical-motion, that's fine with me.  If not, your
position-of-pixel-in-text suggestion will suffer from the same
problem.

I also briefly considered writing a primitive such as
position-of-pixel-in-text, but eventually decided against it, because
I don't think that's what Lars needs.  Here's why.

First, Lars has no "text" in the sense that there's no buffer text to
render.  What he has is a _spec_ for the layout, in the form of HTML.
That spec provides a bunch of strings that need to be rendered under
certain constraints, but these strings are not inserted into any
buffer by the time the code we are discussing runs, and the move_it_*
functions we use in vertical-motion and elsewhere won't help us,
because they do need a buffer to iterate over.

(Of course, he could insert each string into a scratch buffer, but
that's a waste, and doesn't solve the other problem, described below.)

Second, there's a subtlety in move_it_* functions that was never
explicitly raised in this discussion, but which becomes rather
important if you want to consider reusing the move_it_* functions for
this use case: they produce glyphs in the _visual_ order.  So if the
text to be rendered includes R2L characters, you might break text
between screen lines incorrectly.  (If this isn't clear enough, I can
elaborate.)  Also, you will have to deal with the situation where
(position-of-pixel-in-text FROM TO) is _less_ than
(position-of-pixel-in-text FROM (1- TO)).  That is a complication that
Lars would surely like to avoid, I presume.  By contrast,
font-get-glyphs works in the logical order.

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> I explicitly mentioned vertical-motion in one of my previous messages,
> and explained there how to use it to find buffer position that
> corresponds to a given pixel coordinate.  Lars said it wasn't what he
> needed, I don't know why.

I think I asked whether it was another one of those functions that only
report the position after redisplay has taken place, and if so, it's not
usable for this use case.

If it does report the position before redisplay has happened, that would
be very helpful.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  emacs-devel@gnu.org
> Date: Mon, 02 Feb 2015 20:46:51 +1100
> 
> > I explicitly mentioned vertical-motion in one of my previous messages,
> > and explained there how to use it to find buffer position that
> > corresponds to a given pixel coordinate.  Lars said it wasn't what he
> > needed, I don't know why.
> 
> I think I asked whether it was another one of those functions that only
> report the position after redisplay has taken place, and if so, it's not
> usable for this use case.
> 
> If it does report the position before redisplay has happened

It does.

In general, very few functions require an up-to-date display for doing
their job; those that do have that fact stated in their doc strings.

I hope you noticed my comments about the visual vs logical order:
vertical-motion is one of the functions that work in visual order.

Re: Pixel-based display functions

[Stefan Monnier]

>> I just don't want Elisp code to compute pixel sizes based on
>> glyph info.  It's what the C code already does.  So whenever Elisp code
>> tries to do that, it's a mistake (not so much because we should do it
>> in C, but because we end up having to reproduce in new code what
>> existing code already does, which is exactly what's going on with this
>> whole discussion about guessing which chunks of text will use the same
>> font and how we should handle chars which aren't covered by the font
>> etc...).

> I don't think Lars's Elisp does what the display engine does.

All I know is that his code will give wrong results if it doesn't
reproduce faithfully enough what the redisplay does.  And that to me is
a clear sign that it should reuse (some part of) the redisplay code,
since it's the simpler and most reliable way to make sure that the
behavior is faithful.

> (Of course, he could insert each string into a scratch buffer, but
> that's a waste, and doesn't solve the other problem, described below.)

I think this "waste" would be negligible.

> Second, there's a subtlety in move_it_* functions that was never
> explicitly raised in this discussion, but which becomes rather
> important if you want to consider reusing the move_it_* functions for
> this use case: they produce glyphs in the _visual_ order.  So if the
> text to be rendered includes R2L characters, you might break text
> between screen lines incorrectly.  (If this isn't clear enough, I can
> elaborate.)  Also, you will have to deal with the situation where
> (position-of-pixel-in-text FROM TO) is _less_ than
> (position-of-pixel-in-text FROM (1- TO)).  That is a complication that
> Lars would surely like to avoid, I presume.  By contrast,
> font-get-glyphs works in the logical order.

I think solving this problem is *hard*.  The problem is not just whether
you work on the logical or visual order, but in order to get the right
behavior when filling bidi text inside columns, is that you'll have to
partly reimplement the bidi code.  Maybe the best approach would be to
make position-of-pixel-in-text return some info about the reordering
that redisplay performs.


        Stefan


PS: who still would prefer refilling single-column text in the redisplay
itself, since it makes it lazy, and makes it work right even when the
buffer is displayed in different windows/frames with different fonts.

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: larsi@gnus.org,  emacs-devel@gnu.org
> Date: Mon, 02 Feb 2015 11:43:23 -0500
> 
> > I don't think Lars's Elisp does what the display engine does.
> 
> All I know is that his code will give wrong results if it doesn't
> reproduce faithfully enough what the redisplay does.

Not necessarily, since an HTML browser doesn't need to implement all
the gazillion display features that redisplay needs to handle.  Not
even close.  It just needs to render text with faces and sometimes
with images.

> And that to me is a clear sign that it should reuse (some part of)
> the redisplay code

I agree, and it does.  font-get-glyphs, window-text-pixel-size,
vertical-motion, etc. all reuse the display engine code.

> > (Of course, he could insert each string into a scratch buffer, but
> > that's a waste, and doesn't solve the other problem, described below.)
> 
> I think this "waste" would be negligible.

Maybe, maybe not.  It all depends on how many small strings will be
needed to be rendered.

> > Second, there's a subtlety in move_it_* functions that was never
> > explicitly raised in this discussion, but which becomes rather
> > important if you want to consider reusing the move_it_* functions for
> > this use case: they produce glyphs in the _visual_ order.  So if the
> > text to be rendered includes R2L characters, you might break text
> > between screen lines incorrectly.  (If this isn't clear enough, I can
> > elaborate.)  Also, you will have to deal with the situation where
> > (position-of-pixel-in-text FROM TO) is _less_ than
> > (position-of-pixel-in-text FROM (1- TO)).  That is a complication that
> > Lars would surely like to avoid, I presume.  By contrast,
> > font-get-glyphs works in the logical order.
> 
> I think solving this problem is *hard*.

I explicitly suggested a way that avoids the need to solve it.

> The problem is not just whether you work on the logical or visual
> order, but in order to get the right behavior when filling bidi text
> inside columns, is that you'll have to partly reimplement the bidi
> code.

Not really.  Certainly not when the screen lines are L2R and there's
some R2L text in it.  R2L lines are slightly harder, in that you need
to append to such lines a 'space' display property of a suitably
calculated width, to make them flushed to the right.  What other
issues do you envision?  And what do you think the display engine does
to fill bidi text, anyway?

> Maybe the best approach would be to make position-of-pixel-in-text
> return some info about the reordering that redisplay performs.

What information would that be?  Why would an application be
interested in it?

> PS: who still would prefer refilling single-column text in the redisplay
> itself, since it makes it lazy, and makes it work right even when the
> buffer is displayed in different windows/frames with different fonts.

I agree that this feature would be useful.  However, how to use it in
this case is not clear to me: recall that there's no buffer to render
here, only a bunch of strings, each one with some kind of
specification, lifted from the HTML source, regarding their layout in
table cells.  The display engine cannot work with such kind of layout
spec, we'd need to invent some new machinery on top of teaching it how
to wrap at arbitrary pixel coordinate.

Re: Pixel-based display functions

[Stefan Monnier]

> Not necessarily, since an HTML browser doesn't need to implement all
> the gazillion display features that redisplay needs to handle.  Not
> even close.  It just needs to render text with faces and sometimes
> with images.

Right, it only needs to faithfully model the parts that it uses.
That's still a lot of needless and hard work.

> I agree, and it does.  font-get-glyphs, window-text-pixel-size,
> vertical-motion, etc. all reuse the display engine code.

Right.  So we should do our best to make sure this same approach can be
used here.

>> > Second, there's a subtlety in move_it_* functions that was never
>> > explicitly raised in this discussion, but which becomes rather
>> > important if you want to consider reusing the move_it_* functions for
>> > this use case: they produce glyphs in the _visual_ order.  So if the
>> > text to be rendered includes R2L characters, you might break text
>> > between screen lines incorrectly.  (If this isn't clear enough, I can
>> > elaborate.)  Also, you will have to deal with the situation where
>> > (position-of-pixel-in-text FROM TO) is _less_ than
>> > (position-of-pixel-in-text FROM (1- TO)).  That is a complication that
>> > Lars would surely like to avoid, I presume.  By contrast,
>> > font-get-glyphs works in the logical order.
>> I think solving this problem is *hard*.
> I explicitly suggested a way that avoids the need to solve it.

Hmm.. maybe you're right that doing it in the logical order will work.

>> PS: who still would prefer refilling single-column text in the redisplay
>> itself, since it makes it lazy, and makes it work right even when the
>> buffer is displayed in different windows/frames with different fonts.
> I agree that this feature would be useful.  However, how to use it in
> this case is not clear to me: recall that there's no buffer to render
> here,

Hmm... I don't follow.  In the single-column case, SHR can generate the
buffer text directly without worrying about line-wrapping (just setting
up the redisplay line-wrapping according to the expected dimansions of
the column).


        Stefan

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Not really the same, but how about using
>
>   (aref char-script-table (char-after POS))
>
> to find where in the text you have characters from a different script?
> If this is fast enough, it should give you a conservative estimation
> of where to call font-at again.  (It's conservative because the same
> font can, and usually does, cover more than a single script.)

That's a very good idea; thanks.  I'll try using that as a segmenting
function and see whether this thing ends up being fast enough...

> One more issue that I think needs to be handled are characters for
> which there's no font installed on the user's system.  We display them
> as defined by glyphless-char-display-control.  Or maybe you should
> allow the text to overflow in this case: after all, it's not really
> legible, is it?

I stumbled upon this issue a couple of days ago, I think.  It was a
utf-8-encoded web page that had one iso-8859-1 character in it, and
`font-get-glyphs' returned nil for that "character" (which was displayed
as \241 or something).  But these should have a pretty predictable
width, probably.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Sat, 31 Jan 2015 11:42:51 +1100
> 
> > One more issue that I think needs to be handled are characters for
> > which there's no font installed on the user's system.  We display them
> > as defined by glyphless-char-display-control.  Or maybe you should
> > allow the text to overflow in this case: after all, it's not really
> > legible, is it?
> 
> I stumbled upon this issue a couple of days ago, I think.  It was a
> utf-8-encoded web page that had one iso-8859-1 character in it, and
> `font-get-glyphs' returned nil for that "character" (which was displayed
> as \241 or something).  But these should have a pretty predictable
> width, probably.

I didn't mean \241, I meant the characters whose UTF-8 encoding is
longer than one byte.  These are displayed as a box with the Unicode
codepoint in hex inside it.  The size of that is also predictable, but
font-at will return nil for them.

Re: Pixel-based display functions

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> Not really the same, but how about using
>
>   (aref char-script-table (char-after POS))
>
> to find where in the text you have characters from a different script?

It isn't guaranteed that all characters from a script are rendered from
the same font.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: larsi@gnus.org,  emacs-devel@gnu.org
> Date: Sat, 31 Jan 2015 10:04:28 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Not really the same, but how about using
> >
> >   (aref char-script-table (char-after POS))
> >
> > to find where in the text you have characters from a different script?
> 
> It isn't guaranteed that all characters from a script are rendered from
> the same font.

Can you give an example?

Re: Pixel-based display functions

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andreas Schwab <schwab@linux-m68k.org>
>> Cc: larsi@gnus.org,  emacs-devel@gnu.org
>> Date: Sat, 31 Jan 2015 10:04:28 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > Not really the same, but how about using
>> >
>> >   (aref char-script-table (char-after POS))
>> >
>> > to find where in the text you have characters from a different script?
>> 
>> It isn't guaranteed that all characters from a script are rendered from
>> the same font.
>
> Can you give an example?

set-fontset-font can arrange an arbitrary range come from a different
font.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: larsi@gnus.org,  emacs-devel@gnu.org
> Date: Sat, 31 Jan 2015 11:15:20 +0100
> 
> >> >   (aref char-script-table (char-after POS))
> >> >
> >> > to find where in the text you have characters from a different script?
> >> 
> >> It isn't guaranteed that all characters from a script are rendered from
> >> the same font.
> >
> > Can you give an example?
> 
> set-fontset-font can arrange an arbitrary range come from a different
> font.

Do people actually do such things?  If so, in what circumstances?

Re: Pixel-based display functions

[Alexis]

Eli Zaretskii writes:
 
>> set-fontset-font can arrange an arbitrary range come from a different
>> font.
>
> Do people actually do such things?  If so, in what circumstances?

i make use of this:

https://github.com/rolandwalker/unicode-fonts


Alexis.

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Alexis <flexibeast@gmail.com>
> Date: Sat, 31 Jan 2015 23:04:54 +1100
> 
> 
> Eli Zaretskii writes:
>  
> >> set-fontset-font can arrange an arbitrary range come from a different
> >> font.
> >
> > Do people actually do such things?  If so, in what circumstances?
> 
> i make use of this:
> 
> https://github.com/rolandwalker/unicode-fonts

Thanks.  However, that's a 5000-line file, so could you be more
specific -- where do you use there more than one font for the same
Unicode block, and why?

Re: Pixel-based display functions

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> Thanks.  However, that's a 5000-line file, so could you be more
> specific -- where do you use there more than one font for the same
> Unicode block, and why?

Even without set-fontset-font you can have various fonts for the same
script.  For example, ㇀ is displayed with "-unknown-AR PL UKai
CN-normal-normal-normal-*-14-*-*-*-*-0-iso10646-1", but 〿 uses
"-Efont-Efont Biwidth-normal-normal-normal-*-16-*-*-*-d-80-iso10646-1".
Both are cjk-misc.


Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: Alexis <flexibeast@gmail.com>,  emacs-devel@gnu.org
> Date: Sat, 31 Jan 2015 14:25:48 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Thanks.  However, that's a 5000-line file, so could you be more
> > specific -- where do you use there more than one font for the same
> > Unicode block, and why?
> 
> Even without set-fontset-font you can have various fonts for the same
> script.  For example, ㇀ is displayed with "-unknown-AR PL UKai
> CN-normal-normal-normal-*-14-*-*-*-*-0-iso10646-1", but 〿 uses
> "-Efont-Efont Biwidth-normal-normal-normal-*-16-*-*-*-d-80-iso10646-1".
> Both are cjk-misc.

OK, but if there's a small number of rarely-used scripts that are
exceptions from the rule, we could treat those exceptions specially,
by calling font-at there for every character.  If those are rare
enough, the slow-down will be insignificant, I think.

Re: Pixel-based display functions

[Alexis]

Eli Zaretskii writes:

>> From: Alexis <flexibeast@gmail.com>
>> Date: Sat, 31 Jan 2015 23:04:54 +1100
>> 
>> 
>> Eli Zaretskii writes:
>>  
>> >> set-fontset-font can arrange an arbitrary range come from a different
>> >> font.
>> >
>> > Do people actually do such things?  If so, in what circumstances?
>> 
>> i make use of this:
>> 
>> https://github.com/rolandwalker/unicode-fonts
>
> Thanks.  However, that's a 5000-line file, so could you be more
> specific -- where do you use there more than one font for the same
> Unicode block, and why?

i use Emacs both under X and in a terminal, depending on
circumstances. Under X, i use the Xft backend, which, if i understand
correctly, gives me access to fonts not available in a terminal. So if
i'm in a terminal, i'll want to use one font for a particular Unicode
block, but if i'm in X, i'll want to use another font which has 'better'
(i.e. more aesthetically pleasing, to me) glyphs. The `unicode-fonts`
package allows me to specify a list of fonts, ordered by priority
(highest-first), to be used for a particular Unicode block, depending on
contextual availability.

(Also, `unicode-fonts` allowed me to work around an issue i've had
where, under X, Emacs would select a rather ugly outline font - Linux
Biolinum Outline - for displaying Hebrew, rather than using Ezra
SIL. Thus, my init contains:

    (setcdr (assoc "Hebrew" unicode-fonts-block-font-mapping)
            '(("Ezra SIL" "Ezra SIL SR" "Arial Hebrew" "Raanana" "New Peninim MT" "Aharoni" "David" "FrankRuehl" "Gisha" "Levenim MT" "Narkisim" "Rod" "Cardo" "Courier New" "Adobe Hebrew" "Code2000" "Aramaic Imperial Yeb" "Microsoft Sans Serif" "Tahoma" "Lucida Sans Unicode" "Arial Unicode MS" "Arial" "Quivira" "Everson Mono:weight=boeld" "ALPHABETUM Unicode")))

i imagine there is some straightforward, native-to-Emacs, way of doing
this, but since `unicode-fonts` already does most of the legwork to
automatically make sure glyphs are available to display most codepoints,
i just decided to slightly modify the value of
`unicode-fonts-block-font-mapping`.)


Alexis.

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Alexis <flexibeast@gmail.com>
> Date: Sun, 01 Feb 2015 00:37:28 +1100
> 
> i use Emacs both under X and in a terminal, depending on
> circumstances. Under X, i use the Xft backend, which, if i understand
> correctly, gives me access to fonts not available in a terminal. So if
> i'm in a terminal, i'll want to use one font for a particular Unicode
> block, but if i'm in X, i'll want to use another font which has 'better'
> (i.e. more aesthetically pleasing, to me) glyphs. The `unicode-fonts`
> package allows me to specify a list of fonts, ordered by priority
> (highest-first), to be used for a particular Unicode block, depending on
> contextual availability.

AFAIU, each "Unicode block" corresponds to a script, with a couple of
exceptions, one of which is cjk-misc that Andreas mentioned.  So what
you describe above AFAIU does not yet mean you have more than one font
for the same block.

Re: Pixel-based display functions

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 219 bytes --]

> >
> > set-fontset-font can arrange an arbitrary range come from a different
> > font.
>
> Do people actually do such things?  If so, in what circumstances?
>

I use it with a nil range to set a fallback Unicode font.

[-- Attachment #2: Type: text/html, Size: 307 bytes --]

Re: Pixel-based display functions

[Eli Zaretskii]

> Date: Sat, 31 Jan 2015 12:23:51 -0200
> From: Artur Malabarba <bruce.connor.am@gmail.com>
> Cc: Andreas Schwab <schwab@linux-m68k.org>, emacs-devel <emacs-devel@gnu.org>, larsi@gnus.org
> 
> > > set-fontset-font can arrange an arbitrary range come from a different
> > > font.
> >
> > Do people actually do such things? If so, in what circumstances?
> >
> 
> I use it with a nil range to set a fallback Unicode font.

Thanks, I don't think fallbacks are a problem in this context.

Re: Pixel-based display functions

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > Do people actually do such things?  If so, in what circumstances?

Boldface and italic come immediately to mind. ;-)

Among math symbols people may prefer different versions of some of
them (cf TeX's \varphi, for example).  Large character sets like Han
(you may specify Japanese fonts because your audience is Japanese, but
if your text contains Chinese names for example, you may need to
borrow characters from a Chinese font).

Re: Pixel-based display functions

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: Andreas Schwab <schwab@linux-m68k.org>,
>     larsi@gnus.org,
>     emacs-devel@gnu.org
> Date: Sat, 31 Jan 2015 23:55:59 +0900
> 
> Eli Zaretskii writes:
> 
>  > Do people actually do such things?  If so, in what circumstances?
> 
> Boldface and italic come immediately to mind. ;-)

That's not a problem, since there will be a face property there.

> Among math symbols people may prefer different versions of some of
> them (cf TeX's \varphi, for example).

Not sure I understand the situation you describe here.

> Large character sets like Han (you may specify Japanese fonts
> because your audience is Japanese, but if your text contains Chinese
> names for example, you may need to borrow characters from a Chinese
> font).

I get quite a few spam mail with Japanese and Chinese characters in
them, but the former are always Kana, while the latter are Han.

Re: Pixel-based display functions

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > > Among math symbols people may prefer different versions of some of
 > > them (cf TeX's \varphi, for example).
 > 
 > Not sure I understand the situation you describe here.

Run the string "\phi \varphi \epsilon \varepsilon" through TeX.  I
don't recall which fonts I had the issue with but there were two TTF
fonts that were quite similar to each other which each had one of the
variants I liked and one I didn't like.  So I wanted to create a
combined font; it turned out it wasn't possible at the within-face
level in XEmacs.

 > > Large character sets like Han (you may specify Japanese fonts
 > > because your audience is Japanese, but if your text contains Chinese
 > > names for example, you may need to borrow characters from a Chinese
 > > font).
 > 
 > I get quite a few spam mail with Japanese and Chinese characters in
 > them, but the former are always Kana, while the latter are Han.

I can't speak to the spam you receive, and you needn't bother saving
any for me, but I assure you I regularly get mail in Japanese language
encoded as UTF-8 from Chinese, which all consoles I have must mix
fonts to display all characters[1] because all of the Japanese fonts I
use lack many Han, because they are not part of the JIS standard.  I
appreciate being able to choose the fallback fonts in XEmacs rather
than using fontconfig configs.

Incomplete fonts are less of an issue for Chinese fonts, but I doubt
Handa-san will be sympathetic to an argument which deprecates
Japanese. :-)  Be that as is may, AIUI font mixing is not so useful
for Chinese. At least "traditional" fonts have a repertoire which more
or less includes the Japanese repertoire up to glyph variant issues
like \phi vs. ^varphi, and both "traditional" and "simplified" Chinese
fonts seem to all have Japanese kana in them.  In fact since the GB
18030 standard does "#include <Unicode>", at least simplified Chinese
fonts with full coverage in that standard should have all Unicode
characters ... I don't know if there are full- coverage fonts, but I
suppose there are.

Footnotes: 
[1]  It turns out that Chinese often input characters that are
unstandardized cognates of standard Japanese characters, so I can
mostly read them, and sometimes they use Chinese words that "should"
(and may, for all I know) translate directly to Japanese "cognates".
But display engines are more literal-minded and can't make those
substitutions based on glyph shape.

Re: Pixel-based display functions

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Wed, 28 Jan 2015 14:27:17 +1100
> 
> shr needs three functions to be kind-of fast to work:
> 
> 1) A pixel equivalent of `current-column'

You already have it:

  (car (nth 2 (posn-at-point)))

> 2) A pixel equivalent of `move-to-column' (which will be imprecise, of
> course, but that's OK)

You already have it (albeit not necessarily where you'd look ;-):

  (vertical-motion (cons (/ PIXELS (frame-char-width)) 0))

> 3) A pixel equivalent of `string-width'

Didn't we just go through using font-get-glyphs for that?

Or just insert the string into the current buffer and use
posn-at-point to see how many pixels it took.

Do the above fix your problems?

Re: Pixel-based display functions

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Lars Ingebrigtsen <larsi@gnus.org>
>> Date: Wed, 28 Jan 2015 14:27:17 +1100
>> 
>> shr needs three functions to be kind-of fast to work:
>> 
>> 1) A pixel equivalent of `current-column'
>
> You already have it:
>
>   (car (nth 2 (posn-at-point)))

Isn't that one of those functions that says what redisplay has already
computed?  So it's kinda useless.

(with-temp-buffer
  (insert (propertize "hello" 'face 'variable-pitch))
  (car (nth 2 (posn-at-point))))
=> nil

>> 2) A pixel equivalent of `move-to-column' (which will be imprecise, of
>> course, but that's OK)
>
> You already have it (albeit not necessarily where you'd look ;-):
>
>   (vertical-motion (cons (/ PIXELS (frame-char-width)) 0))

Is that another one of those that only work after redisplay?

>> 3) A pixel equivalent of `string-width'
>
> Didn't we just go through using font-get-glyphs for that?

Yes, and it's too slow to use.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Then somebody will have to get cracking on making the Emacs display
  > engine more expressive so that that's possible, I guess?

  > (Yes, Emacs can display proportional fonts and fonts of different sizes,
  > but until you can fold (etc) proportional text (and text with a mixture
  > of font sizes) in a pretty manner, that's more of a toy than anything
  > else.)

This is a very important feature to add to Emacs.  It includes
being able to do width and position calculations in units finer
than characters.  Also, there should be a way to adjust intercharacter
spacing to make the trailing margin line up.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Juri Linkov]

> `M-x eww RET
> http://www.gnu.org/software/emacs/manual/html_node/emacs/index.html RET'
>
> and you're off.  The `n'/`p'/`u' commands work as in Info already, so
> the only thing that's missing is the index (which is trivial to add).

Another thing that's missing is a multi-page search like in Info.
It's trivial to add it with this small patch:

diff --git a/lisp/net/eww.el b/lisp/net/eww.el
index 9d787d3..74a0f4b 100644
--- a/lisp/net/eww.el
+++ b/lisp/net/eww.el
@@ -705,6 +705,8 @@ (define-derived-mode eww-mode nil "eww"
    (setq-local tool-bar-map eww-tool-bar-map))
   ;; desktop support
   (setq-local desktop-save-buffer 'eww-desktop-misc-data)
+  ;; multi-page isearch support
+  (setq-local multi-isearch-next-buffer-function 'eww-isearch-next-buffer)
   (buffer-disable-undo)
   (setq buffer-read-only t))
 
@@ -1884,6 +1886,17 @@ (add-to-list 'desktop-locals-to-save
 (add-to-list 'desktop-buffer-mode-handlers
              '(eww-mode . eww-restore-desktop))
 
+;;; Isearch support
+
+(defun eww-isearch-next-buffer (&optional buffer wrap)
+  "Go to the next page to search using `rel' attribute for navigation."
+  (if wrap
+      (eww-top-url)
+    (if isearch-forward
+	(eww-next-url)
+      (eww-previous-url)))
+  (current-buffer))
+
 (provide 'eww)
 
 ;;; eww.el ends here

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > 4. Emacs Lisp code to browse HTML-Info files.

  > I think this will be most difficult part, so better focus on this part
  > than on the #3 which should be a piece of cake once we have an Elisp
  > solution for the rendering.

I think #4 will be straightforward once #1-3 are done.  The crucial decision
is designing the format for the Info data in HTML.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Nic Ferrier]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > > 4. Emacs Lisp code to browse HTML-Info files.
>
>   > I think this will be most difficult part, so better focus on this part
>   > than on the #3 which should be a piece of cake once we have an Elisp
>   > solution for the rendering.
>
> I think #4 will be straightforward once #1-3 are done.  The crucial decision
> is designing the format for the Info data in HTML.

#2 (Something to generate HTML-Info from Texinfo input) is the really
hard bit because mostly it is in C right now. It can be done by wrapping
the existing Texinfo, like I am doing right now. But that results in a
more complex system than we have now.

But I think a good thing to do would be to develop #4 along with #1 and
#3. So they are all linked.


Nic

Re: HTML-Info design

[Eli Zaretskii]

> From: Nic Ferrier <nferrier@ferrier.me.uk>
> Date: Mon, 29 Dec 2014 00:13:59 +0000
> Cc: eliz@gnu.org, stephen@xemacs.org, Stefan Monnier <monnier@iro.umontreal.ca>,
> 	emacs-devel@gnu.org
> 
> #2 (Something to generate HTML-Info from Texinfo input) is the really
> hard bit because mostly it is in C right now.

No, it's in Perl, and is said to be easily customizable.

Re: HTML-Info design

[Stefan Monnier]

>> I think this will be most difficult part, so better focus on this part
>> than on the #3 which should be a piece of cake once we have an Elisp
>> solution for the rendering.
> I think #4 will be straightforward once #1-3 are done.

Lars hinted at the fact that #4 might require changes to the redisplay.
So I think "straightforward" is very optimistic.


        Stefan

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > I think #4 will be straightforward once #1-3 are done.

  > Lars hinted at the fact that #4 might require changes to the redisplay.
  > So I think "straightforward" is very optimistic.

Making it display as well as current Emacs Info does
is straightforward.  To make it look as nice as a browser
requires the variable-width filling feature that Lars mentioned,
but that's independent of making it work at all.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Juri Linkov]

>   > What would be success?
>
> My idea of the goal is
>
> 1. A spec for HTML-Info format, saying how the Info data such as
> menus, indices and node structure navigation are represented
> in HTML.

These is already a node structure in HTML generated by makeinfo
in form of <link rel="next">, <link rel="previous">, <link rel="up">.
Menus are represented by class="menu", and indices by class="index".

> 2. Something to generate HTML-Info from Texinfo input.

It would be great to include JavaScript in the output generated by makeinfo.
To be able to do this, prepared JavaScript files should be included
in the Texinfo distribution.  Then visiting an HTML-Info either
locally or from a web site in a web browser supporting JavaScript
will allow HTML-Info navigation, search and other features of Info.

> 3. An extension for Firefox to implement Info-style commands
> using that data.

A Firefox extension requires the users to install it,
so this won't be a convenient option to use HTML-Info manuals.

> 4. Emacs Lisp code to browse HTML-Info files.

There are at least two places to use such code:

1. creating an extension for eww (e.g. eww-info.el)
   to support HTML-Info search, indexing, etc.

2. using HTML in info.el (or creating a separate info-html.el)
   that will use shr to better rendering in Info
   like was demonstrated in http://debbugs.gnu.org/14670#14

> Any comments on this plan?
>
> Are you interested in working on it?

If needed, I could help with the latter.

Re: HTML-Info design

[Stefan Monnier]

> A Firefox extension requires the users to install it,
> so this won't be a convenient option to use HTML-Info manuals.

A an extension installed separately lets the user choose which version
to install, lets her hack on it easily, ...

A version bundled in the HTML files would force the user to use exactly
that code with no practical control over it.  Even if that Javascript
code is GPL'd, in practice the user doesn't have much more option to
modify it than in a Tivo device.

Remember what the first "F" in "FSF" stands for?


        Stefan

Re: HTML-Info design

[Lars Magne Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> A version bundled in the HTML files would force the user to use exactly
> that code with no practical control over it.  Even if that Javascript
> code is GPL'd, in practice the user doesn't have much more option to
> modify it than in a Tivo device.

It depends on how you do it.  You could have a button in the info page
that says "actuvate JS?"  And you could provide greasemonkey scripts to
do the same.

JS doesn't have to be something forced unto the user.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: HTML-Info design

[Nic Ferrier]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> A Firefox extension requires the users to install it,
>> so this won't be a convenient option to use HTML-Info manuals.
>
> A an extension installed separately lets the user choose which version
> to install, lets her hack on it easily, ...
>
> A version bundled in the HTML files would force the user to use exactly
> that code with no practical control over it.  Even if that Javascript
> code is GPL'd, in practice the user doesn't have much more option to
> modify it than in a Tivo device.

I favour a separation between the content and the app.

If you look at it, my existing webapp achieves this. It doesn't do
anything with that separation... but it does have the separation
there. So the same "app" could be used to look at multiple manual
sources presuming they all conformed to what the app expects (no
different from any other software situation of course).

I don't particularly see that the separation needs to be achieved with
an extension. There are other ways. But as we already said, it's
pointless arguing about this right now.


> Remember what the first "F" in "FSF" stands for?

This thread seems to exist to just annoy us all.


Nic

Re: HTML-Info design

[Stefan Monnier]

> If you look at it, my existing webapp achieves this.

AFAICT, its design is indeed compatible with the desire to provide a
reader "extension" separately from the manual itself.

> But as we already said, it's pointless arguing about this right now.

Indeed.


        Stefan

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > These is already a node structure in HTML generated by makeinfo
  > in form of <link rel="next">, <link rel="previous">, <link rel="up">.
  > Menus are represented by class="menu", and indices by class="index".

We should not feel any obligation to stick with that old form of Info.
The goal is to provide the most Info-like possible behavior
in a browser.  This may require a different representation.

Part of what Nic is working on is to determine what representation is best.
He can use the same format or a different one.

  > It would be great to include JavaScript in the output generated by makeinfo.

I've decided against this.

  > > 3. An extension for Firefox to implement Info-style commands
  > > using that data.

  > A Firefox extension requires the users to install it,
  > so this won't be a convenient option to use HTML-Info manuals.

I've made this decision on ethical grounds.  The ethical issue
overrides that practical one (which is a small one, after all).


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Lennart Borgman]

On Tue, Dec 30, 2014 at 12:24 AM, Richard Stallman <rms@gnu.org> wrote:
>
>   > It would be great to include JavaScript in the output generated by makeinfo.
>
> I've decided against this.

I wonder if that is a misunderstanding? A JavaScript (EcmaScript)
solution could be valuable for browsing the HTML documentation stored
locally.

A search engine solution could give a similar functionality as part of
an online search of the HTML documentation on the web.

(A search engine can be used locally to, but that looks rather
inconvenient to me.)


>   > > 3. An extension for Firefox to implement Info-style commands
>   > > using that data.
>
>   > A Firefox extension requires the users to install it,
>   > so this won't be a convenient option to use HTML-Info manuals.
>
> I've made this decision on ethical grounds.  The ethical issue
> overrides that practical one (which is a small one, after all).

What is unethical about using EcmaScript instead? Or, do I misunderstand you?

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > I've decided against this.

  > I wonder if that is a misunderstanding? 

You may have misunderstood the decision which I have stated here
several times.

Do you want to work on designing and coding this project?
If so, it is worth more effort to explain this so that you get it.
Otherwise, I have other work I'd rather do.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Lennart Borgman]

On Tue, Dec 30, 2014 at 8:51 PM, Richard Stallman <rms@gnu.org> wrote:
>
>   > > I've decided against this.
>
>   > I wonder if that is a misunderstanding?
>
> You may have misunderstood the decision which I have stated here
> several times.

So then I take it as you have decided against any web search for the manuals.


> Do you want to work on designing and coding this project?

I offered to help with the web search. (And in fact did set up that,
temporary.) Sorry, I do not have time for other coding right now.

Re: HTML-Info design

[Lennart Borgman]

On Tue, Dec 23, 2014 at 9:41 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>
> We are talking about replacing an existing browser, one that is
> developed for decades and is chock-full of useful features.  I use
> many of them every day.  People can say it's "samizdat" all they want,
> but for me it is a tremendously useful tool.

I would rather think of it as adding that kind of functionality to the
web browser version. It is the functionality that is important, of
course, not how it will look.

All those years that info has been developed and married with the
content has given a lot of experience, useful experiences. For me the
question is: Can we transform that knowledge to that web browser
version? That might help not only Emacs, but also other projects.

Of course that has to be merged with the internet search that is now
the main way for people to search for information.

This is not an easy project. Because of that there is no room for quarrels. ;-)


> If you want to convince
> me that the replacement will be worthy, you need to show that those
> important features, or at least some non-trivial subset thereof, can
> be had with your suggested approach.  And given my JS ignorance, the
> only way of showing that is by implementing it.  Because I have no
> idea what will it take to write a multi-file regexp search in JS.

A very common situation. A very important type of situations.

I would say implement part of it. And then talk carefully about the details.

Re: HTML-Info design

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

> We are talking about replacing an existing browser, one that is
> developed for decades and is chock-full of useful features.

Not really.  It's pretty much the same since it was developed first.
Indexing and search have admittedly been polished quite a bit in the
last 10 years.  Though it's not as much the indexing that has been
polished but rather Emacs completion in general.  Images have been added
and happen to work fast and well in Emacs info though not anywhere else.

The Info file format is bland and has not been changed over a long time,
formatting of links is incoherent and unpredictable, paragraph
formatting around @example tends to be inconsistent and so on.

Navigation and display is instantaneous and good, the results are as
readable as one would expect from an editor specializing on working with
texts.

That's all, but it's essential.  It doesn't suck where it counts, and
for some reason that's not what the HTML browsing based universe has
managed so far.  And the browsers _have_, in contrast to Info, been
actively developed for decades.  And yet I could not name a single thing
in which they are significantly better than the first browsers let loose
on the HTML from Texinfo were.

For M-x info RET, I could name probably half a dozen things.  That's
pitiful, but less pitiful than none.

-- 
David Kastrup

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Please let's stop arguing about whether we "need" to replace Info
format.  I'm convinced that HTML-Info is a good idea, and I am
inviting people to work on it.  This is my decision.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Ted Zlatanov]

On Wed, 24 Dec 2014 04:56:56 -0500 Richard Stallman <rms@gnu.org> wrote: 

RS> Please let's stop arguing about whether we "need" to replace Info
RS> format.  I'm convinced that HTML-Info is a good idea, and I am
RS> inviting people to work on it.  This is my decision.

...work here (emacs-devel) or elsewhere? Will the spec be done by a
group or an individual before it's open? Will it be a new project or
part of an existing one?

Thanks
Ted

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > ...work here (emacs-devel) or elsewhere? Will the spec be done by a
  > group or an individual before it's open? Will it be a new project or
  > part of an existing one?

I see no reason for it to be done on this list.  As for the rest,
it depends on the people who want to do it.

Would those interested please write me, but not to this list?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Ted Zlatanov]

On Wed, 24 Dec 2014 13:00:52 -0500 Richard Stallman <rms@gnu.org> wrote: 

> Ted wrote:
>> ...work here (emacs-devel) or elsewhere? Will the spec be done by a
>> group or an individual before it's open? Will it be a new project or
>> part of an existing one?

RS> I see no reason for it to be done on this list.  As for the rest,
RS> it depends on the people who want to do it.

RS> Would those interested please write me, but not to this list?

Last request: could you or someone else post a final notice here, when
these logistics are settled, to tell interested observers where they can
follow the progress?

Thanks!
Ted

Re: HTML-Info design

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Last request: could you or someone else post a final notice here, when
  > these logistics are settled, to tell interested observers where they can
  > follow the progress?

I will leave details like that to the people that work on this.
Right now I am just waiting for volunteers.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: HTML-Info design

[Stefan Monnier]

> I'm saying I'm not convince this is a viable way of providing a
> replacement for info.el.

Maybe I misunderstood other people's stance, but AFAICT Nic's work is
not meant as a replacement for info.el.


        Stefan

Re: HTML-Info design

[Stephen J. Turnbull]

Stefan Monnier writes:

 > But this is largely unrelated to the "inside Emacs manual browsing",
 > because there's no way Emacs will be able to render this kind of
 > HTML+Javascript efficiently any time soon, I think.

Of course it can -- by hardcoding the Javascript (and CSS!) 
*functionality* in EWW (or a derived mode specifically for browsing
Info-HTML).  You don't need to be able to parse and interpret
Javascript or CSS to provide equivalent functionality to the
Javascript and CSS recommended by Emacs.

Of course this violates DRY to some extent.

 > So, Nic's work is not competing against info.el, and it's actually
 > a work that tends to vote in favor of keeping Texinfo as the source
 > documentation format.

Agree.

Re: HTML-Info design

[Stephen J. Turnbull]

Nic Ferrier writes:

 > Yes, it's pulled down from the GNU HTML manuals online. So a proxy
 > call basically. I've been slowly working on a better way (a direct
 > way) of delivering the content. That's a relatively difficult
 > logistics job. A matter of building everyone's documentation and
 > putting it in a place.

Could we AJAX-ify (as Lennart suggested earlier) by pulling down the
first 10k (or whatever is enough to display the menu), and then grab
the rest in the background?  Also a logistics job (in a different way)
but it would be really cool if users could point this at arbitrary
texi2any HTML output!

Re: HTML-Info design

[Nic Ferrier]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Could we AJAX-ify (as Lennart suggested earlier) by pulling down the
> first 10k (or whatever is enough to display the menu), and then grab
> the rest in the background?  Also a logistics job (in a different way)
> but it would be really cool if users could point this at arbitrary
> texi2any HTML output!

I think that would not be optimal.

Info is handily built into nodes. The problem with my current gnudoc app
is that it's proxying from GNU's website, so there are two HTTP
roundtrips. It would all be fine if there was just one for each node.

And consider that the information is very cachable (and I already do
cache it heavily) so most people won't be paying the cost of even a disc
load.

The thing is this work (the logistics of all repos for documentation)
has to be done to extend the manuals beyond the Emacs ones. So it has to
be done anyway to progress my project.


Nic

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Tom]

Stephen J. Turnbull <stephen <at> xemacs.org> writes:
> 
> So what you get with a web-friendly manual is accessibility (though
> somewhat degraded) for those who *don't* have the manuals installed
> locally.  

Regarding accessibility the current HTML manuals could also be
made more accessible (i.e. work like info). It just involves adding
some JS code to the HTML.

As a test I took the current emacs HTML manual (for simplicity I used
the everything on a single page version, so you may have to wait
for it load completely) and added a simple goto feature to it,
like the one in info.

You can press 'g' and a dialog pops up, you type something, the
nodename completions appear and you can press enter to jump
to the desired node. You can cancel the dialog with esc, pop
it up again by pressing 'g' again, etc. So it works like an
interactive local application.

Here it is:

http://emacstest.byethost12.com/goto.html

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Tom writes:

 > As a test I took the current emacs HTML manual (for simplicity I used
 > the everything on a single page version, so you may have to wait
 > for it load completely)

In two tries (in private browsing windows, so shouldn't be cached) it
took ~2 and ~4 seconds to load respectively.  The ToC was visible in
both cases in under 2 seconds.  (Safari on Mac OS X "Yosemite")

That would be acceptable if I just kept a browser window open all the
time.

I wonder if you could write an Emacs function to prompt for a node
name and "send" the request to the browser window.  I'm not sure how
to make that practical (I'd want the completion feature, although I
don't like the current implementation, see below).

 > and added a simple goto feature to it, like the one in info.

Thanks!

 > You can press 'g' and a dialog pops up, you type something, the
 > nodename completions appear and you can press enter to jump
 > to the desired node.

It is instantaneous, as advertised.

Doesn't work properly in Safari - I have to actually select something
from the menu, can't type and go.  (Yes, I understand this is a quick
proof of concept, but in Emacs completion is normally automatic and my
fingers expect that -- this behavior would get very old very fast.)

It doesn't work as I would want even if the completion is unique.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Tom]

Stephen J. Turnbull <stephen <at> xemacs.org> writes:
> 
> Doesn't work properly in Safari - I have to actually select something
> from the menu, can't type and go.  (Yes, I understand this is a quick
> proof of concept, but in Emacs completion is normally automatic and my
> fingers expect that -- this behavior would get very old very fast.)
> 
> It doesn't work as I would want even if the completion is unique.
> 

Naturally, it could be much better. As you also mentioned this is 
only a quick hack I put together to demonstrate that the HTML doc can 
work like an interactive app.

This is all the JS code I wrote for this:

http://emacstest.byethost12.com/info.js

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Tom writes:

 > > It doesn't work as I would want even if the completion is unique.
 > 
 > Naturally, it could be much better.

Of course.  I was more wondering if that was specific to my browser's
implementation of ecmascript or if that just requires extension of the
code you wrote.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Tom]

Stephen J. Turnbull <stephen <at> xemacs.org> writes:
> 
> Of course.  I was more wondering if that was specific to my browser's
> implementation of ecmascript or if that just requires extension of the
> code you wrote.
> 

If you mean not being able to jump to a node without selecting something
then it's simply not implemented. I added only the bare minimum of
implementation to make the demo work, so I implemented only the
select handler which is called when an item is selected in the completion
list.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Sat, Dec 20, 2014 at 7:05 PM, Tom <adatgyujto@gmail.com> wrote:
>
> You can press 'g' and a dialog pops up, you type something, the
> nodename completions appear and you can press enter to jump
> to the desired node. You can cancel the dialog with esc, pop
> it up again by pressing 'g' again, etc. So it works like an
> interactive local application.
>
> Here it is:
>
> http://emacstest.byethost12.com/goto.html

Avast will not let me visit that page. It complains about URL:Mal.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Tom]

Lennart Borgman <lennart.borgman <at> gmail.com> writes:

> 
> Avast will not let me visit that page. It complains about URL:Mal.
> 

No idea why.

It's a free webhost and I registered on it only to host the demo.
Probably someone used it to host some malware and that's why the 
main domain is marked as suspicious.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Mike Gerwitz]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sat, Dec 20, 2014 at 18:05:53 +0000, Tom wrote:
> Regarding accessibility the current HTML manuals could also be
> made more accessible (i.e. work like info). It just involves adding
> some JS code to the HTML.

I consider adding JS to be more of a kluge than a solution; while this
does add additional functionality, it is specific to the page (not
handled by the web browser), non-standard, and wouldn't function in a
nice text-based browser.

Even as a web developer, I rarely enable JavaScript and consider a web
page to be broken if it requires it.  In this case, the page would
function just fine without (and would therefore not be broken), but the
experience would differ.

If the goal (according to ESR) is to appeal to the younger crowd, this
wouldn't do that.

I see no problem with our current format, which can generate decent HTML
(though it could be improved upon), and which can be cached by the
browser for instant page views to already-visited nodes.  Users who do
care about keyboard-based navigation can use either access keys[0], or
install an extension (I use Vimperator).  I would not want a web page to
dictate that for me.

[0]: http://mdn.beonex.com/en/HTML/Global_attributes.html

- -- 
Mike Gerwitz
Free Software Hacker | GNU Maintainer
http://mikegerwitz.com
FSF Member #5804 | GPG Key ID: 0x8EE30EAB
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAEBAgAGBQJUlx4nAAoJEPIruBWO4w6r0egP/1gaoXr+lUBmw0SPH/NpWkjG
U/eJtvbLsChJt5KrdQDnEytu8ez/4l6ixWDj7QKQIJQY0HTYysscitr7jGVdL740
Gp77gyb+xje1Uek+Z1giYqqfhvSwjadn09YYAkNpFcFCxvzCK616WBBlXR3Kjb6f
yhvm1sLFOjJ3NHQsMN2K8WeDpZLvpUveGT05lopoGLOH8xGpHNlRbXKcLXvzVZpj
PizR5/u0XxDSpxav/f3tOckhDQyh201h08OVKoVdI86JGdz51I3HTYfChbGVJnpt
HkL5GLKl6MzS8xqgTvaK5Pzeo1nQUs3Q+x+Q3Db4aDwvNZ/Ps63qUeYi/ktJBzDC
Uipg9j6bfzDwg0jQzX1WAc68zaOdrcd8kp3DnUtnSkAXVWeKkCpASaCQW2CzL+gI
Q552H6bnQz2HWW26w0Yrq1tuwOyd8b0DDzU1fTUkpd8/B0vf/fCB4n48sWq26df2
uCwNX6y6waFAR8/c9bpppuY/s7yf6b/oEorP8vzKlJtQY8DJPTUXmmBvX54zneId
aYfPRLs88bV6t2MYYDeW+Z4N5ECX+lCBGVdjLUyKdKPW2NwG34Mg/vg+u9Kg8/mt
17whHkEooBsqSHa86vslHqVz2qLklgKRlidMJClwpy8VNd/HKIP5z8cbmwQ08Lci
8t/kT3vTwnQyS+rzjwW+
=isH6
-----END PGP SIGNATURE-----

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Tom]

Mike Gerwitz <mikegerwitz <at> gnu.org> writes:

> 
> I consider adding JS to be more of a kluge than a solution; while this
> does add additional functionality, it is specific to the page (not
> handled by the web browser), non-standard, and wouldn't function in a
> nice text-based browser.

GUI Emacs has features which don't work on a terminal, yet they are
nice to have. In a text browser it works as a simple HTML page, while 
a modern GUI browser can provide the interactive feature to the user.


> Even as a web developer, I rarely enable JavaScript and consider a web
> page to be broken if it requires it.  In this case, the page would
> function just fine without (and would therefore not be broken), but the
> experience would differ.
> 
> If the goal (according to ESR) is to appeal to the younger crowd, this
> wouldn't do that.

The younger crowd expects interactive web pages (e.g. jumping to 
manual nodes with completion), because they are used to interactive
features on other pages (Gmail, facebook, etc.)

And they expect it to work out of the box, because they know the 
browser can do it. E.g. they won't install an extension just to 
browse Emacs manual nodes interactively when they know the browser
can do the same natively on other pages.

> I see no problem with our current format, which can generate decent HTML
> (though it could be improved upon), and which can be cached by the
> browser for instant page views to already-visited nodes.  Users who do
> care about keyboard-based navigation can use either access keys[0], or
> install an extension (I use Vimperator).  I would not want a web page to
> dictate that for me.

G as hotkey was chosen to replicate the emacs info experience. An emacs
user would naturally want to use the same keys as in info to browse the 
HTML documentation. Why use something else when you can use the same
keys?

RE: Have you all gone crazy? Was: On being web-friendly and why info must die

[Drew Adams]

> The younger crowd expects interactive web pages (e.g. jumping
> to manual nodes with completion), because they are used to
> interactive features on other pages (Gmail, facebook, etc.)

Fine.  And it is not only "the younger crowd" that is used to
using interactive web pages.

And if this is about providing access to arbitrary GNU manuals
using HTML on the web then sure, why not add such features to
our HTML manuals on line?

But for the Emacs and Elisp manuals, in particular, I think that
one of the goals should be to also steer users to *ask Emacs*
itself.

IOW, let users know that within Emacs they have access to the
Emacs manuals, and that that is really the way to go (for the
reasons already discussed, in particular, index features).

Yes, this is perhaps a side point.  But it is a concern I have.
When a URL sends a user to a manual node on the web, I would
like to see a feature that points back to the manual in Emacs,
e.g., a reminder.

Too many new Emacs users (old as well as young), I think,
reflexively ask their questions about Emacs *first* on forums,
mailing lists, Stack Exchange, etc.  And often just because
that's how they ask about other things.

That's OK if it's a habit or they somehow find it easier,
but they should at least be made aware that they *can* access
the same information from within Emacs, and that they will
get more help and learn more that way than via the manuals
on line.

The best way we can help them in this regard is to let them
know that Emacs itself offers great help for learning about
Emacs, and one of the best such aids are the Emacs and Elisp
manuals - *within Emacs*.

Having the information in these manuals at your fingertips
while you use Emacs is an giant plus.  Providing better Web
use of the manuals is of course a good idea.  But we should
not neglect inviting users to consult the manuals from Emacs.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Drew Adams <drew.adams@oracle.com> writes:

>> The younger crowd expects interactive web pages (e.g. jumping to
>> manual nodes with completion), because they are used to interactive
>> features on other pages (Gmail, facebook, etc.)
>
> Fine.  And it is not only "the younger crowd" that is used to
> using interactive web pages.

Frankly, that's not really useful since
a) every "interactive web page" is different
b) they work badly with automated fetchers, so you cannot usefully
   aggregate them
c) they tend to require enabling known security problems like Flash

> But for the Emacs and Elisp manuals, in particular, I think that
> one of the goals should be to also steer users to *ask Emacs*
> itself.
>
> IOW, let users know that within Emacs they have access to the
> Emacs manuals, and that that is really the way to go (for the
> reasons already discussed, in particular, index features).
>
> Yes, this is perhaps a side point.  But it is a concern I have.
> When a URL sends a user to a manual node on the web, I would
> like to see a feature that points back to the manual in Emacs,
> e.g., a reminder.

We will also want to increase the attractivity of the average HTML
manual rendition so that they are actually linked to and come up in web
searches.

> That's OK if it's a habit or they somehow find it easier,
> but they should at least be made aware that they *can* access
> the same information from within Emacs, and that they will
> get more help and learn more that way than via the manuals
> on line.

Well, one problem is that the manuals are really effective for learning
a lot in one learning session.  And that's a good deal for getting
better with using Emacs.  But it doesn't match modern attention spans.
And that's an inherent problem we have with selling Emacs, and it will
continue to cost us "market percentages".

> The best way we can help them in this regard is to let them
> know that Emacs itself offers great help for learning about
> Emacs, and one of the best such aids are the Emacs and Elisp
> manuals - *within Emacs*.
>
> Having the information in these manuals at your fingertips
> while you use Emacs is an giant plus.  Providing better Web
> use of the manuals is of course a good idea.  But we should
> not neglect inviting users to consult the manuals from Emacs.

Certainly.  But I think that would primarily concern the Emacs-related
manuals, so we should probably see to it that one can get the Texinfo
conversion process and/or our personal manual CSS sheets to make this a
general feature of the Emacs manuals in particular.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Tom]

David Kastrup <dak <at> gnu.org> writes:
> 
> Well, one problem is that the manuals are really effective for learning
> a lot in one learning session.  And that's a good deal for getting
> better with using Emacs.  But it doesn't match modern attention spans.

Exactly. Users want to get the information fast and it is much
faster to search for something in google than trying to find
the relevant section of the manual.

The manual is great for looikng up again something which you know
where to find. E.g. what kind of text properties are there?

C-h i -> m elisp -> g -> text properties

But if you want to lookup something unfamiliar then google is much
more efficient.

I wonder how many people read the manual from the beginning to the
end like a book. These days (when average attention span is very
short due to years of filtering through huge volumes of information
on the net) I'd guess not many.

RE: Have you all gone crazy? Was: On being web-friendly and why info must die

[Drew Adams]

> > Well, one problem is that the manuals are really effective for
> > learning a lot in one learning session.  And that's a good deal
> > for getting better with using Emacs.  But it doesn't match
> > modern attention spans.
> 
> Exactly. Users want to get the information fast and it is much
> faster to search for something in google than trying to find
> the relevant section of the manual.

And why is that a problem?  What is wrong with people using a
web search engine like Google to get access to information
about Emacs?

(This is about the use of web search engines; it is not about
the company Google and any political issues surrounding it.)

And what do you suppose users actually find when they do that?
<drum-roll> ... </drum-roll)  Links to the GNU Emacs manuals,
among other things.  Precisely *because* Google indexing and
search is not stupid.  It is based on the information that
people find most useful.

> The manual is great for looikng up again something which you know
> where to find. E.g. what kind of text properties are there?
> C-h i -> m elisp -> g -> text properties
> But if you want to lookup something unfamiliar then google is much
> more efficient.

Something unfamiliar like what, for instance?  It's not a
rhetorical question.  I'm sure there are such things that
are hard to look up in the manuals, but an example
might be illustrative.  (And it might even help you file a
bug report, to improve the manual's index.)

In many cases, I've found that `i' in Info, supplemented by
better completion (in the case of Icicles, being able to use
multiple patterns - unordered, complement pattern matches,
etc.) can be very helpful.

As others have already argued, there is no substitute for a
human-created index.  (We can argue that more, if you are not
convinced and you think that indexing is just a derivative of
search - e.g. "full-text indexing".)

But I won't argue that there is no substitute for something
like Google.  And I will add that I hope we are not aiming
to replace it anytime soon. ;-)

There is no reason to pose Google search either as a threat
to Emacs or as something opposed to using Info within Emacs.
There is no reason for Emacs users not to use a web search
engine to get access to information about Emacs.

> I wonder how many people read the manual from the beginning
> to the end like a book. 

Who cares?  Why would that even be a consideration?  But
if you really don't know, the answer is: very, very, very
very few - if any.

> These days 

Oh, please.  "These days, these days,..."  As if "these
days" were something new.

This gets tiresome at the end.  There have *NEVER* been
legions of people who "read the manual from the beginning
to the end like a book."  Few people have *EVER* read *ANY*
reference book, especially a large one, from cover to cover.

So what?  There is nothing new about the fact that people
often dip into reference books to remind themselves about
particular information or to learn more about a particular
topic.

"These days..."  Sheesh.  As if this were a discovery.

> (when average attention span is very short due to years
> of filtering through huge volumes of information
> on the net) I'd guess not many.

Uh, "these days", Google does most of the filtering.

I am not impressed by the "years of filtering through
huge volumes of information" that today's users manage
to carry out.

On the contrary.  The new phenomenon, if there really is
one, is a relative *lack* of effort spent researching and
filtering information.  It is so simple to just ask others.

With the result that Stack Exchange sites (for example)
end up with zillions of butt-ugly, half-comprehensible,
simpleton questions that other well-meaning users and
moderators have to close, under the category of
"TRY to find the f---ing answer yourself first; THEN ask
for help here."

Catering to THAT problem should not waste 3 seconds of
our time here.  The problem there is not our manuals or
Google, and that problem is not one we should be losing
sleep over.  There are better improvements to be worked on.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Drew Adams <drew.adams@oracle.com> writes:

>> > Well, one problem is that the manuals are really effective for
>> > learning a lot in one learning session.  And that's a good deal
>> > for getting better with using Emacs.  But it doesn't match
>> > modern attention spans.
>> 
>> Exactly. Users want to get the information fast and it is much
>> faster to search for something in google than trying to find
>> the relevant section of the manual.
>
> And why is that a problem?  What is wrong with people using a
> web search engine like Google to get access to information
> about Emacs?

Not all information is good, not all information matches the version you
are using.

One of the top hits for AUCTeX I remember being annoyed at contained
information that was bad advice even when it was new, something like a
dozen years before the web search returned it in a high place, turned
bad not too long after that and it belonged to a student page that was
long frozen in time with nobody to look after it any more, the
corresponding mail addresses being either dead or non-responsive.

We can't fix the "web".

-- 
David Kastrup

RE: Have you all gone crazy? Was: On being web-friendly and why info must die

[Drew Adams]

> >> Users want to get the information fast and it is much
> >> faster to search for something in google than trying to
> >> find the relevant section of the manual.
> >
> > And why is that a problem?  What is wrong with people using
> > a web search engine like Google to get access to information
> > about Emacs?
> 
> Not all information is good, not all information matches the version
> you are using.

No, of course not.  And some people can google better than
others.  And people can learn to google better.  And some
things are not so easy to find by googling anyway.  And yes,
it can be easy to get not-so-good information sometimes.
And it is possible not to be aware that the info found is
not so good.

Still.  It's not because a tool that can be helpful is not
also a panacea that it is something to be avoided.  Web
search engines are not the problem here, IMO.

> One of the top hits for AUCTeX I remember being annoyed at contained
> information that was bad advice even when it was new, something like
> a dozen years before the web search returned it in a high place,
> turned bad not too long after that and it belonged to a student
> page that was long frozen in time with nobody to look after it any
> more, the corresponding mail addresses being either dead or
> non-responsive.

If *most* of the top hits for a reasonably good query are
poor then yes, we can perhaps do a better job of making
better information more visible.

Typically, this kind of problem is, on the average and over
time, taken care of by itself, because more people find other
info more useful.  Web-search indexing is certainly not
perfect, but it is much better than it used to be.  And it
is quite useful overall.

> We can't fix the "web".

No, we can't.  But that is not really a problem we should
worry about here.  Not under the topic "on being web-friendly",
in any case.

RE: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Drew Adams writes:

 > On the contrary.  The new phenomenon, if there really is
 > one, is a relative *lack* of effort spent researching and
 > filtering information.  It is so simple to just ask others.

It is new.  Before the internet, people relied on local expertise and
asking face to face, and if there was none, gave up.  StackOverflow
and friends are basically the inverse of spam: ask the universe at no
cost to yourself.

I agree with you that not catering to those folks is the right way to
go.  Among other things, in the rare case that they offer
contributions, they usually suck. ;-)  Just do what Emacs wants to do.

RE: Have you all gone crazy? Was: On being web-friendly and why info must die

[Drew Adams]

>  > On the contrary.  The new phenomenon, if there really is
>  > one, is a relative *lack* of effort spent researching and
>  > filtering information.  It is so simple to just ask others.
> 
> It is new.  Before the internet, people relied on local expertise
> and asking face to face, and if there was none, gave up.
> StackOverflow and friends are basically the inverse of spam:
> ask the universe at no cost to yourself.

That mischaracterizes "StackOverflow and friends", I'm afraid.
But yes, you get the point.

It is more accurate to say that some people use StackOverflow
& friends that way.  Their attempts to do so are only
moderately successful, however.  Users of SO & friends who
do not act that way are rewarded with more and better help
in the long run.

Another consideration is that asking a good question takes
not only some up-front effort but also some experience and
knowledge of how to express oneself clearly.  And language
(English) is sometimes an obstacle, if not a barrier.  And
in some of our countries the education system is not as
good (for most) as it once was.

It is actually a *good* sign that people, especially those
who have difficulty expressing themselves, do not hesitate
to ask when they have a question.  We need a lot more of that.

(In some contexts, especially in some fairly traditional,
formal education settings, students are taught not to ask
but to shut up and respect.  It is a good thing that more
young people do not hesitate to ask, in order to understand
better.  Question authority.  Question anything.)

And users can sometimes learn to ask better questions in
such contexts, which generally means *first* asking Google,
Emacs, and other resources oneself.  And the feedback
provided by StackOverflow & friends tends to help users
get better at it, if they pay attention at all.

Never has it been easier for an individual to "ask the
universe".  And that's a *good* thing.  It's just that
there are better ways to ask than to pose an undeveloped
question on a question board.

> I agree with you that not catering to those folks is the
> right way to go.  Among other things, in the rare case that
> they offer contributions, they usually suck. ;-)  Just do
> what Emacs wants to do.

We agree, but I would add this: Even users who reflexively
ask poor questions, without any effort, can improve.  As
they learn how to ask better questions - which by definition
require some up-front effort in research or at least more
careful explanation - they will be rewarded.

And they sometimes do become helpful contributors, giving
back to others who have their own questions, etc.

Emacs should not cater to such poor behavior.  And it
certainly should not accept it as a "new norm" that we
must somehow get in sync with in order to attract new
blood.  But Emacs can (continue to) try to show the way.

Improve Emacs manuals on the web, sure.  But let's not
forget to use that web presence to help users learn that
the best way to consult the manuals is to *ask Emacs*.

This is not obvious.  There are few, if any, interactive
contexts that are helpful the way Emacs is.  People do
not expect it, and they start out ignorant of it.

Aiming *primarily* for perfectly interactive web versions
of the manuals, even if we could emulate all of the Emacs
Info features on the web, would be a mistake.  Such
improvement can be a goal, but another goal should be to
make sure we point the way to the manuals in Emacs itself.

RE: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Drew Adams writes:

 > Aiming *primarily* for perfectly interactive web versions
 > of the manuals, even if we could emulate all of the Emacs
 > Info features on the web, would be a mistake.  Such
 > improvement can be a goal, but another goal should be to
 > make sure we point the way to the manuals in Emacs itself.

I don't think anybody with real leverage on the decisions is thinking
in terms of aiming at "interactive web versions".  The point is to
make an interactive version targeting Emacs that is more compatible
with web presentation, and that gives browser-preferring Emacs users
the choice of getting their docs in a browser.  There is also the
purported benefit of more efficient and effective authoring if a
"better" source language is used.  I'm not sure there really *is* a
significant benefit of that kind, in fact I tend to doubt it.  But
I'll be happy to adopt if somebody else does the work of developing
one! :-)

The rest is tl;dr, but it was already written and bandwidth is cheap
nowadays. :-)

 > > It is new.  Before the internet, people relied on local expertise
 > > and asking face to face, and if there was none, gave up.
 > > StackOverflow and friends are basically the inverse of spam:
 > > ask the universe at no cost to yourself.
 > 
 > That mischaracterizes "StackOverflow and friends", I'm afraid.

Exaggeration for effect.  The cost-benefit analysis for the
questioners, however, stands.

 > It is more accurate to say that some people use StackOverflow
 > & friends that way.  Their attempts to do so are only
 > moderately successful, however.

They are fully successful in keeping me from using SO, Q *or* A. ;-)
But then, I'm generally pretty good at asking the kind of questions
that the people I want to talk to enjoy answering.

Despite the deprecation by the baby boomer generation, I suppose crowd
sourcing (wikis and web fora like SO) have had significant successes.
Still, we should be able to do better.

 > It is actually a *good* sign that people, especially those
 > who have difficulty expressing themselves, do not hesitate
 > to ask when they have a question.  We need a lot more of that.

True, but we also need more people to jump over the desk and start
answering.

 > (In some contexts, especially in some fairly traditional,
 > formal education settings, students are taught not to ask
 > but to shut up and respect.

Tell me about.  Most of the students I advise are Chinese, and the
rest are Japanese.[1]  But I teach them how to ask questions in a way
that empowers them and pleases Those Who Have Some Answers.  Not
always successful, but as often as not they get it, even if two
years in a master program isn't enough for them to learn to do it
consistently without supervision.

It doesn't give me a good reputation among my colleagues; many
consider my students to be so many questioning PITAs.  On average,
they are (and still a slight negative at graduation).

 > Never has it been easier for an individual to "ask the
 > universe".  And that's a *good* thing.

I'm not sure I agree.  Spam is still spam, even if it's in a good
cause.  Maybe *worse* when it's in a good cause. :-/  Specifically, we
need to consider the effect on the answerers as well as on the
questioners.  In my environment, a significant increase in questioning
behavior is clearly a blade that cuts both ways.  (I think my
colleagues by and large prefer to do bureaucratic paperwork!)

Footnotes: 
[1]  People who "get it" are everywhere, AFAICT in proportion to
population.  But those two educational *systems* are diametrically
opposed to educing the ability to "get it" for the average student.
There are lots of teachers who can't help themselves, and mentor
despite the obstacles raised by the systems.  But not enough. :-(

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Drew Adams writes:
>
>  > That mischaracterizes "StackOverflow and friends", I'm afraid.
>
> Exaggeration for effect.

We are all mature and civilized adults on this list so there is no need
for exaggeration.  Instead it is best practice to calmly repeat your
point until the other throws up his hands in disgust and finds something
better to do.  You can see the results in Emacs' code base.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Drew Adams <drew.adams@oracle.com> writes:

> Another consideration is that asking a good question takes
> not only some up-front effort but also some experience and
> knowledge of how to express oneself clearly.  And language
> (English) is sometimes an obstacle, if not a barrier.

<URL:http://english.stackexchange.com/questions/210225/the-word-for-a-man-who-hunts-a-dangerous-mountain-cat-without-prophylactic>

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Harry Putnam]

Tom <adatgyujto@gmail.com> writes:

> But if you want to lookup something unfamiliar then google is much
> more efficient.

I don't find that to be true...  I usually end up with so much info
turned up there is no way I'll even scan it all, plus a very lot of
it is either so dated as to nearly be irrelevant or just hives off
into something unrelated with the right catch words.

The info manuals, especially when read thru emacs, are much more focused
compared to the scatter gun mess on google.

The `i' search beats the snot out of any kind of + - or other wannabe
filter possible on google.

Then there is the `s' (regular expression) search you cannot even come
close to on google.

In particular the emacs manual has had many an hour spent by lots of
people setting up a truly comprehensive `'' (index) search.

Further more, if you do find a shortcoming in the info manual or the
searching, there are folks who will fix it... try that on google.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> From: Tom <adatgyujto@gmail.com>
> Date: Sun, 21 Dec 2014 21:32:13 +0000 (UTC)
> 
> Users want to get the information fast and it is much faster to
> search for something in google than trying to find the relevant
> section of the manual.

That's simply incorrect, if you use the 'i' command in Info, which is
the main way of searching an Info manual.

> The manual is great for looikng up again something which you know
> where to find. E.g. what kind of text properties are there?
> 
> C-h i -> m elisp -> g -> text properties
> 
> But if you want to lookup something unfamiliar then google is much
> more efficient.

Give me an example of what you want to find, and I will show you how
to get there faster than with Google.

Once again, with Google, you never know whether the info you see is up
to date or even correct.  That's one definite advantage of looking up
in manuals installed on your system -- you _kinow_ it's correct and
accurate (barring docs bugs).

> I wonder how many people read the manual from the beginning to the
> end like a book. These days (when average attention span is very
> short due to years of filtering through huge volumes of information
> on the net) I'd guess not many.

Irrelevant.  Finding information quickly and efficiently calls for a
very different kind of reading than when you read the entire manual.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Mon, Dec 22, 2014 at 4:36 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Tom <adatgyujto@gmail.com>
>> Date: Sun, 21 Dec 2014 21:32:13 +0000 (UTC)
>>
>> Users want to get the information fast and it is much faster to
>> search for something in google than trying to find the relevant
>> section of the manual.
>
> That's simply incorrect, if you use the 'i' command in Info, which is
> the main way of searching an Info manual.

That is perhaps more an opinion? I definitively think users who know
Google search well can find the information very quickly that way. And
new users - who we want to help, of course - probably are very good at
that.

That is my opinion. ;-)


> Once again, with Google, you never know whether the info you see is up
> to date or even correct.

This is not true. You do not have to search the whole internet just
because you use Google. You could do something like this:

   Google: "site:www.gnu.org/software/emacs/24.2/manual/ some thing"

If that folder exists there, of course.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Yuri Khan]

On Mon, Dec 22, 2014 at 9:54 AM, Lennart Borgman
<lennart.borgman@gmail.com> wrote:

>>> Users want to get the information fast and it is much faster to
>>> search for something in google than trying to find the relevant
>>> section of the manual.
>>
>> That's simply incorrect, if you use the 'i' command in Info, which is
>> the main way of searching an Info manual.
>
> That is perhaps more an opinion? I definitively think users who know
> Google search well can find the information very quickly that way. And
> new users - who we want to help, of course - probably are very good at
> that.

I use Google to search for information about Emacs, unless I know
exactly what I’m looking for.

E.g. I use <f1> k and <f1> f and <f1> v, and to a lesser extent <f1>
w, to ask Emacs itself about a particular key, function, variable or
command when I know the name exactly or well enough to use completion.
In these cases, I want authoritative information.

On the other hand, when my question is more like “How do I <do X> in
Emacs”, I’m not specifically looking for a page in the Emacs manual.
Rather, I want a page in the manual, plus a range of ways how other
people do X, and a range of opinions on why X is the wrong thing to
want to do. Google gives me that. Info doesn’t. *By searching with
Google, I extend the scope of my search beyond the Emacs manual.*


Additionally, the HTML rendition of the Emacs manual is more
convenient to read for me, because of these differences:

* The HTML version wraps to the size of my browser. The Info version
is hard-wrapped for 72 columns.
* The HTML version uses my preferred proportional font for prose and
my preferred monospace font for code. The Info version is monospace
throughout, except for headings.
* The HTML version uses text styles to highlight different kinds of
text (keys, commands, paths, arguments, etc.). The Info version uses
the spacing grave accent and the straight single quote and all-caps
formatting.
* The HTML version uses typographic quotes “”. The Info version uses
straight quotes "".

To summarize the above, the HTML version “feels like” an electronic
version of a well-typeset printed book. The Info version feels like an
electronic version of a samizdat book typed on a typewriter.
*Readability counts.*

* Firefox provides pixelwise autoscrolling on middle mouse button, and
opens links in new tabs when middle-clicked. Emacs Info has no
pixelwise scrolling, no autoscrolling, and prefers to have no more
than one *info* buffer.


Replacing the Info output format with HTML and replacing the Emacs
Info browser with an Emacs HTML-Info browser might help with the
readability issue. Improving the Emacs display engine might provide a
better reading experience. But the search scope issue requires an
all-encompassing Web search engine.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> Date: Mon, 22 Dec 2014 12:25:19 +0700
> From: Yuri Khan <yuri.v.khan@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>, Tom <adatgyujto@gmail.com>, 
> 	Emacs-Devel devel <emacs-devel@gnu.org>
> 
> I use Google to search for information about Emacs, unless I know
> exactly what I’m looking for.

That's a mistake.  The Info's 'i' command is precisely the means to
use when you do NOT know exactly what you are looking for.  I urge you
to try that next time: at 'i's prompt type some word or phrase that
you think relates to the subject you are after, and see what happens.
You can also use TAB to see possible completions, once you've typed
some part of the subject.  If the first hit is not what you want, type
',' to see more hits.

The Info manuals are indexed up front with this usage pattern in mind,
and you'd be surprised how efficient this search can be.  Well, with
good manuals, anyway.  (Emacs manuals are good.)  We add index entries
all the time to continuously improve the indexing.

> On the other hand, when my question is more like “How do I <do X> in
> Emacs”, I’m not specifically looking for a page in the Emacs manual.
> Rather, I want a page in the manual, plus a range of ways how other
> people do X, and a range of opinions on why X is the wrong thing to
> want to do. Google gives me that. Info doesn’t. *By searching with
> Google, I extend the scope of my search beyond the Emacs manual.*

You also get a load of crap.  Just sifting through all that trying to
make up your mind what is true and what's wrong will take you on a
small research trip.

Instead, start with looking up the subject in the manual, using the
'i' command and the hyper-links in the nodes where 'i' takes you.
_Then_, after you know what the manuals say, by all means use Google
or some other search engine to expand your knowledge.  The difference
is that you no longer need to find the documentation itself, just how
the feature is used and what are its alternatives (some of that is
also in the manual, btw).

> Additionally, the HTML rendition of the Emacs manual is more
> convenient to read for me, because of these differences:
> 
> * The HTML version wraps to the size of my browser. The Info version
> is hard-wrapped for 72 columns.

I encourage you (or anyone else) to enhance info.el, which will remove
or hide the newlines from the explanatory text, and then use word-wrap
and wrap-prefix to get the same effect as you see in HTML browsers.
(Not that I understand why it would be more readable to have the
description in 200-column lines, but if someone wants such a feature,
why not?)  The only not-entirely-trivial part here is to identify the
lines where the newlines should be kept, like examples, list items,
etc.  But there are enough clues in the text to identify those, in the
same manner as we identify the section headings.

> * The HTML version uses my preferred proportional font for prose and
> my preferred monospace font for code. The Info version is monospace
> throughout, except for headings.

Likewise: should be easy to do this for Info.  Patches are welcome.

> * The HTML version uses text styles to highlight different kinds of
> text (keys, commands, paths, arguments, etc.). The Info version uses
> the spacing grave accent and the straight single quote and all-caps
> formatting.
> * The HTML version uses typographic quotes “”. The Info version uses
> straight quotes "".

Some of this is simply untrue nowadays, I guess you haven't looked at
an Info manual for a few years.  The rest (like using CAPS for
arguments) is again not too hard to teach Info to do.  Once more,
patches welcome.

> To summarize the above, the HTML version “feels like” an electronic
> version of a well-typeset printed book. The Info version feels like an
> electronic version of a samizdat book typed on a typewriter.
> *Readability counts.*

If this is an itch to scratch, I invite you and others to scratch it.
Should be a fine project, and not a hard one, either.  Volunteers are
welcome.

> * Firefox provides pixelwise autoscrolling on middle mouse button, and
> opens links in new tabs when middle-clicked. Emacs Info has no
> pixelwise scrolling

Emacs does support pixelwise scrolling, you can see it in action when
scrolling a large image or very large text.  Making this work for
"normal" text is not hard, it's just that no one did it.  Again,
volunteers are welcome.  (I can provide hints and help on this one, if
necessary.)

> no autoscrolling

Not sure what that means.  Emacs certainly auto-scrolls when point
enters the scroll-margin.

> and prefers to have no more than one *info* buffer.

Not true.  I have between 40 and 50 *info* buffers in my Emacs session
at any given time (I wrote the info-display-manual command to help me
manage them efficiently), and see no problems with that.

> Replacing the Info output format with HTML and replacing the Emacs
> Info browser with an Emacs HTML-Info browser might help with the
> readability issue.

As was written many times here, HTML-Info browser you are talking
about doesn't exist.  It needs to be coded first.  Existing HTML
browsers lack a few important features, they were identified in these
discussions.  It is not useful to compare a live, working program with
an _idea_ of a program.  It certainly makes no sense to claim the idea
is much better than the program.

> Improving the Emacs display engine might provide a better reading
> experience.

The Emacs display engine doesn't need to be improved to support the
features you miss.  What we need is motivated individual(s) who would
sit down and code this in Lisp using _existing_ features.  I hope you
will be that volunteer, or that someone else will come soon, because
frankly, I enjoy much more seeing Emacs improve and helping others
improve it than talk-talk-talk about that.

> But the search scope issue requires an all-encompassing Web search
> engine.

I suggest to give another chance to the Info's 'i' command.  You might
even like it.  If the initial experience is not good enough, come back
and tell why, maybe we could help; after all, using Google efficiently
also takes some practice.  (FWIW, I usually land right on the spot
using 'i' on my first attempt, sometimes the second, and it's not
because I remember all the index entries by heart, far from it.)

Please note that I never said Info is a replacement for searching the
net.  That is a red herring.  What I am saying is that whenever you
need accurate information about some Emacs feature, you should first
look up its description in the manuals.  Then, armed with that
knowledge, go search the net for more info, if you need to.  Most
probably your Google query will be different if you follow this
paradigm.

Btw, if you, or someone else, are ambitious, I can suggest a larger
and more challenging project: add a front end to Info's search
capabilities that can accept queries in more-or-less natural language,
not unlike Google.  Examples of such queries:

  . "how do I do SOMETHING?"
  . "what is THAT-THING?"
  . "look for SUBJECT but excluding THIS-CRAP"

etc.  Bonus points for maintaining a database of user-specific
preferences and personal style of queries, and applying that to future
queries.

Interested?

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Yuri Khan]

On Mon, Dec 22, 2014 at 9:43 PM, Eli Zaretskii <eliz@gnu.org> wrote:

>> I use Google to search for information about Emacs, unless I know
>> exactly what I’m looking for.
>
> That's a mistake.  The Info's 'i' command is precisely the means to
> use when you do NOT know exactly what you are looking for.  I urge you
> to try that next time: at 'i's prompt type some word or phrase that
> you think relates to the subject you are after, and see what happens.

This is limited to the features that are currently installed. The
question “how do I do X” does not always mean “how do I do X using
features I already have installed” — rather frequently it means “what
can I install in order to do X”.

> The Info manuals are indexed up front with this usage pattern in mind,
> and you'd be surprised how efficient this search can be.  Well, with
> good manuals, anyway.  (Emacs manuals are good.)  We add index entries
> all the time to continuously improve the indexing.

What good is an efficient search facility if it is limited to good manuals?

I program in several languages, not only and not primarily Elisp. I
want to have a single search habit which works for all languages,
libraries and tools that I use. Typing “gg <tool name> <feature>” into
my Firefox’s address bar gives me that. Info, only if the relevant
Info manuals exist, are installed and contain the information I want.

I come from Windows background and I used to be used to well-indexed
MSDN Library manuals. (This was before they replaced their hand-made
indexes with machine-generated crap.) I know the value of a good index
and I miss them. But unless all tools I need to use come with Info
manuals, I will still have to search the Web.

> I encourage you (or anyone else) to enhance info.el, which will remove
> or hide the newlines from the explanatory text, and then use word-wrap
> and wrap-prefix to get the same effect as you see in HTML browsers.
> (Not that I understand why it would be more readable to have the
> description in 200-column lines, but if someone wants such a feature,
> why not?)  The only not-entirely-trivial part here is to identify the
> lines where the newlines should be kept, like examples, list items,
> etc.  But there are enough clues in the text to identify those, in the
> same manner as we identify the section headings.

You are suggesting that I solve a backward problem — inferring
structure given a hard-wrapped text rendition. And, as much as I can
infer without reading the Info source, it’s all like that — first
render to an unparseable format, then heuristically infer structure.
Why do that when it’s possible to just not lose structural information
at all?

And, where do you get that mythical 200-column width? A 24" display
accommodates two side-by-side frames, 100 columns each.

>> * The HTML version uses my preferred proportional font for prose and
>> my preferred monospace font for code. The Info version is monospace
>> throughout, except for headings.
>
> Likewise: should be easy to do this for Info.  Patches are welcome.

I might do that *if* Info were sufficiently better for me than
Google-indexed HTML. As it stands, it is not.

>> * The HTML version uses text styles to highlight different kinds of
>> text (keys, commands, paths, arguments, etc.). The Info version uses
>> the spacing grave accent and the straight single quote and all-caps
>> formatting.
>> * The HTML version uses typographic quotes “”. The Info version uses
>> straight quotes "".
>
> Some of this is simply untrue nowadays, I guess you haven't looked at
> an Info manual for a few years.

Emacs manual, emacs24-common-non-dfsg 24.3+1-1 as packaged in Ubuntu
Trusty on 2013-04-13. Ok, maybe it’s ancient; I don’t know.

>> To summarize the above, the HTML version “feels like” an electronic
>> version of a well-typeset printed book. The Info version feels like an
>> electronic version of a samizdat book typed on a typewriter.
>> *Readability counts.*
>
> If this is an itch to scratch, I invite you and others to scratch it.
> Should be a fine project, and not a hard one, either.  Volunteers are
> welcome.

I see no reason to. Improving the HTML output to support indexes is a
much more productive goal.

>> no autoscrolling
>
> Not sure what that means.  Emacs certainly auto-scrolls when point
> enters the scroll-margin.

In browser parlance, “autoscrolling” means that you can press the
middle mouse button and the content starts to scroll.

It comes in two minor variations.

* You can press and release, then a circle appears around the point
where you middle-clicked, and as soon as you move the mouse outside
this circle, the content starts to scroll at a constant speed in the
direction where you moved the mouse. Move it farther, and it scrolls
faster. Move it closer, and it scrolls slower. Move it back inside the
circle, and it stops but is ready to start scrolling again if you move
outside. Middle-click again to stop.

* Or, you can press and hold. Then it works the same way as described
above, except that releasing the middle button exits the scroll mode.

It’s somewhat similar to wheel scrolling but different.

> As was written many times here, HTML-Info browser you are talking
> about doesn't exist.  It needs to be coded first.  Existing HTML
> browsers lack a few important features, they were identified in these
> discussions.

Works for me, except for the wonderful hand-crafted indexes.


> Btw, if you, or someone else, are ambitious, I can suggest a larger
> and more challenging project: add a front end to Info's search
> capabilities that can accept queries in more-or-less natural language,
> not unlike Google.  Examples of such queries:
>
>   . "how do I do SOMETHING?"
>   . "what is THAT-THING?"
>   . "look for SUBJECT but excluding THIS-CRAP"
>
> etc.  Bonus points for maintaining a database of user-specific
> preferences and personal style of queries, and applying that to future
> queries.
>
> Interested?

Might be a good research project for a candidate dissertation in
linguistics/programming. Requires much more time than I’m prepared to
invest; sorry.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> Date: Tue, 23 Dec 2014 01:26:20 +0700
> From: Yuri Khan <yuri.v.khan@gmail.com>
> Cc: Lennart Borgman <lennart.borgman@gmail.com>, Tom <adatgyujto@gmail.com>, 
> 	Emacs developers <emacs-devel@gnu.org>
> 
> On Mon, Dec 22, 2014 at 9:43 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> >> I use Google to search for information about Emacs, unless I know
> >> exactly what I’m looking for.
> >
> > That's a mistake.  The Info's 'i' command is precisely the means to
> > use when you do NOT know exactly what you are looking for.  I urge you
> > to try that next time: at 'i's prompt type some word or phrase that
> > you think relates to the subject you are after, and see what happens.
> 
> This is limited to the features that are currently installed. The
> question “how do I do X” does not always mean “how do I do X using
> features I already have installed” — rather frequently it means “what
> can I install in order to do X”.

That's a far cry from your original statement, see above.  I'm
guessing that Emacs belongs to the features that are installed on your
system, yes?

> > The Info manuals are indexed up front with this usage pattern in mind,
> > and you'd be surprised how efficient this search can be.  Well, with
> > good manuals, anyway.  (Emacs manuals are good.)  We add index entries
> > all the time to continuously improve the indexing.
> 
> What good is an efficient search facility if it is limited to good manuals?

Are we still talking about Emacs here?  Because that's what this
discussion is about, right?  Solving the problems of the rest of the
world might take a little longer.

> I program in several languages, not only and not primarily Elisp. I
> want to have a single search habit which works for all languages,
> libraries and tools that I use. Typing “gg <tool name> <feature>” into
> my Firefox’s address bar gives me that. Info, only if the relevant
> Info manuals exist, are installed and contain the information I want.

Suit yourself, but IME limiting your habits to a single tool will
yield a limited solution.  This area is not yet developed enough to
have one-fits-all solutions, so using the best tool for each job is
still better.

> I know the value of a good index and I miss them. But unless all
> tools I need to use come with Info manuals, I will still have to
> search the Web.

Yes, you will.  I see no problems with that.  There will always be
dark corners not described in any manual.

> > I encourage you (or anyone else) to enhance info.el, which will remove
> > or hide the newlines from the explanatory text, and then use word-wrap
> > and wrap-prefix to get the same effect as you see in HTML browsers.
> > (Not that I understand why it would be more readable to have the
> > description in 200-column lines, but if someone wants such a feature,
> > why not?)  The only not-entirely-trivial part here is to identify the
> > lines where the newlines should be kept, like examples, list items,
> > etc.  But there are enough clues in the text to identify those, in the
> > same manner as we identify the section headings.
> 
> You are suggesting that I solve a backward problem — inferring
> structure given a hard-wrapped text rendition. And, as much as I can
> infer without reading the Info source, it’s all like that — first
> render to an unparseable format, then heuristically infer structure.
> Why do that when it’s possible to just not lose structural information
> at all?

You could start with HTML output of makeinfo, if that makes the job
easier.  I don't think anyone will mind.

> >> * The HTML version uses my preferred proportional font for prose and
> >> my preferred monospace font for code. The Info version is monospace
> >> throughout, except for headings.
> >
> > Likewise: should be easy to do this for Info.  Patches are welcome.
> 
> I might do that *if* Info were sufficiently better for me than
> Google-indexed HTML. As it stands, it is not.

Why not?  Above you didn't give any arguments for that, you just said
that you'll need Google anyway.

> >   . "how do I do SOMETHING?"
> >   . "what is THAT-THING?"
> >   . "look for SUBJECT but excluding THIS-CRAP"
> >
> > etc.  Bonus points for maintaining a database of user-specific
> > preferences and personal style of queries, and applying that to future
> > queries.
> >
> > Interested?
> 
> Might be a good research project for a candidate dissertation in
> linguistics/programming. Requires much more time than I’m prepared to
> invest; sorry.

Sigh.  If the time and energy wasted on these endless discussions why
Emacs and Info are so bad were used for development, we might have
been there already.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Yuri Khan]

On Tue, Dec 23, 2014 at 12:48 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> The
>> question “how do I do X” does not always mean “how do I do X using
>> features I already have installed” — rather frequently it means “what
>> can I install in order to do X”.
>
> That's a far cry from your original statement, see above.  I'm
> guessing that Emacs belongs to the features that are installed on your
> system, yes?

Emacs does. Ten thousand packages for Emacs do not, yet.

Example: I want to be able to perform automated refactoring of Python
code. The answer I’m seeking is “go install the ropemacs package” but
Info won’t find that.

>> > I encourage you (or anyone else) to enhance info.el
> You could start with HTML output of makeinfo, if that makes the job
> easier.  I don't think anyone will mind.

The HTML output of makeinfo already works for me well enough. I just
choose to view it in Firefox rather than Emacs and cope with Google’s
automatic indexes instead of hand-tuned Info indexes.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Paul Eggert]

Yuri Khan wrote:

> the HTML version “feels like” an electronic
> version of a well-typeset printed book. The Info version feels like an
> electronic version of a samizdat book typed on a typewriter.
> *Readability counts.*

This is a fair point.  It's partly because the process we use to generate an 
Emacs distribution still uses Texinfo 4 to generate the info files.  We should 
at least use Texinfo 5, so that the info files have proper quotes, which would 
be an improvement (although, unfortunately, it would not address your other 
points, where our info readers indeed have problems).

> Improving the Emacs display engine might provide a
> better reading experience. But the search scope issue requires an
> all-encompassing Web search engine.

I agree that better use of search engines would be a big improvement.  Others 
have argued in this thread that our manual indexes suffice, but I'm afraid my 
experience is otherwise.

Just today, for example, I needed to come up to speed on how composite glyphs 
work in Emacs, a topic I knew little about; at first I didn't even remember what 
they were.  I hardly ever use the Emacs manuals' indexes due to my 
unsatisfactory past experiences with them, but since they're a current topic of 
discussion in this thread I thought I'd try them again. I went to the Emacs 
manual in info mode and typed 'i composite RET'; the response was "No 
`composite' in index" (complete with those amateurish circa-1980 
not-really-quotes -- "samizdat", I'll have to remember that description :-). 
Since I'm supposedly expert I switched to the for-experts-only Elisp manual and 
typed 'i composite RET', and got sent to an irrelevant page about composite 
types.  In short, the info-mode indexes were utterly a failure for this example. 
  Which isn't surprising, since none of the Emacs manuals talk about composite 
glyphs anywhere.  (I verified this by using other tools, as this was faster than 
using info mode would have been.)

In contrast, Google searches were reasonably helpful.  The search '"composite 
glyph"' quickly got me up to speed on the general topic, and '"composite glyph" 
Emacs' gave me helpful bug reports that let me intuit relevant issues reasonably 
well.

If one prefers a traditional manual, the use-Google approach can be *really* 
annoying, as it tends to throw up all sorts of trash, and I understand the 
annoyance.  Really, I do.  But what can I say?  It often works way better, and 
we should be exploiting this technology rather than limiting ourselves to the 
not-particularly-successful tactic of asking people to send us bug reports and 
fixes for our manuals.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> Date: Mon, 22 Dec 2014 17:41:01 -0800
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: Eli Zaretskii <eliz@gnu.org>, Tom <adatgyujto@gmail.com>, 
>  Emacs-Devel devel <emacs-devel@gnu.org>
> 
> Just today, for example, I needed to come up to speed on how composite glyphs 
> work in Emacs, a topic I knew little about; at first I didn't even remember what 
> they were.

So now we are down to saying Info is not good because it cannot find
information that no one cared to write in a manual?  How reasonable is
that?  Did you never happen to search in Google for information that
simply isn't there?  When all you get is hits in acronym lists and
irrelevant sites in Chinese?  Does that mean Google is good for
nothing?

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > So now we are down to saying Info is not good because it cannot find
 > information that no one cared to write in a manual?

Calm down, Eli.  Paul is on the side of the angels, he just wants to
add a new useful tool (web search), not obsolete an old useful tool
(Info).

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> Date: Tue, 23 Dec 2014 14:01:30 +0900
> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: Paul Eggert <eggert@cs.ucla.edu>,
>     emacs-devel@gnu.org,
>     lennart.borgman@gmail.com,
>     adatgyujto@gmail.com,
>     yuri.v.khan@gmail.com
> 
> Paul is on the side of the angels, he just wants to add a new useful
> tool (web search)

That's a no-brainer.  No one said anything to the contrary, so there's
no need to argue with that non-existing argument.  Web search is a
valuable tool when you want to expand on the information you found in
the manuals.  However, for information that is in the manuals, Info is
usually faster and more accurate.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Eli Zaretskii writes:
 > > Date: Tue, 23 Dec 2014 14:01:30 +0900
 > > From: "Stephen J. Turnbull" <stephen@xemacs.org>

 > > Paul is on the side of the angels, he just wants to add a new useful
 > > tool (web search)
 > 
 > That's a no-brainer.

Apparently the word "just" was not.  Despite his disclaimers, you took
his post as a deprecation of manuals in Info format.

 > No one said anything to the contrary, so there's no need to argue
 > with that non-existing argument.

Please slow down, Eli.  Nobody said there was an argument with that.
My intention was to imply that you seem to be misinterpreting his post
which was intended to say that, and not intended to say that web search
should *replace* Info indexes.  Obviously the courtesy was wasted and
I should have said "you seem to be misinterpreting" explicitly.

 > Web search is a valuable tool when you want to expand on the
 > information you found in the manuals.  However, for information
 > that is in the manuals, Info is usually faster and more accurate.

"Nobody said anything to the contrary, so why are you arguing with
that nonexistent argument?"

Some people are arguing that for them and for people like them Emacs
manuals at present do *not* have the necessary information indexed,
and that therefore web search is a relatively cheap way to improve
access to information about Emacs.  A few deprecate Info manuals[1],
it's true, but surely Paul doesn't.

Footnotes: 
[1]  Because they believe "people like them" to be the great majority,
and apparently that people like you will adapt to the new format at
little cost -- I believe the second half is quite false.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: yuri.v.khan@gmail.com,
>     eggert@cs.ucla.edu,
>     lennart.borgman@gmail.com,
>     adatgyujto@gmail.com,
>     emacs-devel@gnu.org
> Date: Wed, 24 Dec 2014 11:58:16 +0900
> 
>  > > Paul is on the side of the angels, he just wants to add a new useful
>  > > tool (web search)
>  > 
>  > That's a no-brainer.
> 
> Apparently the word "just" was not.  Despite his disclaimers, you took
> his post as a deprecation of manuals in Info format.

Yes.  And I stand by that interpretation.

>  > No one said anything to the contrary, so there's no need to argue
>  > with that non-existing argument.
> 
> Please slow down, Eli.  Nobody said there was an argument with that.
> My intention was to imply that you seem to be misinterpreting his post
> which was intended to say that, and not intended to say that web search
> should *replace* Info indexes.

Based on what Paul wrote since then (cf. "inferior technology"), I
think my interpretation was not a mistake.

>  > Web search is a valuable tool when you want to expand on the
>  > information you found in the manuals.  However, for information
>  > that is in the manuals, Info is usually faster and more accurate.
> 
> "Nobody said anything to the contrary, so why are you arguing with
> that nonexistent argument?"

Because someone did say that.  Several someone's, in fact.

> Some people are arguing that for them and for people like them Emacs
> manuals at present do *not* have the necessary information indexed,
> and that therefore web search is a relatively cheap way to improve
> access to information about Emacs.  A few deprecate Info manuals[1],
> it's true, but surely Paul doesn't.

I have no argument with those "some" (and actually did some work
yesterday to improve on that), but violently disagree with those
"few".  I don't think I've misinterpreted to which group Paul belongs,
but that doesn't really matter, as my problem is with the views of
those "few", not with them personally.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Paul Eggert writes:

 > If one prefers a traditional manual, the use-Google approach can be
 > *really* annoying, as it tends to throw up all sorts of trash, and
 > I understand the annoyance.  Really, I do.  But what can I say?  It
 > often works way better, and we should be exploiting this technology
 > rather than limiting ourselves to the not-particularly-successful
 > tactic of asking people to send us bug reports and fixes for our
 > manuals.

Google search is (a) SaaS (although it is free as in Japanese sake)
and (b) privacy-invading.  You (and I!) may not care but Richard does.
We should probably try duckduckgo as default.

RE: Have you all gone crazy? Was: On being web-friendly and why info must die

[Drew Adams]

> how composite glyphs work in Emacs...  I went to the Emacs manual
> in info mode and typed 'i composite RET'; the response was "No
> `composite' in index"

To begin with, I would expect that the Emacs manual would be a bad
choice for information about composite glyphs.  (Char composition
is covered briefly in the Emacs manual, and if you type `compos
TAB' then you see candidate `compose character'.  But that's not
the same thing.)

> Since I'm supposedly expert I switched to the for-experts-only
> Elisp manual 

It is not at all for-experts-only.  It is simply the Lisp manual
for Emacs.

> and typed 'i composite RET', and got sent to an irrelevant page
> about composite types.

FWIW, had you typed `compos TAB', that would have been expanded to
`composit', and you would have seen candidates `composition (text
property)' and `composition property, and point display', together
with the irrelevant entry you followed, `composite types
(customization)'.

> In short, the info-mode indexes were utterly a failure for
> this example.

Eggert-mode + info-mode was utterly a failure for this example. ;-)

> Which isn't surprising, since none of the Emacs manuals talk about
> composite glyphs anywhere.  (I verified this by using other tools,
> as this was faster than using info mode would have been.)

Hm.  They might not cover the information you want, but it's not
true that they do not talk about composite glyphs anywhere.

 `composition'
  This text property is used to display a sequence of characters
  as a single glyph composed from components.  But the value of the
  property itself is completely internal to Emacs and should not be
  manipulated directly by, for instance, `put-text-property'.

But yes, that is not very much info about composite glyphs.

If the info you are looking for is *not in the manuals*,
then it is certainly not a failure of their indexing that
consulting the indexes didn't help you find that info.
That's a no-brainer.

> In contrast, Google searches were reasonably helpful.

No, not wrt the Emacs manual; not per your example.

No one has claimed that the Emacs manuals replace
Internet-wide searches for information about Emacs.
Or that they should or could do that.  Red herring.

It's not about one or the other, manuals vs Internet.
There will always be more information about Emacs on the
Internet than is contained in the Emacs manuals - the
manuals are on the Internet.

> The search '"composite glyph"' quickly got me up to speed on
> the general topic, and '"composite glyph" Emacs' gave me helpful
> bug reports that let me intuit relevant issues reasonably well.

No one denies that Internet searching is powerful and helpful.
But it didn't help you, in this example, to find something in
the Emacs manuals - something that isn't there.

IOW, hardly a demonstration of the failings of Info indexing.

> If one prefers a traditional manual, the use-Google approach
> can be *really* annoying, as it tends to throw up all sorts of
> trash, and I understand the annoyance.  Really, I do.

Do you also understand that it's not about a choice *between*
"traditional manual" and "the use-Google approach"?  That's a
false choice.  No one is out to replace googling.

There can be more than one tool in your tool belt, just as
there is more than one book in the library.  Choose both,
not between.

Sticking with the library, your little anecdote is like this
one: I went to the library and looked in the index of a book
for information about TVs.  I didn't find anything about TVs
there.  I even searched every page of that book, but I found
nothing about TVs.  Damn!

Then I asked the librarian for help, and s?he pointed me
(immediately!) to plenty of magazines and books that talk
about TVs.  Clearly that first book's index is total crap -
utterly a failure.

> But what can I say?  It often works way better,

Yes, of course it works better at some things - many things.
Most things.  It indexes the entire web!  Everyday.  Very
well.  That's really not the point.

Is it better at navigating the Emacs manuals?  Is it better
at helping you find information in the Emacs manuals?  Those
are reasonable & interesting questions that might lead to
our improving a user's experience with the manuals.

But arguing that googling is powerful is a waste of time.
And telling us that if you google poorly you can end up 
with noise is also a waste of time.  We all google.
We all know its strengths and weaknesses.

> we should be exploiting this technology

What's your proposition for exploiting the technology?
I exploit it everyday, when I search for things.  Just
what did you have in mind?

Google and other search engines already index our
manuals for full-text search.  You can already use web
search to find information in the manuals.  That's not
a missing feature.

What is missing in the web versions of the manuals has
already been discussed somewhat.  Some of what's missing
could perhaps be provided, even including Info-style
navigation and indexing perhaps.

And even including the feature of *being inside Emacs*,
if you browse and search the web using Emacs.

I don't do the latter, personally.  But if I end up in
the web manual from some Internet search, I do sometimes
copy the node name and then yank it into my local Emacs
manual (Info), for further navigation and investigation.

(That's essentially the manual inverse of what the code
does that I sent a couple days ago, which makes the link
in the other direction, from Info node to web-manual node.)

And I don't move back to Info inside Emacs just because
Info has better navigation and investigation features
for the manuals (for now, at least).  I do it also
because *Emacs is where I want to be* when I am consuming
information about Emacs - because I use the information
with Emacs.

That feature of local Info should also be a no-brainer,
when it comes to the Emacs manuals: you want to *use*
the information about Emacs for Emacs, with Emacs.

> rather than limiting ourselves to the
> not-particularly-successful tactic of asking people to
> send us bug reports and fixes for our manuals.

Again, a strange, black-&-white either-or.  No one has
argued that we should limit ourselves to soliciting bug
reports and fixes for the manuals.

But had you found that there *was* a section of the manual
that covered composite glyphs the way you wanted, and had
you felt it was not easy to find that section using `i',
then I would hope that you would take the time to suggest
an index entry or two for it.

You can do both: file bug reports and search the Internet.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Paul Eggert]

Drew Adams wrote:
> You can do both: file bug reports and search the Internet.

Sure, but there are good reasons that searching is way more popular than filing 
bug reports.  Although traditional indexed manuals needn't be discarded if 
already written, other ways of navigating through Emacs are increasingly 
cost-effective, and this suggests that we should shift some of our limited 
development resources away from the tedium of writing and indexing the manuals. 
  This shifting is already happening, and it's something we should welcome 
rather than decry.

Composite glyphs are admittedly a poorly-documented topic in the Emacs manuals, 
so here's an example using a well-documented topic.  Let's say I want the time 
of day in Emacs.  If I visit the Elisp manual in info mode and type 'i time of 
day RET' Emacs responds "No `timestamp of day' in index" (there's that ugly 
1980s-style quoting again!) and fails, even though there's a perfectly good 
section called "Time of Day" in the Elisp manual -- what's up with that and why 
did Emacs insist on changing "time" to "timestamp" and then getting lost?

In contrast, the Google search 'Emacs "time of day"' yields a first hit that's 
precisely what's needed.

This was the very first example I tried while writing this email -- it's not a 
contrived example.  I hope it helps to explain why search engines are far more 
popular for this sort of thing.  It's not merely that users know search engines 
better than they know Emacs info mode.  It's that the search engines are 
typically better for most users.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > so here's an example using a well-documented topic.  Let's say I want the time 
  > of day in Emacs.  If I visit the Elisp manual in info mode and type 'i time of 
  > day RET' Emacs responds "No `timestamp of day' in index" (there's that ugly 
  > 1980s-style quoting again!) and fails,

Would you like to fix this?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Paul Eggert]

Richard Stallman wrote:
> Would you like to fix this?

I suppose I could help fix the diagnostic's obsolete quoting format, yes.  More 
important, perhaps I could add infrastructure to cause diagnostics to be 
translated from English into the user's preferred language.  (The infrastructure 
would provide a mechanism for other people to supply translations.)  Other GNU 
applications have done these things for years, but Emacs mostly does not, and 
this is an obstacle to its use.

As far as fixing that particular index entry, though, I'm afraid I'd rather not. 
  There are so many things to fix in Emacs, and so little time!  This sort of 
fix doesn't make the cut, as the manual's indexes are becoming less important.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I think the manual's indexes are important.  People cited flaws in
them here as arguments for using a search engine instead.  If the flaws
are enough to argue for that, they are enough to fix.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> Date: Tue, 23 Dec 2014 10:34:04 -0500
> From: Richard Stallman <rms@gnu.org>
> Cc: adatgyujto@gmail.com, lennart.borgman@gmail.com, emacs-devel@gnu.org,
> 	eliz@gnu.org, yuri.v.khan@gmail.com, drew.adams@oracle.com
> 
>   > so here's an example using a well-documented topic.  Let's say I want the time 
>   > of day in Emacs.  If I visit the Elisp manual in info mode and type 'i time of 
>   > day RET' Emacs responds "No `timestamp of day' in index" (there's that ugly 
>   > 1980s-style quoting again!) and fails,
> 
> Would you like to fix this?

Which part?  The indexing part is already fixed.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >   If I visit the Elisp manual in info mode and type 'i time of 
  > >   > day RET' Emacs responds "No `timestamp of day' in index" (there's that ugly 
  > >   > 1980s-style quoting again!) and fails,
  > > 
  > > Would you like to fix this?

  > Which part?  The indexing part is already fixed.

The change from "time" to "timestamp" is what seems bad.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> Date: Mon, 22 Dec 2014 23:32:39 -0800
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: Eli Zaretskii <eliz@gnu.org>, Tom <adatgyujto@gmail.com>, 
>  Emacs-Devel devel <emacs-devel@gnu.org>
> 
> Composite glyphs are admittedly a poorly-documented topic in the
> Emacs manuals, so here's an example using a well-documented
> topic. Let's say I want the time of day in Emacs. If I visit the
> Elisp manual in info mode and type 'i time of day RET' Emacs
> responds "No `timestamp of day' in index" (there's that ugly
> 1980s-style quoting again!) and fails, even though there's a
> perfectly good section called "Time of Day" in the Elisp manual --
> what's up with that

It is generally advisable to have in each node an index entry which
matches that node's name or the name of its section.  There are 880
nodes in the ELisp manual, out of which 116, including "Time of Day",
didn't follow that rule (they do now) -- not bad for "samizdat"!

> and why did Emacs insist on changing "time" to "timestamp" and then
> getting lost?

That's our smart completion at work for you, nothing related to Info
in particular.  Feel free to file a bug report.

Anyway, when index search fails for the phrase you type (and you
should try more than one, like 2 or 3 before you give up), what I
normally do is back up a little and use a shorter phrase.  In this
case, I'd type "i time TAB", and that probably would have been enough
in this case to show you what you are after, among a short enough list
of candidates.

Failing that (which would already warrant a bug report), ...

> In contrast, the Google search 'Emacs "time of day"' yields a first
> hit that's precisely what's needed.

...I'd use the Info equivalent of that, "s time of day RET", which
would have instantaneously found you what you were after.  Info also
knows how to search for a string (or regexp).

> This was the very first example I tried while writing this email --
> it's not a contrived example. I hope it helps to explain why search
> engines are far more popular for this sort of thing. It's not merely
> that users know search engines better than they know Emacs info
> mode. It's that the search engines are typically better for most
> users.

I see no evidence here of Google being better for this particular
task.  You just gave up too soon, that's all.

Granted, Info does not pretend to be a sophisticated search engine
anywhere near Google.  But for the job it needs to do it is quite
adequate, and this example shows that clearly, even though it exposed
a (temporary) weakness in our indexing.

And of course, I agree with Drew: no one said you should use either
Google or Info; that is a red herring.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Paul Eggert]

Eli Zaretskii wrote:

> You just gave up too soon, that's all.

No I didn't.  I switched from the info index (which didn't work for me first 
time) to a search engine (which did).  Why should I insist on slowing myself 
down with inferior technology?

Of course this was just one example, but it's representative.  It's long been my 
experience that search engines work better than traditional indexes for most of 
my questions about Emacs functionality.  And I don't think my experience is at 
all atypical.  Feel free to tweak the manually-maintained index, but in the end 
I expect the tweaks won't benefit most users.

> There are 880
> nodes in the ELisp manual, out of which 116, including "Time of Day",
> didn't follow that rule (they do now)

Thanks, that's undoubtedly an improvement to the manually-maintained index, but 
it underscores a problem with the way we're currently using Texinfo.  Here's an 
extract from the current emacs-24 documentation source:

* Standard Abbrev Tables::      Abbrev tables used by various major modes.
...
@node Standard Abbrev Tables
@section Standard Abbrev Tables
@cindex standard abbrev tables
...
* Standard Abbrev Tables::  Abbrev tables used by various major modes.

Although this kind of repetition may be needed in the *output* of makeinfo, it 
shouldn't be necessary in its *input*.  I know why each input line is there, and 
of course there are arguments for doing it in this annoyingly prolix and 
error-prone way, but overall it's clearly a mess that gets in the way of real 
improvements, and contributors shouldn't have to put up with this sort of thing.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Andy Moreton]

On Tue 23 Dec 2014, Paul Eggert wrote:

> Eli Zaretskii wrote:
>
>> You just gave up too soon, that's all.
>
> No I didn't.  I switched from the info index (which didn't work for me first
> time) to a search engine (which did).  Why should I insist on slowing myself
> down with inferior technology?

Your sample size is too small to have any statistical significance. I
have usually found search engines to be vastly inferior to a manual with
a good index, but both mechanisms are useful and both are needed.

> Of course this was just one example, but it's representative.  It's long been
> my experience that search engines work better than traditional indexes for
> most of my questions about Emacs functionality.  And I don't think my
> experience is at all atypical.  Feel free to tweak the manually-maintained
> index, but in the end I expect the tweaks won't benefit most users.

The popular search engines are close to useless for focussed searches
these days, as they are far more interested in delivering advertising
than in producing useful results.

Using a search engine usually results in several pages of irrelevant
links, each of which must be followed in the forlorn hope that any of
them contain anything pertinent and up to date.

Everyone seems to agree that texinfo could do with some performance
improvements. While the info format could happily be jettisoned in
favour of something else, the discussion in this thread seems mostly
about bikeshedding something new and incompatible with any existing
tooling.

The markup used for documentation is not an impediment to writing
documentation - on this ESR is flat out wrong. Eli's offer to add texinfo
markup to documentation written by those without knowledge of info will
result in no extra work, as that offer will not be taken up by anyone.

The use of Texinfo or any other markup format is irrelevant to the
production of high quality documentation. The creation of the initial
plain text prose  is the heart of the problem. Markup and editorial
fixes can be added to existing text, but changing the markup does not
magically produce a working text from thin air.

    AndyM

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> Date: Tue, 23 Dec 2014 13:08:39 -0800
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: drew.adams@oracle.com, yuri.v.khan@gmail.com, 
>  lennart.borgman@gmail.com, adatgyujto@gmail.com, emacs-devel@gnu.org
> 
> I switched from the info index (which didn't work for me first
> time) to a search engine (which did).

You switched too soon, without trying with Info the same approach you
did with Web search -- a search for a fixed string.

> Why should I insist on slowing myself down with inferior
> technology?

Because it's not inferior, at least not in this use case.  It has the
same capabilities for this specific use case, and the hits it brings
you are guaranteed to be accurate for the version of the software you
have installed.

You are free to use whatever you like, of course, but I see no basis
for your assertion that using Info before trying the Web is not a good
idea.

>     There are 880
>     nodes in the ELisp manual, out of which 116, including "Time of Day",
>     didn't follow that rule (they do now)
> 
> 
> Thanks, that's undoubtedly an improvement to the manually-maintained
> index, but it underscores a problem with the way we're currently
> using Texinfo.

If anything, it underscores how well we are using it.

> Here's an extract from the current emacs-24
> documentation source:
> 
> 
> * Standard Abbrev Tables::      Abbrev tables used by various major modes.
> ...
> @node Standard Abbrev Tables
> @section Standard Abbrev Tables
> @cindex standard abbrev tables
> ...
> * Standard Abbrev Tables::  Abbrev tables used by various major modes.

> Although this kind of repetition may be needed in the *output* of
> makeinfo, it shouldn't be necessary in its *input*.

It's not always repetition.  That index entry, like any good index
entry, needs a human judgment call.

But if we care we could ask the Texinfo maintainers to add a feature
where each @node contributes to the Concept Index.  Feel free to
report this to them.

> overall it's clearly a mess that gets in the way of real
> improvements, and contributors shouldn't have to put up with this
> sort of thing.

That's an exaggeration, I think.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Lennart Borgman writes:

 > That is perhaps more an opinion? I definitively think users who
 > know Google search well can find the information very quickly that
 > way.

I use Google a lot for various different purposes, and yes, it has an
excellent thesaurus behind it.  You *can* just type "emacs cut" into
Google and expect to get almost as good results as "emacs kill" in
many circumstances.  (But not in this case.  I  get completely
different results, oriented to very different audiences.  It's not
obvious to me that the "How to use Emacs like Nano in 30 seconds"
answers are very good for users -- I think they should probably be
using Nano in the first place if such answers are sufficient.)

 > And new users - who we want to help, of course - probably are very
 > good at that.

Not in my experience.  The same people who are unwilling to spend an
hour learning to use Info also tend to have poor Google foo, and tend
to believe anything that they read in a source from the first page of
results.  YMMV, of course, but that's my experience.

I'll grant you that my experience with Windows and Mac help is "db;gf"
("don't bother; Google first"), and I'm definitely a n00b with respect
to those OSes and the WIMP-y software written for them (25 years after
starting with them ;-).  The analogous effect *could* be true for
Emacs vs new users, but I hope not.  The Emacs manuals have much more,
and much more informative, content.

 > This is not true. You do not have to search the whole internet just
 > because you use Google. You could do something like this:
 > 
 >    Google: "site:www.gnu.org/software/emacs/24.2/manual/ some thing"

A truly horrible search to type, of course.

 > If that folder exists there, of course.

A problem that Info "i" doesn't have.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> I'll grant you that my experience with Windows and Mac help is "db;gf"
> ("don't bother; Google first"),

I've set up a dual boot computer for my father recently, and the amount
of stuff like "use some arcane modifier key combination in order to get
some system menu in an obscure place populated with more, absolutely
essential configuration options" was a total surprise to me.  The
preinstalled help available is completely and utterly useless for that.

Then you Google for the necessary steps, and the respective information
is found in some step-by-step blogs from a power user.  Nothing from
Microsoft itself.

And those users call UNIX arcane.  I suspect that this kind of "let's do
some secret stuff and tell nobody about it until he begs for it in
person" attitude has colored off somewhat on projects like GNOME3.
I mean, if the market leader gets away with it, they must be doing
something right?

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Mon, Dec 22, 2014 at 7:41 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
> Lennart Borgman writes:
>
>  >
>  >    Google: "site:www.gnu.org/software/emacs/24.2/manual/ some thing"
>
> A truly horrible search to type, of course.

;-) -- You do not do that, normally. Two alternatives:

1. You open an equivalent URL. From within Emacs for example. Or from
a web page.
2. You create a Google CSE. Works very well upto 200 documents (and
perhaps a few hundred more.)

>  > If that folder exists there, of course.
>
> A problem that Info "i" doesn't have.

You can use 1 above to get around that. Or 2, if you have several
Google CSE:s, one for each Emacs version.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> On Mon, Dec 22, 2014 at 7:41 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
>> Lennart Borgman writes:
>>
>>  >
>>  >    Google: "site:www.gnu.org/software/emacs/24.2/manual/ some thing"
>>
>> A truly horrible search to type, of course.
>
> ;-) -- You do not do that, normally. Two alternatives:
>
> 1. You open an equivalent URL. From within Emacs for example. Or from
> a web page.
> 2. You create a Google CSE. Works very well upto 200 documents (and
> perhaps a few hundred more.)

The LilyPond manual pages have a search box that works like the first
option, I think.

Against using a clear text search or the index in Info, it sucks really,
really, really bad.  Part of the problem is that followup
searches/refinements pretty fast end up on some other page.  Part of the
problem is that, well, searching for something on a huge page that is
actually already loaded works not all that well anyway, but then you may
end up on the split version, too.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Mon, Dec 22, 2014 at 1:08 PM, David Kastrup <dak@gnu.org> wrote:
> Lennart Borgman <lennart.borgman@gmail.com> writes:
>
>> On Mon, Dec 22, 2014 at 7:41 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
>>> Lennart Borgman writes:
>>>
>>>  >
>>>  >    Google: "site:www.gnu.org/software/emacs/24.2/manual/ some thing"
>>>
>>> A truly horrible search to type, of course.
>>
>> ;-) -- You do not do that, normally. Two alternatives:
>>
>> 1. You open an equivalent URL. From within Emacs for example. Or from
>> a web page.
>> 2. You create a Google CSE. Works very well upto 200 documents (and
>> perhaps a few hundred more.)
>
> The LilyPond manual pages have a search box that works like the first
> option, I think.

Do you have a link to that on the web?


> Against using a clear text search or the index in Info, it sucks really,
> really, really bad.  Part of the problem is that followup
> searches/refinements pretty fast end up on some other page.

I can't see how that can happen.


> Part of the
> problem is that, well, searching for something on a huge page that is
> actually already loaded works not all that well anyway, but then you may
> end up on the split version, too.

I do not understand how that relates to what we are talking about here.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> On Mon, Dec 22, 2014 at 1:08 PM, David Kastrup <dak@gnu.org> wrote:
>> Lennart Borgman <lennart.borgman@gmail.com> writes:
>>
>>> On Mon, Dec 22, 2014 at 7:41 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
>>>> Lennart Borgman writes:
>>>>
>>>>  >
>>>>  >    Google: "site:www.gnu.org/software/emacs/24.2/manual/ some thing"
>>>>
>>>> A truly horrible search to type, of course.
>>>
>>> ;-) -- You do not do that, normally. Two alternatives:
>>>
>>> 1. You open an equivalent URL. From within Emacs for example. Or from
>>> a web page.
>>> 2. You create a Google CSE. Works very well upto 200 documents (and
>>> perhaps a few hundred more.)
>>
>> The LilyPond manual pages have a search box that works like the first
>> option, I think.
>
> Do you have a link to that on the web?
>
>
>> Against using a clear text search or the index in Info, it sucks really,
>> really, really bad.  Part of the problem is that followup
>> searches/refinements pretty fast end up on some other page.
>
> I can't see how that can happen.

Shrug.  Try the page
<URL:http://lilypond.org/doc/v2.19/Documentation/extending/>.  Enter
into the Search box on the left "ly:context-property".  You get to some
Google search page and the original page is left.  The Google search
terms include a site restriction, but if you do something like C-a in
order to overtype the search term, of course you land somewhere else.

>> Part of the problem is that, well, searching for something on a huge
>> page that is actually already loaded works not all that well anyway,
>> but then you may end up on the split version, too.
>
> I do not understand how that relates to what we are talking about here.

Search engines suck for navigating some document you already have open
in your browser.

And that's what is touted as the way to make everything easier.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Mon, Dec 22, 2014 at 2:13 PM, David Kastrup <dak@gnu.org> wrote:
> Lennart Borgman <lennart.borgman@gmail.com> writes:
>
>>>>
>>>> 1. You open an equivalent URL. From within Emacs for example. Or from
>>>> a web page.
>>>> 2. You create a Google CSE. Works very well upto 200 documents (and
>>>> perhaps a few hundred more.)
>>>
>>> The LilyPond manual pages have a search box that works like the first
>>> option, I think.
>>
>> Do you have a link to that on the web?
>>
>>
>>> Against using a clear text search or the index in Info, it sucks really,
>>> really, really bad.  Part of the problem is that followup
>>> searches/refinements pretty fast end up on some other page.
>>
>> I can't see how that can happen.
>
> Shrug.  Try the page
> <URL:http://lilypond.org/doc/v2.19/Documentation/extending/>.  Enter
> into the Search box on the left "ly:context-property".  You get to some
> Google search page and the original page is left.  The Google search
> terms include a site restriction, but if you do something like C-a in
> order to overtype the search term, of course you land somewhere else.

If you type C-a there you kind of ask for it. ;-)

Yes, it is a bit inconvenient in that regard. Google CSE can take care
of that (the "site:" part is hidden), but the gratis version of CSE is
limited to the number of pages I mentioned.

This is one of the reasons I mentioned http://opensearchserver.com/ in
this thread. Another reason is that I found the gratis CSE offer from
Google a bit unstable. What will they do tomorrow?

(Quite inconvenient, BTW, that they the LilyPond page does not open
the search result in a new window. I can see no good reason for not
using a new window there. But that is the responsibility of those
writing the page at LilyPond.)


>>> Part of the problem is that, well, searching for something on a huge
>>> page that is actually already loaded works not all that well anyway,
>>> but then you may end up on the split version, too.
>>
>> I do not understand how that relates to what we are talking about here.
>
> Search engines suck for navigating some document you already have open
> in your browser.

Really. Because they are not involved at all. ;-)

You can design some JavaScript for that.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > 2. You create a Google CSE. Works very well upto 200 documents (and
  > perhaps a few hundred more.)

What is a Google CSE?

Do you need a Google account to use that?
It is bad to let Google know who you are.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Tue, Dec 23, 2014 at 4:54 AM, Richard Stallman <rms@gnu.org> wrote:
>
>   > 2. You create a Google CSE. Works very well upto 200 documents (and
>   > perhaps a few hundred more.)
>
> What is a Google CSE?

Google CSE = Google Custom Search Engine.

It allows you to search just the pages you want. That could be a
certain host. Or a folder on a host. (Google CSE allows you to do a
bit more, but that is the basic.)

Like all Google search their revenue is based on advertising. You can
(as a maintainer for that CSE) get rid of that by paying. (And some
non-profit organisations can get rid of it for free.)


> Do you need a Google account to use that?
> It is bad to let Google know who you are.

You need a Google account to set it up. (They can't avoid that even if
they wanted to!) You do not need such an account to just use the CSE
to search.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I'd rather do this sort of deal with duckduckgo, if we are to do it.
But using a search engine requires a network connectin.

I have a feeling that we can avoid the need, because the sort of
synonym table we need is probably not so big after all.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Richard Stallman writes:

 > I'd rather do this sort of deal with duckduckgo, if we are to do it.
 > But using a search engine requires a network connectin.
 > 
 > I have a feeling that we can avoid the need, because the sort of
 > synonym table we need is probably not so big after all.

The problem is not the size of the table, it's aggregating user
queries into priorities for indexing.  This is the bread and butter of
search engines, while being hard to implement with decentralized, and
especially disconnected, users.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

   > > I have a feeling that we can avoid the need, because the sort of
   > > synonym table we need is probably not so big after all.

  > The problem is not the size of the table,

We are miscommunicating.  I was not talking about the size of the data.
I was talking about the work of collecting the entries it should have.

It won't be so hard, because there won't be so many entries.

					      it's aggregating user
  > queries into priorities for indexing.

We can get the suggstions from users.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Richard Stallman writes:

 >   > The problem is not the size of the table,
 > 
 > We are miscommunicating.  I was not talking about the size of the
 > data.

Neither was I.

 > I was talking about the work of collecting the entries it should
 > have.

So was I.

 > It won't be so hard, because there won't be so many entries.

That's why it's already done, and nobody has found anything missing
after 40 years of collecting entries, right?  Unfortunately, wrong.
People find missing entries every day (and rarely report them; they
just use Google instead).

 > 					      it's aggregating user
 >   > queries into priorities for indexing.
 > 
 > We can get the suggstions from users.

You can, but they'll be of relatively low quality for the purpose
because the kind of folks who especially need more indexing are those
least attached to the Emacs project (on average).

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Richard Stallman writes:
>
>  >   > The problem is not the size of the table,
>  > 
>  > We are miscommunicating.  I was not talking about the size of the
>  > data.
>
> Neither was I.
>
>  > I was talking about the work of collecting the entries it should
>  > have.
>
> So was I.
>
>  > It won't be so hard, because there won't be so many entries.
>
> That's why it's already done, and nobody has found anything missing
> after 40 years of collecting entries, right?  Unfortunately, wrong.
> People find missing entries every day (and rarely report them; they
> just use Google instead).

Google substitutes better quality with graceful failure based on
heuristics.  That works for some things better than others.  It
certainly will not help much replacing a "concept index" since the whole
point of a concept index is _not_ to be based on exact keywords.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Tom]

David Kastrup <dak <at> gnu.org> writes:
> 
> Google substitutes better quality with graceful failure based on
> heuristics.  That works for some things better than others.  It
> certainly will not help much replacing a "concept index" since the whole
> point of a concept index is _not_ to be based on exact keywords.
> 

Which is also true for Google, beacuse it automatically finds
alternate keywords (e.g. close file -> kill buffer) and it does it
better than any manually compiled index, because it takes into 
account lots more variations.

I think the main point is what Stephen brough up earlier, that
trying to add all variations to the index is a waste of time,
because new users can (and do) use Google to find things,
while experienced users know where to look in the manual, and
they can also use Google of course.

So the effort of expanding the index to contain more variation
of terminology is much better spent on improving other areas
of Emacs instead.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Tom <adatgyujto@gmail.com> writes:

> David Kastrup <dak <at> gnu.org> writes:
>> 
>> Google substitutes better quality with graceful failure based on
>> heuristics.  That works for some things better than others.  It
>> certainly will not help much replacing a "concept index" since the whole
>> point of a concept index is _not_ to be based on exact keywords.
>> 
>
> Which is also true for Google, beacuse it automatically finds
> alternate keywords (e.g. close file -> kill buffer) and it does it
> better than any manually compiled index, because it takes into 
> account lots more variations.

It puts up a lot more false positives, and sifting through them can make
this quite useless.  Particularly if we are talking about words that
have also common meanings.  Try Googling for the relation between "cat"
and "less".

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lars Ingebrigtsen]

David Kastrup <dak@gnu.org> writes:

> It puts up a lot more false positives, and sifting through them can make
> this quite useless.  Particularly if we are talking about words that
> have also common meanings.  Try Googling for the relation between "cat"
> and "less".

First hit on Google on the search "cat less":

http://staff.science.nus.edu.sg/~shing/unixlab/files03.html

:-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Tom]

David Kastrup <dak <at> gnu.org> writes:
> 
> It puts up a lot more false positives, and sifting through them can make
> this quite useless.  Particularly if we are talking about words that
> have also common meanings.  Try Googling for the relation between "cat"
> and "less".
> 

First hit is this for "cat less":
http://staff.science.nus.edu.sg/~shing/unixlab/files03.html


In my experience when searching for emacs related stuff in the vast
majority of cases google gives me useful results in the first hits.

I practically always find what I'm looking for at the first try, so
google is very usable to quickly find the relevant pages of emacs
documentatiton.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Tom <adatgyujto@gmail.com> writes:

> David Kastrup <dak <at> gnu.org> writes:
>> 
>> It puts up a lot more false positives, and sifting through them can make
>> this quite useless.  Particularly if we are talking about words that
>> have also common meanings.  Try Googling for the relation between "cat"
>> and "less".
>> 
>
> First hit is this for "cat less":
> http://staff.science.nus.edu.sg/~shing/unixlab/files03.html
>
>
> In my experience when searching for emacs related stuff in the vast
> majority of cases google gives me useful results in the first hits.

Try "Install auctex on windows".

Hit #6 (rather high on the list, and it's #4 on Duckduckgo) is
"http://www.ssc.wisc.edu/~dvanness/auctex.htm" talking about AucTeX 9.9p
which was released in 1999.  The instructions are all completely
useless.

The first two hits are from the official documentation, with the
remaining hits running into the obscure pretty fast.  So why prefer this
search over the official documentation?

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> From: Tom <adatgyujto@gmail.com>
> Date: Fri, 26 Dec 2014 13:30:12 +0000 (UTC)
> 
> experienced users know where to look in the manual

That's profoundly false, except for people with phenomenal memory.
Each of the two main manuals prints in many hundreds of pages, to say
nothing about the auxiliary manuals we have just for Emacs; and then
there are GCC, Binutils, GDB, Texinfo, Make, etc.  It is inhuman to
try to remember where things are in each manual.  And also unneeded,
because we do have indices.

> So the effort of expanding the index to contain more variation
> of terminology is much better spent on improving other areas
> of Emacs instead.

Whenever I improve an index, I do that so the next time I will find
things easier.  So it does help me, and is not wasted effort.  I hope
it will also help others, of course.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I think the main point is what Stephen brough up earlier, that
  > trying to add all variations to the index is a waste of time,
  > because new users can (and do) use Google to find things,

The reason I don't want to adopt this position is that it implicitly
assumes we should treat "use Google" as the recommended solution.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

RE: Have you all gone crazy? Was: On being web-friendly and why info must die

[Drew Adams]

> The reason I don't want to adopt this position is that it implicitly
> assumes we should treat "use Google" as the recommended solution.

I'm guessing that your objection here is not to recommending web search but to recommending the use of Google.

FWIW, "googling" is commonly used as a generic term for web-searching (using a web search engine).  Like it or not.

Just as "kleenex" became a generic name for facial tissues and "scotch tape" became a generic name for transparent tape, "google" as a verb is likely here to stay.

You can decide to fight that use as a verb - that's a choice.  You can suggest and hope that people will use a verb such as "web-search" instead.

But "googling" does not imply using www.google.com, even if, because of Google's overwhelming presence, "googling" also has that connotation.

People will know what you mean, if you use "web-search" as a verb instead of "google".  That's one viable alternative.  (What they will understand by the verb "to web-search" is "to google".)

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > The reason I don't want to adopt this position is that it implicitly
  > > assumes we should treat "use Google" as the recommended solution.

  > I'm guessing that your objection here is not to recommending web
  > search but to recommending the use of Google.

I object to recommending web search as the primary way to find
documentation.  I should have stated this more clearly but I already
said it a week ago.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Sat, Dec 27, 2014 at 11:54 PM, Richard Stallman <rms@gnu.org> wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > > The reason I don't want to adopt this position is that it implicitly
>   > > assumes we should treat "use Google" as the recommended solution.
>
>   > I'm guessing that your objection here is not to recommending web
>   > search but to recommending the use of Google.
>
> I object to recommending web search as the primary way to find
> documentation.  I should have stated this more clearly but I already
> said it a week ago.

It is unclear what you object to. Is it:

1) Using a service like Google etc to find documentation.

2) Using internet as a communication channel to a (search) server set
up and controlled by FSF to find documentation?

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stefan Monnier]

> It is unclear what you object to. Is it:
> 1) Using a service like Google etc to find documentation.
> 2) Using internet as a communication channel to a (search) server set
> up and controlled by FSF to find documentation?

Controlled by the FSF is of no help.  Controlled by a user-setup server
would be acceptable from a Freedom point of view, but to me the main
issue is that it should work when disconnected.


        Stefan

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Sun, Dec 28, 2014 at 2:07 AM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> It is unclear what you object to. Is it:
>> 1) Using a service like Google etc to find documentation.
>> 2) Using internet as a communication channel to a (search) server set
>> up and controlled by FSF to find documentation?
>
> Controlled by the FSF is of no help.  Controlled by a user-setup server
> would be acceptable from a Freedom point of view, but to me the main
> issue is that it should work when disconnected.

If that is the main issue then you can have a local copy of a search
server on your computer. Just half a GB. ;-)

Some years ago that would have been ridiculous. Today it is not a big
problem for many any user with a good internet connection.

Though you can probably use Apache Lucy to get a much smaller footprint.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It is unclear what you object to. Is it:

  > 1) Using a service like Google etc to find documentation.

  > 2) Using internet as a communication channel to a (search) server set
  > up and controlled by FSF to find documentation?

#2 is bad because it makes people need a net connection.  Anything
that makes people need a net connection for their own computing is
bad.

#2 makes them dependent on the FSF, which is also bad.

#1 is similar; the only real difference is that it makes people
dependent on Google instead of the FSF.  Google is worse in practice,
but in principle this dependence is bad no matter who people are
dependent on.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Mon, Dec 29, 2014 at 12:57 AM, Richard Stallman <rms@gnu.org> wrote:
>
>   > It is unclear what you object to. Is it:
>
>   > 1) Using a service like Google etc to find documentation.
>
>   > 2) Using internet as a communication channel to a (search) server set
>   > up and controlled by FSF to find documentation?
>
> #2 is bad because it makes people need a net connection.  Anything
> that makes people need a net connection for their own computing is
> bad.
>
> #2 makes them dependent on the FSF, which is also bad.
>
> #1 is similar; the only real difference is that it makes people
> dependent on Google instead of the FSF.  Google is worse in practice,
> but in principle this dependence is bad no matter who people are
> dependent on.


A web connection is needed, yes. Or you need to copy the whole web
search server to your local disk.

Anyway, here is a web search for Emacs documentation using OSS:

    https://dl.dropboxusercontent.com/u/848981/it/oss/oss-emacs.html

The autocompletion there is just what you get by default from OSS. You
could replace that with the handmade strings in the Info index if you
want to.

I have only added the files in Emacs documentation to the index. You
could add Emacs Wiki and other sources too, of course.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[rekado]

> The autocompletion there is just what you get by default from OSS. You
> could replace that with the handmade strings in the Info index if you
> want to.

The default autocompletion includes things like

    health care fraud settlement
    racial bias
    police is suing the government
    ...

which don't seem useful in the context of Emacs documentation.

-- rekado

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Mon, Dec 29, 2014 at 8:33 AM, rekado <rekado@elephly.net> wrote:
>> The autocompletion there is just what you get by default from OSS. You
>> could replace that with the handmade strings in the Info index if you
>> want to.
>
> The default autocompletion includes things like
>
>     health care fraud settlement
>     racial bias
>     police is suing the government
>     ...
>
> which don't seem useful in the context of Emacs documentation.
>
> -- rekado


;-) -- Wrong index. Fixed.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I've decided to reject the idea of depending on a net connection
for searching our documentation.  Please take this as decided.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Tue, Dec 30, 2014 at 12:23 AM, Richard Stallman <rms@gnu.org> wrote:
>
> I've decided to reject the idea of depending on a net connection
> for searching our documentation.  Please take this as decided.

I do accept that.

But that does not mean such an alternative can not be valuable too.
New users will probably mostly use that approach to begin with.

What I have tried to show is that it is quite easy to setup. And flexible.

Since I did most of the work for another purpose it is already done.
(Nearly. I need to automate some more things, though.)

At the moment I wonder how a search engine can be integrated with what
Nic is trying to do and the work on the Info index that Eli has done.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> On Tue, Dec 30, 2014 at 12:23 AM, Richard Stallman <rms@gnu.org> wrote:
>>
>> I've decided to reject the idea of depending on a net connection
>> for searching our documentation.  Please take this as decided.
>
> I do accept that.
>
> But that does not mean such an alternative can not be valuable too.
> New users will probably mostly use that approach to begin with.

This feels like discussing the nourishment value of steak in a
vegetarian diet, saying that new vegetarians will most probably start on
steak.

I have a hard time reading any sense into "I do accept that" in this
context.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Tue, Dec 30, 2014 at 8:05 AM, David Kastrup <dak@gnu.org> wrote:
> Lennart Borgman <lennart.borgman@gmail.com> writes:
>
>> On Tue, Dec 30, 2014 at 12:23 AM, Richard Stallman <rms@gnu.org> wrote:
>>>
>>> I've decided to reject the idea of depending on a net connection
>>> for searching our documentation.  Please take this as decided.
>>
>> I do accept that.
>>
>> But that does not mean such an alternative can not be valuable too.
>> New users will probably mostly use that approach to begin with.
>
> This feels like discussing the nourishment value of steak in a
> vegetarian diet, saying that new vegetarians will most probably start on
> steak.
>
> I have a hard time reading any sense into "I do accept that" in this
> context.

That is your problem. Not ours.

RE: Have you all gone crazy? Was: On being web-friendly and why info must die

[Drew Adams]

>   > > The reason I don't want to adopt this position is that it
>   > > implicitly assumes we should treat "use Google" as the
>   > > recommended solution.
> 
>   > I'm guessing that your objection here is not to recommending web
>   > search but to recommending the use of Google.
> 
> I object to recommending web search as the primary way to find
> documentation.  I should have stated this more clearly but I already
> said it a week ago.

I see.  Good.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >  > It won't be so hard, because there won't be so many entries.

  > That's why it's already done, and nobody has found anything missing
  > after 40 years of collecting entries, right?

Please avoid sarcasm.  It communicates hostility but no pertinent
substance.

  >   Unfortunately, wrong.
  > People find missing entries every day (and rarely report them; they
  > just use Google instead).

Not cogent.

  > You can, but they'll be of relatively low quality for the purpose
  > because the kind of folks who especially need more indexing are those
  > least attached to the Emacs project (on average).

These are just the sort of people for whom we would like to improve
the index, so the "quality" of a suggestion means its usefulness to
them.

You are not being constructive, just spreading negativity.
Please take it away from here.


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Richard Stallman writes:

  > You are not being constructive, just spreading negativity.

I'm hardly negative about the Emacs manuals or their indexes.  Both
are very good IMO, despite some critics' claims.  But by that very
token, they're going to require either inordinate effort or great
creativity to improve very much.  I don't think specific effort is
justified; the folks who have kaizenned them into their current
excellence will continue to improve them (and cooperating modules such
as completion) when the occasion arises, as has happened in this
thread.

 > Please take it away from here.

OK.

Sayonara

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,
>     Tom <adatgyujto@gmail.com>,
>     Emacs-Devel devel <emacs-devel@gnu.org>
> Date: Mon, 22 Dec 2014 15:41:51 +0900
> 
> I use Google a lot for various different purposes, and yes, it has an
> excellent thesaurus behind it.  You *can* just type "emacs cut" into
> Google and expect to get almost as good results as "emacs kill" in
> many circumstances.

For comparison, in an Emacs manual, typing "i cut RET" gets you to a
section named ``"Cut and Paste" Operations on Graphical Displays''.

> (But not in this case.  I  get completely
> different results, oriented to very different audiences.

By contrast, the above simple use of 'i' puts me _exactly_ where I
want to be.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > > (But not in this case.  I  get completely
 > > different results, oriented to very different audiences.
 > 
 > By contrast, the above simple use of 'i' puts me _exactly_ where I
 > want to be.

But Eli, unless you've got a psychological condition I haven't
noticed, you are at most one of those audiences.  And even if you
didn't write that node and indexing markup, you could have, no?  Of
course you're not surprised!  That's hardly a fair test.

Don't get me wrong: I have always been happy with the results of "i",
80% or 90% of the time it does indeed take me exactly where I want to
be, and much of the rest of the time I end up in some unintended place
that is nevertheless educational (and usefully so).  But I also did
the whole tutorial the first time I used Emacs, and often get started
with a new program by reading a few chapters in sequence from the
reference manual.  So I'm hardly a reliable witness on what "typical"
users would expect.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: lennart.borgman@gmail.com,
>     adatgyujto@gmail.com,
>     emacs-devel@gnu.org
> Date: Tue, 23 Dec 2014 09:29:43 +0900
> 
> Eli Zaretskii writes:
> 
>  > > (But not in this case.  I  get completely
>  > > different results, oriented to very different audiences.
>  > 
>  > By contrast, the above simple use of 'i' puts me _exactly_ where I
>  > want to be.
> 
> But Eli, unless you've got a psychological condition I haven't
> noticed, you are at most one of those audiences.  And even if you
> didn't write that node and indexing markup, you could have, no?  Of
> course you're not surprised!  That's hardly a fair test.
> 
> Don't get me wrong: I have always been happy with the results of "i",
> 80% or 90% of the time it does indeed take me exactly where I want to
> be, and much of the rest of the time I end up in some unintended place
> that is nevertheless educational (and usefully so).  But I also did
> the whole tutorial the first time I used Emacs, and often get started
> with a new program by reading a few chapters in sequence from the
> reference manual.  So I'm hardly a reliable witness on what "typical"
> users would expect.

I think a "typical" user will want to be in that node as well, when
she wants to read about "cut and paste".

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > I think a "typical" user will want to be in that node as well, when
 > she wants to read about "cut and paste".

You mean

    12.3 “Cut and Paste” Operations on Graphical Displays

    In most graphical desktop environments, you can transfer data
    (usually text) between different applications using a system
    facility called the clipboard. 

right?  I agree that the *user* wants to be there, but I'm not sure
*we* want her to be there.  In particular, on "i cut" I would be very
tempted to land her in the parent, "Killing and Moving Text".  YMMV,
but when I introduce students to Emacs, I do want them to read at
least a little about the power of "killing" vs the limitations of
"cutting"!

RE: Have you all gone crazy? Was: On being web-friendly and why info must die

[Drew Adams]

>  > she wants to read about "cut and paste".
> You mean 12.3 “Cut and Paste” Operations on Graphical Displays
> right?  I agree that the *user* wants to be there, but I'm not sure
> *we* want her to be there.  In particular, on "i cut" I would be
> very tempted to land her in the parent, "Killing and Moving Text".  

Send a bug report suggesting the change you think should be made
to the target for that particular index entry.  Irrelevant to this
discussion.

RE: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Drew Adams writes:

 > >  > she wants to read about "cut and paste".

 > > You mean 12.3 “Cut and Paste” Operations on Graphical Displays
 > > right?  I agree that the *user* wants to be there, but I'm not sure
 > > *we* want her to be there.  In particular, on "i cut" I would be
 > > very tempted to land her in the parent, "Killing and Moving Text".  
 > 
 > Irrelevant to this discussion.

Look who's talking!

But no, this *is* relevant to this discussion.  The point (which was
made explicitly) is that Eli and I disgree on "the right" node for the
index; in such cases Google style "implicit" indexing based on user
behavior which provides a variety of choices may be more useful than
Info-style indexing based on focused choices by knowledgeable editors.

As Paul Eggert points out, the two approaches are complementary for
most users, with different users tending to prefer one or the other to
some degree.

RE: Have you all gone crazy? Was: On being web-friendly and why info must die

[Drew Adams]

>  > >  > she wants to read about "cut and paste".
> 
>  > > You mean 12.3 “Cut and Paste” Operations on Graphical Displays
>  > > right?  I agree that the *user* wants to be there, but I'm not
>  > > sure *we* want her to be there.  In particular, on "i cut" I 
>  > > would be very tempted to land her in the parent, "Killing and
>  > > Moving Text".
>  >
>  > Irrelevant to this discussion.
> 
> Look who's talking!

Sigh.  ad hominem (again).  Please point to one thing I've said
here that is irrelevant to this discussion - but off list, as
that too is irrelevant here.  Stick to arguments about the topic.
The topic is not me or you.

> But no, this *is* relevant to this discussion.  The point (which
> was made explicitly) is that Eli and I disgree on "the right" node
> for the index;

No, that is not the point.  If you think a different node is more
appropriate, file a bug report.  Eli is not right all the time. ;-)

But what is this about "the right node" for "cut"?  Do you think
there is or should be only one?

Most manually defined indexes allow for multiple targets for the
same index entry, although it is usually helpful to qualify them
using secondary entries.

Simply "cut" as input and index entry could point to several places
in the Emacs manual.  Preferably they would be qualified, to help
you decide which is closer to what you are looking for.

And lo and behold, so there are.  Already.  There are several
such related "cut" index entries:

cut                   - "Cut and Paste" Operations on Graphical Display
cut and paste         - Glossary
cutting text          - Deletion and Killing
X cutting and pasting - Cut and Paste with Other Window Applications

(The last one does not appear for vanilla Emacs `i cut' completion,
but it does for Icicles and probably other completion enhancers.)

Those designing indexes make judgment calls, yes.  And you might
disagree with any given judgment, yes.  Likewise, search-engine
scoring.

> in such cases Google style "implicit" indexing based on user
> behavior which provides a variety of choices may be more useful
> than Info-style indexing based on focused choices by knowledgeable
> editors.

No one denies that full-text search indexing can be useful.

And manually designed indexes provide "a variety of choices" as well.

In this particular case, googling - just entering "emacs cut" -
returns this as the only hit for the Emacs manual: `"Cut and Paste"
Operations on Graphical Display'.  (The same hit that you ended
up at by `i cut RET'.)  That is the only hit for the Emacs manual
among the first 20 pages (200 hits), at least.

However, to be fair, if you google "emacs manual cut" then you do
get much better results.  There are 5 hits among the first 10 that
target www.gnu.org for the manual, one of which is the "Cut and
Paste" node.  And yes, there are 2 hits among the first 10 that
target your prefered node (but not at gnu.org).

Anyway, the point is not that googling cannot help you find
information about Emacs - including information that is in the
manual.  The point is that we need not (and should not) choose
only one or the other.

You have both available, right now.  And if you find that for some
information in the manual `i' doesn't cut the mustard and googling
does a better job, then please do file a bug report.

RE: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Drew Adams writes:

 > Sigh.  ad hominem (again).

"Look who's talking" is not an argument, it is an expletive.

 > No, that is not the point.  If you think a different node is more
 > appropriate, file a bug report.  Eli is not right all the time. ;-)

The first statement is invalid because it's not your call to make
about my posts, and the second two, though valid, are irrelevant to
this subthread.

Once again, the point of *my* posts in this subthread is that opinions
(and needs) differ, and therefore multiple applications (or output
formats) for viewing and indexing documentation may be beneficial.
Eli's argument may be persuasive because, after all, his proposals are
good for Emacs much of the time, and I wanted to point out that the
generalization from getting it exactly right for one person to doing
pretty well for everyone is logically invalid in this case.

Regards,

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Eli's argument may be persuasive because, after all, his proposals are
> good for Emacs much of the time, and I wanted to point out that the
> generalization from getting it exactly right for one person to doing
> pretty well for everyone is logically invalid in this case.

Well, Emacs Info is catered to one purpose: getting it pretty much
exactly right for a very limited set of tasks is an attainable goal.
Getting all HTML browsers exactly right for the same set of tasks is
much harder since it's rather vaguely right for several orders of
magnitude more tasks.

Emacs is a platform, and it supports doing Info in a manner very much
tied into the platform.  HTML+Browser vaguely is also a platform, but
hand-catering it to a particular task, particularly given a multitude of
unknown browsers and language standards, is much more fuzzy.

And compared to HTML/JavaScript/whatever, Emacs Lisp is remarkably
concise, efficient, well-defined and clear.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I think we don't need more argument about how good the indices
in our manuals are.  Could we drop that, please?

Practical suggestions to improve indices are welcome.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >   YMMV,
  > but when I introduce students to Emacs, I do want them to read at
  > least a little about the power of "killing" vs the limitations of
  > "cutting"!

Perhaps the node "Cut and Paste" should say a line or two suggesting
you look at the containing chapter with keyboard commands for the same
operations.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> Date: Tue, 23 Dec 2014 10:34:02 -0500
> From: Richard Stallman <rms@gnu.org>
> CC: eliz@gnu.org, lennart.borgman@gmail.com, adatgyujto@gmail.com,
> 	emacs-devel@gnu.org
> 
> Perhaps the node "Cut and Paste" should say a line or two suggesting
> you look at the containing chapter with keyboard commands for the same
> operations.

It already does.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> Date: Tue, 23 Dec 2014 13:43:31 +0900
> From: "Stephen J. Turnbull" <turnbull@sk.tsukuba.ac.jp>
> Cc: lennart.borgman@gmail.com,
>     adatgyujto@gmail.com,
>     emacs-devel@gnu.org
> 
> Eli Zaretskii writes:

>  > I think a "typical" user will want to be in that node as well, when
>  > she wants to read about "cut and paste".
> 
> You mean
> 
>     12.3 “Cut and Paste” Operations on Graphical Displays
> 
>     In most graphical desktop environments, you can transfer data
>     (usually text) between different applications using a system
>     facility called the clipboard. 
> 
> right?

And its subsections.

> I agree that the *user* wants to be there, but I'm not sure
> *we* want her to be there.  In particular, on "i cut" I would be very
> tempted to land her in the parent, "Killing and Moving Text".  YMMV,
> but when I introduce students to Emacs, I do want them to read at
> least a little about the power of "killing" vs the limitations of
> "cutting"!

Did you read the text that follows?  If not, please do, it mentions
killing and yanking right away.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > Did you read the text that follows?  If not, please do, it mentions
 > killing and yanking right away.

This is beside the point, but yes, I did read that text and the
subsections.  I think that in fact someone arriving at "Cut and Paste"
knowing very little about Emacs would be generally confused by mere
mentions of those concepts when cut/copy/paste are so much more
intuitive.  The parent does explain that Emacs can do the simple
operations but has a more powerful notion than the "cut" implemented
in most GUI applications.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[David Kastrup]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> This is not true. You do not have to search the whole internet just
> because you use Google. You could do something like this:
>
>    Google: "site:www.gnu.org/software/emacs/24.2/manual/ some thing"
>
> If that folder exists there, of course.

A sizable number of bug reports on LilyPond, a reasonably actively
developed and still evolving project, comes from users who try something
from the manual and find out it doesn't work.

Here usually the problem is the other way round: Google manages to turn
up the newest manuals for the current development version, and people
work with the next-to-previous stable version.

The link the people will reference contains the version number right in
it.  And yet a considerable number of people gets tripped up by that.

If you want to argue based on a mythical "fully competent computer
user", the results will not be representative.

-- 
David Kastrup

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> From: Lennart Borgman <lennart.borgman@gmail.com>
> Date: Mon, 22 Dec 2014 04:54:43 +0100
> Cc: Tom <adatgyujto@gmail.com>, Emacs-Devel devel <emacs-devel@gnu.org>
> 
> On Mon, Dec 22, 2014 at 4:36 AM, Eli Zaretskii <address@hidden> wrote:
> >> From: Tom <address@hidden>
> >> Date: Sun, 21 Dec 2014 21:32:13 +0000 (UTC)
> >>
> >> Users want to get the information fast and it is much faster to
> >> search for something in google than trying to find the relevant
> >> section of the manual.
> >
> > That's simply incorrect, if you use the 'i' command in Info, which is
> > the main way of searching an Info manual.
> 
> That is perhaps more an opinion?

No, it's experience.

> I definitively think users who know Google search well can find the
> information very quickly that way. And new users - who we want to
> help, of course - probably are very good at that.

You need to be good at both.  One does not come instead of the other.
A wise person will use each one where appropriate, and sometimes
both.  Please don't present this as an "either-or" situation.  No one
said that you should use either Google or Info.

> > Once again, with Google, you never know whether the info you see is up
> > to date or even correct.
> 
> This is not true. You do not have to search the whole internet just
> because you use Google. You could do something like this:
> 
>    Google: "site:www.gnu.org/software/emacs/24.2/manual/ some thing"
> 
> If that folder exists there, of course.

How do you know it does exist?

And if you do know, how is this different from going to the same
manual and using the text search (the 's' command) there?  It isn't.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Tom]

Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> And if you do know, how is this different from going to the same
> manual and using the text search (the 's' command) there?  It isn't.
> 

The main difference is google finds alternative phrases too.

E.g. suppose you want to read about closing buffers. You try
i and it has no completion for closing buffer. If you try to search
for "close buffer" in the emacs manual then no matches come up.
(Note I tried it on 24.1, a newer emacs might give better results.)

The reason for this is emacs calls it killing a buffer. But the
feature is generally called closing in other apps (close window,
close file, etc.), so a new user probably uses this for searching.

So in order to find something in info you often need to know the
term emacs uses for it. Info is a great reference, becase it is quick to
look up something when you already know the term. But if you have only
a vague idea then it may not be so easy to find what you are looking
for.

With google on the other hand if you search for "emacs close buffer"
(no quotes) then kill buffer is the top result. You can also search for 
"emacs close file", because a new user may be used to this term
(close file) from other apps and google gives useful results for this
too.

You can add these alternatives to the index, but in practice you
can't compete with google with a manually compiled index, because
you can add just so many alternatives, while google does the same
mechanically and intelligently (stemming, thesaurus, etc.), so
it will always have an advantage.

The most useful solution could be submitting a search to google
when info gives no results and if there is a manual hit in the
search results then determine the info page from the manual link
and show the page info. But I understand it won't happen, because it's
SaaS.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Lennart Borgman]

On Mon, Dec 22, 2014 at 5:03 PM, Tom <adatgyujto@gmail.com> wrote:
> Eli Zaretskii <eliz <at> gnu.org> writes:
>>
>> And if you do know, how is this different from going to the same
>> manual and using the text search (the 's' command) there?  It isn't.
>>
>
> The main difference is google finds alternative phrases too.

Plus the power of AND, of course, etc.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> From: Lennart Borgman <lennart.borgman@gmail.com>
> Date: Mon, 22 Dec 2014 17:11:38 +0100
> Cc: Emacs-Devel devel <emacs-devel@gnu.org>
> 
> On Mon, Dec 22, 2014 at 5:03 PM, Tom <adatgyujto@gmail.com> wrote:
> > Eli Zaretskii <eliz <at> gnu.org> writes:
> >>
> >> And if you do know, how is this different from going to the same
> >> manual and using the text search (the 's' command) there?  It isn't.
> >>
> >
> > The main difference is google finds alternative phrases too.
> 
> Plus the power of AND, of course, etc.

That's not needed in Info, because you don't search combinations of
words.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> From: Tom <adatgyujto@gmail.com>
> Date: Mon, 22 Dec 2014 16:03:57 +0000 (UTC)
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> > 
> > And if you do know, how is this different from going to the same
> > manual and using the text search (the 's' command) there?  It isn't.
> > 
> 
> The main difference is google finds alternative phrases too.

A good manual should have them already indexed.

> E.g. suppose you want to read about closing buffers. You try
> i and it has no completion for closing buffer. If you try to search
> for "close buffer" in the emacs manual then no matches come up.
> (Note I tried it on 24.1, a newer emacs might give better results.)

Please submit a documentation bug, and we will have it.

> So in order to find something in info you often need to know the
> term emacs uses for it.

Not as rule.  E.g., "cut" is already there, as are many other popular
terms.  Where some terminology isn't in the index, you are encouraged
to submit bug reports.

But patches to add such a capability to info.el will be most welcome.

> Info is a great reference, becase it is quick to look up something
> when you already know the term. But if you have only a vague idea
> then it may not be so easy to find what you are looking for.

The index doesn't include only Emacs terminology, or just terms.  It
includes phrases and words that a reader might have in mind when she
is looking for something.  IOW, good indexing is supposed to solve
precisely this kind of problems.  And if you look at the various
@cindex entries in the manual, you will see this in action.

Of course, there's always a place for improvement.  Google improve
their techniques as well.

> You can add these alternatives to the index, but in practice you
> can't compete with google with a manually compiled index, because
> you can add just so many alternatives, while google does the same
> mechanically and intelligently (stemming, thesaurus, etc.), so
> it will always have an advantage.

So let's add such a front end to Emacs as well.  Should be a nice
project, I think.

(Btw, "mechanically" and "intelligently" contradict each other.)

In any case, with Google you are still at the disadvantage that many
hits are not what you want, outdated, and just plain incorrect.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > > You can add these alternatives to the index, but in practice you
 > > can't compete with google with a manually compiled index, because
 > > you can add just so many alternatives, while google does the same
 > > mechanically and intelligently (stemming, thesaurus, etc.), so
 > > it will always have an advantage.
 > 
 > So let's add such a front end to Emacs as well.  Should be a nice
 > project, I think.

The problem is how do you assemble the results into an "index" for the
manual?  The scary thing about Google is that it is reindexing the
Emacs manual constantly based on click trails (I assume), and that
information is immediately available to all Google users (for some
reasonably short value of "immediate").

 > (Btw, "mechanically" and "intelligently" contradict each other.)

No, they just apply to different facets of the solution: "intelligent"
refers to the design, "mechanical" to the implementation.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: Tom <adatgyujto@gmail.com>,
>     emacs-devel@gnu.org
> Date: Tue, 23 Dec 2014 09:35:20 +0900
> 
> Eli Zaretskii writes:
> 
>  > > You can add these alternatives to the index, but in practice you
>  > > can't compete with google with a manually compiled index, because
>  > > you can add just so many alternatives, while google does the same
>  > > mechanically and intelligently (stemming, thesaurus, etc.), so
>  > > it will always have an advantage.
>  > 
>  > So let's add such a front end to Emacs as well.  Should be a nice
>  > project, I think.
> 
> The problem is how do you assemble the results into an "index" for the
> manual?

That's one issue that needs to be solved as part of the project.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > > The problem is how do you assemble the results [of analysis of
 > > user browsing behavior] into an "index" for the manual?
 > 
 > That's one issue that needs to be solved as part of the project.

IMO, it's insoluble, without violating some auxiliary principles of
the GNU Project.  You would need to arrange for Emacs to "phone home"
with the results every once in a while.  That's become more acceptable
to the general population, as many large software vendors (including
Mozilla) do it (on an opt-in basis).  I haven't thought about it
carefully myself, but I would guess Richard is uncomfortable at best
with such schemes.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

It is ok to invite the user to send a precomposed email message with
whatever information would help us.  The user would say yes or no
on each occasion.

I say "email message", but it would be equally ethical with any other
method of transmission.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> Date: Tue, 23 Dec 2014 13:47:45 +0900
> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: adatgyujto@gmail.com,
>     emacs-devel@gnu.org
> 
> Eli Zaretskii writes:
> 
>  > > The problem is how do you assemble the results [of analysis of
>  > > user browsing behavior] into an "index" for the manual?
>  > 
>  > That's one issue that needs to be solved as part of the project.
> 
> IMO, it's insoluble, without violating some auxiliary principles of
> the GNU Project.

Only because you are trying to solve a problem that doesn't need
solving.  No one said we need to communicate personal behavior traits
back to the master manual.  They can stay on the local machine.  Not
to mention the fact that this feature is not the most important part
of such a front end, at least not in my eyes.

I don't know if anyone is thinking about working on this, but if they
do, my advice would be to start light, and first get the basic problem
of NL to query conversion solved.  Don't be intimidated by the
seemingly huge and complex problem, and don't be hypnotized by the
mammoth called "Google".  Keep in mind that the problem which needs to
be solved here is much smaller and simpler than the one Google solved
and continues to solve every day: we have a much narrower domain with
a much smaller and simpler terminology.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Eli Zaretskii writes:
 > > Date: Tue, 23 Dec 2014 13:47:45 +0900
 > > From: "Stephen J. Turnbull" <stephen@xemacs.org>
 > > Eli Zaretskii writes:
 > > 
 > >  > > The problem is how do you assemble the results [of analysis of
 > >  > > user browsing behavior] into an "index" for the manual?
 > >  > 
 > >  > That's one issue that needs to be solved as part of the project.
 > > 
 > > IMO, it's insoluble, without violating some auxiliary principles of
 > > the GNU Project.
 > 
 > Only because you are trying to solve a problem that doesn't need
 > solving.  No one said we need to communicate personal behavior traits
 > back to the master manual.

I thought I did say that, and I am asserting it now.  Improving
indicies is *not* a matter of automatically constructing macros for
common keystrokes of a particular user.  The reason people are pushing
web access so hard is that they are focusing on newer users who don't
know appropriate search terms yet, and may not have the documentation
installed or even know how to access it if it is installed.  Maybe
it's peculiar to my population of students, but my experience with
college students and the bottom half of graduate students is that they
suck *badly* at picking good search terms, even in areas where they
allegedly have interest and background.  I see no reason to suppose
that typical programmers are much better, especially given that much
Emacs terminology is (nowadays) idiosyncratic.

Yes, personal indicies can help, but they don't solve the problem for
first-time users.  What can help for the first time search is
following the dynamic search behavior of many new users and
correlating initial search terms with eventual target nodes -- but it
only helps first-timers if it makes it back to the master index.

 > Not to mention the fact that this feature is not the most important
 > part of such a front end, at least not in my eyes.

That's a valid argument, but one I disagree with (maybe not *the most*
important, but I consider it very important).

 > Keep in mind that the problem which needs to be solved here is much
 > smaller and simpler than the one Google solved and continues to
 > solve every day: we have a much narrower domain with a much smaller
 > and simpler terminology.

Start with "doctor.el", right?  Good luck, guys! :-)

RE: Have you all gone crazy? Was: On being web-friendly and why info must die

[Drew Adams]

> >> The younger crowd expects interactive web pages (e.g. jumping
> >> to manual nodes with completion), because they are used to
> >> interactive features on other pages (Gmail, facebook, etc.)
> >
> > Fine.  And it is not only "the younger crowd" that is used to
> > using interactive web pages.
> 
> Frankly, that's not really useful since
> a) every "interactive web page" is different
> b) they work badly with automated fetchers, so you cannot usefully
>    aggregate them
> c) they tend to require enabling known security problems like Flash

Yes, I agree.

> > That's OK if it's a habit or they somehow find it easier,
> > but they should at least be made aware that they *can* access
> > the same information from within Emacs, and that they will
> > get more help and learn more that way than via the manuals
> > on line.
> 
> Well, one problem is that the manuals are really effective for
> learning a lot in one learning session.  And that's a good deal
> for getting better with using Emacs.  But it doesn't match
> modern attention spans.

I wouldn't use the word "modern" in that context. ;-)
But OK, let's take as given, for the sake of discussion at least,
that attention spans of people are in fact shorter nowadays.

I don't see the problem you are getting at.  Why does it follow
that because the manuals are "really effective for learning a
lot" they are not also very effective for learning a little?

IMO, using the manuals within Emacs is effective for learning
a little, as well as a lot.

But perhaps your point here is something like this:

The manuals are essentially reference manuals, with some
guidance thrown in.  They are not user guides or tutorials.
They are not how-to videos.  They are not FAQs.  And so on.

In sum, they are good for looking up reference information,
but they are not so good for teaching/learning.

Is that it?  If so, then I would agree.  For many users,
reading a reference manual is not the best way to *learn*.
(For some of us it is a good way, but for many others it
is not so good.)

But the answer to that problem is to add such learning aids,
to supplement the manuals.  I don't see this as a failing
of the manuals, and certainly not of their form as Info
within Emacs.

> And that's an inherent problem we have with selling Emacs,
> and it will continue to cost us "market percentages".

Assuming I got your point (I'm not sure), I don't agree that
it is an inherent problem.  It might be a problem to some
extent now (and I'm not even sure of that), but it is not
an inherent problem.  There are already lots of Emacs blogs,
tutorials, wiki how-to's, etc. that help users learn Emacs.

I really don't see the lack of such learning aids as "an
inherent problem we have selling Emacs."

> > The best way we can help them in this regard is to let them
> > know that Emacs itself offers great help for learning about
> > Emacs, and one of the best such aids are the Emacs and Elisp
> > manuals - *within Emacs*.
> >
> > Having the information in these manuals at your fingertips
> > while you use Emacs is an giant plus.  Providing better Web
> > use of the manuals is of course a good idea.  But we should
> > not neglect inviting users to consult the manuals from Emacs.
> 
> Certainly.  But I think that would primarily concern the Emacs-
> related manuals, 

That's exactly what I said, at the outset.  This is a separate
concern from the need to provide a better web experience for
GNU manuals in general.  My point was that the Emacs and Elisp
manuals are special, in that using them inside Emacs is
particularly rewarding.

> so we should probably see to it that one can get the Texinfo
> conversion process and/or our personal manual CSS sheets to
> make this a general feature of the Emacs manuals in particular.

I agreed with that already.  In doing that, I would like us to
nevertheless consider the concern I raised, which is Emacs-specific.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Most GNU manuals are designed to be introductions _and_ references.
They are written so you can read them straight through to learn about
a topic if you know nothing about it.

The Emacs Lisp Reference Manual is an exception -- because we have
a separate introduction for that topic, so the reference manual does not
need to try to serve the same purpose.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Date: Sat, 20 Dec 2014 16:30:37 +0900
> Cc: "Allen S. Rout" <asr@ufl.edu>, emacs-devel@gnu.org
> 
> Eric is stirring up nothing but trouble with his intemperate
> vituperations.  Karl is a little more circumspect, but he is also
> going to fail.  Of all free software philosophers, even more so than
> RMS, *those two* should be well aware of the distinction between the
> *software* and the *project*.  A work of free software is the world's,
> 'tis true, but nary a project be that be not "owned" by its
> participants.

I agree completely with what Stephen wrote, and that was also my gut
feeling even before reading his post.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Karl Fogel]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:
>Eric is stirring up nothing but trouble with his intemperate
>vituperations.  Karl is a little more circumspect, but he is also
>going to fail.  Of all free software philosophers, even more so than
>RMS, *those two* should be well aware of the distinction between the
>*software* and the *project*.  A work of free software is the world's,
>'tis true, but nary a project be that be not "owned" by its
>participants.
>
>The right way to stir things up is to appeal to the choir, not to the
>tourists gawking at the icons in the back of the hall.  The criterion
>for appeal of a new documentation format is clear: present a proof of
>concept translation of a "representative" Emacs manual[1] to the new
>source format, along with built manuals in the target format(s) and
>any tools needed to implement the desired navigation features.  The
>cost is high, but experience shows that worthwhile moves usually have
>redundant costs being paid.
>
>For example, I've observed 6 VCS transitions closely.  In 3 cases
>(including the current move of Emacs to git), the choice was based on
>consensus of the involved developers, and only one conversion was done
>(but note that Eric's conversion was not based on one of the existing
>git mirrors, and was done a couple dozen times I guess).  In the other
>3 cases, multiple repos were presented for consideration -- a lot of
>redundant effort from one point of view.
>
>In other cases (3 cases of issue tracker introduction), it was
>universally agreed that "some" was better than "none".  In two cases,
>projects just took the first thing that had a volunteer to implement
>and run the tracker.  In the case of Emacs, however, there was a
>strong demand that the existing email-centric workflow be extended,
>and the only candidate with a proof-of-concept implementation that
>satisfied that requirement was the current debbugs tracker.  That
>despite protests that Bugzilla, Roundup, Trac, etc "can be" configured
>to be controlled by email.  But no implementation was presented, and
>debbugs won by default.
>
>I suspect a careful study would substantiate such anecdotes.  For the
>documentation format, the core members of the project clearly consider
>the existing Texinfo manuals to be adequate (and often, excellent).
>So there's no hurry to produce a proof of concept -- but I would say
>one is necessary, and the cost is not exorbitant according to common
>practice.

Spot on, IMHO.  Another way to look at it:

Sometimes one can get a consensus that a transition would probably be a good idea, *in advance* of anyone doing the sample transition work.  That advance consensus can then help motivate people to work on the change, because they have confidence that their work will eventually be used for the real transition.  E.g., if there were broad consensus that a Web-based bug tracker (with APIs and email controllability) would *probably* be a good thing to move to from Debbugs, then someone might be more willing to work on it.

What a critical mass of Emacs developers currently express is a slightly different stance, though: "Debbugs is good enough until someone comes along and shows a concrete implementation of something else that is unequivocally better."  So now if someone's going to do all that work to demonstrate a different system, they're doing it under a much greater risk that their work will end up being wasted.  The lever can be on one of two settings: "We stay here until something clearly better is shown" vs "We'd probably like to change, unless we unexpectedly discover that there's nowhere better to go."  For many parts of the Emacs project, the lever is in the former position.  I believe that in other projects the lever is in the latter position rather more often (than it is in Emacs), and consequent
 ly people are more motivated to work on those things -- because they feel there is less chance their work will be rejected in the end.

Stephen, FWIW I don't think I was displaying any confusion between the software and the project (as per your first paragraph) :-).  I'm quite aware of what it would take here, and merely regret that I don't, alas, have time right now to work on these things in a way that would be effective for the Emacs project.

Best,
-K

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stephen J. Turnbull]

Karl Fogel writes:
 > "Stephen J. Turnbull" <stephen@xemacs.org> writes:

 > >I suspect a careful study would substantiate [the] anecdotes
 > >[presented in an earlier post].  For the documentation format, the
 > >core members of the project clearly consider the existing Texinfo
 > >manuals to be a dequate (and often, excellent).  So there's no
 > >hurry to produce a proof of concept -- but I would say one is
 > >necessary, and the cost is not exorbitant according to common
 > >practice.
 > 
 > Spot on, IMHO.  Another way to look at it:
 > 
 > Sometimes one can get a consensus that a transition would probably
 > be a good idea, *in advance* of anyone doing the sample transition
 > work.  That advance consensus can then help motivate people to work
 > on the change, because they have confidence that their work will
 > eventually be used for the real transition.

"Can", yes.  Does, I dunno.  The high-quality large contributions I
know in some detail were mostly done "speculatively", because there
was a need in the software -- not because there was demand from the
project.[1]  If you want control over integration, you typically need
to either become trusted core, or you need to fork.  Especially in
Emacs.  Little has changed here since the 1980s in that respect.

 > E.g., if there were broad consensus that a Web-based bug tracker
 > (with APIs and email controllability) would *probably* be a good
 > thing to move to from Debbugs, then someone might be more willing
 > to work on it.

That's a poor example, IMO.  There's clearly no consensus, and there
won't be one because the core really does prefer the email-based
workflow.  No?  As I see it, anyway, here there's only one choice:
build the damn mousetrap, and *show* the core that it catches mice
without bruising their fingernails.  If *you* don't have faith the
mousetrap will be enough better, then it probably isn't enough better.

 > What a critical mass of Emacs developers currently express is a
 > slightly different stance, though: "Debbugs is good enough until
 > someone comes along and shows a concrete implementation of
 > something else that is unequivocally better."  So now if someone's
 > going to do all that work to demonstrate a different system,
 > they're doing it under a much greater risk that their work will end
 > up being wasted.

I'm with Fred Brooks: build one to throw away.  "You will, anyway."
Cf. Eric -- he didn't throw just one repo away, he junked one every
couple days for a few weeks.

 > I believe that in other projects the lever is in the latter
 > position rather more often (than it is in Emacs), and consequently
 > people are more motivated to work on those things -- because they
 > feel there is less chance their work will be rejected in the end.

But there's a good reason for that: few projects are as old as Emacs,
few works are of the quality of Emacs.  *Many* of the projects are
brand new; anything is better than nothing.  (That's not entirely
true; one of my GSoC projects was mandated by the org to use Podio for
workflow.  Nothing is *definitely* better than vanilla Podio.  But I
digress.)  Very few are as solid, or have as well-established
traditions and workflows, as does Emacs.

Python is very similar to Emacs in terms of age and solidity, and
there is a lunatic fringe there advocating random changes.[2]  But
real progress on anything like workflow changes in Python takes at
least as much time as in Emacs, multiple Python Enhancement Proposals,
often discussion at one, sometimes two of the annual Language Summits,
and often multiple demonstration implementations.  But (aside from the
aforementioned fringe) nobody there thinks this process unnatural.

 > Stephen, FWIW I don't think I was displaying any confusion between
 > the software and the project (as per your first paragraph) :-).
 > I'm quite aware of what it would take here, and merely regret that
 > I don't, alas, have time right now to work on these things in a way
 > that would be effective for the Emacs project.

I concede that you know your mind better than I do.  But given that
you *did* "write the book", I at least have trouble distinguishing
your "I wish"es from your "You should"s.  If you see what I mean. :-)

Anyway, I apologize and will work on my own temper in the New Year.
Happy Holidays to you and all the denizens of emacs-devel!


Footnotes: 
[1]  "In the software" is not a great expression -- as I see it, this
applies to infrastructure projects too (especially things like repo
mirrors in a different VCS, but including trackers and testing
infrastructure), but the wording is poor for that.  Even the tracker
to some extent, although that (and mailing lists) are pretty much the
most un-decentralizable aspects.

[2]  I don't consider any of the people currently advocating radical
changes for Emacs to be lunatics at all.  The culture is very
different from Python.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stefan Monnier]

> That's a poor example, IMO.  There's clearly no consensus, and there
> won't be one because the core really does prefer the email-based
> workflow.

Actually, I'm definitely not wedded to an email based
bug-tracking system.  But I don't want a system where I need to use
a web-browser.  And I'd also much prefer keeping the ability to access
bug reports and queue changes to them while disconnected.


        Stefan

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Karl Fogel]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:
>I concede that you know your mind better than I do.  But given that
>you *did* "write the book", I at least have trouble distinguishing
>your "I wish"es from your "You should"s.  If you see what I mean. :-)
>
>Anyway, I apologize and will work on my own temper in the New Year.
>Happy Holidays to you and all the denizens of emacs-devel!

Oh, that's my fault -- you were never intemperate.  I phrased some "I wish"es such that they leaned too far toward "You should"s, in earlier posts.  The scary thing about the Internet is that there's no message-sized backspace key.  I'll try to do better on that in the New Year too :-).

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Jay Belanger]

> I have watched several of the recent threads regarding changing the
> documentation tool chain for no reason whatsoever from the sideline.
> And I really can't understand what's going on.

There have been a couple reasons listed.
First, to increase the web presence.  To be honest, I don't know what
this has to do with texinfo at all.  (Maybe I missed something.)
Second, because texinfo version 5 is slow.

This second problem seems to be the only real issue, and it seems that
we should try to fix it.  Richard put some pressure on the bzr
developers when we were using that; perhaps he should put some pressure
on the texinfo developers.  (I think that Stefan and Eli have already
brought it up to the texinfo developers.)

> If the only remaining reason for this change is to attract new, young
> and hip contributors,

That's been brought up, but it's also pretty clear that if someone wants
to submit doc changes in some sort of markdown, there are people here
who will translate it to texinfo.

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Jose E. Marchesi]

    > I have watched several of the recent threads regarding changing the
    > documentation tool chain for no reason whatsoever from the sideline.
    > And I really can't understand what's going on.
    
    There have been a couple reasons listed.
    First, to increase the web presence.  To be honest, I don't know what
    this has to do with texinfo at all.  (Maybe I missed something.)
    Second, because texinfo version 5 is slow.
    
    This second problem seems to be the only real issue, and it seems that
    we should try to fix it.  Richard put some pressure on the bzr
    developers when we were using that; perhaps he should put some pressure
    on the texinfo developers.  (I think that Stefan and Eli have already
    brought it up to the texinfo developers.)

I peeked at the texinfo svn repo and it looks like the texinfo
developers are actively working on a C parser that would replace the
current perl-based parser...

From
http://svn.savannah.gnu.org/viewvc/trunk/parsetexi/README?revision=5957&root=texinfo&view=markup:

"This is an experimental program intended to replicate the functionality
 in tp/Texinfo/Parser.pm.  How it will be integrated into makeinfo is
 still unknown.

 makeinfo in this directory wraps texi2any-C.pl, which is tp/texi2any.pl
 changed to use the module in the Parsetexi subdirectory instead of
 Texinfo::Parser."

Re: Have you all gone crazy? Was: On being web-friendly and why info must die

[Stefan Monnier]

> And I really can't understand what's going on.

Just ignore it and hack on,


        Stefan