Flipping url-show-status default

Flipping url-show-status default

[Lars Magne Ingebrigtsen]

The url library is very chatty.  I think most users would get a more
pleasant experience if url-show-status defaulted to nil.  I mean, our
other network libraries don't issue tons of messages every time you
transfer data, so I don't really see why url fetching should be
different.

This would avoid flashing tons of these messages when you read
HTML-based news, for instance:

Reading [image/gif]... 52 bytes of 45 bytes (116%)
Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%)
Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%) [5 times]
Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%)
Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%)
Reading [text/plain]... 25 bytes of 15 bytes (167%)
Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%)
Reading [text/plain]... 25 bytes of 15 bytes (167%)
Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%)
Reading [text/plain]... 25 bytes of 15 bytes (167%)


-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Flipping url-show-status default

[Ted Zlatanov]

On Fri, 01 Oct 2010 16:20:45 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> The url library is very chatty.  I think most users would get a more
LMI> pleasant experience if url-show-status defaulted to nil.  I mean, our
LMI> other network libraries don't issue tons of messages every time you
LMI> transfer data, so I don't really see why url fetching should be
LMI> different.

LMI> This would avoid flashing tons of these messages when you read
LMI> HTML-based news, for instance:

LMI> Reading [image/gif]... 52 bytes of 45 bytes (116%)
LMI> Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%)
LMI> Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%) [5 times]
LMI> Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%)
LMI> Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%)
LMI> Reading [text/plain]... 25 bytes of 15 bytes (167%)
LMI> Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%)
LMI> Reading [text/plain]... 25 bytes of 15 bytes (167%)
LMI> Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%)
LMI> Reading [text/plain]... 25 bytes of 15 bytes (167%)

Seconded, this is not useful information.

Ted

Re: Flipping url-show-status default

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> The url library is very chatty.  I think most users would get a more
> pleasant experience if url-show-status defaulted to nil.  I mean, our
> other network libraries don't issue tons of messages every time you
> transfer data, so I don't really see why url fetching should be
> different.

I don't think setting url-show-status to nil is the right thing; it's
pretty important to show something, particularly for synchronous
fetching, so that the user knows the connection isn't hung.  But we can
probably comment out some of the less important messages in url-http.el.

Re: Flipping url-show-status default

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> I don't think setting url-show-status to nil is the right thing; it's
> pretty important to show something, particularly for synchronous
> fetching, so that the user knows the connection isn't hung.  But we can
> probably comment out some of the less important messages in url-http.el.

Why do we want to do this at such a low level in the url fetching
library when we don't do it in other libraries?  Surely it must be up to
the caller of the url fetching library to message about these things if
they think it's necessary.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Flipping url-show-status default

[Ted Zlatanov]

On Fri, 01 Oct 2010 14:13:27 -0400 Chong Yidong <cyd@stupidchicken.com> wrote: 

CY> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>> The url library is very chatty.  I think most users would get a more
>> pleasant experience if url-show-status defaulted to nil.  I mean, our
>> other network libraries don't issue tons of messages every time you
>> transfer data, so I don't really see why url fetching should be
>> different.

CY> I don't think setting url-show-status to nil is the right thing; it's
CY> pretty important to show something, particularly for synchronous
CY> fetching, so that the user knows the connection isn't hung.  But we can
CY> probably comment out some of the less important messages in url-http.el.

Why not show something if the connection is hung, which is the
exceptional case?  % completed, byte count, and MIME type are completely
useless information in the vast majority of the use cases.  So I would
make it `show-url-status' a choice of nil, t, or 'error with 'error
being the default.

The "wait" cursor is not a good indicator for data retrieval.  Maybe the
mode line can be updated with a numeric indicator like line-number-mode?
It doesn't even have to be a number.  A pulsating icon to indicate an
ongoing transfer would look good; it can have an overlaid A/T to
indicate the number of active and total (active+hung) connections.  In
text mode this can be done with spinning bars or something equally
obnoxious :)

Ted

Re: Flipping url-show-status default

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

>> I don't think setting url-show-status to nil is the right thing; it's
>> pretty important to show something, particularly for synchronous
>> fetching, so that the user knows the connection isn't hung.  But we can
>> probably comment out some of the less important messages in url-http.el.
>
> Why do we want to do this at such a low level in the url fetching
> library when we don't do it in other libraries?  Surely it must be up to
> the caller of the url fetching library to message about these things if
> they think it's necessary.

This would involve specifying a callback function for each intermediate
step; callers would have to specify their own callback function, but
pretty much every caller would want to do the same thing, i.e. report
how much of the operation is done).  If the URL reporting is too
verbose, I think it makes more sense to tweak it, rather than opening up
an additional mostly useless interface.

Re: Flipping url-show-status default

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> This would involve specifying a callback function for each intermediate
> step; callers would have to specify their own callback function, but
> pretty much every caller would want to do the same thing, i.e. report
> how much of the operation is done).

The caller could send in a `silent' flag, which would be useful
especially for async operation.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Flipping url-show-status default

[Chong Yidong]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Why not show something if the connection is hung, which is the
> exceptional case?  % completed, byte count, and MIME type are completely
> useless information in the vast majority of the use cases.  So I would
> make it `show-url-status' a choice of nil, t, or 'error with 'error
> being the default.

I think it makes sense to eliminate (by default) most of those messages
except for percent complete, and to report the latter using the standard
Emacs progress-reporter mechanism.

> The "wait" cursor is not a good indicator for data retrieval.  Maybe the
> mode line can be updated with a numeric indicator like line-number-mode?
> It doesn't even have to be a number.  A pulsating icon to indicate an
> ongoing transfer would look good; it can have an overlaid A/T to
> indicate the number of active and total (active+hung) connections.  In
> text mode this can be done with spinning bars or something equally
> obnoxious :)

If anyone wants to improve the progress reporter, feel free.  One idea
I've had is for it to use the background of the echo area as a progress
bar (this will need some redisplay hacking, of course).

Re: Flipping url-show-status default

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Chong Yidong <cyd@stupidchicken.com> writes:
>
>> This would involve specifying a callback function for each intermediate
>> step; callers would have to specify their own callback function, but
>> pretty much every caller would want to do the same thing, i.e. report
>> how much of the operation is done).
>
> The caller could send in a `silent' flag, which would be useful
> especially for async operation.

Yes, that would be fine.  (I guess this would work by binding
url-show-status to nil locally in the url buffer).

Re: Flipping url-show-status default

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> Yes, that would be fine.  (I guess this would work by binding
> url-show-status to nil locally in the url buffer).

No, that didn't work, unfortunately, since after poking around in the
code for a couple of hours, I found that the buffers were created and
discarded and then created again.  And besides, many of the messaging
calls didn't take place from the process buffers.

So I created a new slot in the URL defstruct to signal silence, and
tweaked a couple functions to respect that.

It works for me, but the code is very involved, so who knows...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Flipping url-show-status default

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> It works for me, but the code is very involved, so who knows...

Oh, and you may need to do a "make clean" first to get it to compile,
since the URL struct is defined in a file that's compiled after the file
that uses the struct...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Flipping url-show-status default

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Oh, and you may need to do a "make clean" first to get it to compile,
> since the URL struct is defined in a file that's compiled after the file
> that uses the struct...

I mean, "make bootstrap".

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Flipping url-show-status default

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Chong Yidong <cyd@stupidchicken.com> writes:
>
> No, that didn't work, unfortunately, since after poking around in the
> code for a couple of hours, I found that the buffers were created and
> discarded and then created again.  And besides, many of the messaging
> calls didn't take place from the process buffers.
>
> So I created a new slot in the URL defstruct to signal silence, and
> tweaked a couple functions to respect that.
>
> It works for me, but the code is very involved, so who knows...

What's this about?

+     (message "hei: %s" url-current-object)
+     (with-current-buffer (get-buffer-create "back")
+       (let ((standard-output (current-buffer)))
+ 	(backtrace)))

Re: Flipping url-show-status default

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Date: Fri, 01 Oct 2010 18:21:52 -0400
> Cc: emacs-devel@gnu.org
> 
> If anyone wants to improve the progress reporter, feel free.  One idea
> I've had is for it to use the background of the echo area as a progress
> bar (this will need some redisplay hacking, of course).

You mean, like in ebrowse-show-progress?

Re: Flipping url-show-status default

[James Cloos]

>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:

TZ> Why not show something if the connection is hung, which is the
TZ> exceptional case?  % completed, byte count, and MIME type are
TZ> completely useless information in the vast majority of the use
TZ> cases.

Most browsers show that info, especially % completed.  IME that is
a good and useful thing.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: Flipping url-show-status default

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> What's this about?
>
> +     (message "hei: %s" url-current-object)
> +     (with-current-buffer (get-buffer-create "back")
> +       (let ((standard-output (current-buffer)))
> + 	(backtrace)))

Sorry; leftover debugging code.  Now gone.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Flipping url-show-status default

[Ted Zlatanov]

On Sat, 02 Oct 2010 03:34:36 -0400 James Cloos <cloos@jhcloos.com> wrote: 

>>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:
TZ> Why not show something if the connection is hung, which is the
TZ> exceptional case?  % completed, byte count, and MIME type are
TZ> completely useless information in the vast majority of the use
TZ> cases.

JC> Most browsers show that info, especially % completed.  IME that is
JC> a good and useful thing.

If you're loading a large file, yes (in the download manager, not in the
main window).  But the vast majority of use cases for HTTP are tiny
files and a progress bar for each one would be pointless.

Google Chrome, for instance, tells you it's loading the page, but not
each element within it.  There is no progress bar.  Mozilla Firefox
shows a tiny unobstrusive inaccurate progress bar for the whole page
in the lower left corner.

My proposal was to show the URL download progress in an unobstrusive
way, not to hide it altogether.

Ted

Re: Flipping url-show-status default

[Ted Zlatanov]

On Fri, 01 Oct 2010 18:21:52 -0400 Chong Yidong <cyd@stupidchicken.com> wrote: 

CY> If anyone wants to improve the progress reporter, feel free.  One idea
CY> I've had is for it to use the background of the echo area as a progress
CY> bar (this will need some redisplay hacking, of course).

In graphical mode, I don't think the echo area should be used for
progress.  It makes text harder to read if there's two backgrounds and
this would commandeer a large portion of screen real estate.  I'd put it
in the modeline as a tiny icon or text indicator.

A simple improvement to the current state would be to use SVG icons
instead of the spinning and progress text indicators.  I have hacked up
something for my Gnus article counts (tzz-make-ungradient here returns a
SVG data string set up for a 32x28 icon showing the number 324):

(when (image-type-available-p 'svg)
  (with-temp-buffer
    (let* ((svg-data (tzz-make-ungradient 32 28 324))
           (image (create-image svg-data 'svg t :ascent 'center))
           (props `(display ,image intangible ,image)))
      (insert "i")
      (add-text-properties (point-min) (point-max) props)
      (buffer-string))))

So this can be an immediate improvement.  Then these icons can be moved
to whatever screen area we decide to use (I assume SVG images can be
shown in the modeline but haven't tested it).

I can't work on it currently (gotta finish GnuTLS, then auth-source.el,
then gnus-sync.el, etc. :) but if anyone is interested I can send them
the full code.  If not I'll get to it eventually.

Ted

Re: Flipping url-show-status default

[James Cloos]

>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:

TZ> If you're loading a large file, yes (in the download manager, not in
TZ> the main window).  But the vast majority of use cases for HTTP are
TZ> tiny files and a progress bar for each one would be pointless.

Large is relative.  The variances of pipe bandwidth and latency are huge.  

And most images on the web are larger octet-wise -- often horribly
larger -- than they need to be.  The practice of optimizing images
for the web evaporated when site design moved from coders to graphic
artists.  Even the all-cell-phone-all-the-time kids don't seem to
understand the need to optimize download time.

TZ> Google Chrome, for instance, tells you it's loading the page, but
TZ> not each element within it.  There is no progress bar.

Chrome is not friendly.

TZ> Mozilla Firefox shows a tiny unobstrusive inaccurate progress bar for
TZ> the whole page in the lower left corner.

And the biggest change the ff nuts did to Mozilla was to ruin the UI.

Some useful improvements occurred in the low level code, but the UI
changes were almost all regressive.  (The URL bar background colour
change for TLS is the only UI progression I can think of.)

TZ> My proposal was to show the URL download progress in an unobstrusive
TZ> way, not to hide it altogether.

I missed that.  Appologies.

But when one is behind a tiny pipe (a straw, if you will) detailed
progress info is imperative.

My preference is sequential pipelined downloads, with n/m file count an
percentage of current file progress info (%age can be either numeric or
graphic).

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: Flipping url-show-status default

[Ted Zlatanov]

On Sat, 02 Oct 2010 15:21:59 -0400 James Cloos <cloos@jhcloos.com> wrote: 

>>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:
TZ> If you're loading a large file, yes (in the download manager, not in
TZ> the main window).  But the vast majority of use cases for HTTP are
TZ> tiny files and a progress bar for each one would be pointless.

TZ> Google Chrome, for instance, tells you it's loading the page, but
TZ> not each element within it.  There is no progress bar.

JC> Chrome is not friendly.

TZ> Mozilla Firefox shows a tiny unobstrusive inaccurate progress bar for
TZ> the whole page in the lower left corner.

JC> And the biggest change the ff nuts did to Mozilla was to ruin the UI.

I think you're underestimating the amount of time and expertise that
went into the Chrome and Firefox UI design.  They are quite good.

JC> But when one is behind a tiny pipe (a straw, if you will) detailed
JC> progress info is imperative.

JC> My preference is sequential pipelined downloads, with n/m file count an
JC> percentage of current file progress info (%age can be either numeric or
JC> graphic).

An SVG icon can display all this.  I'll try to put something together.

Ted

Re: Flipping url-show-status default

[James Cloos]

>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:

TZ> I think you're underestimating the amount of time and expertise that
TZ> went into the Chrome and Firefox UI design.  They are quite good.

I use ff for sites with faulty ui detection logic and seamonkey as my
primary gui browser.  It is precisely true that the ff ui is crap
compared to sm (which tries to keep the mozilla ui).  Everything they
could do to make the ui worse is done, from removing regularly-used
menu items, to making the context menus harder to use, to breaking the
keyboard ui, et al.

Nearly every "cool" idea they came up with makes it harder to get work
done.  And the one good change they made wasn't even their idea;
changing the appearance of the uri bar for encrypted links was proposed
years earlier in the crypto community.  Kudos to the ff team for finally
getting the concept into gecko, and for many of the lower-level changes
they made.  But very much not for the majority of their ui changes.

The ff team may *think* they improved the zilla ui, but anyone who has
extensively used both can attest that that is a fallacy.

Following /their/ tradition of ui redesign is a poor idea.

If emacs' progress info is too noisy to follow, then I'm all for
cleaning it.  Just please do not eliminate it, or simplify it
beyond what it needs.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: Flipping url-show-status default

[Davis Herring]

> Reading [image/gif]... 52 bytes of 45 bytes (116%)
> Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%)
> Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%) [5
> times]
> Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%)
> Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%)
> Reading [text/plain]... 25 bytes of 15 bytes (167%)
> Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%)
> Reading [text/plain]... 25 bytes of 15 bytes (167%)
> Reading [text/html; charset=UTF-8]... 330 bytes of 321 bytes (103%)
> Reading [text/plain]... 25 bytes of 15 bytes (167%)

Two things:

1. Why not just show these messages but not record them in *Messages*?
2. Maybe we could save bandwidth by stopping at 100%.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: Flipping url-show-status default

[Lars Magne Ingebrigtsen]

"Davis Herring" <herring@lanl.gov> writes:

> 1. Why not just show these messages but not record them in *Messages*?

Because they trample over messages that may actually be worthwhile for
the user to see.

> 2. Maybe we could save bandwidth by stopping at 100%.

:-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen