Re: What's the problem?

[Miles Bader <miles@lsi.nec.co.jp>]

Simon Josefsson <jas@extundo.com> writes:
> I'm using the Agent in recent Gnus, so almost all data is locally
> cached (except for flag updates via IMAP and new-mail checks in NNTP),
> so for me the delays I see are mostly CPU bound.

Not sure about agent mode -- the last time I tried it, it was so
completely awful that I quickly stopped.

> > What I'm trying to say is that the `threading support' need not be
> > particularly good, or general-purpose.  Probably something could be
> > hacked up right _now_, without any additional core functions, using
> > clever programming and emacs timers, by changing the worst-offending
> > part of the gnus code into something event driven.
> 
> I don't understand.  How would making the summary buffer generation
> asynchronous stop Emacs from locking up during computations?  For me,
> the summary buffer generation is CPU bound in elisp, not IO bound.

Yes, I understand.  I'd guess that a typical very long computation in
gnus is probably in the form of loops that do something fairly simple
many, many times, with other shorter calculations between them.  So
you'd like gnus to somehow yield cpu time for the user every iteration,
or every 100 iterations, or something like that.  In a cooperative
multi-tasking system, you'd just stick in a (yield) or something, but in
emacs this would be very hard to support.

So, instead, lets say you have a function like:

   (define gnus-fill-in-summary-buffer (...)
     (let (...some bindings...)
       (do-some-fairly-quick-calculation)
       (dolist (var1 potentially-big-long-list-1)
         ...something-1...)
       (dolist (var2 potentially-big-long-list)
         ...something-2...)))

change it into something like:

   (define gnus-fill-in-summary-buffer (...)
     (let ((some-state ...))
       (do-some-fairly-quick-calculation)
       (with-background-queue (bgq some-state)
         (bgq-dolist (var1 potentially-big-long-list-1 bgq)
            ...something-1...)
         (bgq-dolist (var2 potentially-big-long-list-2 bgq)
            ...something-2...))))

where `with-background-queue' and `bgq-dolist' (1) just package up their
body code into lambda(s), [effectively] shove them on the end of some
queue in bgq, and return immediately, and (2) explicitly refuse to make
any guarantees about non-global bindings; the value here called
`some-state' is stashed away in bgq somewhere, and can be used to
communicate values internal to the calculation.

IOW, it's annoying for the gnus programmer, but very easy for the emacs
implementation.

Some questions are (1) are there only a few points within gnus that
represent most of the annoying delays (2) do these places have fairly
simple code structures that could be split up into pieces like the above
example, and (3) do these pieces not depend on the long-term consistency
of any global state.

> I think there are two separate problems here.  They can and need to be
> solved individually.

I don't know.  A framework like the above could pretty simply handle the
I/O bound portion too, though.

-Miles
-- 
`To alcohol!  The cause of, and solution to,
 all of life's problems' --Homer J. Simpson