From mboxrd@z Thu Jan 1 00:00:00 1970 Path: main.gmane.org!not-for-mail From: Miles Bader Newsgroups: gmane.emacs.devel Subject: Re: What's the problem? Date: 10 Dec 2003 16:19:11 +0900 Sender: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Message-ID: References: <4nllpt3hr3.fsf@lockgroove.bwh.harvard.edu> <5bad69zd43.fsf@lister.roxen.com> <4noeuon378.fsf@lockgroove.bwh.harvard.edu> <4ny8tsgxy6.fsf@lockgroove.bwh.harvard.edu> <4nhe0ggv0u.fsf@lockgroove.bwh.harvard.edu> <4nk75bwjaf.fsf@lockgroove.bwh.harvard.edu> <4nsmjv8d32.fsf@collins.bwh.harvard.edu> <87iskpbloe.fsf@mail.jurta.org> NNTP-Posting-Host: deer.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1071040853 5132 80.91.224.253 (10 Dec 2003 07:20:53 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Wed, 10 Dec 2003 07:20:53 +0000 (UTC) Cc: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Wed Dec 10 08:20:48 2003 Return-path: Original-Received: from quimby.gnus.org ([80.91.224.244]) by deer.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 1ATyeO-00016y-00 for ; Wed, 10 Dec 2003 08:20:48 +0100 Original-Received: from monty-python.gnu.org ([199.232.76.173]) by quimby.gnus.org with esmtp (Exim 3.35 #1 (Debian)) id 1ATyeO-0002KC-00 for ; Wed, 10 Dec 2003 08:20:48 +0100 Original-Received: from localhost ([127.0.0.1] helo=monty-python.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.24) id 1ATzb9-0002O0-Mt for emacs-devel@quimby.gnus.org; Wed, 10 Dec 2003 03:21:31 -0500 Original-Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1ATzb2-0002Na-JK for emacs-devel@gnu.org; Wed, 10 Dec 2003 03:21:24 -0500 Original-Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1ATzaW-0002Di-OR for emacs-devel@gnu.org; Wed, 10 Dec 2003 03:21:23 -0500 Original-Received: from [202.32.8.214] (helo=TYO201.gate.nec.co.jp) by monty-python.gnu.org with esmtp (Exim 4.24) id 1ATzaW-0002BO-1S; Wed, 10 Dec 2003 03:20:52 -0500 Original-Received: from mailgate3.nec.co.jp ([10.7.69.160]) by TYO201.gate.nec.co.jp (8.11.7/3.7W01080315) with ESMTP id hBA7JEc17158; Wed, 10 Dec 2003 16:19:14 +0900 (JST) Original-Received: (from root@localhost) by mailgate3.nec.co.jp (8.11.7/3.7W-MAILGATE-NEC) id hBA7JDE08674; Wed, 10 Dec 2003 16:19:13 +0900 (JST) Original-Received: from edtmg04.lsi.nec.co.jp ([10.26.17.201]) by mailsv3.nec.co.jp (8.11.7/3.7W-MAILSV4-NEC) with ESMTP id hBA7JDL21315; Wed, 10 Dec 2003 16:19:13 +0900 (JST) Original-Received: from mcsss2.ucom.lsi.nec.co.jp (localhost [127.0.0.1]) by edtmg04.lsi.nec.co.jp (8.9.3p2+3.2W/3.7W_EDC_Ver.1.0) with ESMTP id QAA06294; Wed, 10 Dec 2003 16:19:12 +0900 (JST) Original-Received: from localhost (mcspd15 [10.30.114.174]) by mcsss2.ucom.lsi.nec.co.jp (8.12.10/8.12.8/EDcg v2.01-mc/1046780839) with ESMTP id hBA7JC7Q029320; Wed, 10 Dec 2003 16:19:12 +0900 (JST) Original-Received: by localhost (Postfix, from userid 31295) id A6ED110099; Wed, 10 Dec 2003 16:19:11 +0900 (JST) Original-To: Juri Linkov System-Type: i686-pc-linux-gnu Blat: Foop In-Reply-To: Original-Lines: 73 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Emacs development discussions. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Xref: main.gmane.org gmane.emacs.devel:18608 X-Report-Spam: http://spam.gmane.org/gmane.emacs.devel:18608 Simon Josefsson writes: > I'm using the Agent in recent Gnus, so almost all data is locally > cached (except for flag updates via IMAP and new-mail checks in NNTP), > so for me the delays I see are mostly CPU bound. Not sure about agent mode -- the last time I tried it, it was so completely awful that I quickly stopped. > > What I'm trying to say is that the `threading support' need not be > > particularly good, or general-purpose. Probably something could be > > hacked up right _now_, without any additional core functions, using > > clever programming and emacs timers, by changing the worst-offending > > part of the gnus code into something event driven. > > I don't understand. How would making the summary buffer generation > asynchronous stop Emacs from locking up during computations? For me, > the summary buffer generation is CPU bound in elisp, not IO bound. Yes, I understand. I'd guess that a typical very long computation in gnus is probably in the form of loops that do something fairly simple many, many times, with other shorter calculations between them. So you'd like gnus to somehow yield cpu time for the user every iteration, or every 100 iterations, or something like that. In a cooperative multi-tasking system, you'd just stick in a (yield) or something, but in emacs this would be very hard to support. So, instead, lets say you have a function like: (define gnus-fill-in-summary-buffer (...) (let (...some bindings...) (do-some-fairly-quick-calculation) (dolist (var1 potentially-big-long-list-1) ...something-1...) (dolist (var2 potentially-big-long-list) ...something-2...))) change it into something like: (define gnus-fill-in-summary-buffer (...) (let ((some-state ...)) (do-some-fairly-quick-calculation) (with-background-queue (bgq some-state) (bgq-dolist (var1 potentially-big-long-list-1 bgq) ...something-1...) (bgq-dolist (var2 potentially-big-long-list-2 bgq) ...something-2...)))) where `with-background-queue' and `bgq-dolist' (1) just package up their body code into lambda(s), [effectively] shove them on the end of some queue in bgq, and return immediately, and (2) explicitly refuse to make any guarantees about non-global bindings; the value here called `some-state' is stashed away in bgq somewhere, and can be used to communicate values internal to the calculation. IOW, it's annoying for the gnus programmer, but very easy for the emacs implementation. Some questions are (1) are there only a few points within gnus that represent most of the annoying delays (2) do these places have fairly simple code structures that could be split up into pieces like the above example, and (3) do these pieces not depend on the long-term consistency of any global state. > I think there are two separate problems here. They can and need to be > solved individually. I don't know. A framework like the above could pretty simply handle the I/O bound portion too, though. -Miles -- `To alcohol! The cause of, and solution to, all of life's problems' --Homer J. Simpson