From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: keramida@ceid.upatras.gr (Giorgos Keramidas) Newsgroups: gmane.emacs.devel,gmane.emacs.gnus.general Subject: Re: Emacs trunk and Gnus master are fully sync'd now Date: Tue, 09 Jul 2013 22:41:51 +0200 Message-ID: <67um8rbo6bpisw.fsf@saturn.laptop> References: <67um8rzju4ykdj.fsf@saturn.laptop> <87a9lz1zpj.fsf@building.gnus.org> <87ppuv4rs1.fsf@randomsample.de> <67um8robae33lb.fsf@saturn.laptop> <877gh24094.fsf@randomsample.de> <67um8ry59hnqe3.fsf@saturn.laptop> <8738ro52qr.fsf@randomsample.de> <67um8robackn33.fsf@saturn.laptop> <87k3kz692t.fsf@randomsample.de> <67um8rzjtvtx98.fsf@saturn.laptop> <874nc3o7ej.fsf@randomsample.de> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1373402548 27618 80.91.229.3 (9 Jul 2013 20:42:28 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 9 Jul 2013 20:42:28 +0000 (UTC) Cc: Katsumi Yamaoka , ding@gnus.org, emacs-devel@gnu.org To: Lars Ingebrigtsen Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Jul 09 22:42:29 2013 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Uwejw-0008SI-KZ for ged-emacs-devel@m.gmane.org; Tue, 09 Jul 2013 22:42:24 +0200 Original-Received: from localhost ([::1]:41553 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Uwejw-0008Eh-7b for ged-emacs-devel@m.gmane.org; Tue, 09 Jul 2013 16:42:24 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:38430) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Uwejp-0008AA-LJ for emacs-devel@gnu.org; Tue, 09 Jul 2013 16:42:19 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Uwejn-0002Ng-A0 for emacs-devel@gnu.org; Tue, 09 Jul 2013 16:42:17 -0400 Original-Received: from tux-cave.hellug.gr ([195.134.99.74]:40046) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Uwejm-0002Mq-Sz for emacs-devel@gnu.org; Tue, 09 Jul 2013 16:42:15 -0400 X-Hellug-MailScanner-From: gkeramidas@gmail.com X-Hellug-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-2.9, required 5, autolearn=not spam, ALL_TRUSTED -1.00, BAYES_00 -1.90) X-Hellug-MailScanner: Found to be clean X-Hellug-MailScanner-ID: r69KfxXL029791 Original-Received: from giorgos.local (217-162-217-29.dynamic.hispeed.ch [217.162.217.29]) (authenticated bits=0) by tux-cave.hellug.gr (8.14.4/8.14.4/Debian-4) with ESMTP id r69KfxXL029791 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 9 Jul 2013 23:42:08 +0300 Original-Received: by giorgos.local (Postfix, from userid 1800799460) id 195C36EDA4C; Tue, 9 Jul 2013 22:41:51 +0200 (CEST) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (darwin) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 195.134.99.74 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:161788 gmane.emacs.gnus.general:83519 Archived-At: On Tue, 09 Jul 2013 21:33:24 +0200, David Engster wrote: > Giorgos Keramidas writes: >> So the unexist ranges *are* there. But as you said the information >> seems to be lost somewhere between the time `.newsrc.eld' loads and the >> time Gnus queries the imap server for group state. > > Yes, and I found the problem. I could not reproduce this initially > because it only exists in Emacs trunk, not in Gnus git. After I switched > to Emacs trunk, I saw the same thing. > > It's actually really simple: The problem is `gnus-clean-old-newsrc', > where 'unexist' marks will be removed when you upgrade from an older > Gnus version, probably because the format has changed. However, in Emacs > trunk, `gnus-newsrc-file-version' is "Gnus v5.13", which is always > considered *smaller* than "Ma Gnus v0.03", so it will always remove the > marks: > > (when (or force > (< (gnus-continuum-version gnus-newsrc-file-version) > (gnus-continuum-version "Ma Gnus v0.03"))) > ;; Remove old `exist' marks from old nnimap groups. > > So this has to be changed or maybe completely removed. I'm not sure; I > guess it *could* happen that someone running Ma Gnus v0.02 upgrades to > Gnus from Emacs 24.4. Great! Now we know at least why this is triggered. Looking at the diffs between emacs-24 and trunk of Emacs I can see that this function indeed only exists in the trunk: % 0709 21:49 gkeramidas@giorgos:~/git/emacs/lisp/gnus$ git diff emacs-24..trunk -- gnus-start.el % [...] % @@ -2291,7 +2301,27 @@ If FORCE is non-nil, the .newsrc file is read." % (gnus-message 5 "Reading %s...done" newsrc-file))) % % ;; Convert old to new. % - (gnus-convert-old-newsrc)))) % + (gnus-convert-old-newsrc) % + (gnus-clean-old-newsrc)))) % + % +(defun gnus-clean-old-newsrc (&optional force) % + (when gnus-newsrc-file-version % + ;; Remove totally bogus `unexists' entries. The name is % + ;; `unexist'. % + (dolist (info (cdr gnus-newsrc-alist)) % + (let ((exist (assoc 'unexists (gnus-info-marks info)))) % + (when exist % + (gnus-info-set-marks % + info (delete exist (gnus-info-marks info)))))) % + (when (or force % + (< (gnus-continuum-version gnus-newsrc-file-version) % + (gnus-continuum-version "Ma Gnus v0.03"))) % + ;; Remove old `exist' marks from old nnimap groups. % + (dolist (info (cdr gnus-newsrc-alist)) % + (let ((exist (assoc 'unexist (gnus-info-marks info)))) % + (when exist % + (gnus-info-set-marks % + info (delete exist (gnus-info-marks info))))))))) % % (defun gnus-convert-old-newsrc () % "Convert old newsrc formats into the current format, if needed." If I'm reading this function right, then _every_ time someone runs Gnus from the trunk of Emacs they would have to do an 'initial sync' of all their groups, because "Gnus v5.13" will never be >= "Ma Gnus.*". Doing an initial sync for large sets of IMAP groups means that the users will have to download several MBytes of data _every time_ they start Gnus. We should probably fix this some way toa void so large startup downloads and, as a result, slow startup times. The time different during the startup of Gnus and the amount of data downloaded is too large to ignore it. Running the following snippet in emacs-24 and trunk I got the following results a few minutes ago: (let ((info (format "%s" (list (format-time-string "%H:%M:%S Loading Gnus" (current-time)) (gnus) (format-time-string "%H:%M:%S Finished loading Gnus" (current-time)))))) (with-current-buffer "*scratch*" (insert info))) Output: | Branch | Start Time | End Time | Duration | Downloaded | |-----------------+------------+----------+----------+------------| | Emacs-24 branch | 22:03:24 | 22:03:45 | 21 sec | 148k | | trunk branch | 22:04:43 | 22:06:11 | 88 sec | 18350k | A 4x delay in startup time and more than 10x difference in downloaded data is really _huge_.