From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: prj@po.cwru.edu (Paul Jarc) Newsgroups: gmane.emacs.devel Subject: Re: [PATCH 2/2] nnmaildir: Use a 'num' file, instead of a directory Date: Sat, 26 Jun 2010 13:33:42 -0400 Organization: What did you have in mind? A short, blunt, human pyramid? Message-ID: References: NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: dough.gmane.org 1277573798 10877 80.91.229.12 (26 Jun 2010 17:36:38 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Sat, 26 Jun 2010 17:36:38 +0000 (UTC) To: Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Jun 26 19:36:36 2010 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1OSZIx-0008PW-1E for ged-emacs-devel@m.gmane.org; Sat, 26 Jun 2010 19:36:35 +0200 Original-Received: from localhost ([127.0.0.1]:41640 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OSZIw-00014e-1w for ged-emacs-devel@m.gmane.org; Sat, 26 Jun 2010 13:36:34 -0400 Original-Received: from [140.186.70.92] (port=56258 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OSZIo-000144-FH for emacs-devel@gnu.org; Sat, 26 Jun 2010 13:36:27 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OSZGG-0001e3-6y for emacs-devel@gnu.org; Sat, 26 Jun 2010 13:33:50 -0400 Original-Received: from mpv2.tis.cwru.edu ([129.22.105.37]:34656) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OSZGG-0001dz-1s for emacs-devel@gnu.org; Sat, 26 Jun 2010 13:33:48 -0400 Original-Received: from mpv5.TIS.cwru.edu (mpv5.tis.cwru.edu [129.22.105.51]) by mpv2.tis.cwru.edu (MOS 4.1.9-GA) with ESMTP id AOB46370 for ; Sat, 26 Jun 2010 13:33:46 -0400 Original-Received: from multivac.cwru.edu (multivac.its.CWRU.Edu [129.22.114.26]) by mpv5.TIS.cwru.edu (MOS 3.10.9-GA) with SMTP id FBA73779 for ; Sat, 26 Jun 2010 13:33:46 -0400 (EDT) Original-Received: (qmail 24101 invoked by uid 500); 26 Jun 2010 17:34:07 -0000 In-Reply-To: (John Wiegley's message of "Thu, 27 May 2010 15:09:08 -0600") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux) Mail-Copies-To: nobody Mail-Followup-To: Original-Lines: 35 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:126444 Archived-At: Ted Zlatanov wrote: > On Thu, 27 May 2010 15:09:08 -0600 John Wiegley wrote: > > JW> This patch causes nnmaildir to use a "num" file to track the next > JW> available article number, rather than creating N empty files. > JW> However, this only works for new groups. Existing groups, which use > JW> the directory, will continue to. If you want to switch to using > JW> file-based numbering, you must convert your Maildir using the > JW> attached script (it's also copied to the commit description in the > JW> patch). > > IIRC (and Paul Jarc may want to comment) something similar was discussed > in http://thread.gmane.org/gmane.emacs.gnus.general/66245/focus=66247 Yes, although John hasn't said whether performance is his motivation. > I have recently experienced slow nnmaildir performance I suspect the performance problem is with the nov/ directory, rather than num/. With the current nov/ structure, it's quick and easy to map from filenames to article numbers, but I think we usually need the inverse operation. As it is, to map from an article number to a filename, we need to read the contents of all the nov/ files. That's done just once and the results are cached, so it's not too horrific, but we still need to check timestamps to see if the files have changed, and it takes a lot of memory for large groups. It's been a while since I looked at it, though--there may be some operations where we do need to go from the filename to the number. So then it might be useful to add hard links so each nov/ file could be accessed by either its article number or filename. The filename would also have to be added to the contents of those files somehow. paul