From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 190B8431FB6 for ; Tue, 7 Oct 2014 00:57:19 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -0.7 X-Spam-Level: X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nhCCKu3eRI+9 for ; Tue, 7 Oct 2014 00:57:14 -0700 (PDT) Received: from mail-wg0-f49.google.com (mail-wg0-f49.google.com [74.125.82.49]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by olra.theworths.org (Postfix) with ESMTPS id 2B21E431FAE for ; Tue, 7 Oct 2014 00:57:14 -0700 (PDT) Received: by mail-wg0-f49.google.com with SMTP id x12so8538054wgg.20 for ; Tue, 07 Oct 2014 00:57:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:to:subject:in-reply-to:references:user-agent :from:date:message-id:mime-version:content-type; bh=F8Q2p2uCHjkXkNFu+8eMwVRPI9n0G/ButLRhBMNo6/w=; b=Kwyd0QoSPFiOM4r7raPBMimZl0dKeVnYayvGtTKhc7LYGH0eNp1VRThsGz5GOPdJEI TzkNCge6SqAXV44x9b45hhjWR7/VoSPDLr6StiG4ZBi4A0HQJucS0ECry0PAh7UOAXsj GG3ng19M4GPwec9rgQvSXTyutF0Biff7dBGk6qnfkD7XqGF7g5mUJxpoq/+F16v9h/c1 POdF3o96SwKVQwgx2KAZUq1ccEAxFoS6OsqqIfZ/XWj9fOgDGA6ULB15UahjPWWrhf9J lDizaDidVNcBZBDRiduj3JGplWh0KaRT71UXOawaO6ENE5SeswsfnxjVMpdX9wSJZczv wOTg== X-Gm-Message-State: ALoCoQnAWptzOvjC3ngQ9uwy2UFb1mVFqxL/A3eGRPh7Zlr2//Ws90WQikRSNPJ+QAOcJnCvcov6 X-Received: by 10.194.119.9 with SMTP id kq9mr2617393wjb.25.1412668632947; Tue, 07 Oct 2014 00:57:12 -0700 (PDT) Received: from localhost ([2a01:348:1a2:1:bc6b:bac4:2954:d414]) by mx.google.com with ESMTPSA id fs13sm13703833wic.19.2014.10.07.00.57.11 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Oct 2014 00:57:12 -0700 (PDT) To: notmuch@notmuchmail.org Subject: Re: RFC: notmuch-cache.el: simple caching of MIME parts In-Reply-To: References: User-Agent: Notmuch/0.18.1 (http://notmuchmail.org) Emacs/24.3.1 (x86_64-apple-darwin13.3.0) From: David Edmondson Date: Tue, 07 Oct 2014 08:57:09 +0100 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 07:57:19 -0000 --=-=-= Content-Type: text/plain On Tue, Oct 07 2014, David Edmondson wrote: > I've been using remote-notmuch over a 3G connection quite a lot > recently. Dragging down the same MIME part more than once is annoying, > so here is a very simple filesystem based cache of MIME parts for > notmuch. It's integrated using defadvice for now, but a cleaner approach > is obviously possible. > > No suggestion that this should be integrated - it's just a toy to prompt > discussion. Bah. Attached the auto-save file :-p --=-=-= Content-Type: application/emacs-lisp Content-Disposition: attachment; filename=notmuch-cache.el Content-Transfer-Encoding: quoted-printable ;; notmuch-cache.el --- a simple local filesystem cache of MIME parts ;; ;; Copyright =C2=A9 David Edmondson ;; ;; This file is not part of Notmuch. ;; ;; Notmuch is free software: you can redistribute it and/or modify it ;; under the terms of the GNU General Public License as published by ;; the Free Software Foundation, either version 3 of the License, or ;; (at your option) any later version. ;; ;; Notmuch is distributed in the hope that it will be useful, but ;; WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; General Public License for more details. ;; ;; You should have received a copy of the GNU General Public License ;; along with Notmuch. If not, see . ;; ;; Authors: David Edmondson ;; Things to consider: ;; ;; - Keep an in-memory cache of parts? What would be an appropriate ;; way to limit the size of that cache? ;; ;; - In a long thread with mostly non-matching messages (i.e. new ;; message in old thread), the default show output includes the text ;; of all of the non-matching parts. Given that we can now insert ;; previously unshown parts on the fly, maybe non-matching body ;; parts should not be included in the default output, and code ;; added to glue them in when a non-matching message is ;; opened. (Would this be simpler if we convert simple text/plain ;; messages to look like MIME messages with a single text/plain ;; part?) ;; ;; - There is no cache management functionality. Just delete the ;; directory and its' contents for now. ;; Stats: (defvar notmuch-cache-stats [0 0 0] "Statistics about the notmuch part cache. This is an array with three elements: - the number of requests to the cache, - the number of reads satisified from the cache, - the number of writes to the cache.") (defun notmuch-cache-stat-incr (nth) (aset notmuch-cache-stats nth (+ 1 (aref notmuch-cache-stats nth)))) ;; The cache: (defcustom notmuch-cache-directory "~/.notmuch-cache/parts" "A directory used to store cached message parts." :type 'directory) (defun notmuch-cache-filename (id part) (expand-file-name (format "%s/%s/%d" notmuch-cache-directory id part))) (defun notmuch-cache-get (id part) "Return a cached copy of ID's MIME PART, or nil." (notmuch-cache-stat-incr 0) ;; Request. =20=20 (let ((filename (notmuch-cache-filename id part))) (if (not (file-exists-p filename)) nil (notmuch-cache-stat-incr 1) ;; Read. (let ((coding-system-for-read 'no-conversion)) (with-temp-buffer (insert-file-contents (notmuch-cache-filename id part)) (buffer-substring (point-min) (point-max))))))) (defun notmuch-cache-put (id part content) "Cache a copy of ID's MIME PART using CONTENT." (let ((filename (notmuch-cache-filename id part))) (unless (file-exists-p filename) (notmuch-cache-stat-incr 2) ;; Write. (make-directory (file-name-directory filename) t) (let ((coding-system-for-write 'no-conversion)) (with-temp-buffer (insert content) (write-region (point-min) (point-max) (notmuch-cache-filename id part)))= )))) ;; Integration: (defadvice notmuch-get-bodypart-internal (around notmuch-bodypart-cache act= ivate) "Use a local cache of notmuch body parts." (let ((content (notmuch-cache-get query part-number))) (if content (setq ad-return-value content) ad-do-it (when ad-return-value (notmuch-cache-put query part-number ad-return-value))))) --=-=-=--