From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?Q?=C3=93scar_Fuentes?= Newsgroups: gmane.emacs.devel Subject: Re: sqlite3 Date: Wed, 15 Dec 2021 01:13:31 +0100 Message-ID: <875yrq67ys.fsf@telefonica.net> References: <874k7ljwkr.fsf@gnus.org> <87fsr5cuzq.fsf@ericabrahamsen.net> <878rwx8mdn.fsf@gnu.org> <87zgpdhw4d.fsf@yahoo.com> <2F63580E-FF58-45D0-9DBB-389ED64C0F11@mit.edu> <83v8zzw867.fsf@gnu.org> <87wnk7kkxp.fsf@red-bean.com> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="9531"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) To: emacs-devel@gnu.org Cancel-Lock: sha1:JWBl6OOymPWf8eC237iIbdhnQU4= Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Dec 15 01:28:32 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mxI9o-0002GV-1I for ged-emacs-devel@m.gmane-mx.org; Wed, 15 Dec 2021 01:28:32 +0100 Original-Received: from localhost ([::1]:36052 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxI9m-0003i0-AE for ged-emacs-devel@m.gmane-mx.org; Tue, 14 Dec 2021 19:28:30 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:53060) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxHvT-00074k-BX for emacs-devel@gnu.org; Tue, 14 Dec 2021 19:13:43 -0500 Original-Received: from ciao.gmane.io ([116.202.254.214]:50292) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxHvQ-0005qY-W7 for emacs-devel@gnu.org; Tue, 14 Dec 2021 19:13:43 -0500 Original-Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1mxHvO-0004N6-Lt for emacs-devel@gnu.org; Wed, 15 Dec 2021 01:13:38 +0100 X-Injected-Via-Gmane: http://gmane.org/ Received-SPF: pass client-ip=116.202.254.214; envelope-from=ged-emacs-devel@m.gmane-mx.org; helo=ciao.gmane.io X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.248, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:281955 Archived-At: Alan Mackenzie writes: > In my mail program, mutt, I load my entire mail collection into memory > every time I start it. That's around 140,000 mails, it's over 2 GB, and > it loads and threads in 7 or 8 seconds. And you are concerned about the bloat added by sqlite! ;-) What you describe is a huge waste of RAM and time, unless you need to scan all those mails many times on every session. And even then it still would be a waste of RAM. (I wonder how that affects the time required for a gc cycle?) If using a proper database you could have almost zero RAM overhead and almost zero init time overhead. This practice of loading everything into RAM only goes so far. Doing what you describe would be unacceptable to me. It is important to provide a method for efficiently handling large amounts of stored data. It will benefit some existing packages and make feasible new features.