From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Arthur Miller Newsgroups: gmane.emacs.devel Subject: Re: Against sqlite3!!! Date: Tue, 07 Dec 2021 19:43:30 +0100 Message-ID: References: <41E1B5BD-879C-430C-8BA3-3A5354AF2928@mit.edu> <87tufkwga9.fsf@gnu.org> <28E143D2-5C8F-4CDC-B6CE-15F672A00AB1@mit.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="36738"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) Cc: "emacs-devel@gnu.org" , Zhu Zihao , Stefan Monnier , Tassilo Horn To: Qiantan Hong Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Dec 07 19:48:49 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mufWB-0009I7-QO for ged-emacs-devel@m.gmane-mx.org; Tue, 07 Dec 2021 19:48:49 +0100 Original-Received: from localhost ([::1]:45158 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mufWA-0001UI-7Z for ged-emacs-devel@m.gmane-mx.org; Tue, 07 Dec 2021 13:48:46 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:37490) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mufRL-000790-BH for emacs-devel@gnu.org; Tue, 07 Dec 2021 13:43:47 -0500 Original-Received: from mail-vi1eur05olkn2065.outbound.protection.outlook.com ([40.92.90.65]:15201 helo=EUR05-VI1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mufRB-0001GU-44; Tue, 07 Dec 2021 13:43:43 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Kp0waGKZZ2afwnH24pOvgZtwsS0vpz3VFvwIWlgMgFw4m3pchkB5lfUM6o7de48RLETLo6baa44rcI8YxqCgHup0rIlM7KpERelbuU5Ap0H8JaO++KEJndo1eJ6/ln/aQTf3+/6dsMI1N2llO0SX949MAi2agEQ16uw7vdKP9Z3odwv4j/DhYAxhwM5pDDKfLmOPB6wim9FkWN47VhysWRmW3iLgL0Y02kJCs8839oqCW7ZY0HA807Y8nMC47zvH8Xq3Mhp9ut8I2FiKTiLgQ/7JZLk2qN9hF6oIM+guyAkdK78b5iLzQC8QoIApUoDbZlMRrDvei7hCe4Y12+BN/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zOrMpxMKcDWBvQCR8zeMBMVSiL0DS38i7pqM+jKEeTM=; b=QgfFacQP+UaWr7q7kxC21Ih9aDxdG96LG6fii2DBSj9vEsJDWwyVF9G2gPIDpelvEIIW79/p4lwpJvabMzN1bzSS3Rbml50wc9wO7caTxlWdgHNZsEBFgSN4eSVBj/F+KPHRMmFfyvBhYEbz8f/wEtjvtlW/gXZ8ZzK+we8dotJzMnhO5rcirD6AIBIIb8oR3DlZjDjLY+P0pIrX7HX34yZ0rvPp7pGDLrrlFTrrmnzj5n/s1MqTgyEuoeKSLa08aq1lg0U7SRLGaK8qdsYGI54KdCZj6RFz4RkyqM+rkgeYasK2unumInZXMBt4mjkILOc0hg9JhfnWR6sT8r9SpA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=live.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zOrMpxMKcDWBvQCR8zeMBMVSiL0DS38i7pqM+jKEeTM=; b=n/cnRe5ABhwSpUkaJEtSvckvyF5gzOFjHA2tZTvxTbySPnqdRE89ySw8wvKeD3HRrqnWI+Pez5JipK13ZTHD8N1TuwASuGeD7PHgwnIV4pyfR46lUQp0KmbkSYxOchKF+1dvxmflLSJfKjLgFxHm7gypA0Fj6dyk1bT7PaFWu9cneBjxOC9NWahZA4/AyJY6hrfFEzJhZM++ckHURiC2VGC50BxGCJvFZIQS0LLTdcfHFg969pi+K691e/0xsjzpBpGuDXnpIZqcl4g2LD+IufOjK2h3AUFjpJ8ZShO9fOqkibNq3LD6ixXbpP02jOA3/Vl/4PcBuQ9NKHd4ucyGzQ== Original-Received: from AM9PR09MB4977.eurprd09.prod.outlook.com (2603:10a6:20b:304::20) by AM0PR09MB2385.eurprd09.prod.outlook.com (2603:10a6:208:dc::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4755.11; Tue, 7 Dec 2021 18:43:33 +0000 Original-Received: from AM9PR09MB4977.eurprd09.prod.outlook.com ([fe80::b012:5fd9:9447:d64a]) by AM9PR09MB4977.eurprd09.prod.outlook.com ([fe80::b012:5fd9:9447:d64a%4]) with mapi id 15.20.4755.022; Tue, 7 Dec 2021 18:43:33 +0000 In-Reply-To: <28E143D2-5C8F-4CDC-B6CE-15F672A00AB1@mit.edu> (Qiantan Hong's message of "Tue, 7 Dec 2021 16:35:47 +0000") X-TMN: [LSBfOgHsY1gF4HDdfiqdnQ56DvoSubm3] X-ClientProxiedBy: GV3P280CA0053.SWEP280.PROD.OUTLOOK.COM (2603:10a6:150:9::21) To AM9PR09MB4977.eurprd09.prod.outlook.com (2603:10a6:20b:304::20) X-Microsoft-Original-Message-ID: <87a6hcqmr1.fsf@live.com> X-MS-Exchange-MessageSentRepresentingType: 1 Original-Received: from pascal.homepc (81.232.177.30) by GV3P280CA0053.SWEP280.PROD.OUTLOOK.COM (2603:10a6:150:9::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4755.11 via Frontend Transport; Tue, 7 Dec 2021 18:43:31 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ebef061a-77b2-42f8-f606-08d9b9b17780 X-MS-TrafficTypeDiagnostic: AM0PR09MB2385:EE_ X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: pzkFuE0GDuslsVsCQSTpBjKcDPj0O3mpFnG4C5f59G4Q+cl4MH3IkdNEW71hzcVNhnysclrNq4wwiZSZTqrwTnVlUuhLVJV6Oq2sx6kFDmb8MTuLaBQFOcx/Vu+Lbl2JtOMIKK+7BWQP61KPgT4GMmbHZvUB2KSiDPWkrPPXWfqozfjcppQJrl4sby0x8So/eqs5acdNQNB5Dbo1Me/lMNYYtEk7Lwtpkho7ylN99g0/R7myE7T5vguROqfS3kBc70rpI6f5WFTwXVQjAa/mUdMV3HTRCnuQZ1SnFwv8uXRBPoExH+2CSHIcHX9+rvBk7U1UY/0cOYH3Pfh3ZDLN6U/yUwPcDYAdZqZ3GXBQC3JP85OnC1cERsLy3K50z176BtStAYEryzwHtTi5Ha2WynCugOHhaOxtZSwPN0qMKYHU4zua3NIkZJrrw48bMVPoKO4LY1yBNvHq/gn9gvrYXKZYvAk6wjLVLBo5SXTTnIB8wtZ+nIQD5PAl9M4h3FIDK0OAIGQ9I7GM7QZ5PYb2as3l4L279aMoaqTMlsKXcEDsXgCPBPbOTa15eJS4ofs6KfliNn/5xkW5YowB0nLtBw== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SXZobU1Uc01VVksyYlNsZHVqTFZnUndXSWI4SDV5ak15dXNpazRyOXZ6L2xn?= =?utf-8?B?QS9CZHBPQUtmQWMybHNyR1RGdkMrODZHaGtwTTNXQVBRczNIUSs0N1YxNEVX?= =?utf-8?B?RWordnhGK0VkSTYzSGdqMDYrMWpaVTBMU2RPb2tqczhMMURNRitBNFR6Ry8x?= =?utf-8?B?ZzFLN0dXRzh0eGF6dXgyYkR2V2h5ZUIzVjE2aVF3bkluZzQ2dkNCKzk1YWkr?= =?utf-8?B?blNzc3hFQm1XSmRSSWx4b216OEFBZ3ZEWS9PRW1wdUpielpuTklwTE9VZ2dT?= =?utf-8?B?REd4NCt2VGZUaHd1MGRWbTBRamJXajRWOW5tK2swQlJoZU85TzFhVUsweWNv?= =?utf-8?B?OGNzRHg4dStIVTY3RGtZWnJVenpEVDcxUWZ0c2VjYkFHTDcvdFFSdzJkUGtq?= =?utf-8?B?aTdjR0V1ZWt4YkhzTFprVUVpenByL2FuMndhQkN2UTVTU2VuSXNaS0NJTmxI?= =?utf-8?B?eS9BcGFBbE1tK0xtNUdZbjJRSHNKMjBiNkRiRU9UaWdXdUZLTWo2eVRyMEph?= =?utf-8?B?MGVaT0F0TWwxelNpNGRha1QzN0d6YzdWcXVjbmd0bDVVNjB6akNzV0c0UU50?= =?utf-8?B?VWR5VXdwT2MzaElsQUZmWng4cjhmZzZxLytrNGZXSEVTQXRQa3VudW1PMjRX?= =?utf-8?B?N05Ea1c0SlNid1dMakdoN2JIM0J4Wm9uQTExY1 X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-64da6.templateTenant X-MS-Exchange-CrossTenant-Network-Message-Id: ebef061a-77b2-42f8-f606-08d9b9b17780 X-MS-Exchange-CrossTenant-AuthSource: AM9PR09MB4977.eurprd09.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Dec 2021 18:43:33.1822 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR09MB2385 Received-SPF: pass client-ip=40.92.90.65; envelope-from=arthur.miller@live.com; helo=EUR05-VI1-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:281272 Archived-At: Qiantan Hong writes: >> That would be quite a >> disadvantage compared to a DB approach where I'd naturally expect that >> only the value I'm asking for is loaded. Sqlite does not read one key per request, it reads at least a page. So if y= our system uses 64k pages, sqlite will read at least 64k at once. Then you also= have system doing disk i/o, caching nodes and so on; so you are never really get= ting just "the value you are asking for to be loaded". > It does, but I=E2=80=99ve done some benchmark and it loads 10k entries in= 0.02~0.03=20 > seconds. 100k entries takes <0.5s. > I=E2=80=99d say it should be suffice for most Emacs application I know of= . > Nobody is using Emacs for trillions of business records. > > On the other hand save operation is fully incremental and don=E2=80=99t > even need to be invoked explicitly. > > Being said that, if we really find out loading is not fast enough, I migh= t > come up with some way to load it in segments lazily. I doubt if that will > ever become necessary. > >> Also, how would it ensure consistency when I have 2 parallel emacs >> sessions (like one for mail/irc and one for programming/editing) where >> session 1 modifies the value of key A and the other of key B? It looks >> like the values of the kv-store that gets saved later will win. In case >> that's the emacs session which has modified B, it'll revert A to the >> state before the other session modified it, no? > Since it records a log of deltas instead of printing the whole data struc= ture, > different key won=E2=80=99t interfere. > > Being said that, currently it probably won=E2=80=99t work because UNIX ap= pend > is not atomic and will probably be interleaved into nonsense. > There=E2=80=99re various workarounds, lock file being one, but I like > the idea of keeping only one =E2=80=9Ccontroller=E2=80=9D instance with e= xclusive > access to the file more. You can do same as sqlite does: just lock entire file (db). Sqlite locks en= tire db while writes are performed. It can be configured to allow concurrent rea= ds but only if there are no ongoing writes.