From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS6315 166.70.0.0/16 X-Spam-Status: No, score=-3.7 required=3.0 tests=AWL,BAYES_00, RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from out01.mta.xmission.com (out01.mta.xmission.com [166.70.13.231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 9AD031F597; Mon, 16 Jul 2018 15:20:35 +0000 (UTC) Received: from in01.mta.xmission.com ([166.70.13.51]) by out01.mta.xmission.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.87) (envelope-from ) id 1ff5If-0002sw-UI; Mon, 16 Jul 2018 09:20:33 -0600 Received: from [97.119.167.31] (helo=x220.xmission.com) by in01.mta.xmission.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.87) (envelope-from ) id 1ff5If-0003zN-Ae; Mon, 16 Jul 2018 09:20:33 -0600 From: ebiederm@xmission.com (Eric W. Biederman) To: Eric Wong Cc: meta@public-inbox.org References: <87k1q1bky6.fsf@xmission.com> <20180712014715.dn5aouayoa3uejp4@dcvr> <87k1q07dyc.fsf@xmission.com> <20180712230946.mqv3yjw4aabf7xrf@dcvr.yhbt.net> <878t6f1ch7.fsf@xmission.com> <87h8l2ykb4.fsf@xmission.com> <20180713222200.GB27845@dcvr> <87a7qtwsih.fsf@xmission.com> <20180715031823.GA24609@dcvr> Date: Mon, 16 Jul 2018 10:20:27 -0500 In-Reply-To: <20180715031823.GA24609@dcvr> (Eric Wong's message of "Sun, 15 Jul 2018 03:18:23 +0000") Message-ID: <87tvozqkas.fsf@xmission.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-XM-SPF: eid=1ff5If-0003zN-Ae;;;mid=<87tvozqkas.fsf@xmission.com>;;;hst=in01.mta.xmission.com;;;ip=97.119.167.31;;;frm=ebiederm@xmission.com;;;spf=neutral X-XM-AID: U2FsdGVkX1+v1PVPnBGkvL6BFJaJNJ0wP2VElSyb70g= X-SA-Exim-Connect-IP: 97.119.167.31 X-SA-Exim-Mail-From: ebiederm@xmission.com Subject: Re: msgmap serial number regeneration [was: Q: V2 format] X-SA-Exim-Version: 4.2.1 (built Thu, 05 May 2016 13:38:54 -0600) X-SA-Exim-Scanned: Yes (on in01.mta.xmission.com) List-Id: Eric Wong writes: > "Eric W. Biederman" wrote: >> I believe we can modify the msg number assignment to assign numbers to >> deletes as well as adds. Short of the same Message-ID coming up twice >> that should be enough for the current backwards loop to assign message >> ids reliably. And even Message-IDs comming up twice is handle-able. > > OK, I would likely accept a patch to fix that. > > A note about Message-ID uniqueness... The v2 code will generate > a new, truly unique Message-ID on duplicates and use that in > msgmap instead what was in the message. It's gross, but I needed > to do that to allow all messages to be accessible via Message-ID > over NNTP, because: > > a) some legit messages reuse Message-IDs :< > > b) some broken mailers (including some versions of git-send-email) > put multiple Message-IDs in the same message, so the code > needs to handle messages with any number of Message-IDs > anyways. I will send the patch along shortly. I mispoke when I said the problem could be fixed by assigning numbers to deletes. The actual problem was that not every add was assigned a number. So the fix simpler than I expected. It is interesting to note that INSERT DEL INSERT In sqlite does not reuse numbers in the primary key. So not reassigning numbers is what the local sqlite data base does as well. I need to track down what the v1 bug with add-remove-add was. I think the way I have updated the code I won't need the bug fix for v1. But I haven't checked that scenario yet. I also need to write a test case sigh. But in practice I have this working for git mailling list archive. Eric