From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id 4IphEDj6j2N/bwAAbAwnHQ (envelope-from ) for ; Wed, 07 Dec 2022 03:28:08 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id OiBnDzj6j2MZUgAAG6o9tA (envelope-from ) for ; Wed, 07 Dec 2022 03:28:08 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 0F4A01C7BA for ; Wed, 7 Dec 2022 03:28:07 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2k9p-00064m-Az; Tue, 06 Dec 2022 21:27:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2k9o-00064R-0h for help-guix@gnu.org; Tue, 06 Dec 2022 21:27:36 -0500 Received: from mout01.posteo.de ([185.67.36.65]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2k9k-0003TV-6y for help-guix@gnu.org; Tue, 06 Dec 2022 21:27:35 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 81D08240026 for ; Wed, 7 Dec 2022 03:27:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.de; s=2017; t=1670380048; bh=NDrg1JhnIEfpfDChcBWkzz4JbpyEa+dOhxgqSmElFvc=; h=From:To:Cc:Subject:Date:From; b=i8uyBASo9Zd69Rxs3FR4GNTODl4DDtrBAvGhEtML7x5VITxG5fjpqlQPMt6tvFpx9 boZxfYvWlILe4Mgfw5lGW3wMFwNcpiFtOerTED1yq+Hz+38dCMpnqYENoF0y6uM4Mm 9CNQkl0YjzAwSEuuesvzD+d7mpuEVea+HHH++wDYF2tXVrl+1JJ3gdCvdKaeF5smvq tV4lXNLGQml8gf6G26B04qRdBfCdE3PRwd7dU+rzg8onvgyIHdswxRdB0me9dEdXQv vPFCsPniwXxqjG3+wv2bmGQOwswjj+dEysnIjfwIdZY4PlDxAcYNqnO3dhMhxP9PJY rAM0tDmXy1bpQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4NRh5B5S4Jz6tmH; Wed, 7 Dec 2022 03:27:26 +0100 (CET) References: <878rka8pcj.fsf@posteo.de> <864jua5jd6.fsf@gmail.com> <87y1rlfinp.fsf@kyleam.com> From: Mekeor Melire To: Kyle Meyer Cc: zimoun , help-guix@gnu.org Subject: Re: This is how you import a mailing list archives (e.g. guix-patches) into mu4e Date: Wed, 07 Dec 2022 02:19:26 +0000 In-reply-to: <87y1rlfinp.fsf@kyleam.com> Message-ID: <87wn74kkxl.fsf@posteo.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=185.67.36.65; envelope-from=mekeor@posteo.de; helo=mout01.posteo.de X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: help-guix-bounces+larch=yhetil.org@gnu.org Authentication-Results: aspmx1.migadu.com; none X-Migadu-Scanner: scn0.migadu.com X-TUID: IDkcAfI+Shoi --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable 2022-12-05 20:03 kyle@kyleam.com: > On 12/05/22 09:45:41 +0100, zimoun wrote: > > Personally, I use =E2=80=9Cgit clone=E2=80=9D from a public-inbox insta= nce [1]. > > > > git clone --mirror https://yhetil.org/guix-patches/1 \ > > guix-patches/git/1.git > > > > where =E2=80=991=E2=80=99 can be also replace by =E2=80=990=E2=80=99 fo= r the very old ones. > > In the case of guix-patches, there's not a 0.git. I started it at 1.git > to leave open the possibility of adding 0.git with the messages I was > missing from the beginning of the list's history (from Feb 12, 2017 to > March 27, 2017). I'm not sure I'm ever going to do so at this point > and, even if I did, reserving 0.git doesn't have much advantage over > just adding the old messages on top of the existing epoch, so I probably > should have just started it at 0.git. > > > Then the conversion from Git commit to maildir is done by a small script > > [2], where all the job reads: > > > > --8<---------------cut here---------------start------------->8--- > > # Extract the message from each commit in the range and store it > > # in the Maildir for notmuch to consume. > > $git rev-list $range | while read sha; do > > # XXXX: fatal: path 'm' does not exist in > > # and it can also raise issues with notmuch, as: > > # Note: Ignoring non-mail file: $maildir/new/$sha > > A tree can either have m or d ("deleted" messages): > > https://public-inbox.org/public-inbox-v2-format.html > > So you should be able to avoid this error by skipping d's. > > > $git show $sha:m > $maildir/new/$sha > > done > > --8<---------------cut here---------------end--------------->8--- > > > > (Maybe better could be done and more robust are around.) > > No need to change what works, of course, but > https://public-inbox.org/clients.html mentions l2md and impibe as tools > for converting public-inbox archives to Maildir. > > * https://git.kernel.org/pub/scm/linux/kernel/git/dborkman/l2md.git > * https://leahneukirchen.org/dotfiles/bin/impibe > > In terms of cloning archives, plain cloning and fetching with Git is > fine, but, if you have public-inbox locally, you can instead use > public-inbox-clone and public-inbox-fetch, which will handle some > details for you (e.g, cloning underlying epochs and recognizing that new > epochs have been added): > > $ public-inbox-clone https://yhetil.org/guix-patches > > Another option for fetching that's nice if you're mirror multiple repos > is grokmirror: > > * https://git.kernel.org/pub/scm/utils/grokmirror/grokmirror.git/about/ > * example setup for guix: https://yhetil.org/guix-patches/878scww903.fsf= @kyleam.com/ > > Both grokmirror and public-inbox-clone/fetch make use of the manifests > that are published for public-inbox archives: > > $ curl -fSsL https://yhetil.org/manifest.js.gz | gzip -d | \ > jq -r 'keys | .[] | select(contains("guix"))' > /guix-bugs/git/0.git > /guix-devel/git/0.git > /guix-patches/git/1.git > /guix-science/git/0.git > /guix-user/git/0.git > > Then there's of course also public-inbox's lei (local email interface). > I won't get into that, but, for anyone interested, here are messages > where I've given some examples: > > * https://yhetil.org/emacs-devel/87wnh22w7o.fsf@kyleam.com > * https://yhetil.org/guix-devel/87y1zcljq3.fsf@kyleam.com Thank you both for sharing your approaches. As I personally use isync / mbsync to fetch the emails of my email-accounts via IMAP into maildirs (which I then proceed to index with mu and read with mu4e), I prefer to use the same tech stack to fetch the mailing-list archives. And I now accomplished this with the attached isync-configuration. Note that, as Kyle noted, I needed to use the "0" path sometimes, and the "1" path other times. Also, you might want to set and adapt MaxMessages to fit your needs. --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=isync-config IMAPAccount guix-bugs Host news.yhetil.org User arbitrary AuthMechs ANONYMOUS SSLType IMAPS SSLVersions TLSv1.2 IMAPStore guix-bugs-remote Account guix-bugs MaildirStore guix-bugs-local Inbox /home/user/path/to/maildirs/guix-bugs/inbox Path /home/user/path/to/maildirs/guix-bugs/ SubFolders Verbatim Channel guix-bugs Far :guix-bugs-remote:yhetil.gnu.guix.bugs.0 Near :guix-bugs-local: CopyArrivalDate yes Create Near ExpireUnread yes # MaxMessages 100000 Sync Pull SyncState * IMAPAccount guix-devel Host news.yhetil.org User arbitrary AuthMechs ANONYMOUS SSLType IMAPS SSLVersions TLSv1.2 IMAPStore guix-devel-remote Account guix-devel MaildirStore guix-devel-local Inbox /home/user/path/to/maildirs/guix-devel/inbox Path /home/user/path/to/maildirs/guix-devel/ SubFolders Verbatim Channel guix-devel Far :guix-devel-remote:yhetil.gnu.guix.devel.1 Near :guix-devel-local: CopyArrivalDate yes Create Near ExpireUnread yes # MaxMessages 100000 Sync Pull SyncState * IMAPAccount guix-help Host news.yhetil.org User arbitrary AuthMechs ANONYMOUS SSLType IMAPS SSLVersions TLSv1.2 IMAPStore guix-help-remote Account guix-help MaildirStore guix-help-local Inbox /home/user/path/to/maildirs/guix-help/inbox Path /home/user/path/to/maildirs/guix-help/ SubFolders Verbatim Channel guix-help Far :guix-help-remote:yhetil.gnu.guix.user.0 Near :guix-help-local: CopyArrivalDate yes Create Near ExpireUnread yes # MaxMessages 100000 Sync Pull SyncState * IMAPAccount guix-patches Host news.yhetil.org User arbitrary AuthMechs ANONYMOUS SSLType IMAPS SSLVersions TLSv1.2 IMAPStore guix-patches-remote Account guix-patches MaildirStore guix-patches-local Inbox /home/user/path/to/maildirs/guix-patches/inbox Path /home/user/path/to/maildirs/guix-patches/ SubFolders Verbatim Channel guix-patches Far :guix-patches-remote:yhetil.gnu.guix.patches.1 Near :guix-patches-local: CopyArrivalDate yes Create Near ExpireUnread yes # MaxMessages 100000 Sync Pull SyncState * Group mailing-lists Channel guix-bugs Channel guix-devel Channel guix-help Channel guix-patches --=-=-=--