From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id yNscCKAROWK1ZQEAgWs5BA (envelope-from ) for ; Tue, 22 Mar 2022 01:00:32 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id 6BDLBKAROWK1IgEAauVa8A (envelope-from ) for ; Tue, 22 Mar 2022 01:00:32 +0100 Received: from mail.notmuchmail.org (yantan.tethera.net [135.181.149.255]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id CBCC23E169 for ; Tue, 22 Mar 2022 01:00:28 +0100 (CET) Received: from yantan.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id C94F35F6DD; Tue, 22 Mar 2022 00:00:24 +0000 (UTC) Received: from arlo.cworth.org (arlo.cworth.org [50.126.95.6]) by mail.notmuchmail.org (Postfix) with ESMTPS id 910F55F6CF for ; Tue, 22 Mar 2022 00:00:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cworth.org; s=mail; t=1647907220; bh=3P96Bp52BkVT2jVc4BJXB42MuJErAon98XVHLixMCc8=; h=To:Cc:Subject:In-Reply-To:References:From:Date:From; b=sQUJoKZOXvsItyanTsdC+GCq9BIPmrexVVMgoC8N8xSEQ2zCWWVhGGhsjd8X51v/r BkC+hp6aTnnU28iyE/wZahyhX/YIBuIC5A8A/YS5obN8iKk0zf4ZjKPTjYiKTxx1JY SjvD5C5UyjeBZIM1DRcsQqVihmcf+dDLqFGbWioTbgiNczwpfBOktuyusGC5yidvrS SXe+pW8RdF61vI60RuHjX1kcR++SBmwX4n4XRuBCjqJXvhGN8eninieoLcr3qicBtz 7zNfiTbyoBJmBeV2CG8Qc8Li6yn+bCDYC2OBjYnaGUsdDBiuR+LNKqEX7aaipYhLHJ o3jlfx2df/pRg== Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 05F4B6DE02CD for ; Mon, 21 Mar 2022 17:00:20 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 71-s0NfFGsC9; Mon, 21 Mar 2022 17:00:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cworth.org; s=mail; t=1647907219; bh=3P96Bp52BkVT2jVc4BJXB42MuJErAon98XVHLixMCc8=; h=To:Cc:Subject:In-Reply-To:References:From:Date:From; b=ycY2BLKCC7fwq8y3ijj3/XqtXWk9gmrjWXhT8zscHTQ0JBCzAGkpJKA+ziDvME/4C Y/O7TmubFssYItGTgVI3xeF16o3utxfdHORo8GHo0RZ9CHPNvv/QpKgGn21WIFMuk3 XT6/HdNKxdVqkLnXQT+unOI4cJNmlzU96+angm0BZlx0I6IwTg1RpmOy+YVsmscY2E tLb1xy9wtH08WRO7vIRazLem0VVeCcgMJcDMiWCSgLh/8KNChqi84rFhSWpCsMFjES e63PjXZmuyrejvhpWACxmmpGIsmIC16hNBTShAXE1vOXPgRy87I0CschWaWL26SA99 qLY7FzczZzb/g== Received: from wondoo.home.cworth.org (unknown [10.0.0.225]) (Authenticated sender: cworth) by arlo.cworth.org (Postfix) with ESMTPSA id 7EED96DE0225; Mon, 21 Mar 2022 17:00:19 -0700 (PDT) Received: from wondoo (localhost [IPv6:::1]) by wondoo.home.cworth.org (Postfix) with ESMTPS id 5F3341FE00CF; Mon, 21 Mar 2022 17:00:19 -0700 (PDT) To: Tobias Waldekranz Subject: Thanks for notmuch-lore In-Reply-To: <8735l2b7ui.fsf@waldekranz.com> References: <20220131154655.1614770-1-tobias@waldekranz.com> <20220131154655.1614770-2-tobias@waldekranz.com> <20220201170634.wnxy3s7f6jnmt737@skbuf> <87a6fabbtb.fsf@waldekranz.com> <20220201201141.u3qhhq75bo3xmpiq@skbuf> <8735l2b7ui.fsf@waldekranz.com> Sender: cworth@cworth.org From: Carl Worth Date: Mon, 21 Mar 2022 17:00:04 -0700 Message-ID: <87mthiua9n.fsf@wondoo.home.cworth.org> MIME-Version: 1.0 Message-ID-Hash: UF72TECXCN5CKQVTQARX2FZMGCH6XHYP X-Message-ID-Hash: UF72TECXCN5CKQVTQARX2FZMGCH6XHYP X-MailFrom: cworth@cworth.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: notmuch@notmuchmail.org X-Mailman-Version: 3.3.3 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: multipart/mixed; boundary="===============1429380440535873143==" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: DE ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1647907231; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-owner:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=OQGmCLw2n94EmVLDSC1L4vbOSxJ4QzbqZApH9K6lSnE=; b=aIUzUzKWq7c/Vwn7weZ7pQ/ZDhfQRZLS0pjxojmmIi7Mc/SJysrw4oIXVAus2qjtXv/fiu 2HgPWQRqsY2ox/lN3ZzE5V0kVm2keM5sooTdOO26ix6awR/jVebyQnArcB2hQQ/2dysYF/ eKeAYMUAClAfW04+PzafoKEENbYQqJWjk8QIurMuBWd03MfDO0kPKZAZo5pQgUuSndhLST XtpvdLf9ctc8A6+Ou2eEZ2y19v3WpNHyoDpSRmLwgtZJw140rSH322YEFeuJhhzIi2v+O5 iuwOT6GLbM8bvYVicsaTN8cpMN8w9W1IspLGMAKtylkiMnsC5wWLXYh1RhNB6Q== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1647907231; a=rsa-sha256; cv=none; b=aRmA40PBijMXZQt26O/qym3reyQMvgT/fMTg+RT5fNX+O4JnUn2G+8YYlTNSPAnJdwXQRY 5Ij4LDDgotC4AxAjnfv7QxCNu4+itutHLmkw+0ZHeDL4wscltc2rfC8bWJWo5A9lAtb3hl oNO4ySAydvPmAnQH0QDf1r97FLwSL4geE/AvUt/3gBVasO2VM9uEsYuKHZCufc9lHUaXBA KfrSSQbODgLpIzbBAaPZ/i6PyFq9WlvNHBmsIoSQ7aorwgB5LWMRxykMCj9CGoXft/71rs iFtOGzRQforJSDJoHRIbbC96c5FxyKXpXqMFoHsb5svSRhr811AgCukJ2vxOFw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=cworth.org header.s=mail header.b=sQUJoKZO; dkim=fail ("body hash did not verify") header.d=cworth.org header.s=mail header.b=ycY2BLKC; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 135.181.149.255 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: 0.71 Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=cworth.org header.s=mail header.b=sQUJoKZO; dkim=fail ("body hash did not verify") header.d=cworth.org header.s=mail header.b=ycY2BLKC; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 135.181.149.255 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: CBCC23E169 X-Spam-Score: 0.71 X-Migadu-Scanner: scn1.migadu.com X-TUID: f+RjM3y3pfo8 --===============1429380440535873143== Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" --=-=-= Content-Type: text/plain On Tue, Feb 01 2022, Tobias Waldekranz wrote: > I actually gave up on getting my mailinglists from my email provider, > now I just download it directly from lore. I hacked together a script > that will scrape a public-inbox repo and convert it to a Maildir: > > https://github.com/wkz/notmuch-lore Thanks for sharing this, Tobias. I needed exactly this today, and was happy to have found this. It looks like you've coded something to efficiently do the work that's needed periodically, (fetch new emails from the public-inbox git repository, convert them to maildir files, and prune away git state other than a pointer to what's been converted already). What I'm missing is the piece to convert over the entire archive from the past. I can fetch it all easily enough with public-inbox-clone. Maybe what I want could be captured in a tool named something like: public-inbox-export --output=maildir After which I'd be all bootstrapped and ready to use your notmuch-lore pre-new hook. > As you can tell from the name, it is tailored for plugging into notmuch, > but the guts are pretty generic. Indeed. And it looks like all the code I would need for the export I described above is right there in your script. It's as simple as: git rev-list | while read sha; do $git show $sha:m > $maildir/new/$sha done So, next I should go put together a patch against public-inbox to add that. Thanks again, -Carl PS. I debated whether to CC lkml where the original message I was replying to was from originally. I decided against it and almost just emailed Tobias alone, but I really do want discussion like this to be archived in public. So I CCed the notmuch mailing list at least. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEzvjdyAGcpm1OMJTzYAIzup5U3GEFAmI5EYQACgkQYAIzup5U 3GEx4A//Rz3x4E3+Kax5JFi61qo0EfPNEmEKmj3IGMX+YmPwXaEeh5eCxChM4oLk NF60+Rr2F+rbW5LQpCL8CeKrkMmH29ZT1NUC6u0KrYr7BjVCEFQiQndI3P8tpMLR JT1shhX8Z2BXHKtBTFfkCVb9VjRE408u2OrCjJN0G1ciS7UbNaagdc4my8TNPNsN WMi1+q/Nw0KsgnFRkrAeS1SsIxH3GVWWTI5xaqKXhn4H0Yj2WUKFz0VUoFbAHpMH weqXszn1Vdre4ZSc2sUEh5w7S4g9XazRIQN31qPWulS3EhveN8+Ql9YxoSHnHIYJ I8aAceO8ipVRstSDd+lnniyibR0b04PA3JQf/8TloN+QqZIcme78q/30oO7HIPwK vWMNUmWpl91C6JWEPUU7b+Uz6ed68eqaTyo07b9lydoSEUgiYyWO/EEsnLRZM6fB UnEv4h3lHff7TEBgR9E6qstBA3t+blPTHgD6x8BzDrfRNByB+BRfNMPkH4sbORZ0 ieT/NGsoFJNElUnhxVAX4CsTre+/PeSAdT9TpJtWttweyH1+/kR9RcI/hZIjlevv QoMMh5Aj/irtLFkOY0/e0KW78C+J9qrmF6OIu1vljKa9hDZ9uR7MNQiT7bB37QSV Vw4ElffiS20kObZ3gIqxdacXDW+tVArG84CeKv97nJB0KX8QGa8= =0PMz -----END PGP SIGNATURE----- --=-=-=-- --===============1429380440535873143== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============1429380440535873143==--