From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 226061F8C8; Fri, 17 Sep 2021 00:22:05 +0000 (UTC) Date: Fri, 17 Sep 2021 00:22:05 +0000 From: Eric Wong To: Luis Chamberlain Cc: meta@public-inbox.org Subject: Re: lei import on epochs [was: make menuconfig interface for lei / grok-pull] Message-ID: <20210917002204.GA13112@dcvr> References: <20210916210902.GA28336@dcvr> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: List-Id: Luis Chamberlain wrote: > On Thu, Sep 16, 2021 at 09:09:02PM +0000, Eric Wong wrote: > > Luis Chamberlain wrote: > > > # The next two don't work > > > lei import import https://lore.kernel.org/linux-fsdevel/git/0.git > > > > Btw, extra "import" aside; I never intended to support the above > > case. However, your post got me thinking we could and probably > > should... :> > > > > (epochs are git storage with a loose limit of ~1G packed data, > > "slices" are public-inbox-imapd folders capped at 50K to > > accomodate limitations of existing clients/filesystems) > > So... by default all epochs are pulled? If so, yeah geesh. > I don't want to pull all linux-kernel epochs. Only if doing > R&D on that list would I need it. Right now, "lei import" doesn't read nor understand git repos at all. If you meant add-external, yes, all epochs are pulled when using "lei add-external --mirror" (and public-inbox-clone). Yes, there should be a way to do partial mirrors, just haven't gotten around to it... "lei import" currently reads IMAP, NNTP, Maildir, mbox*, etc. It also understands various https?://*/(raw,t.mbox.gz) URLs used by PublicInbox::WWW, since they're (gzipped) mboxrd. But none of that is reading from git repos directly... > Maybe a --query-epochs and then a --use-epoch 0 or whatever > as well? Since epochs expose the gittyness of lei, perhaps a limited form of commitish-like ranges used by git would work: lei add-external --mirror $URL --epoch=0..9 lei add-external --mirror $URL --epoch=5.. lei add-external --mirror $URL --epoch=~2.. # like HEAD~2.. That only works for contiguous ranges, though; but I'm also unsure if discontiguous ranges make sense to anyone. There's other syntaxes (e.g. (sed|Perl)-like), of course, but I think git-like here would be most natural. Thoughts?