From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 21A0F1F47D for ; Sun, 5 Mar 2023 22:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1678054691; bh=TsOEjesMh3B1KwNSAElBGM0AeobqZV4TGOY71o7CtVs=; h=From:To:Subject:Date:From; b=i0vaGXdOY1I/4MeyMDlAnK+lj7R6qibsNwLhcnVNg3eUrjOZv6ootILn9ECX2LGHp QGcUbJLOkJNcyrHk9b5cNLGoc7bKkwcPcUR21JIBjyPCx7GlTLvl/Z+wBttiK6+UFm aDH8JRizTU4hyJfZ8dvj3ZT/vr+uLprCa0dhdxk4= From: Eric Wong To: meta@public-inbox.org Subject: [PATCH] doc: update public-inbox-clone examples and help Date: Sun, 5 Mar 2023 22:18:11 +0000 Message-Id: <20230305221811.3846334-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Basically, public-inbox-clone has become grok-pull without config files nor absolute paths. --- Documentation/public-inbox-clone.pod | 62 ++++++++++++++++++++++------ script/public-inbox-clone | 12 ++++-- 2 files changed, 57 insertions(+), 17 deletions(-) diff --git a/Documentation/public-inbox-clone.pod b/Documentation/public-inbox-clone.pod index 7774d4ed..af4e8e95 100644 --- a/Documentation/public-inbox-clone.pod +++ b/Documentation/public-inbox-clone.pod @@ -15,14 +15,16 @@ making the initial clone of a remote HTTP(S) public-inbox. It allows cloning multi-epoch v2 inboxes with a single command and zero configuration. -In public-inbox 2.0+, public-inbox-clone can also mirror multiple -inboxes or code repositories using manifest.js.gz files like -L from grokmirror. +In public-inbox 2.0+, public-inbox-clone can create and maintain +a mirror of multiple inboxes or code repositories using manifest.js.gz +files like L from grokmirror. L is +NOT required when using this mode. It does not run L nor L. Those commands must be run separately if serving/searching the mirror is required. As-is, -public-inbox-clone is suitable for creating a git-only backup. +public-inbox-clone is suitable for creating a git-only backup +without Xapian and SQLite indices. When cloning a single inbox, public-inbox-clone creates a Makefile with handy targets to update the inbox once indexed. @@ -31,11 +33,12 @@ not be rewritten by L unless it is removed completely. public-inbox-clone does not use nor require any extra -configuration files (not even C<~/.public-inbox/config>). +configuration files (not even C<~/.public-inbox/config>), +but it can download snippets suitable for adding to any +L file. L may be used to keep a single C -up-to-date. When using manifest.js.gz, public-inbox-clone is used -to maintain the multi-repository mirror. +up-to-date. For v2 inboxes, it will create a C<$INBOX_DIR/manifest.js.gz> file to speed up subsequent L. @@ -63,17 +66,17 @@ Default: C<0..~0> or C<0..> or C<..~0> =item --include=PATTERN -When cloning a top-level with multiple inboxes, only clone inboxes and -repositories matching a given wildcard pattern (using C<*?> and C<[]> is -supported). +When cloning a top-level with multiple inboxes via manifest, +only clone inboxes and repositories matching a given wildcard pattern +(using C<*?> and C<[]> is supported). This is a new option in public-inbox 2.0+ =item --exclude=PATTERN -When cloning a top-level with multiple inboxes, ignore inboxes and -repositories matching the given wildcard pattern. Supports the same -wildcards as L +When cloning a top-level with multiple inboxes via manifest, +ignore inboxes and repositories matching the given wildcard pattern. +Supports the same wildcards as L This is a new option in public-inbox 2.0+ @@ -204,6 +207,39 @@ Whether to wrap L and L commands with L. Default: C +=item -j JOBS +=item --jobs=JOBS + +The number of parallel processes to spawn at once for various network +operations using L and/or L. + +=back + +=head1 EXAMPLES + +=for comment +Sticking to smaller projects in examples to minimize load on servers + +=over + +=item To mirror the most recent epochs of dwarves and LTTng inboxes: + + public-inbox-clone --epoch=~0 \ + --include='*lttng*' --include='*dwarves' \ + https://80x24.org/lore/ /path/to/inbox-mirror + +C may be used instead of C + +=item To mirror all code repos of the sparse project: + + public-inbox-clone --objstore= --project-list= --prune \ + --include='*sparse*' --inbox-config=never \ + --remote-manifest=https://80x24.org/lore/pub/manifest.js.gz \ + https://80x24.org/lore/ /path/to/code-mirror + +C may be used instead of C +and the C<--remote-manifest> option can be omitted. + =back =head1 CONTACT diff --git a/script/public-inbox-clone b/script/public-inbox-clone index 10ad3487..5b365df7 100755 --- a/script/public-inbox-clone +++ b/script/public-inbox-clone @@ -6,21 +6,25 @@ use v5.12; use Getopt::Long qw(:config gnu_getopt no_ignore_case auto_abbrev); my $opt = {}; my $help = <