Figuring out `public-inbox-fetch`

Figuring out `public-inbox-fetch`

[Dom (shymega) Rodriguez]

[-- Attachment #1: Type: text/plain, Size: 2164 bytes --]

Hi.

I've recently been setting up a `public-inbox` mirror on my workstation.

I've cloned the epochs for a few Linux kernel lists, and Spectrum, and
used `public-inbox-init` to add the lists to the configuration in
`$HOME`.

However, I'm finding the documentation rather lacking (happy to make
patches once I figure it out) in terms of examples in some instances.

For example, the `public-inbox-fetch` command, I've pointed the `-C`
argument to say, `linux-mediatek/git/0.git`, and get the following
output:

````
$ public-inbox-fetch -C projects/mailing-lists/linux-mediatek/git/0.git
# inbox URL: http://lore.kernel.org/linux-mediatek/
# /etc/profiles/per-user/shymega/bin/curl -Sf -R -o m-xfHa.tmp http://lore.kernel.org/linux-mediatek/manifest.js.gz
   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                  Dload  Upload   Total   Spent    Left  Speed
   0     0    0     0    0     0      0      0 --:--:-- --:-100   162  100   162    0     0   5290      0 --:--:-- --:--:-- --:--:--  5400
# git --git-dir=/home/shymega/projects/mailing-lists/linux-mediatek/git/0.git fetch
warning: redirecting to https://lore.kernel.org/linux-mediatek/0/
# git clone --mirror http://lore.kernel.org/linux-mediatek/git/1.git /home/shymega/projects/mailing-lists/linux-mediatek/git/1.git
Cloning into bare repository '/home/shymega/projects/mailing-lists/linux-mediatek/git/1.git'...
remote: Not Found
fatal: repository 'http://lore.kernel.org/linux-mediatek/git/1.git/' not found
error: could not lock config file /home/shymega/projects/mailing-lists/linux-mediatek/git/1.git/config: No such file or directory
git config -f /home/shymega/projects/mailing-lists/linux-mediatek/git/1.git/config include.path ../../all.git/config failed: $?=65280
````

Am I on the right path here? I'm not sure what I'm doing wrong.

Sorry for the 'newbie' question. Although, we all were at one point or
another.

Other than this, I'm relatively happy with `public-inbox` mirroring. I'm
making a developer-oriented multi-'adapter' NNTP server at the moment,
which I'm hoping to use myself.

Thank you.

Best wishes,
--
Dom Rodriguez

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Figuring out `public-inbox-fetch`

[Eric Wong]

"Dom (shymega) Rodriguez" <shymega@shymega.org.uk> wrote:
> Hi.
> 
> I've recently been setting up a `public-inbox` mirror on my workstation.
> 
> I've cloned the epochs for a few Linux kernel lists, and Spectrum, and
> used `public-inbox-init` to add the lists to the configuration in
> `$HOME`.
> 
> However, I'm finding the documentation rather lacking (happy to make
> patches once I figure it out) in terms of examples in some instances.
> 
> For example, the `public-inbox-fetch` command, I've pointed the `-C`
> argument to say, `linux-mediatek/git/0.git`, and get the following
> output:
> 
> ````
> $ public-inbox-fetch -C projects/mailing-lists/linux-mediatek/git/0.git

That should be pointed to the inbox directory, not the git epoch,
(although epoch seems to work, too...):

$ public-inbox-fetch -C projects/mailing-lists/linux-mediatek/

You shouldn't need to go into deeper directory levels in normal day-to-day use.

> # inbox URL: http://lore.kernel.org/linux-mediatek/
> # /etc/profiles/per-user/shymega/bin/curl -Sf -R -o m-xfHa.tmp http://lore.kernel.org/linux-mediatek/manifest.js.gz
>   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
>                                  Dload  Upload   Total   Spent    Left  Speed
>   0     0    0     0    0     0      0      0 --:--:-- --:-100   162  100   162    0     0   5290      0 --:--:-- --:--:-- --:--:--  5400
> # git --git-dir=/home/shymega/projects/mailing-lists/linux-mediatek/git/0.git fetch
> warning: redirecting to https://lore.kernel.org/linux-mediatek/0/
> # git clone --mirror http://lore.kernel.org/linux-mediatek/git/1.git /home/shymega/projects/mailing-lists/linux-mediatek/git/1.git
> Cloning into bare repository '/home/shymega/projects/mailing-lists/linux-mediatek/git/1.git'...
> remote: Not Found
> fatal: repository 'http://lore.kernel.org/linux-mediatek/git/1.git/' not found
> error: could not lock config file /home/shymega/projects/mailing-lists/linux-mediatek/git/1.git/config: No such file or directory
> git config -f /home/shymega/projects/mailing-lists/linux-mediatek/git/1.git/config include.path ../../all.git/config failed: $?=65280

Everything said, I'm not sure why it's trying 1.git here when
linux-mediatek only has the 0.git epoch...

Viewing the manifest via:
curl https://lore.kernel.org/linux-mediatek/manifest.js.gz | zcat | less
confirms there's only one epoch..

> Am I on the right path here? I'm not sure what I'm doing wrong.

I wonder if something got mangled by lore doing the redirects from http to
https.  Perhaps changing the url in git/0.git/config to use https will help
*shrug*

You can also try my slower lore mirror https://yhbt.net/lore/linux-mediatek/
I use for dogfooding new changes.

Also, are you using public-inbox.git or the v1.9.0 release?
I really need to get the 2.0 features done and released but real life
has been sucking more and more every year :x

> Sorry for the 'newbie' question. Although, we all were at one point or
> another.

No worries, it's good for pointing out flaws in documentation :>

> Other than this, I'm relatively happy with `public-inbox` mirroring. I'm
> making a developer-oriented multi-'adapter' NNTP server at the moment,
> which I'm hoping to use myself.

Cool :>

Re: Figuring out `public-inbox-fetch`

[Dom (shymega) Rodriguez]

[-- Attachment #1: Type: text/plain, Size: 2194 bytes --]

On 05.09.2024 19:33, Eric Wong wrote:
>"Dom (shymega) Rodriguez" <shymega@shymega.org.uk> wrote:
>> Hi.
>>
>> I've recently been setting up a `public-inbox` mirror on my workstation.
>>
>> I've cloned the epochs for a few Linux kernel lists, and Spectrum, and
>> used `public-inbox-init` to add the lists to the configuration in
>> `$HOME`.
>>
>> However, I'm finding the documentation rather lacking (happy to make
>> patches once I figure it out) in terms of examples in some instances.
>>
>> For example, the `public-inbox-fetch` command, I've pointed the `-C`
>> argument to say, `linux-mediatek/git/0.git`, and get the following
>> output:
>>
>> ````
>> $ public-inbox-fetch -C projects/mailing-lists/linux-mediatek/git/0.git
>
>That should be pointed to the inbox directory, not the git epoch,
>(although epoch seems to work, too...):
>
>$ public-inbox-fetch -C projects/mailing-lists/linux-mediatek/
>
>You shouldn't need to go into deeper directory levels in normal day-to-day use.

Ah, okay.

>> [ ... snip ... ]

>> Am I on the right path here? I'm not sure what I'm doing wrong.
>
>I wonder if something got mangled by lore doing the redirects from http to
>https.  Perhaps changing the url in git/0.git/config to use https will help
>*shrug*

Changing it to HTTPS did the trick! 

>You can also try my slower lore mirror https://yhbt.net/lore/linux-mediatek/
>I use for dogfooding new changes.
>
>Also, are you using public-inbox.git or the v1.9.0 release?
>I really need to get the 2.0 features done and released but real life
>has been sucking more and more every year :x

I'm on v1.9.0 on NixOS. Is that alright to use?

>> Sorry for the 'newbie' question. Although, we all were at one point or
>> another.
>
>No worries, it's good for pointing out flaws in documentation :>

Absolutely.

>> Other than this, I'm relatively happy with `public-inbox` mirroring. I'm
>> making a developer-oriented multi-'adapter' NNTP server at the moment,
>> which I'm hoping to use myself.
>
>Cool :>

Thank you for your help :)

Best wishes,
--
Dom Rodriguez
GPG Fingerprint: EB0D 45E6 D0DC 1BA1 A2B5  FC24 72DC F123 1E54 BD43

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Figuring out `public-inbox-fetch`

[Eric Wong]

"Dom (shymega) Rodriguez" <shymega@shymega.org.uk> wrote:
> On 05.09.2024 19:33, Eric Wong wrote:
> > I wonder if something got mangled by lore doing the redirects from http to
> > https.  Perhaps changing the url in git/0.git/config to use https will help
> > *shrug*
> 
> Changing it to HTTPS did the trick!

Good to know.  I'm not sure if the bug is on the lore redirect
configuration or something else, though.

> > Also, are you using public-inbox.git or the v1.9.0 release?
> > I really need to get the 2.0 features done and released but real life
> > has been sucking more and more every year :x
> 
> I'm on v1.9.0 on NixOS. Is that alright to use?

Yes, 1.9.0 is fine.  There's been major changes in that area
(mainly public-inbox-clone) since the last release, though.