public-inbox-ssoma fails with lkml archive

public-inbox-ssoma fails with lkml archive

[Yaron Scheffer]

Hi. I've been trying to get ssoma to fetch LKML archives into my mailbox using
the git repo at `https://lore.kernel.org/lkml/6`. Running `ssoma sync` shows
the git fetch completing successfully, but nothing shows up in my Maildir.
Performing the same procedure does however work with the PI archive at `git
://public-inbox.org/meta`.

My perl was rusty 15 years ago and it hasn't gotten any better with time,
but I've debugged this a little and the gittish things in Extractor.pm
led me to what looks like a smoking gun:

On the clone of git://public-inbox.org/meta:

    # git log --name-only
    commit 0fc21086aabb4ff9e9318a8ec3c6b58b24e9618a (HEAD -> master)
    ...
    c7/16e8798bc662c5fb52d05416a27897b49f0634

    commit d307195d89b21619de79722793cfaa7698aa5cf4
    ...
    99/a639f7c6842e6c69f489f9c60af1be5e3f152d

On the clone of https://lore.kernel.org/lkml/6:

    # git log --name-only
    commit fa44fffb6a81e1a6b9cceccf8c08074d4d5d83ca (HEAD -> master)
    ...
    m

    commit 9845bd1f6b14185eaa77eaceea212ee0d04da673
    ...
    m

so the lkml archive doesn't follow (mandatory, I presume) naming scheme as
PI/meta. I'm not sure if this is an issue in upstream or the deployment,
but right now ssoma doesn't work with the lkml archive.

Yaron

Re: public-inbox-ssoma fails with lkml archive

[Konstantin Ryabitsev]

[-- Attachment #1: Type: text/plain, Size: 485 bytes --]

On Tue, Oct 23, 2018 at 07:21:42AM +0000, Yaron Scheffer wrote:
>so the lkml archive doesn't follow (mandatory, I presume) naming scheme 
>as
>PI/meta. I'm not sure if this is an issue in upstream or the deployment,
>but right now ssoma doesn't work with the lkml archive.

The LKML archive uses the v2 format that was written specifically to 
handle huge archives. I don't believe ssoma works with v2-formatted 
repositories, but I'll let Eric provide the detail here.

-K

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: public-inbox-ssoma fails with lkml archive

[Yaron Scheffer]

On Tuesday, October 23, 2018 5:33 PM, Konstantin Ryabitsev wrote:

> The LKML archive uses the v2 format that was written specifically
> to handle huge archives. I don't believe ssoma works with v2-formatted
> repositories, but I'll let Eric provide the detail here.

Thanks for clarifying. I lost several hours trying to debug this,
I wish ssoma would die with an error when it detects a V2 archive.
Is there a version tag somewhere under .git which can be used for
this?

Re: public-inbox-ssoma fails with lkml archive

[Eric Wong]

Yaron Scheffer <yscheffer@protonmail.com> wrote:
> On Tuesday, October 23, 2018 5:33 PM, Konstantin Ryabitsev wrote:
> 
> > The LKML archive uses the v2 format that was written specifically
> > to handle huge archives. I don't believe ssoma works with v2-formatted
> > repositories, but I'll let Eric provide the detail here.
> 
> Thanks for clarifying. I lost several hours trying to debug this,
> I wish ssoma would die with an error when it detects a V2 archive.
> Is there a version tag somewhere under .git which can be used for
> this?

I'm very sorry about wasting your time.  I've made
https://public-inbox.org/meta/20181219095057.21862-2-e@80x24.org/
which should warn on v2.

Also I'm sorry about the delay in replying to this message; too
many hardware failures this year and other crap to deal with.


Zbfg bs gur oynzr tbrf gb znvevk naq Tbbtyr pbqrfrnepu(*)
since the Terabytes Written count on my drives were insane.
I should start sticking my personal mail into public-inbox :x


(*) yeah, still kinda trying to be nice and not speak ill of other
    Free Software projects...