bug#43528: Download all issues so I can search?

bug#43528: Download all issues so I can search?

[Yasuaki Kudo]

Hi,

If possible, I would like to download all guix issues in the tracker (in other words, database dump?) so I can just use full text search locally.   Is it possible?

Cheers,
Yasu

bug#43528: Download all issues so I can search?

[Tobias Geerinckx-Rice via Bug reports for GNU Guix]

Hi!

On 2020-09-20 9:30, Yasuaki Kudo wrote:
> Hi,
> 
> If possible, I would like to download all guix issues in the tracker
> (in other words, database dump?) so I can just use full text search
> locally.   Is it possible?

Probably!  This is basically what Mumi, the software behind 
issues.guix.gnu.org, does: pull all bug-guix and guix-patches mail from 
debbugs.gnu.org and index it locally using Mu.

I don't *think* it uses any secret API/authentication/whitelisted IPs 
etc. but am not very familiar with its innards.

If you can read Scheme, you can probably figure it out from 
<https://git.elephly.net/gitweb.cgi?p=software/mumi.git>.

Hoping to've been of some help,

T G-R

Sent from a Web browser. Excuse or enjoy my brevity.

bug#43528: Download all issues so I can search?

[Ricardo Wurmus]

Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix@gnu.org> writes:

> Hi!
>
> On 2020-09-20 9:30, Yasuaki Kudo wrote:
>> Hi,
>> If possible, I would like to download all guix issues in the tracker
>> (in other words, database dump?) so I can just use full text search
>> locally.   Is it possible?
>
> Probably!  This is basically what Mumi, the software behind
> issues.guix.gnu.org, does: pull all bug-guix and guix-patches mail
> from debbugs.gnu.org and index it locally using Mu.
>
> I don't *think* it uses any secret API/authentication/whitelisted IPs
> etc. but am not very familiar with its innards.

Actually it does exactly that (whitelisted IP doing rsync), ever since
The Indicident.  I used to just fetch all mbox files for all Guix bugs,
but this put the Debbugs servers under excessive load.  After the
server’s IP was blocked from accessing Debbugs we negotiated rsync
access.

We do have full text search on issues.guix.gnu.org.  It was brought to
my attention that it’s not yielding expected results since a while,
though, so this would have to be investigated.

-- 
Ricardo

bug#43528: Download all issues so I can search?

[Michael Rohleder]

[-- Attachment #1: Type: text/plain, Size: 276 bytes --]

Here's [1] an archive of the ML in mbox format.
Maybe this helps if you want to grep for things.


[1] https://lists.gnu.org/archive/mbox/bug-guix/

-- 
Das Leben kann nur rückwärts verstanden werden,
aber es muss vorwärts gelebt werden.
  – Søren Kierkegaard

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]

bug#43528: Download all issues so I can search?

[Maxim Cournoyer]

Michael Rohleder <mike@rohleder.de> writes:

> Here's [1] an archive of the ML in mbox format.
> Maybe this helps if you want to grep for things.
>
>
> [1] https://lists.gnu.org/archive/mbox/bug-guix/

That is indeed a good suggestion.

Closing.

Thanks,

Maxim

bug#43528: Download all issues so I can search?

[Bengt Richter]

On +2020-10-08 21:41:14 -0400, Maxim Cournoyer wrote:
> Michael Rohleder <mike@rohleder.de> writes:
> 
> > Here's [1] an archive of the ML in mbox format.
> > Maybe this helps if you want to grep for things.
> >
> >
> > [1] https://lists.gnu.org/archive/mbox/bug-guix/
> 
> That is indeed a good suggestion.
> 
> Closing.
> 
> Thanks,
> 
> Maxim
> 
> 
>

I discovered that the server supports incremental download using the '-c'
option of wget (so it only downloads what was added to the archive
since you last did it), e.g.,
    wget -c https://lists.gnu.org/archive/mbox/bug-guix/2020-10
then
    mutt -f 2020-10 # (or gnus equivalent I guess) looks normal.

Probably not news to most devs, but I thought it nice that it works.

(For me anyway :)

-- 
Regards,
Bengt Richter

bug#43528: Download all issues so I can search?

[zimoun]

Dear,

On Sun, 20 Sep 2020 at 11:17, Yasuaki Kudo <yasu@yasuaki.com> wrote:

> If possible, I would like to download all guix issues in the tracker (in other words, database dump?) so I can just use full text search locally.   Is it possible?

Here [1], unofficial public-inbox mirrors so you can "git clone" all
the issues and search offline.  Otherwise, please report the searching
issue you could hit with the searching facilities of the official
<issues.guix.gnu.org>.

[1] https://yhetil.org/


All the best,
simon