unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Getting a copy of the Cuirass (berlin.guixsd.org) database
@ 2019-02-10 13:21 Christopher Baines
  2019-02-13 21:00 ` Björn Höfling
  0 siblings, 1 reply; 7+ messages in thread
From: Christopher Baines @ 2019-02-10 13:21 UTC (permalink / raw)
  To: guix-devel

[-- Attachment #1: Type: text/plain, Size: 358 bytes --]

Hey,

Would someone be able to make a copy of the Cuirass database available?
It might be useful when tinkering with Cuirass, as it contains some
realistic data.

I don't know much about SQLite, but it looks like there's a ".backup"
command [1] which might be useful.

1: https://sqlite.org/cli.html#special_commands_to_sqlite3_dot_commands_

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Getting a copy of the Cuirass (berlin.guixsd.org) database
  2019-02-10 13:21 Getting a copy of the Cuirass (berlin.guixsd.org) database Christopher Baines
@ 2019-02-13 21:00 ` Björn Höfling
  2019-02-14  0:07   ` Christopher Baines
  2019-02-21 20:30   ` Danny Milosavljevic
  0 siblings, 2 replies; 7+ messages in thread
From: Björn Höfling @ 2019-02-13 21:00 UTC (permalink / raw)
  To: guix-devel

[-- Attachment #1: Type: text/plain, Size: 605 bytes --]

On Sun, 10 Feb 2019 13:21:57 +0000
Christopher Baines <mail@cbaines.net> wrote:

> Would someone be able to make a copy of the Cuirass database
> available? It might be useful when tinkering with Cuirass, as it
> contains some realistic data.

Hi Chris,

I'm not sure if anybody else responded to you, but I have a database
around. I got it last year from Clement Lassieur. It is 1.6 GB in size,
I made a tar.xz an that is only 86 MB! I wonder if there is anything
wrong or it is just thaaat compressible! Anyway, you can try it out
here:

http://bjoernhoefling.de/berlin.tar.xz

Björn

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Getting a copy of the Cuirass (berlin.guixsd.org) database
  2019-02-13 21:00 ` Björn Höfling
@ 2019-02-14  0:07   ` Christopher Baines
  2019-02-21 20:30   ` Danny Milosavljevic
  1 sibling, 0 replies; 7+ messages in thread
From: Christopher Baines @ 2019-02-14  0:07 UTC (permalink / raw)
  To: Björn Höfling; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 693 bytes --]


Björn Höfling <bjoern.hoefling@bjoernhoefling.de> writes:

> On Sun, 10 Feb 2019 13:21:57 +0000
> Christopher Baines <mail@cbaines.net> wrote:
>
>> Would someone be able to make a copy of the Cuirass database
>> available? It might be useful when tinkering with Cuirass, as it
>> contains some realistic data.
>
> Hi Chris,
>
> I'm not sure if anybody else responded to you, but I have a database
> around. I got it last year from Clement Lassieur. It is 1.6 GB in size,
> I made a tar.xz an that is only 86 MB! I wonder if there is anything
> wrong or it is just thaaat compressible! Anyway, you can try it out
> here:

Awesome, thanks Björn. I've downloaded it now :)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Getting a copy of the Cuirass (berlin.guixsd.org) database
  2019-02-13 21:00 ` Björn Höfling
  2019-02-14  0:07   ` Christopher Baines
@ 2019-02-21 20:30   ` Danny Milosavljevic
  2019-02-22 16:54     ` Björn Höfling
  2019-03-06 12:55     ` Ludovic Courtès
  1 sibling, 2 replies; 7+ messages in thread
From: Danny Milosavljevic @ 2019-02-21 20:30 UTC (permalink / raw)
  To: Björn Höfling; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 697 bytes --]

Hi Björn,

On Wed, 13 Feb 2019 22:00:06 +0100
Björn Höfling <bjoern.hoefling@bjoernhoefling.de> wrote:

> I'm not sure if anybody else responded to you, but I have a database
> around. I got it last year from Clement Lassieur. It is 1.6 GB in size,
> I made a tar.xz an that is only 86 MB! I wonder if there is anything
> wrong or it is just thaaat compressible!

I think we didn't set up the infrastructure to VACUUM the database.

The default in sqlite is not to VACUUM ever, so we probably should
either stop the entire thing and vacuum it once a week or we could
set up autovacuum to incremental or full (the latter would vacuum
at each commit) if we ever ran out of space.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Getting a copy of the Cuirass (berlin.guixsd.org) database
  2019-02-21 20:30   ` Danny Milosavljevic
@ 2019-02-22 16:54     ` Björn Höfling
  2019-03-06 12:55     ` Ludovic Courtès
  1 sibling, 0 replies; 7+ messages in thread
From: Björn Höfling @ 2019-02-22 16:54 UTC (permalink / raw)
  To: Danny Milosavljevic; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 892 bytes --]

On Thu, 21 Feb 2019 21:30:39 +0100
Danny Milosavljevic <dannym@scratchpost.org> wrote:

> Hi Björn,
> 
> On Wed, 13 Feb 2019 22:00:06 +0100
> Björn Höfling <bjoern.hoefling@bjoernhoefling.de> wrote:
> 
> > I'm not sure if anybody else responded to you, but I have a database
> > around. I got it last year from Clement Lassieur. It is 1.6 GB in
> > size, I made a tar.xz an that is only 86 MB! I wonder if there is
> > anything wrong or it is just thaaat compressible!  
> 
> I think we didn't set up the infrastructure to VACUUM the database.
> 
> The default in sqlite is not to VACUUM ever, so we probably should
> either stop the entire thing and vacuum it once a week or we could
> set up autovacuum to incremental or full (the latter would vacuum
> at each commit) if we ever ran out of space.

That would at least tell why it is so good compressable.

Björn

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Getting a copy of the Cuirass (berlin.guixsd.org) database
  2019-02-21 20:30   ` Danny Milosavljevic
  2019-02-22 16:54     ` Björn Höfling
@ 2019-03-06 12:55     ` Ludovic Courtès
  2019-03-06 19:08       ` Christopher Baines
  1 sibling, 1 reply; 7+ messages in thread
From: Ludovic Courtès @ 2019-03-06 12:55 UTC (permalink / raw)
  To: Danny Milosavljevic; +Cc: guix-devel

Hi Danny,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> On Wed, 13 Feb 2019 22:00:06 +0100
> Björn Höfling <bjoern.hoefling@bjoernhoefling.de> wrote:
>
>> I'm not sure if anybody else responded to you, but I have a database
>> around. I got it last year from Clement Lassieur. It is 1.6 GB in size,
>> I made a tar.xz an that is only 86 MB! I wonder if there is anything
>> wrong or it is just thaaat compressible!
>
> I think we didn't set up the infrastructure to VACUUM the database.
>
> The default in sqlite is not to VACUUM ever, so we probably should
> either stop the entire thing and vacuum it once a week or we could
> set up autovacuum to incremental or full (the latter would vacuum
> at each commit) if we ever ran out of space.

What would it take to enable autovacuum?  Sounds like something we
should do.  :-)

Ludo’.

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Getting a copy of the Cuirass (berlin.guixsd.org) database
  2019-03-06 12:55     ` Ludovic Courtès
@ 2019-03-06 19:08       ` Christopher Baines
  0 siblings, 0 replies; 7+ messages in thread
From: Christopher Baines @ 2019-03-06 19:08 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 1294 bytes --]


Ludovic Courtès <ludo@gnu.org> writes:

> Hi Danny,
>
> Danny Milosavljevic <dannym@scratchpost.org> skribis:
>
>> On Wed, 13 Feb 2019 22:00:06 +0100
>> Björn Höfling <bjoern.hoefling@bjoernhoefling.de> wrote:
>>
>>> I'm not sure if anybody else responded to you, but I have a database
>>> around. I got it last year from Clement Lassieur. It is 1.6 GB in size,
>>> I made a tar.xz an that is only 86 MB! I wonder if there is anything
>>> wrong or it is just thaaat compressible!
>>
>> I think we didn't set up the infrastructure to VACUUM the database.
>>
>> The default in sqlite is not to VACUUM ever, so we probably should
>> either stop the entire thing and vacuum it once a week or we could
>> set up autovacuum to incremental or full (the latter would vacuum
>> at each commit) if we ever ran out of space.
>
> What would it take to enable autovacuum?  Sounds like something we
> should do.  :-)

I think vacuum can result in the rowid values changing [1], and in at
least one place (the build ids), the rowid values are exposed to users.

1: https://www.sqlite.org/lang_createtable.html#rowid

I think by creating a explicit column for the id, it's possible to avoid
them changing during the vacuum, so it might be good to do that before
vacuuming.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2019-03-06 19:08 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-02-10 13:21 Getting a copy of the Cuirass (berlin.guixsd.org) database Christopher Baines
2019-02-13 21:00 ` Björn Höfling
2019-02-14  0:07   ` Christopher Baines
2019-02-21 20:30   ` Danny Milosavljevic
2019-02-22 16:54     ` Björn Höfling
2019-03-06 12:55     ` Ludovic Courtès
2019-03-06 19:08       ` Christopher Baines

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).