updating list of substitutes

updating list of substitutes

[Pjotr Prins]

Pretty much every time I want to install a package I get a search for

  updating list of substitutes 

being on a slow internet line this sucks (not everyone has fast
internet! Think outside the US/Europe where internet is often still
metered on mobile lines), besides installing the same software often
install a host of new versions of dependencies. I don't like the
system changing under me - that is not a reproducible setup.

Q1: Do we retain older builds of binaries for some time for download?

Q2: Can we switch off updating list of substitutes? A command line
    switch would do. '--no-update-supstitutes'

Q3: Would it be possible to version the list of substitutes and use
    that for (re)deployment? That way I can truely regenerate an existing
    system.

Pj.

On Sun, Apr 19, 2015 at 10:18:46AM +0200, Pjotr Prins wrote:
> On Sat, Apr 18, 2015 at 11:23:50PM +0200, Ludovic Courtès wrote:
> > Pjotr Prins <pjotr.public12@thebird.nl> skribis:
> > 
> > > Great :). I would make it a little clearer that this is
> > > 'bootstrapping' and hype it a little more that now there is no reason
> > > NOT to install Guix. Only 100Mb on your HDD.
> > 
> > Not sure how to do that, would you like to propose actual text?
> > The thing is, I want it to remain accurate and factual.
> 
> The current text is fine. 
> 
> > What do you meaning by moving a package with dependencies?
> 
> I am thinking about Nix-style closures. But it may only confuse
> things. I don't think the Guix manual covers closures.
> 
> > > BTW when Nix decided to go for a meta-database they lost something. I
> > > know it has good reasons (performance mostly) but it took away the
> > > self-containedness of packages. It would be nice just to be able to
> > > copy/del packages and rebuild the meta information. Do we have
> > > something like that? 
> > 
> > This part is the same as Nix.  The database is here to store meta-data
> > about store items, notably the list of references found in a store item.
> > Determining this list requires scanning all of the store item???s
> > contents, which takes time proportional to the number/size of files it
> > contains, so the database can hardly be avoided.
> 
> Yes, I understand. But would it be possible to regenerate the database
> from an existing /gnu/store? You can see I like to mess around with
> files ;). With closures a rebuild should not be necessary, but as a
> wary system administrator I know I will need it at some point.
> 
> Pj.
> 

-- 

Re: updating list of substitutes

[Ludovic Courtès]

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> Pretty much every time I want to install a package I get a search for
>
>   updating list of substitutes 
>
> being on a slow internet line this sucks (not everyone has fast
> internet! Think outside the US/Europe where internet is often still
> metered on mobile lines), besides installing the same software often
> install a host of new versions of dependencies. I don't like the
> system changing under me - that is not a reproducible setup.
>
> Q1: Do we retain older builds of binaries for some time for download?

Yes, but the amount of time is unspecified.

On hydra.gnu.org it can be quite long in practice, so that would call in
favor of increasing the default TTL in ‘guix substitute’.

In the longer run, we need servers to advertise their TTL (someone
running ‘guix publish’ may have a different TTL.)

> Q2: Can we switch off updating list of substitutes? A command line
>     switch would do. '--no-update-supstitutes'

No.

> Q3: Would it be possible to version the list of substitutes and use
>     that for (re)deployment? That way I can truely regenerate an existing
>     system.

You can always regenerate an existing system.  The list of substitutes
reflects what’s currently available on hydra.gnu.org.  If some
substitutes vanish, Guix automatically falls back to building from
source.

HTH,
Ludo’.

Re: updating list of substitutes

[Pjotr Prins]

On Tue, Apr 21, 2015 at 10:22:48AM +0200, Ludovic Courtès wrote:
> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
> 
> > Pretty much every time I want to install a package I get a search for
> >
> >   updating list of substitutes 
> >
> > being on a slow internet line this sucks (not everyone has fast
> > internet! Think outside the US/Europe where internet is often still
> > metered on mobile lines), besides installing the same software often
> > install a host of new versions of dependencies. I don't like the
> > system changing under me - that is not a reproducible setup.
> >
> > Q1: Do we retain older builds of binaries for some time for download?
> 
> Yes, but the amount of time is unspecified.
> 
> On hydra.gnu.org it can be quite long in practice, so that would call in
> favor of increasing the default TTL in ???guix substitute???.
> 
> In the longer run, we need servers to advertise their TTL (someone
> running ???guix publish??? may have a different TTL.)
> 
> > Q2: Can we switch off updating list of substitutes? A command line
> >     switch would do. '--no-update-supstitutes'
> 
> No.

Can we have that option?

> > Q3: Would it be possible to version the list of substitutes and use
> >     that for (re)deployment? That way I can truely regenerate an existing
> >     system.
> 
> You can always regenerate an existing system.  The list of substitutes
> reflects what???s currently available on hydra.gnu.org.  If some
> substitutes vanish, Guix automatically falls back to building from
> source.

How do I recreate the exact same system from Hydra? Even now if I
install the exact same Ruby-2.2.1 it will install different packages
compared to yesterday.

We could solve it by giving the list of substitutes a HASH and able to
select that and install.

Pj.

Re: updating list of substitutes

[Andreas Enge]

On Tue, Apr 21, 2015 at 10:40:28AM +0200, Pjotr Prins wrote:
> How do I recreate the exact same system from Hydra? Even now if I
> install the exact same Ruby-2.2.1 it will install different packages
> compared to yesterday.

I am lost here. If you use a fixed release or git commit of guix, ruby-2.2.1
should be the same yesterday and today.

The only thing that could happen, if I understand things correctly, is that
with non-deterministic builds and assuming that ruby-2.2.1 has been garbage
collected and rebuilt on hydra, we would have a new store item on hydra
with the same directory name (including the hash), but different content.
However, if your machine has kept the old package, it should not be
redownloaded, as it is the hash in the directory name that counts and not
some hash over its contents. The solution here would be deterministic builds.

As I understand Ludovic's answer, the database cannot be regenerated because
the store may contain corrupted items from a failed and aborted build. Adding
a package to the database in a last step appears to ensure atomicity of the
operation.

Andreas

Re: updating list of substitutes

[Pjotr Prins]

On Tue, Apr 21, 2015 at 11:19:58AM +0200, Andreas Enge wrote:
> On Tue, Apr 21, 2015 at 10:40:28AM +0200, Pjotr Prins wrote:
> > How do I recreate the exact same system from Hydra? Even now if I
> > install the exact same Ruby-2.2.1 it will install different packages
> > compared to yesterday.
> 
> I am lost here. If you use a fixed release or git commit of guix, ruby-2.2.1
> should be the same yesterday and today.


ls /gnu/store/*ruby-2.2.1*
  /gnu/store/gy1dnlh6qhwd40admi3b1mr4r9cn8bww-ruby-2.2.1

ls /var/guix/profiles/per-user/wrk/guix-profile-2-link/bin/ruby
  /var/guix/profiles/per-user/wrk/guix-profile-2-link/bin/ruby -> /gnu/store/gy1dnlh6qhwd40admi3b1mr4r9cn8bww-ruby-2.2.1/bin/ruby

A few days later I install ruby-1.8.7 followed by

guix package -i ruby-2.2.1

The following package will be upgraded:
   ruby 1.8.7-p374 -> 2.2.1     /gnu/store/z8kf6hgln4a7xf68pdnlibl3vcg5rl15-ruby-2.2.1

The following derivations will be built:
   /gnu/store/7k8nsgpvaafljk1wcnpiq3sm0vns64ck-profile.drv
   /gnu/store/b8pasb4kj50x696bln6jpq4myhzkbrg0-ca-certificate-bundle.drv
   /gnu/store/5hnj1f5i41a6vp4xshhrzqcf27vaf0y7-info-dir.drv
The following files will be downloaded:
   /gnu/store/z8kf6hgln4a7xf68pdnlibl3vcg5rl15-ruby-2.2.1
   /gnu/store/xgfynxf3nscq41n27mnd1lf1hb21fc7w-openssl-1.0.2

So openssl has changed the install. That is not reproducible installs
in my book.

> The only thing that could happen, if I understand things correctly, is that
> with non-deterministic builds and assuming that ruby-2.2.1 has been garbage
> collected and rebuilt on hydra, we would have a new store item on hydra
> with the same directory name (including the hash), but different content.
> However, if your machine has kept the old package, it should not be
> redownloaded, as it is the hash in the directory name that counts and not
> some hash over its contents. The solution here would be deterministic builds.
> 
> As I understand Ludovic's answer, the database cannot be regenerated because
> the store may contain corrupted items from a failed and aborted build. Adding
> a package to the database in a last step appears to ensure atomicity of the
> operation.

Yes. In that case, we should make sure that we know an atomic build
has been completed - also outside the DB.

I am not saying what guix does is wrong. I am saying there are
instances where you WANT to rebuild the DB.

Pj.

Re: updating list of substitutes

[Andreas Enge]

On Tue, Apr 21, 2015 at 12:02:16PM +0200, Pjotr Prins wrote:
> ls /var/guix/profiles/per-user/wrk/guix-profile-2-link/bin/ruby
>   /var/guix/profiles/per-user/wrk/guix-profile-2-link/bin/ruby -> /gnu/store/gy1dnlh6qhwd40admi3b1mr4r9cn8bww-ruby-2.2.1/bin/ruby
> 
> A few days later I install ruby-1.8.7 followed by
> guix package -i ruby-2.2.1
> The following package will be upgraded:
>    ruby 1.8.7-p374 -> 2.2.1     /gnu/store/z8kf6hgln4a7xf68pdnlibl3vcg5rl15-ruby-2.2.1

But I suppose that in between, you also did a "git pull; make install" or
"guix pull"? Then it is clear that if you have a different version of guix
installed, it references different packages.

This is not very different from installing different versions of other
distributions, except that with our public git repository, we enable rolling
releases with frequent changes.

Andreas

Re: updating list of substitutes

[Pjotr Prins]

On Tue, Apr 21, 2015 at 02:02:37PM +0200, Andreas Enge wrote:
> On Tue, Apr 21, 2015 at 12:02:16PM +0200, Pjotr Prins wrote:
> > ls /var/guix/profiles/per-user/wrk/guix-profile-2-link/bin/ruby
> >   /var/guix/profiles/per-user/wrk/guix-profile-2-link/bin/ruby -> /gnu/store/gy1dnlh6qhwd40admi3b1mr4r9cn8bww-ruby-2.2.1/bin/ruby
> > 
> > A few days later I install ruby-1.8.7 followed by
> > guix package -i ruby-2.2.1
> > The following package will be upgraded:
> >    ruby 1.8.7-p374 -> 2.2.1     /gnu/store/z8kf6hgln4a7xf68pdnlibl3vcg5rl15-ruby-2.2.1
> 
> But I suppose that in between, you also did a "git pull; make install" or
> "guix pull"? Then it is clear that if you have a different version of guix
> installed, it references different packages.

I don't think so, but I am not 100% sure I did not do a guix pull in
between. I'll show it if it happens again.

> This is not very different from installing different versions of other
> distributions, except that with our public git repository, we enable rolling
> releases with frequent changes.

Which is great. But we need reproducibility too which is tied with a
release that gets updated by pull. Are the releases visibly numbered?
Can we pull a release?

That means updating the substitute list is independent of the
dependencies.  Why do we download it almost every time?

Pj.

Re: updating list of substitutes

[Pjotr Prins]

I wish to thank you guys for bearing with my dumb-ass questions. I
think Guix is totally awesome. I have been tracking Nix for a long
time and Guix makes me happy where Nix did not.

Please keep on answering me. First because I am learning and second
because we are building a record on this ML for others to see. Your
responses have been very valuable, including Ludo creating the
tar-ball install and Ricardo's mail on the database layout.

You should know I have at least three hats, as system administrator,
as software developer and as scientist - these roles have conflicting
goals and demands which is why I can discuss direct state
modifications in the store on one end and ask for reproducible
software on the other ;).

What I have learned is that:

1. We now have a tar-ball install (awesome!)
2. The DB is the authority
3. We can't reconstruct the DB from the store 
4. Every release fixates the dependencies so if we know the exact
   release we can recreate the same dependencies 
5. We reload the list of substitutes after a fixed time

Correct?

One more below on (5):

On Tue, Apr 21, 2015 at 10:40:28AM +0200, Pjotr Prins wrote:
> > > Q1: Do we retain older builds of binaries for some time for download?
> > 
> > Yes, but the amount of time is unspecified.
> > 
> > On hydra.gnu.org it can be quite long in practice, so that would call in
> > favor of increasing the default TTL in ???guix substitute???.
> > 
> > In the longer run, we need servers to advertise their TTL (someone
> > running ???guix publish??? may have a different TTL.)
> > 
> > > Q2: Can we switch off updating list of substitutes? A command line
> > >     switch would do. '--no-update-supstitutes'
> > 
> > No.

Let me rephrase. Can we have a more lazy approach towards fetching
substitutes? Rather than a fixed TTL we could fetch the latest list on
the first failed substitute. I expect, in practise that would save a
lot of downloads which turn out to be very slow, even on decent
internet connections. It also saves load on the build server.

It does mean the initial binary/build overview on package install can
be wrong but since we retain binaries for a long time (in practise) it
would be more likely to change between releases anyway. A simple
message would do:

  INFO: failed download of substitute XXX (you may want to pull the latest release)
  INFO: substitute list downloading...
  INFO: updated substitute list. The following packages will now be built:
    1. etc etc.

I think this is actually safer and fairer than a TTL.

Pj.

Re: updating list of substitutes

[Andreas Enge]

On Wed, Apr 22, 2015 at 01:46:35PM +0200, Pjotr Prins wrote:
> 5. We reload the list of substitutes after a fixed time
> Let me rephrase. Can we have a more lazy approach towards fetching
> substitutes? Rather than a fixed TTL we could fetch the latest list on
> the first failed substitute.

A while ago, the list of substitutes was downloaded even when removing
packages from a profile, which Ludovic fixed if I remember correctly.

But even when installing, one may not need to download anything if the
package is already in the store. So I think the following would be good:
  Determine the list of packages to be installed in a profile with "guix
  package" or to be built with "guix build". If they are all in the store,
  fine, otherwise, check the age of the list of substitutes and if it is
  to old, download it again.

Andreas

Re: updating list of substitutes

[Pjotr Prins]

On Wed, Apr 22, 2015 at 02:24:21PM +0200, Andreas Enge wrote:
> On Wed, Apr 22, 2015 at 01:46:35PM +0200, Pjotr Prins wrote:
> > 5. We reload the list of substitutes after a fixed time
> > Let me rephrase. Can we have a more lazy approach towards fetching
> > substitutes? Rather than a fixed TTL we could fetch the latest list on
> > the first failed substitute.
> 
> A while ago, the list of substitutes was downloaded even when removing
> packages from a profile, which Ludovic fixed if I remember correctly.
> 
> But even when installing, one may not need to download anything if the
> package is already in the store. So I think the following would be good:
>   Determine the list of packages to be installed in a profile with "guix
>   package" or to be built with "guix build". If they are all in the store,
>   fine, otherwise, check the age of the list of substitutes and if it is
>   to old, download it again.

I have the impression this is what is happening now. Why have a TTL in
the first place? 

Pj.
-- 

Re: updating list of substitutes

[Taylan Ulrich Bayırlı/Kammer]

Pjotr Prins <pjotr.public12@thebird.nl> writes:

>> But even when installing, one may not need to download anything if the
>> package is already in the store. So I think the following would be good:
>>   Determine the list of packages to be installed in a profile with "guix
>>   package" or to be built with "guix build". If they are all in the store,
>>   fine, otherwise, check the age of the list of substitutes and if it is
>>   to old, download it again.
>
> I have the impression this is what is happening now. Why have a TTL in
> the first place? 

I think what Andreas describes differs from current behavior in that it
doesn't download the new substitute list (even if the TTL has expired)
if there's nothing to download.

When there is something to download, it's probably preferable to still
check the TTL, so one doesn't constantly re-download the substitute list
when some substitute simply doesn't exist on Hydra.

Taylan

Re: updating list of substitutes

[Mark H Weaver]

Pjotr Prins <pjotr.public12@thebird.nl> writes:

> On Tue, Apr 21, 2015 at 02:02:37PM +0200, Andreas Enge wrote:
>> On Tue, Apr 21, 2015 at 12:02:16PM +0200, Pjotr Prins wrote:
>> > ls /var/guix/profiles/per-user/wrk/guix-profile-2-link/bin/ruby
>> >   /var/guix/profiles/per-user/wrk/guix-profile-2-link/bin/ruby -> /gnu/store/gy1dnlh6qhwd40admi3b1mr4r9cn8bww-ruby-2.2.1/bin/ruby
>> > 
>> > A few days later I install ruby-1.8.7 followed by
>> > guix package -i ruby-2.2.1
>> > The following package will be upgraded:
>> >    ruby 1.8.7-p374 -> 2.2.1     /gnu/store/z8kf6hgln4a7xf68pdnlibl3vcg5rl15-ruby-2.2.1
>> 
>> But I suppose that in between, you also did a "git pull; make install" or
>> "guix pull"? Then it is clear that if you have a different version of guix
>> installed, it references different packages.
>
> I don't think so, but I am not 100% sure I did not do a guix pull in
> between. I'll show it if it happens again.

The different hash indicates that you had done a "guix pull" in between,
or else changed some package that 'ruby' depends on.

To reproduce the exact versions of packages that you had before, then
you need to be using the same version of 'guix' that you used before.

"guix pull" supports only one mode of operation: get the latest version
from the git repo and build it.  It puts that latest version into
$HOME/.config/guix/latest.  You may backup and restore that directory if
you wish.

If you want more flexibility (as I do), you will avoid "guix pull"
entirely and instead build and use guix from a git checkout using
'pre-inst-env'.  Then you can add your own tags, or simply record the
git commit hashes, and then reproduce them at any time by checking out
the desired version of guix and using that one.

Using 'git' also allows you to keep a private branch with your own
arbitrary modifications, and to periodically either merge from our
upstream, cherry-pick from us, or rebase your branch onto our upstream,
whatever you prefer.

      Mark

Re: updating list of substitutes

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> On Wed, Apr 22, 2015 at 01:46:35PM +0200, Pjotr Prins wrote:
>> 5. We reload the list of substitutes after a fixed time
>> Let me rephrase. Can we have a more lazy approach towards fetching
>> substitutes? Rather than a fixed TTL we could fetch the latest list on
>> the first failed substitute.
>
> A while ago, the list of substitutes was downloaded even when removing
> packages from a profile, which Ludovic fixed if I remember correctly.

Not really, see below.

> But even when installing, one may not need to download anything if the
> package is already in the store. So I think the following would be good:
>   Determine the list of packages to be installed in a profile with "guix
>   package" or to be built with "guix build". If they are all in the store,
>   fine, otherwise, check the age of the list of substitutes and if it is
>   to old, download it again.

This is correct, except it’s not just about packages explicitly
specified by the user: anytime the daemon is asked to build something,
it calls out to ‘guix substitute’, which tells it whether a substitute
is available for that thing.

‘guix substitute’ has a cache of what’s available, which it refreshes
when it gets too old (hence the “updating substitute list” message.)

‘guix package -r’ actually asks the daemon to build/download (if needed)
the Guile package that is used to create the new profile.  If that Guile
package is missing, then it is built/downloaded.

HTH,
Ludo’.

Re: updating list of substitutes

[Pjotr Prins]

I am getting really tired of looking at

  substitute-binary: updating list of substitutes from 'http://hydra.gnu.org'...

> What I have learned is that:
> 
> 1. We now have a tar-ball install (awesome!)
> 2. The DB is the authority
> 3. We can't reconstruct the DB from the store 
> 4. Every release fixates the dependencies so if we know the exact
>    release we can recreate the same dependencies 
> 5. We reload the list of substitutes after a fixed time
> 
> Correct?
> 
> One more below on (5):
> 
> On Tue, Apr 21, 2015 at 10:40:28AM +0200, Pjotr Prins wrote:
> > > > Q1: Do we retain older builds of binaries for some time for download?
> > > 
> > > Yes, but the amount of time is unspecified.
> > > 
> > > On hydra.gnu.org it can be quite long in practice, so that would call in
> > > favor of increasing the default TTL in ???guix substitute???.
> > > 
> > > In the longer run, we need servers to advertise their TTL (someone
> > > running ???guix publish??? may have a different TTL.)
> > > 
> > > > Q2: Can we switch off updating list of substitutes? A command line
> > > >     switch would do. '--no-update-supstitutes'
> > > 
> > > No.
> 
> Let me rephrase. Can we have a more lazy approach towards fetching
> substitutes? Rather than a fixed TTL we could fetch the latest list on
> the first failed substitute. I expect, in practise that would save a
> lot of downloads which turn out to be very slow, even on decent
> internet connections. It also saves load on the build server.
> 
> It does mean the initial binary/build overview on package install can
> be wrong but since we retain binaries for a long time (in practise) it
> would be more likely to change between releases anyway. A simple
> message would do:
> 
>   INFO: failed download of substitute XXX (you may want to pull the latest release)
>   INFO: substitute list downloading...
>   INFO: updated substitute list. The following packages will now be built:
>     1. etc etc.
> 
> I think this is actually safer and fairer than a TTL.
> 
> Pj.
> 
> 

-- 

Re: updating list of substitutes

[Ludovic Courtès]

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> I am getting really tired of looking at
>
>   substitute-binary: updating list of substitutes from 'http://hydra.gnu.org'...

Thanks for sharing your tiredness!  ;-)

This message comes from a pre-0.8.2 Guix though.  Now, I’m not claiming
it’s faster, but at least there’s a progress report that gets displayed.

Our poor hydra.gnu.org front-end has been overloaded for way too long.
We’ve been looking at options to fix that, but that has not materialized
yet.  If anyone reading this can help with hardware, please let us know!

  http://www.gnu.org/software/guix/donate

Thanks,
Ludo’.

Re: updating list of substitutes

[Pjotr Prins]

The list of substitutes gets downloaded every time I do someting:

  substitute: updating list of substitutes from 'http://hydra.gnu.org'...   4.6%

and it is slow. Am I doing something wrong?

It appears to me that if the list does not change it should not be
downloaded. Would a simple check on a SHA value not do?

Pj.

Re: updating list of substitutes

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 810 bytes --]

On Sun, 11 Oct 2015 09:46:54 +0200
Pjotr Prins <pjotr.public12@thebird.nl> wrote:

> The list of substitutes gets downloaded every time I do someting:
> 
>   substitute: updating list of substitutes from 'http://hydra.gnu.org'...
> 4.6%
> 
> and it is slow. Am I doing something wrong?
> 
> It appears to me that if the list does not change it should not be
> downloaded. Would a simple check on a SHA value not do?
> 
> Pj.
> 

Without having looked at the code, won't this be even slower when we have
guix archive set up to use gnunet and we have to check even more locations?

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: updating list of substitutes

[Ludovic Courtès]

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> The list of substitutes gets downloaded every time I do someting:
>
>   substitute: updating list of substitutes from 'http://hydra.gnu.org'...   4.6%
>
> and it is slow. Am I doing something wrong?

No!  Slowness is a longstanding issue of hydra.gnu.org, a poor little
VM.  I hope we can address it soon!  See
<https://lists.gnu.org/archive/html/guix-devel/2015-10/msg00172.html>.

> It appears to me that if the list does not change it should not be
> downloaded.

It’s not downloaded “every time.”

When building a package FOO, Guix looks for substitutes for FOO and its
prerequisites (those not already available locally.)  It maintains in
/var/guix/substitute/cache a cache of those lookups.

Positive caches (for substitutes that are available) expire after 36h;
negative caches (substitutes that are missing) expire after 3h.

See (guix scripts substitute) for details; look for ‘ttl’.

Ludo’.

Re: updating list of substitutes

[Pjotr Prins]

On Sun, Oct 11, 2015 at 08:39:32PM +0200, Ludovic Courtès wrote:
> No!  Slowness is a longstanding issue of hydra.gnu.org, a poor little
> VM.  I hope we can address it soon!  See
> <https://lists.gnu.org/archive/html/guix-devel/2015-10/msg00172.html>.

Looking forward to that :)

> > It appears to me that if the list does not change it should not be
> > downloaded.
> 
> It’s not downloaded “every time.”
> 
> When building a package FOO, Guix looks for substitutes for FOO and its
> prerequisites (those not already available locally.)  It maintains in
> /var/guix/substitute/cache a cache of those lookups.

> Positive caches (for substitutes that are available) expire after 36h;
> negative caches (substitutes that are missing) expire after 3h.

The weird thing is that most times I install a new package it does a
lookup. Only rarely it does not. According to the TTL it should then
check every 3hrs at most? Somehow I don't understand why we need to
download the substitute list every time I install a new package. Not
even Debian does that ;) I would think that the list of (prebuild,
right?) substitutes is only updated when some build is triggered.
Could be there are builds triggered while I am installing software
forcing a new list every time. Is that it?

I understand we have a list of prebuilt packages and that the list can
change. But maybe I am too simplistic in my assumptions. I'll look
into that code tomorrow.

Pj.

Re: updating list of substitutes

[Mark H Weaver]

Pjotr Prins <pjotr.public12@thebird.nl> writes:

> On Sun, Oct 11, 2015 at 08:39:32PM +0200, Ludovic Courtès wrote:
>> > It appears to me that if the list does not change it should not be
>> > downloaded.
>> 
>> It’s not downloaded “every time.”
>> 
>> When building a package FOO, Guix looks for substitutes for FOO and its
>> prerequisites (those not already available locally.)  It maintains in
>> /var/guix/substitute/cache a cache of those lookups.
>
>> Positive caches (for substitutes that are available) expire after 36h;
>> negative caches (substitutes that are missing) expire after 3h.
>
> The weird thing is that most times I install a new package it does a
> lookup. Only rarely it does not. According to the TTL it should then
> check every 3hrs at most? Somehow I don't understand why we need to
> download the substitute list every time I install a new package.

The phrase "the substitute list" suggests a single, complete list of all
available substitutes, but there is no such list.  Instead, quoting
Ludovic above:

  "When building a package FOO, Guix looks for substitutes for FOO
   and its prerequisites (those not already available locally.)  It
   maintains in /var/guix/substitute/cache a cache of those lookups."

So, if you build package BAR immediately after building FOO, a different
set of substitutes is queried, and typically that involves more lookups
(unless FOO runtime-depends on BAR).

Does that make sense?

    Regards,
      Mark

Re: updating list of substitutes

[Pjotr Prins]

On Mon, Oct 12, 2015 at 01:15:01AM -0400, Mark H Weaver wrote:
> The phrase "the substitute list" suggests a single, complete list of all
> available substitutes, but there is no such list.  Instead, quoting
> Ludovic above:
> 
>   "When building a package FOO, Guix looks for substitutes for FOO
>    and its prerequisites (those not already available locally.)  It
>    maintains in /var/guix/substitute/cache a cache of those lookups."
> 
> So, if you build package BAR immediately after building FOO, a different
> set of substitutes is queried, and typically that involves more lookups
> (unless FOO runtime-depends on BAR).
> 
> Does that make sense?

Right. So, why don't we have one list for every build? That would save
connecting to the one single server every time and be less fragile.

Pj.

Re: updating list of substitutes

[Mark H Weaver]

Pjotr Prins <pjotr.public12@thebird.nl> writes:

> On Mon, Oct 12, 2015 at 01:15:01AM -0400, Mark H Weaver wrote:
>> The phrase "the substitute list" suggests a single, complete list of all
>> available substitutes, but there is no such list.  Instead, quoting
>> Ludovic above:
>> 
>>   "When building a package FOO, Guix looks for substitutes for FOO
>>    and its prerequisites (those not already available locally.)  It
>>    maintains in /var/guix/substitute/cache a cache of those lookups."
>> 
>> So, if you build package BAR immediately after building FOO, a different
>> set of substitutes is queried, and typically that involves more lookups
>> (unless FOO runtime-depends on BAR).
>> 
>> Does that make sense?
>
> Right. So, why don't we have one list for every build? That would save
> connecting to the one single server every time and be less fragile.

I'm not sure how to make this work well in practice.  One issue is that
the set of available substitutes changes very frequently.  Take a look
at the "Finished at" field of <http://hydra.gnu.org/all> to get an idea.
When Hydra is busy, the set of available substitutes changes every few
minutes, and sometimes several times each minute.  How would we deal
with that?

There are other difficult problems as well, e.g. that the set of
available substitutes includes builds for multiple branches of our git
repo, multiple architectures, and multiple "evaluations" corresponding
to different commits on any given branch of our git repo.

In summary, the full set of available substitutes is typically quite
large and changes frequently, so this approach would entail a lot of
wasted network bandwidth (and load on hydra) to maintain the complete
list of substitutes on every client machine, although only a small
fraction of these would be of interest to any given user.

Does that make sense?

       Mark

Re: updating list of substitutes

[Ludovic Courtès]

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> On Mon, Oct 12, 2015 at 01:15:01AM -0400, Mark H Weaver wrote:
>> The phrase "the substitute list" suggests a single, complete list of all
>> available substitutes, but there is no such list.  Instead, quoting
>> Ludovic above:
>> 
>>   "When building a package FOO, Guix looks for substitutes for FOO
>>    and its prerequisites (those not already available locally.)  It
>>    maintains in /var/guix/substitute/cache a cache of those lookups."
>> 
>> So, if you build package BAR immediately after building FOO, a different
>> set of substitutes is queried, and typically that involves more lookups
>> (unless FOO runtime-depends on BAR).
>> 
>> Does that make sense?
>
> Right. So, why don't we have one list for every build? That would save
> connecting to the one single server every time and be less fragile.

Again, it’s not “every time,” as we’ve seen.  :-)

I’m not sure what you have in mind with “one list for every build.”

Quoth
<http://www.gnu.org/software/guix/manual/html_node/Substitutes.html>:

  Substitutes can be anything resulting from a derivation build (see
  Derivations).

Each derivation has a number of dependencies, and for each dependency
that is not already available locally, we want to know whether
substitutes are available.  This is the list we’re talking about.  It
depends on what’s available in the local store, the state of the local
substitute cache, etc.

Having said that, I’m all for fewer network accesses.  When we have a
new front-end with more disk space, we can hopefully have it retain
build products for longer, and thus increase the TTL.

In the meantime, anyone can run ‘guix publish’ and pass the URL via
‘--substitute-urls’.

HTH!

Ludo’.

Re: updating list of substitutes

[Pjotr Prins]

On Mon, Oct 12, 2015 at 12:31:21PM -0400, Mark H Weaver wrote:
> In summary, the full set of available substitutes is typically quite
> large and changes frequently, so this approach would entail a lot of
> wasted network bandwidth (and load on hydra) to maintain the complete
> list of substitutes on every client machine, although only a small
> fraction of these would be of interest to any given user.
> 
> Does that make sense?

Yes, I had not realised the list changes so often. The technical
solution would be to ship the incremental diffs every time. But I
realise this may be harder than it looks.

I guess we'll have to advertise that people use their own substitue
server as a cache, like Ludo suggests. At least that will scale.

Pj.
-- 

Re: updating list of substitutes

[Pjotr Prins]

On Mon, Oct 12, 2015 at 07:03:51PM +0200, Ludovic Courtès wrote:
> Having said that, I’m all for fewer network accesses.  When we have a
> new front-end with more disk space, we can hopefully have it retain
> build products for longer, and thus increase the TTL.

I just checked the substitute cache in /var/guix/substitute/cache and
these files are small (3K only!). So, all that pain to fetch a 3K file
:). I thought it was network related, but it is generating this cache
file that is expensive.  

So, yes, a fast server may help greatly. And are you sure the
server is able to serve concurrent requests? Maybe the SQL database
is blocking somehow?

> In the meantime, anyone can run ‘guix publish’ and pass the URL via
> ‘--substitute-urls’.

:)

Pj.

Re: updating list of substitutes

[Andreas Enge]

On Tue, Oct 13, 2015 at 02:11:40PM +0200, Pjotr Prins wrote:
> I just checked the substitute cache in /var/guix/substitute/cache and
> these files are small (3K only!). So, all that pain to fetch a 3K file
> :). I thought it was network related, but it is generating this cache
> file that is expensive.  

Well, there is one file per package. Assuming that one would limit the cache
to only one architecture, that would still end up with 2500*3KB=7,5MB,
a somewhat larger file.

Still, my impression (also from using substitutes) is that _access_ to hydra
is slow; once the download starts, even larger files arrive quite quickly
(since nginx caching is enabled). So it may be much more efficient to
download one bigger file once every three hours, than a small file for more
or less each transaction. But this may be complicated to realise; what should
the file be? Of course I am mainly thinking of one branch and its current list
of packages - but which one? And nothing prevents people from remaining on
some old git commit, and hydra will happily serve any package that is still
available in its store. So the implementation would need to change from a
reply to a simple request if some packages are available, to some logic on
the hydra side guessing the list of packages a client might be interested in.

So maybe just switching to a more powerful and reactive hydra machine would
be the better way to solve this problem!

Andreas

PS: And we should stop "updating list of substitutes from 'http://hydra.gnu.org'"
    upon the command "guix package -r upower". Which list is requested during
    this call?

Re: updating list of substitutes

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> Still, my impression (also from using substitutes) is that _access_ to hydra
> is slow;

Yes.  If you haven’t tried yet, compare it with the speed at which ‘guix
publish’ serves those .narinfo URLs on a “real” machine; it’s an order
of magnitude faster, even though hydra.gnu.org has nginx running in the
front and serving things from the cache.

Ludo’.

Re: updating list of substitutes

[Pjotr Prins]

Hi Ludo,

Hydra needs to get kicked ;). Download substitutes is slow. Too slow. 

How much would it cost to get some decent hardware for the substitute
server? Maybe we can do a dedicated round of funding on this list and
go round with a cap at FOSDEM? I am happy to put in $100 if it solves
the issue. Is there no one here who can provide a decent server?

Pj.

On Tue, Oct 13, 2015 at 04:35:45PM +0200, Ludovic Courtès wrote:
> Andreas Enge <andreas@enge.fr> skribis:
> 
> > Still, my impression (also from using substitutes) is that _access_ to hydra
> > is slow;
> 
> Yes.  If you haven’t tried yet, compare it with the speed at which ‘guix
> publish’ serves those .narinfo URLs on a “real” machine; it’s an order
> of magnitude faster, even though hydra.gnu.org has nginx running in the
> front and serving things from the cache.
> 
> Ludo’.
> 

-- 

Re: updating list of substitutes

[Thompson, David]

On Wed, Nov 18, 2015 at 11:15 AM, Pjotr Prins <pjotr.public12@thebird.nl> wrote:
> Hi Ludo,
>
> Hydra needs to get kicked ;). Download substitutes is slow. Too slow.
>
> How much would it cost to get some decent hardware for the substitute
> server? Maybe we can do a dedicated round of funding on this list and
> go round with a cap at FOSDEM? I am happy to put in $100 if it solves
> the issue. Is there no one here who can provide a decent server?

If someone can source the hardware and figure out colocation details,
I will happily throw in some money towards this cause as well.

- Dave

Re: updating list of substitutes

[Alex Sassmannshausen]

Thompson, David writes:

> On Wed, Nov 18, 2015 at 11:15 AM, Pjotr Prins <pjotr.public12@thebird.nl> wrote:
>> Hi Ludo,
>>
>> Hydra needs to get kicked ;). Download substitutes is slow. Too slow.
>>
>> How much would it cost to get some decent hardware for the substitute
>> server? Maybe we can do a dedicated round of funding on this list and
>> go round with a cap at FOSDEM? I am happy to put in $100 if it solves
>> the issue. Is there no one here who can provide a decent server?
>
> If someone can source the hardware and figure out colocation details,
> I will happily throw in some money towards this cause as well.
>
> - Dave

+1

Alex

Funding the build farm

[Ludovic Courtès]

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> How much would it cost to get some decent hardware for the substitute
> server? Maybe we can do a dedicated round of funding on this list and
> go round with a cap at FOSDEM? I am happy to put in $100 if it solves
> the issue. Is there no one here who can provide a decent server?

I know this is ridiculous, but we need a way to collect money.  The FSF
is overworked so the Free Software Fund¹ still isn’t set up for us, and
I’m even concerned about its viability at this point.

So, what can we do?  We could go to another fiscal sponsor for free
software projects in the US, either Conservancy or SPI, Inc., but they
may redirect us to the FSF because we’re GNU.

We could look for something similar in Europe but I don’t know of any.

Or we could just create a bank account (I want to avoid PayPal) and
possibly a non-profit and do it ourselves.  But this has drawbacks, see
<https://lwn.net/Articles/548558/> for instance.

Concrete suggestions are very much welcome!

Then we’ll need to figure out hosting, but I’m a little less concerned.

Ludo’.

¹ https://lists.gnu.org/archive/html/guix-devel/2015-10/msg00172.html

RE: Funding the build farm

[Cook, Malcolm]

Hi,

I don't know what kind of $ we're talking about, nor quite how to interpret all those up and down thumbs at lwn.net, but (once I've finally deployed guix) I'll certainly go to bat with my institute to free up some funding. 

Back to Pjotr's Q: How much would it cost to get some decent hardware for the substitute server?

~Malcolm

> -----Original Message-----
 > From: guix-devel-bounces+mec=stowers.org@gnu.org [mailto:guix-devel-
 > bounces+mec=stowers.org@gnu.org] On Behalf Of Ludovic Courtès
 > Sent: Wednesday, November 18, 2015 12:30 PM
 > To: Pjotr Prins <pjotr.public12@thebird.nl>
 > Cc: guix-devel@gnu.org
 > Subject: Funding the build farm
 > 
 > Pjotr Prins <pjotr.public12@thebird.nl> skribis:
 > 
 > > How much would it cost to get some decent hardware for the substitute
 > > server? Maybe we can do a dedicated round of funding on this list and
 > > go round with a cap at FOSDEM? I am happy to put in $100 if it solves
 > > the issue. Is there no one here who can provide a decent server?
 > 
 > I know this is ridiculous, but we need a way to collect money.  The FSF
 > is overworked so the Free Software Fund¹ still isn’t set up for us, and
 > I’m even concerned about its viability at this point.
 > 
 > So, what can we do?  We could go to another fiscal sponsor for free
 > software projects in the US, either Conservancy or SPI, Inc., but they
 > may redirect us to the FSF because we’re GNU.
 > 
 > We could look for something similar in Europe but I don’t know of any.
 > 
 > Or we could just create a bank account (I want to avoid PayPal) and
 > possibly a non-profit and do it ourselves.  But this has drawbacks, see
 > <https://lwn.net/Articles/548558/> for instance.
 > 
 > Concrete suggestions are very much welcome!
 > 
 > Then we’ll need to figure out hosting, but I’m a little less concerned.
 > 
 > Ludo’.
 > 
 > ¹ https://lists.gnu.org/archive/html/guix-devel/2015-10/msg00172.html

Re: Funding the build farm

[Pjotr Prins]

We could opt for a more powerful VM perhaps? Ludo, what are we looking
for? cores, RAM and HDD space? Maybe an institute or one of the UUG's
could free up a machine? Does it matter where it sits? What ports
should be open? http and ssh? Is Cloud feasible? How much data gets
transferred these days?

For funding we could use any non-profit, really. As long as the books
are balanced :). Maybe one of the list members has a suggestion.

Pj.

On Wed, Nov 18, 2015 at 06:38:09PM +0000, Cook, Malcolm wrote:
> Hi,
> 
> I don't know what kind of $ we're talking about, nor quite how to interpret all those up and down thumbs at lwn.net, but (once I've finally deployed guix) I'll certainly go to bat with my institute to free up some funding. 
> 
> Back to Pjotr's Q: How much would it cost to get some decent hardware for the substitute server?
> 
> ~Malcolm
> 
> > -----Original Message-----
>  > From: guix-devel-bounces+mec=stowers.org@gnu.org [mailto:guix-devel-
>  > bounces+mec=stowers.org@gnu.org] On Behalf Of Ludovic Courtès
>  > Sent: Wednesday, November 18, 2015 12:30 PM
>  > To: Pjotr Prins <pjotr.public12@thebird.nl>
>  > Cc: guix-devel@gnu.org
>  > Subject: Funding the build farm
>  > 
>  > Pjotr Prins <pjotr.public12@thebird.nl> skribis:
>  > 
>  > > How much would it cost to get some decent hardware for the substitute
>  > > server? Maybe we can do a dedicated round of funding on this list and
>  > > go round with a cap at FOSDEM? I am happy to put in $100 if it solves
>  > > the issue. Is there no one here who can provide a decent server?
>  > 
>  > I know this is ridiculous, but we need a way to collect money.  The FSF
>  > is overworked so the Free Software Fund¹ still isn’t set up for us, and
>  > I’m even concerned about its viability at this point.
>  > 
>  > So, what can we do?  We could go to another fiscal sponsor for free
>  > software projects in the US, either Conservancy or SPI, Inc., but they
>  > may redirect us to the FSF because we’re GNU.
>  > 
>  > We could look for something similar in Europe but I don’t know of any.
>  > 
>  > Or we could just create a bank account (I want to avoid PayPal) and
>  > possibly a non-profit and do it ourselves.  But this has drawbacks, see
>  > <https://lwn.net/Articles/548558/> for instance.
>  > 
>  > Concrete suggestions are very much welcome!
>  > 
>  > Then we’ll need to figure out hosting, but I’m a little less concerned.
>  > 
>  > Ludo’.
>  > 
>  > ¹ https://lists.gnu.org/archive/html/guix-devel/2015-10/msg00172.html
> 

-- 

Re: Funding the build farm

[Ludovic Courtès]

"Cook, Malcolm" <MEC@stowers.org> skribis:

> I don't know what kind of $ we're talking about, nor quite how to interpret all those up and down thumbs at lwn.net, but (once I've finally deployed guix) I'll certainly go to bat with my institute to free up some funding. 

Thanks for your support!

Initially, we’re talking about:

  1. One big server, along the lines of 8 cores, 32G of RAM, and a fast
     4T drive (around 4–5k€ maybe?).

  2. Hosting, which needs to be figured out more precisely, but could be
     in the 0–2k€ range per year depending on what we choose.

That’s the back-of-the-envelope calculation.

What do people think?

Ludo’.

RE: Funding the build farm

[Cook, Malcolm]

OK - good - some numbers - I am not yet ready to make an internal pitch for $$, but hope to be within a month.  I have done this for other open source projects before, not quite to this extent, but think I can free up a one-time injection of $omething....  Definitely willing to try.... Film at 11. 

 > -----Original Message-----
 > From: Ludovic Courtès [mailto:ludo@gnu.org]
 > Sent: Wednesday, November 18, 2015 3:21 PM
 > To: Cook, Malcolm <MEC@stowers.org>
 > Cc: Pjotr Prins <pjotr.public12@thebird.nl>; guix-devel@gnu.org
 > Subject: Re: Funding the build farm
 > 
 > "Cook, Malcolm" <MEC@stowers.org> skribis:
 > 
 > > I don't know what kind of $ we're talking about, nor quite how to interpret
 > all those up and down thumbs at lwn.net, but (once I've finally deployed guix)
 > I'll certainly go to bat with my institute to free up some funding.
 > 
 > Thanks for your support!
 > 
 > Initially, we’re talking about:
 > 
 >   1. One big server, along the lines of 8 cores, 32G of RAM, and a fast
 >      4T drive (around 4–5k€ maybe?).
 > 
 >   2. Hosting, which needs to be figured out more precisely, but could be
 >      in the 0–2k€ range per year depending on what we choose.
 > 
 > That’s the back-of-the-envelope calculation.
 > 
 > What do people think?
 > 
 > Ludo’.

Re: Funding the build farm

[Andreas Enge]

On Wed, Nov 18, 2015 at 07:29:55PM +0100, Ludovic Courtès wrote:
> So, what can we do?  We could go to another fiscal sponsor for free
> software projects in the US, either Conservancy or SPI, Inc., but they
> may redirect us to the FSF because we’re GNU.
> 
> We could look for something similar in Europe but I don’t know of any.
> 
> Or we could just create a bank account (I want to avoid PayPal) and
> possibly a non-profit and do it ourselves.  But this has drawbacks, see
> <https://lwn.net/Articles/548558/> for instance.

Definitely I am in favour of creating a French "association loi 1901".
From what I have heard of diverse clubs I am a member of, this is rather
easy; but so far I have not had the time to look into it.

It may still be interesting to also go via the FSF. This would mean bank
accounts in the euro and the dollar currency spaces, and it might be easier
for US donors to deduce donations from their taxes.

Andreas

Re: Funding the build farm

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> On Wed, Nov 18, 2015 at 07:29:55PM +0100, Ludovic Courtès wrote:
>> So, what can we do?  We could go to another fiscal sponsor for free
>> software projects in the US, either Conservancy or SPI, Inc., but they
>> may redirect us to the FSF because we’re GNU.
>> 
>> We could look for something similar in Europe but I don’t know of any.
>> 
>> Or we could just create a bank account (I want to avoid PayPal) and
>> possibly a non-profit and do it ourselves.  But this has drawbacks, see
>> <https://lwn.net/Articles/548558/> for instance.
>
> Definitely I am in favour of creating a French "association loi 1901".
> From what I have heard of diverse clubs I am a member of, this is rather
> easy; but so far I have not had the time to look into it.

Yeah I think it would be fine to do both, and probably more convenient
for donors.

I’m happy to join a bunch of French people to sit down and do the
paperwork.

Ludo’.

Re: Funding the build farm

[Mathieu Lirzin]

ludo@gnu.org (Ludovic Courtès) writes:

> Andreas Enge <andreas@enge.fr> skribis:
>
>> On Wed, Nov 18, 2015 at 07:29:55PM +0100, Ludovic Courtès wrote:
>>> So, what can we do?  We could go to another fiscal sponsor for free
>>> software projects in the US, either Conservancy or SPI, Inc., but they
>>> may redirect us to the FSF because we’re GNU.
>>> 
>>> We could look for something similar in Europe but I don’t know of any.
>>> 
>>> Or we could just create a bank account (I want to avoid PayPal) and
>>> possibly a non-profit and do it ourselves.  But this has drawbacks, see
>>> <https://lwn.net/Articles/548558/> for instance.
>>
>> Definitely I am in favour of creating a French "association loi 1901".
>> From what I have heard of diverse clubs I am a member of, this is rather
>> easy; but so far I have not had the time to look into it.
>
> Yeah I think it would be fine to do both, and probably more convenient
> for donors.
>
> I’m happy to join a bunch of French people to sit down and do the
> paperwork.

I think the paperwork for a French association would need the
rigorousness of a German.  ;)

--
Mathieu Lirzin

Re: Funding the build farm

[John Darrington]

[-- Attachment #1: Type: text/plain, Size: 1347 bytes --]

On Sun, Nov 22, 2015 at 11:53:06AM +0100, Andreas Enge wrote:
     On Wed, Nov 18, 2015 at 07:29:55PM +0100, Ludovic Court??s wrote:
     > So, what can we do?  We could go to another fiscal sponsor for free
     > software projects in the US, either Conservancy or SPI, Inc., but they
     > may redirect us to the FSF because we???re GNU.

     It may still be interesting to also go via the FSF. This would mean bank
     accounts in the euro and the dollar currency spaces, and it might be easier
     for US donors to deduce donations from their taxes.

The FSF does have a EUR bank account.  However, if past experience is anything
to go by, then :

1.  The FSF will have little interest in collecting funds for specific projects.
    They (with a few very special exceptions) only take donations which go into
    central FSF funds to be dispersed as they think fit.

  AND

2.  The Conservancy (which normally DOES do directed funding) refuses to do so for
    GNU projects, for fear of offending the FSF.


This is a crazy situation, but then they are American ....
     
J'
     

-- 
Avoid eavesdropping.  Send strong encryted email.
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Funding the build farm

[Alex Sassmannshausen]

Heya,

John Darrington writes:

> On Sun, Nov 22, 2015 at 11:53:06AM +0100, Andreas Enge wrote:
> [...]
>
> This is a crazy situation, but then they are American ....

Not sure that has anything to do with it…

Alex

Re: Funding the build farm

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1281 bytes --]

On Mon, 23 Nov 2015 20:38:38 +0100
John Darrington <john@darrington.wattle.id.au> wrote:

> On Sun, Nov 22, 2015 at 11:53:06AM +0100, Andreas Enge wrote:
>      On Wed, Nov 18, 2015 at 07:29:55PM +0100, Ludovic Court??s wrote:
>  [...]  
> 
>      It may still be interesting to also go via the FSF. This would mean bank
>      accounts in the euro and the dollar currency spaces, and it might be easier
>      for US donors to deduce donations from their taxes.
> 
> The FSF does have a EUR bank account.  However, if past experience is anything
> to go by, then :
> 
> 1.  The FSF will have little interest in collecting funds for specific projects.
>     They (with a few very special exceptions) only take donations which go into
>     central FSF funds to be dispersed as they think fit.
> 
>   AND
> 
> 2.  The Conservancy (which normally DOES do directed funding) refuses to do so for
>     GNU projects, for fear of offending the FSF.
> 

What about the FSFE? Would they be more amenable or are they just the
European arm of the FSF?

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Funding the build farm

[John Darrington]

[-- Attachment #1: Type: text/plain, Size: 1827 bytes --]

On Tue, Nov 24, 2015 at 04:54:54PM +0200, Efraim Flashner wrote:
     On Mon, 23 Nov 2015 20:38:38 +0100
     John Darrington <john@darrington.wattle.id.au> wrote:
     
     > On Sun, Nov 22, 2015 at 11:53:06AM +0100, Andreas Enge wrote:
     >      On Wed, Nov 18, 2015 at 07:29:55PM +0100, Ludovic Court??s wrote:
     >  [...]  
     > 
     >      It may still be interesting to also go via the FSF. This would mean bank
     >      accounts in the euro and the dollar currency spaces, and it might be easier
     >      for US donors to deduce donations from their taxes.
     > 
     > The FSF does have a EUR bank account.  However, if past experience is anything
     > to go by, then :
     > 
     > 1.  The FSF will have little interest in collecting funds for specific projects.
     >     They (with a few very special exceptions) only take donations which go into
     >     central FSF funds to be dispersed as they think fit.
     > 
     >   AND
     > 
     > 2.  The Conservancy (which normally DOES do directed funding) refuses to do so for
     >     GNU projects, for fear of offending the FSF.
     > 
     
     What about the FSFE? Would they be more amenable or are they just the
     European arm of the FSF?
     
The FSFE have no association with the FSF except that they maintain an informal 
liaison.

A year or so ago, I suggested to Karsten Gerloff, the FSFE president that they 
start such a programme.   His response was that the FSFE was forbidden by law 
(presumably German law - they are an e.V.) from doing that.

J'



-- 
Avoid eavesdropping.  Send strong encryted email.
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Funding the build farm

[Ricardo Wurmus]

Efraim Flashner <efraim@flashner.co.il> writes:

> On Mon, 23 Nov 2015 20:38:38 +0100
> John Darrington <john@darrington.wattle.id.au> wrote:
>
>> On Sun, Nov 22, 2015 at 11:53:06AM +0100, Andreas Enge wrote:
>>      On Wed, Nov 18, 2015 at 07:29:55PM +0100, Ludovic Court??s wrote:
>>  [...]  
>> 
>>      It may still be interesting to also go via the FSF. This would mean bank
>>      accounts in the euro and the dollar currency spaces, and it might be easier
>>      for US donors to deduce donations from their taxes.
>> 
>> The FSF does have a EUR bank account.  However, if past experience is anything
>> to go by, then :
>> 
>> 1.  The FSF will have little interest in collecting funds for specific projects.
>>     They (with a few very special exceptions) only take donations which go into
>>     central FSF funds to be dispersed as they think fit.
>> 
>>   AND
>> 
>> 2.  The Conservancy (which normally DOES do directed funding) refuses to do so for
>>     GNU projects, for fear of offending the FSF.
>> 
>
> What about the FSFE? Would they be more amenable or are they just the
> European arm of the FSF?

The FSFE is a separate organisation with separate infrastructure and
separate finances.

~~ Ricardo