unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Guix publish pipe error from #{read pipe}#
@ 2017-05-16  6:29 Pjotr Prins
  2017-05-16  6:57 ` Andy Wingo
  2017-05-17  7:10 ` Guix publish pipe error from #{read pipe}# - what nginx.conf to use? Pjotr Prins
  0 siblings, 2 replies; 4+ messages in thread
From: Pjotr Prins @ 2017-05-16  6:29 UTC (permalink / raw)
  To: guix-devel

On a recent guix-publish server I get many of these

  gzip: stdin: invalid compressed data--format violated
    r-annotationdbi-1.36.0
    1.7MiB/s 00:02 | 3.0MiB transferredguix substitute: error: corrupt
  input while restoring
    '/gnu/store/g33ns71m61zs5sn758smmc4lnv6h4c4q-r-annotationdbi-1.36.0/site-library/AnnotationDbi/extdata/HG-U95Av2_probe_tab.gz' from #{read pipe}#

I don't think this is a guix problem per se. I have a hunch the people
changed the firewall which may cause this.  Anyone any other ideas?

Pj.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Guix publish pipe error from #{read pipe}#
  2017-05-16  6:29 Guix publish pipe error from #{read pipe}# Pjotr Prins
@ 2017-05-16  6:57 ` Andy Wingo
  2017-05-17  7:10 ` Guix publish pipe error from #{read pipe}# - what nginx.conf to use? Pjotr Prins
  1 sibling, 0 replies; 4+ messages in thread
From: Andy Wingo @ 2017-05-16  6:57 UTC (permalink / raw)
  To: Pjotr Prins; +Cc: guix-devel

On Tue 16 May 2017 08:29, Pjotr Prins <pjotr.public12@thebird.nl> writes:

> On a recent guix-publish server I get many of these
>
>   gzip: stdin: invalid compressed data--format violated
>     r-annotationdbi-1.36.0
>     1.7MiB/s 00:02 | 3.0MiB transferredguix substitute: error: corrupt
>   input while restoring
>     '/gnu/store/g33ns71m61zs5sn758smmc4lnv6h4c4q-r-annotationdbi-1.36.0/site-library/AnnotationDbi/extdata/HG-U95Av2_probe_tab.gz' from #{read pipe}#
>
> I don't think this is a guix problem per se. I have a hunch the people
> changed the firewall which may cause this.  Anyone any other ideas?

IME this means that the .nar or whatever was truncated by some
intermediate cache.  Like if you wget the file and try to gunzip it, it
will give the same error AFAIU.

Andy

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Guix publish pipe error from #{read pipe}# - what nginx.conf to use?
  2017-05-16  6:29 Guix publish pipe error from #{read pipe}# Pjotr Prins
  2017-05-16  6:57 ` Andy Wingo
@ 2017-05-17  7:10 ` Pjotr Prins
  2017-05-17 12:48   ` Ludovic Courtès
  1 sibling, 1 reply; 4+ messages in thread
From: Pjotr Prins @ 2017-05-17  7:10 UTC (permalink / raw)
  To: Pjotr Prins; +Cc: guix-devel

On Tue, May 16, 2017 at 08:29:17AM +0200, Pjotr Prins wrote:
> On a recent guix-publish server I get many of these
> 
>   gzip: stdin: invalid compressed data--format violated
>     r-annotationdbi-1.36.0
>     1.7MiB/s 00:02 | 3.0MiB transferredguix substitute: error: corrupt
>   input while restoring
>     '/gnu/store/g33ns71m61zs5sn758smmc4lnv6h4c4q-r-annotationdbi-1.36.0/site-library/AnnotationDbi/extdata/HG-U95Av2_probe_tab.gz' from #{read pipe}#
> 
> I don't think this is a guix problem per se. I have a hunch the people
> changed the firewall which may cause this.  Anyone any other ideas?

It is an nginx thing. When I go around nginx guix publish works as
expected. Anyone have an example nginx.conf file for guix publish?
Funny thing is that it used to be fine. I think parallel downloads
is messing it up now.

Now we are doing parallel downloads, would it be an idea to do a round
robin on substitute-urls too? Make it optional. May speed downloads up
nicely.

Pj.
-- 

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Guix publish pipe error from #{read pipe}# - what nginx.conf to use?
  2017-05-17  7:10 ` Guix publish pipe error from #{read pipe}# - what nginx.conf to use? Pjotr Prins
@ 2017-05-17 12:48   ` Ludovic Courtès
  0 siblings, 0 replies; 4+ messages in thread
From: Ludovic Courtès @ 2017-05-17 12:48 UTC (permalink / raw)
  To: Pjotr Prins; +Cc: guix-devel

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> On Tue, May 16, 2017 at 08:29:17AM +0200, Pjotr Prins wrote:
>> On a recent guix-publish server I get many of these
>> 
>>   gzip: stdin: invalid compressed data--format violated
>>     r-annotationdbi-1.36.0
>>     1.7MiB/s 00:02 | 3.0MiB transferredguix substitute: error: corrupt
>>   input while restoring
>>     '/gnu/store/g33ns71m61zs5sn758smmc4lnv6h4c4q-r-annotationdbi-1.36.0/site-library/AnnotationDbi/extdata/HG-U95Av2_probe_tab.gz' from #{read pipe}#
>> 
>> I don't think this is a guix problem per se. I have a hunch the people
>> changed the firewall which may cause this.  Anyone any other ideas?
>
> It is an nginx thing. When I go around nginx guix publish works as
> expected. Anyone have an example nginx.conf file for guix publish?

My guess is that you’re running ‘guix publish’ without --cache.  As a
result, ‘guix publish’ compresses archives on the fly, and thus it is
not able to provide a Content-Length HTTP header.

Consequently, your nginx cache cannot detect truncation.  So if for some
reason (e.g., overloaded machine, ‘guix publish’ restart, etc.), the
connection between nginx and ‘guix publish’ is terminated prematurely,
nginx may end up caching a truncated store.

Also, quoth the manual:

     By default, ‘guix publish’ compresses archives on the fly as it
  serves them.  This “on-the-fly” mode is convenient in that it requires
  no setup and is immediately available.  However, when serving lots of
  clients, we recommend using the ‘--cache’ option, which enables caching
  of the archives before they are sent to clients—see below for details.

HTH!

Ludo’.

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2017-05-17 12:48 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-05-16  6:29 Guix publish pipe error from #{read pipe}# Pjotr Prins
2017-05-16  6:57 ` Andy Wingo
2017-05-17  7:10 ` Guix publish pipe error from #{read pipe}# - what nginx.conf to use? Pjotr Prins
2017-05-17 12:48   ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).