* Guix publish pipe error from #{read pipe}# @ 2017-05-16 6:29 Pjotr Prins 2017-05-16 6:57 ` Andy Wingo 2017-05-17 7:10 ` Guix publish pipe error from #{read pipe}# - what nginx.conf to use? Pjotr Prins 0 siblings, 2 replies; 4+ messages in thread From: Pjotr Prins @ 2017-05-16 6:29 UTC (permalink / raw) To: guix-devel On a recent guix-publish server I get many of these gzip: stdin: invalid compressed data--format violated r-annotationdbi-1.36.0 1.7MiB/s 00:02 | 3.0MiB transferredguix substitute: error: corrupt input while restoring '/gnu/store/g33ns71m61zs5sn758smmc4lnv6h4c4q-r-annotationdbi-1.36.0/site-library/AnnotationDbi/extdata/HG-U95Av2_probe_tab.gz' from #{read pipe}# I don't think this is a guix problem per se. I have a hunch the people changed the firewall which may cause this. Anyone any other ideas? Pj. ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Guix publish pipe error from #{read pipe}# 2017-05-16 6:29 Guix publish pipe error from #{read pipe}# Pjotr Prins @ 2017-05-16 6:57 ` Andy Wingo 2017-05-17 7:10 ` Guix publish pipe error from #{read pipe}# - what nginx.conf to use? Pjotr Prins 1 sibling, 0 replies; 4+ messages in thread From: Andy Wingo @ 2017-05-16 6:57 UTC (permalink / raw) To: Pjotr Prins; +Cc: guix-devel On Tue 16 May 2017 08:29, Pjotr Prins <pjotr.public12@thebird.nl> writes: > On a recent guix-publish server I get many of these > > gzip: stdin: invalid compressed data--format violated > r-annotationdbi-1.36.0 > 1.7MiB/s 00:02 | 3.0MiB transferredguix substitute: error: corrupt > input while restoring > '/gnu/store/g33ns71m61zs5sn758smmc4lnv6h4c4q-r-annotationdbi-1.36.0/site-library/AnnotationDbi/extdata/HG-U95Av2_probe_tab.gz' from #{read pipe}# > > I don't think this is a guix problem per se. I have a hunch the people > changed the firewall which may cause this. Anyone any other ideas? IME this means that the .nar or whatever was truncated by some intermediate cache. Like if you wget the file and try to gunzip it, it will give the same error AFAIU. Andy ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Guix publish pipe error from #{read pipe}# - what nginx.conf to use? 2017-05-16 6:29 Guix publish pipe error from #{read pipe}# Pjotr Prins 2017-05-16 6:57 ` Andy Wingo @ 2017-05-17 7:10 ` Pjotr Prins 2017-05-17 12:48 ` Ludovic Courtès 1 sibling, 1 reply; 4+ messages in thread From: Pjotr Prins @ 2017-05-17 7:10 UTC (permalink / raw) To: Pjotr Prins; +Cc: guix-devel On Tue, May 16, 2017 at 08:29:17AM +0200, Pjotr Prins wrote: > On a recent guix-publish server I get many of these > > gzip: stdin: invalid compressed data--format violated > r-annotationdbi-1.36.0 > 1.7MiB/s 00:02 | 3.0MiB transferredguix substitute: error: corrupt > input while restoring > '/gnu/store/g33ns71m61zs5sn758smmc4lnv6h4c4q-r-annotationdbi-1.36.0/site-library/AnnotationDbi/extdata/HG-U95Av2_probe_tab.gz' from #{read pipe}# > > I don't think this is a guix problem per se. I have a hunch the people > changed the firewall which may cause this. Anyone any other ideas? It is an nginx thing. When I go around nginx guix publish works as expected. Anyone have an example nginx.conf file for guix publish? Funny thing is that it used to be fine. I think parallel downloads is messing it up now. Now we are doing parallel downloads, would it be an idea to do a round robin on substitute-urls too? Make it optional. May speed downloads up nicely. Pj. -- ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Guix publish pipe error from #{read pipe}# - what nginx.conf to use? 2017-05-17 7:10 ` Guix publish pipe error from #{read pipe}# - what nginx.conf to use? Pjotr Prins @ 2017-05-17 12:48 ` Ludovic Courtès 0 siblings, 0 replies; 4+ messages in thread From: Ludovic Courtès @ 2017-05-17 12:48 UTC (permalink / raw) To: Pjotr Prins; +Cc: guix-devel Pjotr Prins <pjotr.public12@thebird.nl> skribis: > On Tue, May 16, 2017 at 08:29:17AM +0200, Pjotr Prins wrote: >> On a recent guix-publish server I get many of these >> >> gzip: stdin: invalid compressed data--format violated >> r-annotationdbi-1.36.0 >> 1.7MiB/s 00:02 | 3.0MiB transferredguix substitute: error: corrupt >> input while restoring >> '/gnu/store/g33ns71m61zs5sn758smmc4lnv6h4c4q-r-annotationdbi-1.36.0/site-library/AnnotationDbi/extdata/HG-U95Av2_probe_tab.gz' from #{read pipe}# >> >> I don't think this is a guix problem per se. I have a hunch the people >> changed the firewall which may cause this. Anyone any other ideas? > > It is an nginx thing. When I go around nginx guix publish works as > expected. Anyone have an example nginx.conf file for guix publish? My guess is that you’re running ‘guix publish’ without --cache. As a result, ‘guix publish’ compresses archives on the fly, and thus it is not able to provide a Content-Length HTTP header. Consequently, your nginx cache cannot detect truncation. So if for some reason (e.g., overloaded machine, ‘guix publish’ restart, etc.), the connection between nginx and ‘guix publish’ is terminated prematurely, nginx may end up caching a truncated store. Also, quoth the manual: By default, ‘guix publish’ compresses archives on the fly as it serves them. This “on-the-fly” mode is convenient in that it requires no setup and is immediately available. However, when serving lots of clients, we recommend using the ‘--cache’ option, which enables caching of the archives before they are sent to clients—see below for details. HTH! Ludo’. ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2017-05-17 12:48 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2017-05-16 6:29 Guix publish pipe error from #{read pipe}# Pjotr Prins 2017-05-16 6:57 ` Andy Wingo 2017-05-17 7:10 ` Guix publish pipe error from #{read pipe}# - what nginx.conf to use? Pjotr Prins 2017-05-17 12:48 ` Ludovic Courtès
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).