* [bug#58017] bug#57978: [PATCH 2/2] substitute: Retry downloading when a nar is unavailable.
[not found] ` <20220923061957.5658-2-ludo@gnu.org>
@ 2022-09-23 8:17 ` zimoun
2022-09-24 16:20 ` Ludovic Courtès
[not found] ` <46a88139-3994-71e5-1a78-398d6926ef88@telenet.be>
1 sibling, 1 reply; 3+ messages in thread
From: zimoun @ 2022-09-23 8:17 UTC (permalink / raw)
To: Ludovic Courtès, 58017; +Cc: Attila Lendvai, 57978, Ludovic Courtès
Hi,
On ven., 23 sept. 2022 at 08:19, Ludovic Courtès <ludo@gnu.org> wrote:
> Fixes <https://issues.guix.gnu.org/57978>
> Reported by Attila Lendvai <attila@lendvai.name>.
>
> Previously, if a narinfo was available but its corresponding nar was
> missing (for instance because the narinfo was cached and the server
> became unreachable in the meantime), 'guix substitute --substitute'
> would try to download the nar from its preferred location and abort when
> that fails. This change forces one retry with each of the URLs.
>
> * guix/scripts/substitute.scm (download-nar): Do not catch
> 'http-get-error?' exceptions.
> (system-error?, network-error?, process-substitution/fallback): New
> procedures.
> (process-substitution): Call 'process-substitution/fallback' upon
> 'network-error?'.
> * tests/substitute.scm ("substitute, first URL has narinfo but lacks nar, second URL unauthorized")
> ("substitute, first URL has narinfo but nar is 404, both URLs authorized")
> ("substitute, first URL has narinfo but nar is 404, one URL authorized")
> ("substitute, narinfo is available but nar is missing"): New tests.
LGTM.
> +(test-equal "substitute, first URL has narinfo but nar is 404, one URL authorized"
> + "Substitutable data."
> + (with-narinfo*
> + (string-append %narinfo "Signature: "
> + (signature-field
> + %narinfo
> + #:public-key %wrong-public-key))
> + %main-substitute-directory
> +
> + (with-http-server `((200 ,(string-append %narinfo "Signature: "
> + (signature-field
> + %narinfo
> + #:public-key %wrong-public-key)))
> + (404 "Sorry, nar is missing!"))
> + (let ((url1 (%local-url)))
> + (parameterize ((%http-server-port 0))
> + (with-http-server `((200 ,(string-append %narinfo "Signature: "
> + (signature-field %narinfo)))
> + (404 "Sorry, nar is missing!"))
> + (let ((url2 (%local-url)))
> + (dynamic-wind
> + (const #t)
> + (lambda ()
> + (parameterize ((substitute-urls
> + (list url1 url2
> + (string-append "file://"
> + %main-substitute-directory))))
> + (request-substitution (string-append (%store-prefix)
> + "/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-foo")
> + "substitute-retrieved"))
> + (call-with-input-file "substitute-retrieved" get-string-all))
> + (lambda ()
> + (false-if-exception (delete-file "substitute-retrieved")))))))))))
Although I do not understand this test. Why is 404 appearing twice?
Cheers,
simon
^ permalink raw reply [flat|nested] 3+ messages in thread
* [bug#58017] bug#57978: [PATCH 2/2] substitute: Retry downloading when a nar is unavailable.
2022-09-23 8:17 ` [bug#58017] bug#57978: [PATCH 2/2] substitute: Retry downloading when a nar is unavailable zimoun
@ 2022-09-24 16:20 ` Ludovic Courtès
0 siblings, 0 replies; 3+ messages in thread
From: Ludovic Courtès @ 2022-09-24 16:20 UTC (permalink / raw)
To: zimoun; +Cc: 58017, 57978, Attila Lendvai
Hi!
zimoun <zimon.toutoune@gmail.com> skribis:
>> + (with-narinfo*
>> + (string-append %narinfo "Signature: "
>> + (signature-field
>> + %narinfo
>> + #:public-key %wrong-public-key))
>> + %main-substitute-directory
>> +
>> + (with-http-server `((200 ,(string-append %narinfo "Signature: "
>> + (signature-field
>> + %narinfo
>> + #:public-key %wrong-public-key)))
>> + (404 "Sorry, nar is missing!"))
>> + (let ((url1 (%local-url)))
>> + (parameterize ((%http-server-port 0))
>> + (with-http-server `((200 ,(string-append %narinfo "Signature: "
>> + (signature-field %narinfo)))
>> + (404 "Sorry, nar is missing!"))
>> + (let ((url2 (%local-url)))
>> + (dynamic-wind
>> + (const #t)
>> + (lambda ()
>> + (parameterize ((substitute-urls
>> + (list url1 url2
>> + (string-append "file://"
>> + %main-substitute-directory))))
[...]
> Although I do not understand this test. Why is 404 appearing twice?
That’s because it’s testing with 3 substitute URLs.
Thanks for taking a look!
Ludo’.
^ permalink raw reply [flat|nested] 3+ messages in thread
* [bug#58017] [PATCH 2/2] substitute: Retry downloading when a nar is unavailable.
[not found] ` <46a88139-3994-71e5-1a78-398d6926ef88@telenet.be>
@ 2022-09-24 16:22 ` Ludovic Courtès
0 siblings, 0 replies; 3+ messages in thread
From: Ludovic Courtès @ 2022-09-24 16:22 UTC (permalink / raw)
To: Maxime Devos; +Cc: 58017, 57978, Attila Lendvai, zimoun
Hi Maxime,
Maxime Devos <maximedevos@telenet.be> skribis:
>> +(test-equal "substitute, first URL has narinfo but nar is 404, both URLs authorized"
>> + "Substitutable data."
>> + (with-narinfo*
>> + (string-append %narinfo "Signature: "
>> + (signature-field %narinfo))
>> + %main-substitute-directory
>> +
>> + (with-http-server `((200 ,(string-append %narinfo "Signature: "
>> + (signature-field %narinfo)))
>> + (404 "Sorry, nar is missing!"))
>> + (dynamic-wind
>> + (const #t)
>> + (lambda ()
>> + (parameterize ((substitute-urls
>> + (list (%local-url)
>> + (string-append "file://"
>> + %main-substitute-directory))))
>> + (request-substitution (string-append (%store-prefix)
>> + "/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-foo")
>> + "substitute-retrieved"))
>> + (call-with-input-file "substitute-retrieved" get-string-all))
>> + (lambda ()
>> + (false-if-exception (delete-file "substitute-retrieved")))))))
>
> Shouldn't it only ignore 'file not found' (ENOENT?) exceptions?
By “it”, do you mean ‘dynamic-wind’ should be replaced by a ‘catch’
form?
We could discuss it, but note that this patch just keeps with the style
of existing tests.
> This test, and some others, can be improved by also checking the
> URI. While currently 'with-http-server' does not support that, there
> are (5 months, with the v1 having seen some reviewing and a v2
> available) patches for that at <https://issues.guix.gnu.org/53389>.
>
> That patch also _requires_ always mentioning the URI, if the cover
> letter is correct. It also allows simplifying the use of '%local-url'
> a bit.
Ah, thanks for the reminder! I’ve just spent most of the day reviewing
patches, but not that one…
Ludo’.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-09-24 16:24 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20220923061957.5658-1-ludo@gnu.org>
[not found] ` <20220923061957.5658-2-ludo@gnu.org>
2022-09-23 8:17 ` [bug#58017] bug#57978: [PATCH 2/2] substitute: Retry downloading when a nar is unavailable zimoun
2022-09-24 16:20 ` Ludovic Courtès
[not found] ` <46a88139-3994-71e5-1a78-398d6926ef88@telenet.be>
2022-09-24 16:22 ` [bug#58017] " Ludovic Courtès
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).