unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* how to customize mirror list used in custom channels?
@ 2022-11-25  8:42 zimoun
  2022-11-26 11:11 ` Ludovic Courtès
  0 siblings, 1 reply; 8+ messages in thread
From: zimoun @ 2022-11-25  8:42 UTC (permalink / raw)
  To: Guix Devel

Hi,

I have a custom channel defining a package.  The source of this package
is mirrored.

In ’(guix download)’ module, the list of mirrors are provided by the
list ’%mirrors’.  So I would like to add mine, e.g.,

        (cons
         '(mine
           "https://fast-server.org/sources/"
           "hhtps://kikoo.io/path/to/something/")
         %mirrors)

and then in my package definition,

        (uri "mirror://mine/stuff.tar.gz")

Somehow, I would like to replace the list %mirrors by an extended one.

Well, I am stuck because I am puzzled by the Guile module system. :-)


How to customize the mirror list?

Cheers,
simon


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: how to customize mirror list used in custom channels?
  2022-11-25  8:42 how to customize mirror list used in custom channels? zimoun
@ 2022-11-26 11:11 ` Ludovic Courtès
  2022-11-27 12:09   ` zimoun
  0 siblings, 1 reply; 8+ messages in thread
From: Ludovic Courtès @ 2022-11-26 11:11 UTC (permalink / raw)
  To: zimoun; +Cc: Guix Devel

Hi! :-)

zimoun <zimon.toutoune@gmail.com> skribis:

> In ’(guix download)’ module, the list of mirrors are provided by the
> list ’%mirrors’.  So I would like to add mine, e.g.,
>
>         (cons
>          '(mine
>            "https://fast-server.org/sources/"
>            "hhtps://kikoo.io/path/to/something/")
>          %mirrors)
>
> and then in my package definition,
>
>         (uri "mirror://mine/stuff.tar.gz")
>
> Somehow, I would like to replace the list %mirrors by an extended one.

The easiest solution is to avoid mirror:// and pass a list of regular
URLs:

  (origin
    (method url-fetch)
    (uri (list URL1 URL2 URL3))
    (base32 …))

See for example how ‘qtbase’ does it.

Now, if you really want to extend the set of things recognized, you
could write variant of ‘url-fetch’ that passes a different #:mirrors
argument to ‘built-in-download’.  Maybe ‘url-fetch’ could have a
#:mirrors parameter to simplify it.

But really, the URL list should do the job most of the time.

Ludo’.


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: how to customize mirror list used in custom channels?
  2022-11-26 11:11 ` Ludovic Courtès
@ 2022-11-27 12:09   ` zimoun
  2022-11-28 10:49     ` Ludovic Courtès
  0 siblings, 1 reply; 8+ messages in thread
From: zimoun @ 2022-11-27 12:09 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: Guix Devel

Hi,

On Sat, 26 Nov 2022 at 12:11, Ludovic Courtès <ludo@gnu.org> wrote:

>   (origin
>     (method url-fetch)
>     (uri (list URL1 URL2 URL3))
>     (base32 …))

Neat!  This is nice for one or two packages.


> See for example how ‘qtbase’ does it.

Well, it looks like it is a manual implementation of the Guix mirror
mechanism.


> Now, if you really want to extend the set of things recognized, you
> could write variant of ‘url-fetch’ that passes a different #:mirrors
> argument to ‘built-in-download’.  Maybe ‘url-fetch’ could have a
> #:mirrors parameter to simplify it.

Why is it not possible to extend ’%mirrors?  For instance, it is
possible to extend ’%load-path’.  Why not %mirrors?


Cheers,
simon


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: how to customize mirror list used in custom channels?
  2022-11-27 12:09   ` zimoun
@ 2022-11-28 10:49     ` Ludovic Courtès
  2022-11-28 12:52       ` zimoun
  0 siblings, 1 reply; 8+ messages in thread
From: Ludovic Courtès @ 2022-11-28 10:49 UTC (permalink / raw)
  To: zimoun; +Cc: Guix Devel

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> On Sat, 26 Nov 2022 at 12:11, Ludovic Courtès <ludo@gnu.org> wrote:

[...]

>> Now, if you really want to extend the set of things recognized, you
>> could write variant of ‘url-fetch’ that passes a different #:mirrors
>> argument to ‘built-in-download’.  Maybe ‘url-fetch’ could have a
>> #:mirrors parameter to simplify it.
>
> Why is it not possible to extend ’%mirrors?

What I described above is one way to extend it, just not via ‘set!’.

Ludo’.


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: how to customize mirror list used in custom channels?
  2022-11-28 10:49     ` Ludovic Courtès
@ 2022-11-28 12:52       ` zimoun
  2022-11-29  8:41         ` Ludovic Courtès
  0 siblings, 1 reply; 8+ messages in thread
From: zimoun @ 2022-11-28 12:52 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: Guix Devel

Hi,

On Mon, 28 Nov 2022 at 11:49, Ludovic Courtès <ludo@gnu.org> wrote:
> Hi,
>
> zimoun <zimon.toutoune@gmail.com> skribis:
>
>> On Sat, 26 Nov 2022 at 12:11, Ludovic Courtès <ludo@gnu.org> wrote:
>
> [...]
>
>>> Now, if you really want to extend the set of things recognized, you
>>> could write variant of ‘url-fetch’ that passes a different #:mirrors
>>> argument to ‘built-in-download’.  Maybe ‘url-fetch’ could have a
>>> #:mirrors parameter to simplify it.
>>
>> Why is it not possible to extend ’%mirrors?
>
> What I described above is one way to extend it, just not via ‘set!’.

Just to be sure, one way is this extension mechanism,

--8<---------------cut here---------------start------------->8---
(define (qt-urls component version)
  "Return a list of URLs for VERSION of the Qt5 COMPONENT."
  ;; We can't use a mirror:// scheme because these URLs are not exact copies:
  ;; the layout differs between them.
  (list (string-append "https://download.qt.io/official_releases/qt/"
                       (version-major+minor version) "/" version
                       "/submodules/" component "-everywhere-opensource-src-"
                       version ".tar.xz")
        (string-append "https://download.qt.io/official_releases/qt/"
                       (version-major+minor version) "/" version
                       "/submodules/" component "-everywhere-src-"
                       version ".tar.xz")
        (string-append "https://download.qt.io/archive/qt/"
                       (version-major+minor version) "/" version
                       "/submodules/" component "-everywhere-opensource-src-"
                       version ".tar.xz")
        (let ((directory (string-append "qt5" (string-drop component 2))))
          (string-append "http://sources.buildroot.net/" directory "/"
                         component "-everywhere-opensource-src-" version ".tar.xz"))
        (string-append "https://distfiles.macports.org/qt5/"
                       component "-everywhere-opensource-src-" version ".tar.xz")))
--8<---------------cut here---------------end--------------->8---

then,

    (uri (qt-urls name version))

Right?  Well, for what it is worth, it does not appear to me consistent
with the rest.  We could do the same for everything, and remove
"mirror://" after all. :-)

The other one way is to implement a variant of url-fetch method.  What
would be the interface?  Where the extended list of mirrors would be
provided?


Cheers,
simon


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: how to customize mirror list used in custom channels?
  2022-11-28 12:52       ` zimoun
@ 2022-11-29  8:41         ` Ludovic Courtès
  2022-11-29 19:02           ` zimoun
  0 siblings, 1 reply; 8+ messages in thread
From: Ludovic Courtès @ 2022-11-29  8:41 UTC (permalink / raw)
  To: zimoun; +Cc: Guix Devel

zimoun <zimon.toutoune@gmail.com> skribis:

> Hi,
>
> On Mon, 28 Nov 2022 at 11:49, Ludovic Courtès <ludo@gnu.org> wrote:
>> Hi,
>>
>> zimoun <zimon.toutoune@gmail.com> skribis:
>>
>>> On Sat, 26 Nov 2022 at 12:11, Ludovic Courtès <ludo@gnu.org> wrote:
>>
>> [...]
>>
>>>> Now, if you really want to extend the set of things recognized, you
>>>> could write variant of ‘url-fetch’ that passes a different #:mirrors
>>>> argument to ‘built-in-download’.  Maybe ‘url-fetch’ could have a
>>>> #:mirrors parameter to simplify it.
>>>
>>> Why is it not possible to extend ’%mirrors?
>>
>> What I described above is one way to extend it, just not via ‘set!’.
>
> Just to be sure, one way is this extension mechanism,

I was referring to #:mirrors etc. above.

[...]

> The other one way is to implement a variant of url-fetch method.  What
> would be the interface?  Where the extended list of mirrors would be
> provided?

The more I write, the closer I get to an actual implementation.  :-)
If ‘url-fetch’ takes a #:mirrors, that should allow us to extend the set
of mirrors quite conveniently, along the lines of:

  (define (my-url-fetch . args)
    (apply url-fetch (append args '(#:mirrors …))))

HTH!

Ludo’.


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: how to customize mirror list used in custom channels?
  2022-11-29  8:41         ` Ludovic Courtès
@ 2022-11-29 19:02           ` zimoun
  2022-12-02  9:01             ` Ludovic Courtès
  0 siblings, 1 reply; 8+ messages in thread
From: zimoun @ 2022-11-29 19:02 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: Guix Devel

Hi,

On Tue, 29 Nov 2022 at 09:41, Ludovic Courtès <ludo@gnu.org> wrote:

> The more I write, the closer I get to an actual implementation.  :-)
> If ‘url-fetch’ takes a #:mirrors, that should allow us to extend the set
> of mirrors quite conveniently, along the lines of:
>
>   (define (my-url-fetch . args)
>     (apply url-fetch (append args '(#:mirrors …))))

Well, the ’url-fetch’ defined in (guix build download) already takes
#:mirrors, no?

--8<---------------cut here---------------start------------->8---
(define* (url-fetch url file
                    #:key
                    (timeout 10) (verify-certificate? #t)
                    (mirrors '()) (content-addressed-mirrors '())
                    (disarchive-mirrors '())
                    (hashes '())
                    print-build-trace?)
--8<---------------cut here---------------end--------------->8---

So, I guess you are talking about ’url-fetch*’ from (guix download) and
exported as ’url-fetch’.  Right?  The one which put the bits in the
store. :-)

Does something like that would make sense?

--8<---------------cut here---------------start------------->8---
diff --git a/guix/download.scm b/guix/download.scm
index 2e9ecb43fc..2497ea0f75 100644
--- a/guix/download.scm
+++ b/guix/download.scm
@@ -499,6 +499,7 @@ (define %download-fallback-test
 (define* (url-fetch* url hash-algo hash
                      #:optional name
                      #:key (system (%current-system))
+                     (mirrors %mirrors)
                      (guile (default-guile))
                      executable?)
   "Return a fixed-output derivation that fetches data from URL (a string, or a
@@ -519,7 +520,8 @@ (define file-name
       (_
        (basename url))))
 
-  (let ((uri (and (string? url) (string->uri url))))
+  (let ((uri (and (string? url) (string->uri url)))
+        (mirror-file (plain-file "mirrors" (object->string mirrors))))
     (if (or (and (string? url) (not uri))
             (and uri (memq (uri-scheme uri) '(#f file))))
         (interned-file (if uri (uri-path uri) url)
@@ -540,7 +542,7 @@ (define file-name
                              #:hash-algo hash-algo
                              #:hash hash
                              #:executable? executable?
-                             #:mirrors %mirror-file
+                             #:mirrors mirror-file
                              #:content-addressed-mirrors
                              (match (%download-fallback-test)
                                ((or #f 'content-addressed-mirrors)
--8<---------------cut here---------------end--------------->8---

This change would a world rebuild, right?

Last, note that ’download-to-store’ uses %mirrors so it seems
straightforward here to add ’#:key (mirrors %mirrors)’, I guess.


Cheers,
simon


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: how to customize mirror list used in custom channels?
  2022-11-29 19:02           ` zimoun
@ 2022-12-02  9:01             ` Ludovic Courtès
  0 siblings, 0 replies; 8+ messages in thread
From: Ludovic Courtès @ 2022-12-02  9:01 UTC (permalink / raw)
  To: zimoun; +Cc: Guix Devel

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> Does something like that would make sense?

Yes, that’s the idea.

> diff --git a/guix/download.scm b/guix/download.scm
> index 2e9ecb43fc..2497ea0f75 100644
> --- a/guix/download.scm
> +++ b/guix/download.scm
> @@ -499,6 +499,7 @@ (define %download-fallback-test
>  (define* (url-fetch* url hash-algo hash
>                       #:optional name
>                       #:key (system (%current-system))
> +                     (mirrors %mirrors)
>                       (guile (default-guile))
>                       executable?)
>    "Return a fixed-output derivation that fetches data from URL (a string, or a
> @@ -519,7 +520,8 @@ (define file-name
>        (_
>         (basename url))))
>  
> -  (let ((uri (and (string? url) (string->uri url))))
> +  (let ((uri (and (string? url) (string->uri url)))
> +        (mirror-file (plain-file "mirrors" (object->string mirrors))))

There are subtleties though: we don’t want to create a new <plain-file>
at each call as this would lead to poor performance (lack of
memoization).

> This change would a world rebuild, right?

No, not at all.

Thanks,
Ludo’.


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2022-12-02  9:02 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-25  8:42 how to customize mirror list used in custom channels? zimoun
2022-11-26 11:11 ` Ludovic Courtès
2022-11-27 12:09   ` zimoun
2022-11-28 10:49     ` Ludovic Courtès
2022-11-28 12:52       ` zimoun
2022-11-29  8:41         ` Ludovic Courtès
2022-11-29 19:02           ` zimoun
2022-12-02  9:01             ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).