From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
To: "Ludovic Courtès" <ludo@gnu.org>
Cc: Josselin Poiret <dev@jpoiret.xyz>,
Simon Tournier <zimon.toutoune@gmail.com>,
Mathieu Othacehe <othacehe@gnu.org>,
Tobias Geerinckx-Rice <me@tobias.gr>,
Ricardo Wurmus <rekado@elephly.net>,
65866@debbugs.gnu.org, Christopher Baines <guix@cbaines.net>
Subject: [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts
Date: Wed, 20 Sep 2023 12:05:23 -0400 [thread overview]
Message-ID: <87fs386da4.fsf_-_@gmail.com> (raw)
In-Reply-To: <b25aa2dd644ac85ed72dabf3cb2098fd8c6358d0.1694441831.git.ludo@gnu.org> ("Ludovic Courtès"'s message of "Mon, 11 Sep 2023 16:25:19 +0200")
Hi Ludovic,
Ludovic Courtès <ludo@gnu.org> writes:
> * guix/build/git.scm (git-fetch-with-fallback): New procedure, with code
> taken from…
> * guix/git-download.scm (git-fetch): … here.
> [modules]: Remove modules that are no longer directly used in ‘build’.
> [build]: Use ‘git-fetch-with-fallback’.
[...]
> +
> +(define* (git-fetch-with-fallback url commit directory
> + #:key (git-command "git") recursive?)
> + "Like 'git-fetch', fetch COMMIT from URL into DIRECTORY, but fall back to
> +alternative methods when fetching from URL fails: attempt to download a nar,
> +and if that also fails, download from the Software Heritage archive."
> + (or (git-fetch url commit directory
> + #:recursive? recursive?
> + #:git-command git-command)
> + (download-nar directory)
> +
> + ;; As a last resort, attempt to download from Software Heritage.
> + ;; Disable X.509 certificate verification to avoid depending
> + ;; on nss-certs--we're authenticating the checkout anyway.
> + ;; XXX: Currently recursive checkouts are not supported.
> + (and (not recursive?)
I know this is code moved from elsewhere, but it seems it'd be useful to
fail hard here with a proper error instead of returning #f silently? Or
add support for recursive clones; was is missing to enable that? It's
at least easy from the git CLI.
> + (parameterize ((%verify-swh-certificate? #f))
> + (format (current-error-port)
> + "Trying to download from Software Heritage...~%")
> +
> + (swh-download url commit directory)
> + (when (file-exists?
> + (string-append directory "/.gitattributes"))
> + ;; Perform CR/LF conversion and other changes
> + ;; specificied by '.gitattributes'.
> + (invoke git-command "-C" directory "init")
> + (invoke git-command "-C" directory "config" "--local"
> + "user.email" "you@example.org")
> + (invoke git-command "-C" directory "config" "--local"
> + "user.name" "Your Name")
> + (invoke git-command "-C" directory "add" ".")
> + (invoke git-command "-C" directory "commit" "-am" "init")
> + (invoke git-command "-C" directory "read-tree" "--empty")
> + (invoke git-command "-C" directory "reset" "--hard")
> + (delete-file-recursively
> + (string-append directory "/.git")))))))
I'm not familiar with this code, but was wondering why we need to do
this post processing and handle .gitattributes. I never care about this
on my GNU/Linux machine when using 'git clone'. Perhaps 'git fetch' is
used directly, which is why?
Time passes... Ah! I misread -- that's peculiar to Software Heritage.
> ;;; git.scm ends here
> diff --git a/guix/git-download.scm b/guix/git-download.scm
> index d88f4c40ee..8989b1b463 100644
> --- a/guix/git-download.scm
> +++ b/guix/git-download.scm
> @@ -116,19 +116,16 @@ (define* (git-fetch ref hash-algo hash
> (define modules
> (delete '(guix config)
> (source-module-closure '((guix build git)
> - (guix build utils)
> - (guix build download-nar)
> - (guix swh)))))
> + (guix build utils)))))
>
> (define build
> (with-imported-modules modules
> - (with-extensions (list guile-json gnutls ;for (guix swh)
> + (with-extensions (list guile-json gnutls ;for (guix swh)
> guile-lzlib)
> #~(begin
> (use-modules (guix build git)
> - (guix build utils)
> - (guix build download-nar)
> - (guix swh)
> + ((guix build utils)
> + #:select (set-path-environment-variable))
> (ice-9 match))
>
> (define recursive?
> @@ -151,38 +148,10 @@ (define* (git-fetch ref hash-algo hash
> (setvbuf (current-output-port) 'line)
> (setvbuf (current-error-port) 'line)
>
> - (or (git-fetch (getenv "git url") (getenv "git commit")
> - #$output
> - #:recursive? recursive?
> - #:git-command "git")
> - (download-nar #$output)
> -
> - ;; As a last resort, attempt to download from Software Heritage.
> - ;; Disable X.509 certificate verification to avoid depending
> - ;; on nss-certs--we're authenticating the checkout anyway.
> - ;; XXX: Currently recursive checkouts are not supported.
> - (and (not recursive?)
> - (parameterize ((%verify-swh-certificate? #f))
> - (format (current-error-port)
> - "Trying to download from Software Heritage...~%")
> -
> - (swh-download (getenv "git url") (getenv "git commit")
> - #$output)
> - (when (file-exists?
> - (string-append #$output "/.gitattributes"))
> - ;; Perform CR/LF conversion and other changes
> - ;; specificied by '.gitattributes'.
> - (invoke "git" "-C" #$output "init")
> - (invoke "git" "-C" #$output "config" "--local"
> - "user.email" "you@example.org")
> - (invoke "git" "-C" #$output "config" "--local"
> - "user.name" "Your Name")
> - (invoke "git" "-C" #$output "add" ".")
> - (invoke "git" "-C" #$output "commit" "-am" "init")
> - (invoke "git" "-C" #$output "read-tree" "--empty")
> - (invoke "git" "-C" #$output "reset" "--hard")
> - (delete-file-recursively
> - (string-append #$output "/.git"))))))))))
> + (git-fetch-with-fallback (getenv "git url") (getenv "git commit")
> + #$output
> + #:recursive? recursive?
> + #:git-command "git")))))
>
> (mlet %store-monad ((guile (package->derivation guile system)))
> (gexp->derivation (or name "git-checkout") build
LGTM.
--
Thanks,
Maxim
next prev parent reply other threads:[~2023-09-20 16:06 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-09-11 14:23 [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts Ludovic Courtès
2023-09-11 14:25 ` [bug#65866] [PATCH 1/8] git-download: Move fallback code to (guix build git) Ludovic Courtès
2023-09-20 16:05 ` Maxim Cournoyer [this message]
2023-09-20 16:40 ` [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts Simon Tournier
2023-09-22 21:53 ` Ludovic Courtès
2023-09-11 14:25 ` [bug#65866] [PATCH 2/8] git-download: Honor the ‘GUIX_DOWNLOAD_FALLBACK_TEST’ environment variable Ludovic Courtès
2023-09-20 16:07 ` [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts Maxim Cournoyer
2023-09-11 14:25 ` [bug#65866] [PATCH 3/8] perform-download: Remove unused one-argument clause Ludovic Courtès
2023-09-20 16:09 ` [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts Maxim Cournoyer
2023-09-11 14:25 ` [bug#65866] [PATCH 4/8] daemon: Add “git-download” built-in builder Ludovic Courtès
2023-09-20 17:32 ` [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts Maxim Cournoyer
2023-09-21 7:42 ` Ludovic Courtès
2023-09-22 22:27 ` [bug#65866] [PATCH v2 " Ludovic Courtès
2023-09-22 22:27 ` [bug#65866] [PATCH v2 1/8] git-download: Move fallback code to (guix build git) Ludovic Courtès
2023-09-25 8:15 ` Simon Tournier
2023-09-25 9:24 ` [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts Ludovic Courtès
2023-09-25 12:13 ` Simon Tournier
2023-09-22 22:27 ` [bug#65866] [PATCH v2 2/8] git-download: Honor the ‘GUIX_DOWNLOAD_FALLBACK_TEST’ environment variable Ludovic Courtès
2023-09-22 22:27 ` [bug#65866] [PATCH v2 3/8] perform-download: Remove unused one-argument clause Ludovic Courtès
2023-09-22 22:28 ` [bug#65866] [PATCH v2 4/8] daemon: Add “git-download” built-in builder Ludovic Courtès
2023-09-22 22:28 ` [bug#65866] [PATCH v2 5/8] build: Add dependency on Git Ludovic Courtès
2023-09-25 13:59 ` Simon Tournier
2023-09-26 14:05 ` [bug#65866] Bootstrapping without the daemon and all that Ludovic Courtès
2023-09-26 17:04 ` Simon Tournier
2023-10-12 10:54 ` Ludovic Courtès
2023-10-16 8:46 ` Simon Tournier
2023-09-22 22:28 ` [bug#65866] [PATCH v2 6/8] perform-download: Use the ‘git’ command captured at configure time Ludovic Courtès
2023-09-22 22:28 ` [bug#65866] [PATCH v2 7/8] git-download: Use “builtin:git-download” when available Ludovic Courtès
2023-09-25 8:33 ` Simon Tournier
2023-09-25 9:23 ` [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts Ludovic Courtès
2023-09-25 12:37 ` Simon Tournier
2023-09-25 12:48 ` Simon Tournier
2023-09-25 15:49 ` Maxim Cournoyer
2023-09-26 15:44 ` bug#65866: " Ludovic Courtès
2023-09-26 17:13 ` [bug#65866] " Simon Tournier
2023-10-01 15:02 ` Ludovic Courtès
2023-10-16 9:11 ` [bug#65866] Toward RFC? (was Re: [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts) Simon Tournier
2023-10-30 15:12 ` Ludovic Courtès
2023-09-22 22:28 ` [bug#65866] [PATCH v2 8/8] tests: Assume ‘git’ is always available Ludovic Courtès
2023-09-11 14:25 ` [bug#65866] [PATCH 5/8] build: Add dependency on Git Ludovic Courtès
2023-09-20 17:57 ` [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts Maxim Cournoyer
2023-09-22 22:00 ` Ludovic Courtès
2023-09-25 15:59 ` Maxim Cournoyer
2023-09-11 14:25 ` [bug#65866] [PATCH 6/8] perform-git-download: Use the ‘git’ command captured at configure time Ludovic Courtès
2023-09-20 17:34 ` [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts Maxim Cournoyer
2023-09-11 14:25 ` [bug#65866] [PATCH 7/8] git-download: Use “builtin:git-download” when available Ludovic Courtès
2023-09-20 17:50 ` [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts Maxim Cournoyer
2023-09-22 21:58 ` Ludovic Courtès
2023-09-25 15:56 ` Maxim Cournoyer
2023-09-11 14:25 ` [bug#65866] [PATCH 8/8] tests: Assume ‘git’ is always available Ludovic Courtès
2023-09-20 17:59 ` [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts Maxim Cournoyer
[not found] <877ctljs0m.fsf@inria.fr>
[not found] ` <87pm2osrot.fsf@gnu.org>
2023-09-11 15:16 ` [bug#65866] bug#63331: Guile-GnuTLS/Git circular dependency and built-in git checkouts Vivien Kraus via Guix-patches via
2023-09-11 20:57 ` [bug#65866] [PATCH 0/8] Add built-in builder for Git checkouts Ludovic Courtès
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87fs386da4.fsf_-_@gmail.com \
--to=maxim.cournoyer@gmail.com \
--cc=65866@debbugs.gnu.org \
--cc=dev@jpoiret.xyz \
--cc=guix@cbaines.net \
--cc=ludo@gnu.org \
--cc=me@tobias.gr \
--cc=othacehe@gnu.org \
--cc=rekado@elephly.net \
--cc=zimon.toutoune@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).