[bug#63462] [PATCH maintenance] hydra: build-package-metadata: Update fields in sources.json.

[bug#63462] [PATCH maintenance] hydra: build-package-metadata: Update fields in sources.json.

[Simon Tournier]

This format had been discussed in
<https://gitlab.softwareheritage.org/swh/infra/sysadm-environment/-/issues/4868>.

* hydra/build-package-metadata.scm (origin->json)[resolve]: Add the fields
'outputHashAlgo' and 'outputHashMode'.
---
 hydra/build-package-metadata.scm | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/hydra/build-package-metadata.scm b/hydra/build-package-metadata.scm
index 1ddb409..d110d4d 100755
--- a/hydra/build-package-metadata.scm
+++ b/hydra/build-package-metadata.scm
@@ -1,7 +1,7 @@
 #!/usr/bin/env -S guix repl --
 !#
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2020, 2021 Simon Tournier <zimon.toutoune@gmail.com>
+;;; Copyright © 2020, 2021, 2023 Simon Tournier <zimon.toutoune@gmail.com>
 ;;; Copyright © 2023 Ludovic Courtès <ludo@gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -160,13 +160,23 @@ superseded packages."
                  (else '()))
          ,@(if (or (eq? url-fetch method)
                    (eq? url-fetch/tarbomb method)
-                   (eq? url-fetch/zipbomb method))
+                   (eq? url-fetch/zipbomb method)
+                   (eq? git-fetch method)
+                   (eq? svn-fetch method)
+                   (eq? svn-multi-fetch method)
+                   (eq? hg-fetch method))
                (let* ((content-hash (origin-hash origin))
                       (hash-value (content-hash-value content-hash))
                       (hash-algorithm (content-hash-algorithm content-hash))
                       (algorithm-string (symbol->string hash-algorithm)))
                  `(("integrity" . ,(string-append algorithm-string "-"
-                                                  (base64-encode hash-value)))))
+                                                  (base64-encode hash-value)))
+                   ("outputHashAlgo" . ,algorithm-string)
+                   ("outputHashMode" . ,(if (or (eq? url-fetch method)
+                                                (eq? url-fetch/tarbomb method)
+                                                (eq? url-fetch/zipbomb method))
+                                            "flat"
+                                            "recursive"))))
                '())
          ,@(if (eq? method git-fetch)
                `(("git_ref" . ,(git-reference-commit uri)))

base-commit: 5a6b8cd7286d1c8906beaa35ebba6eff5bae3a79
-- 
2.38.1

[bug#63462] [PATCH maintenance] hydra: build-package-metadata: Update fields in sources.json.

[Ludovic Courtès]

Hi,

Simon Tournier <zimon.toutoune@gmail.com> skribis:

> This format had been discussed in
> <https://gitlab.softwareheritage.org/swh/infra/sysadm-environment/-/issues/4868>.
>
> * hydra/build-package-metadata.scm (origin->json)[resolve]: Add the fields
> 'outputHashAlgo' and 'outputHashMode'.

Neat, thanks for the update!  (I pushed it on your behalf because you’re
not currently authorized in maintenance.git; please email guix-sysadmin
if you’d like to be there.)

>                   `(("integrity" . ,(string-append algorithm-string "-"
> -                                                  (base64-encode hash-value)))))
> +                                                  (base64-encode hash-value)))
> +                   ("outputHashAlgo" . ,algorithm-string)
> +                   ("outputHashMode" . ,(if (or (eq? url-fetch method)
> +                                                (eq? url-fetch/tarbomb method)
> +                                                (eq? url-fetch/zipbomb method))
> +                                            "flat"
> +                                            "recursive"))))

I wish they had opted for something more expressive (and also with a
more meaningful name :-)) than ‘outputHashMode’, similar to what you did
for ‘guix hash -S’.  But anyway, it does the job!

It looks like SWH will be storing nar hashes; will there be endpoints to
query directories by nar hash?

Thanks,
Ludo’.

[bug#63462] [PATCH maintenance] hydra: build-package-metadata: Update fields in sources.json.

[Simon Tournier]

Hi,

On ven., 12 mai 2023 at 13:47, Ludovic Courtès <ludo@gnu.org> wrote:

>                               (I pushed it on your behalf because you’re
> not currently authorized in maintenance.git; please email guix-sysadmin
> if you’d like to be there.)

Ok, I will.


> I wish they had opted for something more expressive (and also with a
> more meaningful name :-)) than ‘outputHashMode’, similar to what you did
> for ‘guix hash -S’.  But anyway, it does the job!

Well, I am biased. ;-)  Another poor naming inherited from Nix [1]. :-)

1: https://nixos.org/manual/nix/stable/language/advanced-attributes.html


> It looks like SWH will be storing nar hashes; will there be endpoints to
> query directories by nar hash?

This is still ongoing.  IIUC, it’s the last item of SWH issue #4868 [2].

2: https://gitlab.softwareheritage.org/swh/infra/sysadm-environment/-/issues/4868


Cheers,
simon

[bug#63462] [PATCH maintenance] hydra: build-package-metadata: Update fields in sources.json.

[Ludovic Courtès]

Hi,

Simon Tournier <zimon.toutoune@gmail.com> skribis:

> On ven., 12 mai 2023 at 13:47, Ludovic Courtès <ludo@gnu.org> wrote:

[...]

>> I wish they had opted for something more expressive (and also with a
>> more meaningful name :-)) than ‘outputHashMode’, similar to what you did
>> for ‘guix hash -S’.  But anyway, it does the job!
>
> Well, I am biased. ;-)  Another poor naming inherited from Nix [1]. :-)

Yeah.

BTW, I’ve restarted the relevant mcron job and it’s rolling out now:

--8<---------------cut here---------------start------------->8---
$ wget -qO- https://guix.gnu.org/sources.json |jq |grep output[A-Z] | head
      "outputHashAlgo": "sha256",
      "outputHashMode": "flat"
      "outputHashAlgo": "sha256",
      "outputHashMode": "flat"
      "outputHashAlgo": "sha256",
      "outputHashMode": "recursive",
      "outputHashAlgo": "sha256",
      "outputHashMode": "flat"
      "outputHashAlgo": "sha256",
      "outputHashMode": "flat"
--8<---------------cut here---------------end--------------->8---

>> It looks like SWH will be storing nar hashes; will there be endpoints to
>> query directories by nar hash?
>
> This is still ongoing.  IIUC, it’s the last item of SWH issue #4868 [2].
>
> 2: https://gitlab.softwareheritage.org/swh/infra/sysadm-environment/-/issues/4868

Nice!

Ludo’.