bug#36781: Website manual generation stopped

bug#36781: Website manual generation stopped

[Julien Lepiller]

Hi guix, it seems that the devel version of the manual is not regenerated as intended. The website says that it's still from July 18th.

bug#36781: Website manual generation stopped

[Ludovic Courtès]

Hi,

Julien Lepiller <julien@lepiller.eu> skribis:

> Hi guix, it seems that the devel version of the manual is not regenerated as intended. The website says that it's still from July 18th.

Indeed it fails like this:

--8<---------------cut here---------------start------------->8---
ludo@berlin ~$ sudo su - static-web-site -c /gnu/store/9w4bbd6gqya2g9zvwgs6qab6aqgbjbd3-update-guix-manual-devel
Backtrace:
           7 (primitive-load "/gnu/store/9w4bbd6gqya2g9zvwgs6qab6aqg…")
In ice-9/eval.scm:
    619:8  6 (_ #f)
   626:19  5 (_ #<directory (guile-user) 2366140>)
In unknown file:
           4 (_ #<procedure 2dd2960 at ice-9/eval.scm:330:13 ()> #<…> …)
In guix/git.scm:
   240:29  3 (update-cached-checkout "https://git.sv.gnu.org/git/gu…" …)
In ice-9/boot-9.scm:
    841:4  2 (with-throw-handler _ _ _)
In git/clone.scm:
     41:8  1 (_ _ _ _)
In ice-9/boot-9.scm:
   752:25  0 (dispatch-exception _ _ _)

ice-9/boot-9.scm:752:25: In procedure dispatch-exception:
Git error: cross host redirect not allowed
--8<---------------cut here---------------end--------------->8---

So I think we have to change the repo URL in berlin.scm.

Ludo’.

bug#36781: Website manual generation stopped

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 2003 bytes --]

Le Fri, 26 Jul 2019 00:54:32 +0200,
Ludovic Courtès <ludo@gnu.org> a écrit :

> Hi,
> 
> Julien Lepiller <julien@lepiller.eu> skribis:
> 
> > Hi guix, it seems that the devel version of the manual is not
> > regenerated as intended. The website says that it's still from July
> > 18th.  
> 
> Indeed it fails like this:
> 
> --8<---------------cut here---------------start------------->8---
> ludo@berlin ~$ sudo su - static-web-site
> -c /gnu/store/9w4bbd6gqya2g9zvwgs6qab6aqgbjbd3-update-guix-manual-devel
> Backtrace: 7 (primitive-load
> "/gnu/store/9w4bbd6gqya2g9zvwgs6qab6aqg…") In ice-9/eval.scm:
>     619:8  6 (_ #f)
>    626:19  5 (_ #<directory (guile-user) 2366140>)
> In unknown file:
>            4 (_ #<procedure 2dd2960 at ice-9/eval.scm:330:13 ()> #<…>
> …) In guix/git.scm:
>    240:29  3 (update-cached-checkout "https://git.sv.gnu.org/git/gu…"
> …) In ice-9/boot-9.scm:
>     841:4  2 (with-throw-handler _ _ _)
> In git/clone.scm:
>      41:8  1 (_ _ _ _)
> In ice-9/boot-9.scm:
>    752:25  0 (dispatch-exception _ _ _)
> 
> ice-9/boot-9.scm:752:25: In procedure dispatch-exception:
> Git error: cross host redirect not allowed
> --8<---------------cut here---------------end--------------->8---
> 
> So I think we have to change the repo URL in berlin.scm.
> 
> Ludo’.

One way I can see to solve that issue is to specify a custome cache
directory name, instead of the default one, which is a hash of the url.
The reason why we use git.sv.gnu.org instead of git.savannah.gnu.org
is that otherwise both repos have the same cache directory, so one wins
over the other. But that hack doesn't scale if we want to generate more
than two manual versions.

Attached is a patch that adds a cache-directory field to the
static-website-configuration record.

Another solution is to fix (guix git) to also add the ref as part of
the hash, so the cache directory is different for two different
branches of the same repository.

[-- Attachment #2: 0001-hydra-Fix-devel-manual-generation.patch --]
[-- Type: text/x-patch, Size: 5270 bytes --]

From e4ff26d3c17ee7be78db4138f41ffb4f50fc2fe0 Mon Sep 17 00:00:00 2001
From: Julien Lepiller <julien@lepiller.eu>
Date: Tue, 30 Jul 2019 18:00:36 +0200
Subject: [PATCH] hydra: Fix devel manual generation.

* hydra/modules/sysadmin/web.scm (static-web-site-configuration): Allow to
pass a custom cache directory.
(static-web-site-mcron-jobs): Pass it to build-program.
(build-program): Accept a cache-directory argument and use it to
customize the name of the checkout directory.
* hydra/berlin.scm (static-web-site-service-type): Use a custom cache-directory
for the manuals of both stable and 'master' releases.
---
 hydra/berlin.scm               | 20 +++++++++-----------
 hydra/modules/sysadmin/web.scm | 12 ++++++++++--
 2 files changed, 19 insertions(+), 13 deletions(-)

diff --git a/hydra/berlin.scm b/hydra/berlin.scm
index 70a917d..513e975 100644
--- a/hydra/berlin.scm
+++ b/hydra/berlin.scm
@@ -2,7 +2,7 @@
 ;; hosted at the MDC.
 
 (use-modules (gnu) (guix) (sysadmin services) (sysadmin people) (sysadmin dns)
-             (sysadmin web))
+             (sysadmin web) (guix git))
 (use-service-modules base databases dns monitoring networking admin shepherd)
 (use-package-modules admin certs emacs linux monitoring
                      ssh tls vim package-management
@@ -217,23 +217,21 @@ fastcgi_param PHP_VALUE \"post_max_size = 16M
                        (git-ref '(branch . "version-1.0.1"))
                        (directory "/srv/guix-manual")
                        (build-file "doc/build.scm")
-		       (environment-variables
-			 '(("GUIX_MANUAL_VERSION" . "1.0.1")
+                       (cache-directory (string-append %repository-cache-directory
+                                                       "/guix-stable-manual"))
+                       (environment-variables
+                         '(("GUIX_MANUAL_VERSION" . "1.0.1")
                            ("GUIX_WEB_SITE_URL" . "/")))))
 
              ;; Manual for 'master'.
              (service static-web-site-service-type
                       (static-web-site-configuration
-                       ;; XXX: Use a different URL to work around the fact
-                       ;; that (guix git) would use a same-named checkout
-                       ;; directory for 'master' and for the branch above.
-                       ;; Since both mcron jobs run at the same time, they
-                       ;; would end up using one branch or the other, in a
-                       ;; non-deterministic way.
-                       (git-url "https://git.sv.gnu.org/git/guix.git")
+                       (git-url "https://git.savannah.gnu.org/git/guix.git")
                        (directory "/srv/guix-manual-devel")
+                       (cache-directory (string-append %repository-cache-directory
+                                                       "/guix-master-manual"))
                        (build-file "doc/build.scm")
-		       (environment-variables
+                       (environment-variables
                         '(("GUIX_WEB_SITE_URL" . "/")))))
 
              (frontend-services %sysadmins
diff --git a/hydra/modules/sysadmin/web.scm b/hydra/modules/sysadmin/web.scm
index b63f160..225b1b4 100644
--- a/hydra/modules/sysadmin/web.scm
+++ b/hydra/modules/sysadmin/web.scm
@@ -44,7 +44,8 @@
                         (file "guix.scm")
                         (ref '(branch . "master"))
                         (name "build-program")
-                        (environment-variables '()))
+                        (environment-variables '())
+                        (cache-directory #f))
   "Return a program that pulls code from URL, builds it by loading FILE from
 that checkout (usually a 'guix.scm' file), and registers the result as
 ROOT (an indirect GC root) upon success.  FILE is loaded in a content where
@@ -67,7 +68,10 @@ that's built with Haunt or similar."
                 (return #t))))
 
           (define-values (checkout commit)
-            (update-cached-checkout #$url #:ref '#$ref))
+            (update-cached-checkout #$url #:ref '#$ref
+                                    #@$(if cache-directory
+                                           `(#:cache-directory ,cache-directory)
+                                           '())))
 
           (define obj
             (let ((variables '#$environment-variables))
@@ -102,6 +106,8 @@ that's built with Haunt or similar."
                (default "guix.scm"))
   (environment-variables static-web-site-configuration-environment-variable
                          (default '()))
+  (cache-directory static-web-site-configuration-cache-directory
+                   (default #f))
   (directory   static-web-site-configuration-directory
                (default "/srv/www")))
 
@@ -113,6 +119,8 @@ that's built with Haunt or similar."
                    #:ref (static-web-site-configuration-git-ref config)
                    #:environment-variables
                    (static-web-site-configuration-environment-variable config)
+                   #:cache-directory
+                   (static-web-site-configuration-cache-directory config)
                    #:name (string-append
                            "update-"
                            (basename
-- 
2.22.0

bug#36781: Website manual generation stopped

[Ludovic Courtès]

Hi Julien,

Julien Lepiller <julien@lepiller.eu> skribis:

> Le Fri, 26 Jul 2019 00:54:32 +0200,
> Ludovic Courtès <ludo@gnu.org> a écrit :

[...]

>> Indeed it fails like this:
>> 
>> --8<---------------cut here---------------start------------->8---
>> ludo@berlin ~$ sudo su - static-web-site
>> -c /gnu/store/9w4bbd6gqya2g9zvwgs6qab6aqgbjbd3-update-guix-manual-devel
>> Backtrace: 7 (primitive-load
>> "/gnu/store/9w4bbd6gqya2g9zvwgs6qab6aqg…") In ice-9/eval.scm:
>>     619:8  6 (_ #f)
>>    626:19  5 (_ #<directory (guile-user) 2366140>)
>> In unknown file:
>>            4 (_ #<procedure 2dd2960 at ice-9/eval.scm:330:13 ()> #<…>
>> …) In guix/git.scm:
>>    240:29  3 (update-cached-checkout "https://git.sv.gnu.org/git/gu…"
>> …) In ice-9/boot-9.scm:
>>     841:4  2 (with-throw-handler _ _ _)
>> In git/clone.scm:
>>      41:8  1 (_ _ _ _)
>> In ice-9/boot-9.scm:
>>    752:25  0 (dispatch-exception _ _ _)
>> 
>> ice-9/boot-9.scm:752:25: In procedure dispatch-exception:
>> Git error: cross host redirect not allowed
>> --8<---------------cut here---------------end--------------->8---
>> 
>> So I think we have to change the repo URL in berlin.scm.
>> 
>> Ludo’.
>
> One way I can see to solve that issue is to specify a custome cache
> directory name, instead of the default one, which is a hash of the url.
> The reason why we use git.sv.gnu.org instead of git.savannah.gnu.org
> is that otherwise both repos have the same cache directory, so one wins
> over the other. But that hack doesn't scale if we want to generate more
> than two manual versions.
>
> Attached is a patch that adds a cache-directory field to the
> static-website-configuration record.

OK.

> Another solution is to fix (guix git) to also add the ref as part of
> the hash, so the cache directory is different for two different
> branches of the same repository.

I thought about doing that.  It’d work but it’d also be slightly
wasteful since branches of a repo typically have a lot in common.

Another option would be to compute the cache directory name like you
write, but only in the (sysadmin web) module.

WDYT?

Thanks,
Ludo’.

bug#36781: Website manual generation stopped

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 53 bytes --]

Closing since the website manual is now up to date :)

[-- Attachment #2: Type: text/html, Size: 94 bytes --]