[bug#30647] [PATCH] guix build: Support '--remote-log-file=PACKAGE'.

[bug#30647] [PATCH] guix build: Support '--remote-log-file=PACKAGE'.

[Oleg Pykhalov]

‘--remote-log-file’ allows to get a URL for a build log file on a substitute
server regardless is it built locally.  ‘--log-file’ returns always local
build log file.

* guix/scripts/build.scm (show-build-log): Split function.
(show-remote-build-log): New function.
(guix-build): Add this.
* doc/guix.texi (Invoking guix build): Document this.
---
 doc/guix.texi          | 18 +++++++++---------
 guix/scripts/build.scm | 31 ++++++++++++++++++++++---------
 2 files changed, 31 insertions(+), 18 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 24db16761..782e532ce 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5812,9 +5812,8 @@ more on GC roots.
 
 @item --log-file
 @cindex build logs, access
-Return the build log file names or URLs for the given
-@var{package-or-derivation}, or raise an error if build logs are
-missing.
+Return the build log file names @var{package-or-derivation}, or raise an
+error if build logs are missing.
 
 This works regardless of how packages or derivations are specified.  For
 instance, the following invocations are equivalent:
@@ -5826,15 +5825,16 @@ guix build --log-file guile
 guix build --log-file -e '(@@ (gnu packages guile) guile-2.0)'
 @end example
 
-If a log is unavailable locally, and unless @code{--no-substitutes} is
-passed, the command looks for a corresponding log on one of the
-substitute servers (as specified with @code{--substitute-urls}.)
+@item --remote-log-file
+@cindex build logs, access
+
+Same as @code{--log-file} but on one of the substitute servers (as
+specified with @code{--substitute-urls}.
 
-So for instance, imagine you want to see the build log of GDB on MIPS,
-but you are actually on an @code{x86_64} machine:
+For example, you want to see the build log of GDB on MIPS:
 
 @example
-$ guix build --log-file gdb -s mips64el-linux
+$ guix build --remote-log-file gdb -s mips64el-linux
 https://hydra.gnu.org/log/@dots{}-gdb-7.10
 @end example
 
diff --git a/guix/scripts/build.scm b/guix/scripts/build.scm
index 57f2d82c5..c45271e50 100644
--- a/guix/scripts/build.scm
+++ b/guix/scripts/build.scm
@@ -601,6 +601,9 @@ must be one of 'package', 'all', or 'transitive'~%")
          (option '("log-file") #f #f
                  (lambda (opt name arg result)
                    (alist-cons 'log-file? #t result)))
+         (option '("remote-log-file") #f #f
+                 (lambda (opt name arg result)
+                   (alist-cons 'remote-log-file? #t result)))
 
          (append %transformation-options
                  %standard-build-options)))
@@ -691,15 +694,20 @@ package '~a' has no source~%")
                 (map (cut transform store <>)
                      (options->things-to-build opts)))))
 
-(define (show-build-log store file urls)
-  "Show the build log for FILE, falling back to remote logs from URLS if
-needed."
-  (let ((log (or (log-file store file)
-                 (log-url store file #:base-urls urls))))
+(define (show-build-log store file)
+  "Show the build log for FILE."
+  (let ((log (log-file store file)))
     (if log
         (format #t "~a~%" log)
         (leave (G_ "no build log for '~a'~%") file))))
 
+(define (show-remote-build-log store file urls)
+  "Show the remote build log for FILE from URLS."
+  (let ((log (log-url store file #:base-urls urls)))
+    (if log
+        (format #t "~a~%" log)
+        (leave (G_ "no remote build log for '~a'~%") file))))
+
 \f
 ;;;
 ;;; Entry point.
@@ -713,6 +721,9 @@ needed."
   (define quiet?
     (assoc-ref opts 'quiet?))
 
+  (define (derivation-file-names drv items)
+    (delete-duplicates (append (map derivation-file-name drv) items)))
+
   (with-error-handling
     ;; Ask for absolute file names so that .drv file names passed from the
     ;; user to 'read-derivation' are absolute when it returns.
@@ -744,6 +755,7 @@ needed."
                                     opts)))
 
             (unless (or (assoc-ref opts 'log-file?)
+                        (assoc-ref opts 'remote-log-file?)
                         (assoc-ref opts 'derivations-only?))
               (show-what-to-build store drv
                                   #:use-substitutes?
@@ -752,10 +764,11 @@ needed."
                                   #:mode mode))
 
             (cond ((assoc-ref opts 'log-file?)
-                   (for-each (cut show-build-log store <> urls)
-                             (delete-duplicates
-                              (append (map derivation-file-name drv)
-                                      items))))
+                   (for-each (cut show-build-log store <>)
+                             (derivation-file-names drv items)))
+                  ((assoc-ref opts 'remote-log-file?)
+                   (for-each (cut show-remote-build-log store <> urls)
+                             (derivation-file-names drv items)))
                   ((assoc-ref opts 'derivations-only?)
                    (format #t "~{~a~%~}" (map derivation-file-name drv))
                    (for-each (cut register-root store <> <>)
-- 
2.16.1

[bug#30647] [PATCH] guix build: Support '--remote-log-file=PACKAGE'.

[Ludovic Courtès]

Hello Oleg,

Oleg Pykhalov <go.wigust@gmail.com> skribis:

> ‘--remote-log-file’ allows to get a URL for a build log file on a substitute
> server regardless is it built locally.  ‘--log-file’ returns always local
> build log file.

What did you think of having ‘--log-file’ transparently fall back to
searching for log files on substitute servers?

I find it handy, but also wondered if it might surprise users that such
a trivially-looking option connects to external servers.  I thought
about having it print something when it does so.  Would address your
concerns?

Thanks,
Ludo’.

[bug#30647] [PATCH] guix build: Support '--remote-log-file=PACKAGE'.

[Oleg Pykhalov]

[-- Attachment #1: Type: text/plain, Size: 1430 bytes --]

Hello Ludovic,

ludo@gnu.org (Ludovic Courtès) writes:

> Oleg Pykhalov <go.wigust@gmail.com> skribis:
>
>> ‘--remote-log-file’ allows to get a URL for a build log file on a substitute
>> server regardless is it built locally.  ‘--log-file’ returns always local
>> build log file.
>
> What did you think of having ‘--log-file’ transparently fall back to
> searching for log files on substitute servers?

Sorry, I don't understand the question.  Does the “fall back” mean the
behavior before a patch?

> I find it handy, but also wondered if it might surprise users that such
> a trivially-looking option connects to external servers.  I thought
> about having it print something when it does so.  Would address your
> concerns?

Do you mean always connect to the external server and print a URL for a
log file in addition to path of local log file?  I don't think mixing
those in one output is good, because for example you cannot do like:
--8<---------------cut here---------------start------------->8---
diff -u <(guix build --log-file hello) <(guix build --remote-log-file hello)
--8<---------------cut here---------------end--------------->8---


As a better approach in addition to ‘--no-substitutes’, maybe we could
implement ‘--only-substitutes’ (as I remember Nix has it)?  Such flag
will return a remote log file and will avoid building packages locally.

Oleg.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[bug#30647] [PATCH] guix build: Support '--remote-log-file=PACKAGE'.

[Ludovic Courtès]

Hello,

Oleg Pykhalov <go.wigust@gmail.com> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Oleg Pykhalov <go.wigust@gmail.com> skribis:
>>
>>> ‘--remote-log-file’ allows to get a URL for a build log file on a substitute
>>> server regardless is it built locally.  ‘--log-file’ returns always local
>>> build log file.
>>
>> What did you think of having ‘--log-file’ transparently fall back to
>> searching for log files on substitute servers?
>
> Sorry, I don't understand the question.  Does the “fall back” mean the
> behavior before a patch?

To put it differently: what do you dislike about the current behavior?

>> I find it handy, but also wondered if it might surprise users that such
>> a trivially-looking option connects to external servers.  I thought
>> about having it print something when it does so.  Would address your
>> concerns?
>
> Do you mean always connect to the external server and print a URL for a
> log file in addition to path of local log file?

No no: keep the current behavior, but print something when we’re looking
for a remote log file (currently it silently checks whether the remote
log file is available.)

> I don't think mixing those in one output is good, because for example
> you cannot do like:
>
> diff -u <(guix build --log-file hello) <(guix build --remote-log-file hello)

I see.  I guess I’ve never wanted that, or rather, when I do, I
explicitly wget the remote log file.  :-)

So I guess I’m unconvinced about the need for a separate
‘--remote-log-file’ option.

What do people think?  Ricardo?

> As a better approach in addition to ‘--no-substitutes’, maybe we could
> implement ‘--only-substitutes’ (as I remember Nix has it)?  Such flag
> will return a remote log file and will avoid building packages locally.

That could be an option, but that’s much more work (not limited to log
file handling.)

Thanks,
Ludo’.

[bug#30647] [PATCH] guix build: Support '--remote-log-file=PACKAGE'.

[Tobias Geerinckx-Rice]

Hullo,

On 2018-02-28 23:13, ludo@gnu.org wrote:
> What did you think of having ‘--log-file’ transparently fall back to
> searching for log files on substitute servers?
> 
> I find it handy, but also wondered if it might surprise users that such
> a trivially-looking option connects to external servers.

It would pleasantly surprise me :-)

(...well, not really ‘surprise’ — just more Guixy network-transparent 
goodness.)

TBH, I don't see the difference between this & all other actions that 
already connect to external servers.

Is this different? Will it ignore ’--no-substitutes’, or ping servers 
outside of user-approved substitute-urls? Does it need to?

> I thought about having it print something when it does so.

Like a progress bar? Seems more eye-catching than yet another warning. 
Or something more permanent in/next to the downloaded log?

Kind regards,

T G-R

Sent from a Web browser. Excuse or enjoy my brevity.

[bug#30647] [PATCH] guix build: Support '--remote-log-file=PACKAGE'.

[Oleg Pykhalov]

[-- Attachment #1: Type: text/plain, Size: 2166 bytes --]

ludo@gnu.org (Ludovic Courtès) writes:

>>>> ‘--remote-log-file’ allows to get a URL for a build log file on a substitute
>>>> server regardless is it built locally.  ‘--log-file’ returns always local
>>>> build log file.
>>>
>>> What did you think of having ‘--log-file’ transparently fall back to
>>> searching for log files on substitute servers?
>
> To put it differently: what do you dislike about the current behavior?

Suppose package build failed locally.  I want to receive a log from a
remote server.  I could do it manually by:

1.  Removing local failed log.
2.  ‘wget’, but I need to know a URL.
3.  Hydra web interface, which is slow (especially multiple packages).

> No no: keep the current behavior, but print something when we’re looking
> for a remote log file (currently it silently checks whether the remote
> log file is available.)

Still not clear to me.  If ‘guix --log-file’ checks for a remote log
file, then it gets a valid URL to a remote build log file for free,
doesn't it?

>> I don't think mixing those in one output is good, because for example
>> you cannot do like:
>>
>> diff -u <(guix build --log-file hello) <(guix build --remote-log-file hello)
>
> I see.  I guess I’ve never wanted that, or rather, when I do, I
> explicitly wget the remote log file.  :-)

Could I ask What's your workflow for ‘wget’?

> So I guess I’m unconvinced about the need for a separate
> ‘--remote-log-file’ option.
>
> What do people think?  Ricardo?

Maybe CC him?  Or is it a bad etiquette for a mailing list, because he
is subscribed?

>> As a better approach in addition to ‘--no-substitutes’, maybe we could
>> implement ‘--only-substitutes’ (as I remember Nix has it)?  Such flag
>> will return a remote log file and will avoid building packages locally.
>
> That could be an option, but that’s much more work (not limited to log
> file handling.)

Yes, but benefits (especially avoid building packages locally) are
worth.

If you don't agree with the patch, I'll not complain and will try to
work on ‘--only-substitutes’.  :-)

Oleg.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[bug#30647] [PATCH] guix build: Support '--remote-log-file=PACKAGE'.

[Ludovic Courtès]

Hey Tobias,

Tobias Geerinckx-Rice <me@tobias.gr> skribis:

> On 2018-02-28 23:13, ludo@gnu.org wrote:
>> What did you think of having ‘--log-file’ transparently fall back to
>> searching for log files on substitute servers?
>>
>> I find it handy, but also wondered if it might surprise users that such
>> a trivially-looking option connects to external servers.
>
> It would pleasantly surprise me :-)

Good.  :-)

> TBH, I don't see the difference between this & all other actions that
> already connect to external servers.
>
> Is this different? Will it ignore ’--no-substitutes’, or ping servers
> outside of user-approved substitute-urls? Does it need to?

It’s not really different, indeed.  (The only thing is that the list of
substitute URLs that is used by ‘--log-file’ can differ from that of
guix-daemon; but that’s a bug, really.)

>> I thought about having it print something when it does so.
>
> Like a progress bar? Seems more eye-catching than yet another
> warning. Or something more permanent in/next to the downloaded log?

‘--log-file’ just probes remote servers without actually downloading the
log, so a progress bar wouldn’t be useful.

I was thinking of a message like:

  info: looking for build log at https://…/log/…-foo

Ludo’.

[bug#30647] [PATCH] guix build: Support '--remote-log-file=PACKAGE'.

[Ludovic Courtès]

Oleg Pykhalov <go.wigust@gmail.com> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>>>>> ‘--remote-log-file’ allows to get a URL for a build log file on a substitute
>>>>> server regardless is it built locally.  ‘--log-file’ returns always local
>>>>> build log file.
>>>>
>>>> What did you think of having ‘--log-file’ transparently fall back to
>>>> searching for log files on substitute servers?
>>
>> To put it differently: what do you dislike about the current behavior?
>
> Suppose package build failed locally.  I want to receive a log from a
> remote server.  I could do it manually by:
>
> 1.  Removing local failed log.
> 2.  ‘wget’, but I need to know a URL.

The URL scheme is documented and easy to use (info "(guix) Invoking guix
publish").  That’s why I don’t find wget to be much of a problem.

>> No no: keep the current behavior, but print something when we’re looking
>> for a remote log file (currently it silently checks whether the remote
>> log file is available.)
>
> Still not clear to me.  If ‘guix --log-file’ checks for a remote log
> file, then it gets a valid URL to a remote build log file for free,
> doesn't it?

Correct.  See ‘log-url’ in (guix scripts build).

>>> I don't think mixing those in one output is good, because for example
>>> you cannot do like:
>>>
>>> diff -u <(guix build --log-file hello) <(guix build --remote-log-file hello)
>>
>> I see.  I guess I’ve never wanted that, or rather, when I do, I
>> explicitly wget the remote log file.  :-)
>
> Could I ask What's your workflow for ‘wget’?

Something like:

  $ guix build foo
  /gnu/store/xyz-foo
  $ wget -O log https://berlin.guixsd.org/log/xyz-foo
  
>>> As a better approach in addition to ‘--no-substitutes’, maybe we could
>>> implement ‘--only-substitutes’ (as I remember Nix has it)?  Such flag
>>> will return a remote log file and will avoid building packages locally.
>>
>> That could be an option, but that’s much more work (not limited to log
>> file handling.)
>
> Yes, but benefits (especially avoid building packages locally) are
> worth.
>
> If you don't agree with the patch, I'll not complain and will try to
> work on ‘--only-substitutes’.  :-)

Heheh.  :-)

IIRC, --only-substitutes in Nix is used together with --upgrade, no?
Some considered it questionable from a security standpoint when we last
discussed it: <https://bugs.gnu.org/26608>.

Thanks,
Ludo’.