[bug#37269] [PATCH] build-system/asdf: Add option to compress programs.

[bug#37269] [PATCH] build-system/asdf: Add option to compress programs.

[Pierre Neidhardt]

* guix/build/lisp-utils.scm (build-program): Add `compress?' key argument.
(generate-executable-for-system): Same.
(generate-executable): Same.
---
 guix/build/lisp-utils.scm | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/guix/build/lisp-utils.scm b/guix/build/lisp-utils.scm
index 97bc6197a3..c7a589c902 100644
--- a/guix/build/lisp-utils.scm
+++ b/guix/build/lisp-utils.scm
@@ -220,12 +220,19 @@ Also load TEST-ASD-FILE if necessary."
   "Return a lisp keyword for the concatenation of STRINGS."
   (string->symbol (apply string-append ":" strings)))
 
-(define (generate-executable-for-system type system)
+(define* (generate-executable-for-system type system #:key compress?)
   "Use LISP to generate an executable, whose TYPE can be 'asdf:image-op or
 'asdf:program-op.  The latter will always be standalone.  Depends on having
 created a \"SYSTEM-exec\" system which contains the entry program."
   (lisp-eval-program
    `((require :asdf)
+     ;; Only SBCL supports compression as of 2019-09-02.
+     ,(if (and compress? (string=? (%lisp-type) "sbcl"))
+          '(defmethod asdf:perform ((o asdf:image-op) (c asdf:system))
+                      (uiop:dump-image (asdf:output-file o c)
+                                       :executable t
+                                       :compression t))
+          '())
      (asdf:operate ',type ,(string-append system "-exec")))))
 
 (define (generate-executable-wrapper-system system dependencies)
@@ -339,6 +346,7 @@ which are not nested."
                         (dependency-prefixes (list (library-output outputs)))
                         (dependencies (list (basename program)))
                         entry-program
+                        compress?
                         #:allow-other-keys)
   "Generate an executable program containing all DEPENDENCIES, and which will
 execute ENTRY-PROGRAM.  The result is placed in PROGRAM.  When executed, it
@@ -350,6 +358,7 @@ retained."
                        #:dependencies dependencies
                        #:dependency-prefixes dependency-prefixes
                        #:entry-program entry-program
+                       #:compress? compress?
                        #:type 'asdf:program-op)
   (let* ((name (basename program))
          (bin-directory (dirname program)))
@@ -382,6 +391,7 @@ DEPENDENCY-PREFIXES to ensure references to those libraries are retained."
                               dependency-prefixes
                               entry-program
                               type
+                              compress?
                               #:allow-other-keys)
   "Generate an executable by using asdf operation TYPE, containing whithin the
 image all DEPENDENCIES, and running ENTRY-PROGRAM in the case of an
@@ -405,7 +415,7 @@ references to those libraries are retained."
                `(((,bin-directory :**/ :*.*.*)
                   (,bin-directory :**/ :*.*.*)))))))
 
-    (generate-executable-for-system type name)
+    (generate-executable-for-system type name #:compress? compress?)
 
     (let* ((after-store-prefix-index
             (string-index out-file #\/
-- 
2.23.0

[bug#37269] Acknowledgement ([PATCH] build-system/asdf: Add option to compress programs.)

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 298 bytes --]

This patch could be used by StumpWM and Next browser to compress the
binary.
For Next, this compresses the binary from ~100 MiB to ~23 MiB.  There is
no real drawback, the startup time is still well under 100 ms.

Andy, Katherine, thoughts?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#37269] Acknowledgement ([PATCH] build-system/asdf: Add option to compress programs.)

[Katherine Cox-Buday]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> This patch could be used by StumpWM and Next browser to compress the
> binary. For Next, this compresses the binary from ~100 MiB to ~23 MiB.
> There is no real drawback, the startup time is still well under
> 100 ms.

This is a great idea and something I lost when I moved from
self-compiling SBCL to Guix's version!

I am also not aware of any drawbacks, and I love that you implemented
this as an option in the build-system. Should we default it to true?

Also, as an aside, I am tracking Next; thank you for this project!

-- 
Katherine

[bug#37269] Acknowledgement ([PATCH] build-system/asdf: Add option to compress programs.)

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 614 bytes --]

Hi,

> I am also not aware of any drawbacks, and I love that you implemented
> this as an option in the build-system. Should we default it to true?

SBCL does not do it by default, so maybe we should respect the default
behaviour of the compiler instead.

I'll go ahead and apply the patch then.  Update SBCL to 1.5.6 and use
CLISP or ECL to build it instead of CCL which is not bootstrappable.

> Also, as an aside, I am tracking Next; thank you for this project!

Thanks!  Lots of good stuff will come this month!
(1.3.1 released today!)

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#37269] Acknowledgement ([PATCH] build-system/asdf: Add option to compress programs.)

[Ludovic Courtès]

Hello!

Katherine Cox-Buday <cox.katherine.e@gmail.com> skribis:

> Pierre Neidhardt <mail@ambrevar.xyz> writes:
>
>> This patch could be used by StumpWM and Next browser to compress the
>> binary. For Next, this compresses the binary from ~100 MiB to ~23 MiB.
>> There is no real drawback, the startup time is still well under
>> 100 ms.
>
> This is a great idea and something I lost when I moved from
> self-compiling SBCL to Guix's version!
>
> I am also not aware of any drawbacks, and I love that you implemented
> this as an option in the build-system. Should we default it to true?

Compressed files are opaque to the garbage collector scanner and to
grafting, so we could end up in situations like:

  https://issues.guix.gnu.org/issue/33848

(Which would be nice to address, BTW! :-))

Thus, I think we’ll have keep compression turned off.

Ludo’.

[bug#37269] Acknowledgement ([PATCH] build-system/asdf: Add option to compress programs.)

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 644 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> Compressed files are opaque to the garbage collector scanner and to
> grafting, so we could end up in situations like:
>
>   https://issues.guix.gnu.org/issue/33848
>
> (Which would be nice to address, BTW! :-))
>
> Thus, I think we’ll have keep compression turned off.

Actually, in this case it doesn't.  I tried with Next and the resulting
package keeps all references to the Common Lisp libraries.
I believe that SBCL only compresses the image, not the sources (which are
embedded as well).

Anyways, I won't turn it off for now.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#37269] Acknowledgement ([PATCH] build-system/asdf: Add option to compress programs.)

[Ludovic Courtès]

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Compressed files are opaque to the garbage collector scanner and to
>> grafting, so we could end up in situations like:
>>
>>   https://issues.guix.gnu.org/issue/33848
>>
>> (Which would be nice to address, BTW! :-))
>>
>> Thus, I think we’ll have keep compression turned off.
>
> Actually, in this case it doesn't.  I tried with Next and the resulting
> package keeps all references to the Common Lisp libraries.
> I believe that SBCL only compresses the image, not the sources (which are
> embedded as well).

OK.  We’d need to see exactly what it does if we decide to enable
it—better safe than sorry!

Thanks,
Ludo’.