* Experimenting with reduced gcc-lib
@ 2022-12-08 21:32 Julien Lepiller
2022-12-15 14:34 ` Ludovic Courtès
0 siblings, 1 reply; 4+ messages in thread
From: Julien Lepiller @ 2022-12-08 21:32 UTC (permalink / raw)
To: guix-devel
[-- Attachment #1: Type: text/plain, Size: 1512 bytes --]
Hi Guix!
I had a look at gcc to try and reduce its footprint. I figured gcc-lib
contained shared and static libraries as well as support libraries for
gdb, many headers and compiler runtime, that are taking a lot of space
when they are not needed at runtime. Reducing its 33.4 MiB would help
create smaller guix packs, have smaller system and profile closures,
etc.
I figured gcc has a --with-slibdir option that is advertised as being
useful to specify a directory for the shared libraries. I decided to
use it and create a new output for the gcc package, shared-lib. In
fact, this option only installs libgcc_s.so in that output, and that's
the only thing required by most packages.
Attached is a patch that divides gcc-lib into gcc-static-lib and
gcc-shared-lib (not very good names, gcc-static-lib contains shared
libs too).
Before the patch we get:
guix size perl
total: 147.7 MiB
After the patch:
./pre-inst-env guix size perl
total: 114.5 MiB
This is because before the patch, gcc-lib is 33.4 MiB, and after the
patch gcc-shared-lib is only 0.2 MiB.
I don't want to push that patch as is, because libstdc++.so is still in
the gcc-static-lib output and will bring all the useless stuff with it.
C++ programs will bring back the remaining 33.2 MiB.
Another possible target is glibc, since the only useful bits in there
are libc.so and ld-linux.so (I think?). This could reduce the closure
of our packs by another ~40 MiB (I think our systems would still need
most of the content of this package).
[-- Attachment #2: 0001-gnu-gcc-Separate-lib-output.patch --]
[-- Type: text/x-patch, Size: 7976 bytes --]
From bff8567b0770455397d44f1ed304a67681b472c9 Mon Sep 17 00:00:00 2001
From: Julien Lepiller <julien@lepiller.eu>
Date: Thu, 8 Dec 2022 22:29:01 +0100
Subject: [PATCH] gnu: gcc: Separate lib output.
* gnu/packages/gcc.scm (gcc-4.7): Replace `lib' output with `static-lib'
and `shared-lib'.
* gnu/packages/commencement.scm (gcc-boot0): Update accordingly.
---
gnu/packages/commencement.scm | 6 +++---
gnu/packages/gcc.scm | 39 +++++++++++++++++++++++++----------
gnu/tests/mail.trs | 0
3 files changed, 31 insertions(+), 14 deletions(-)
create mode 100644 gnu/tests/mail.trs
diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index b4566b41cc..f3fce1750f 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2853,7 +2853,7 @@ (define gcc-boot0
(lambda _ #t))))
(add-after 'install 'symlink-libgcc_eh
(lambda* (#:key outputs #:allow-other-keys)
- (let ((out (assoc-ref outputs "lib")))
+ (let ((out (assoc-ref outputs "static-lib")))
;; Glibc wants to link against libgcc_eh, so provide
;; it.
(with-directory-excursion
@@ -3417,7 +3417,7 @@ (define glibc-final
;; if 'allowed-references' were per-output.
(arguments
`(#:allowed-references
- (,(gexp-input gcc-boot0 "lib")
+ (,(gexp-input gcc-boot0 "shared-lib")
,(kernel-headers-boot0)
,static-bash-for-glibc
,@(if (hurd-system?)
@@ -3523,7 +3523,7 @@ (define gcc-final
`(#:guile ,%bootstrap-guile
#:implicit-inputs? #f
- #:allowed-references ("out" "lib" ,zlib-final
+ #:allowed-references ("out" "shared-lib" "static-lib" ,zlib-final
,glibc-final ,static-bash-for-glibc)
;; Things like libasan.so and libstdc++.so NEED ld.so for some
diff --git a/gnu/packages/gcc.scm b/gnu/packages/gcc.scm
index bb154cac62..010c6102a0 100644
--- a/gnu/packages/gcc.scm
+++ b/gnu/packages/gcc.scm
@@ -106,8 +106,16 @@ (define-public gcc-4.7
'("CC" "CXX" "LD" "AR" "NM" "OBJDUMP" "RANLIB" "STRIP")
'("gcc" "g++" "ld" "ar" "nm" "objdump" "ranlib" "strip"))
'()))))
+ (static-libdir
+ (let ((base '(or (assoc-ref outputs "static-lib")
+ (assoc-ref outputs "out"))))
+ (lambda ()
+ ;; Return the directory that contains lib/libgcc_s.so et al.
+ (if (%current-target-system)
+ `(string-append ,base "/" ,(%current-target-system))
+ base))))
(libdir
- (let ((base '(or (assoc-ref outputs "lib")
+ (let ((base '(or (assoc-ref outputs "shared-lib")
(assoc-ref outputs "out"))))
(lambda ()
;; Return the directory that contains lib/libgcc_s.so et al.
@@ -138,6 +146,12 @@ (define-public gcc-4.7
,(string-append "--with-gxx-include-dir="
(assoc-ref %outputs "out")
"/include/c++")
+ ,(string-append "--libdir="
+ (assoc-ref %outputs "static-lib")
+ "/lib")
+ ,(string-append "--with-slibdir="
+ (assoc-ref %outputs "shared-lib")
+ "/lib")
,(let ((libc (assoc-ref %build-inputs "libc")))
(if libc
@@ -170,7 +184,8 @@ (define-public gcc-4.7
;; Separate out the run-time support libraries because all the
;; dynamic-linked objects depend on it.
(outputs '("out" ;commands, etc. (60+ MiB)
- "lib" ;libgcc_s, libgomp, etc. (15+ MiB)
+ "shared-lib" ;libgcc_s, libgomp, etc. (5+ MiB)
+ "static-lib" ;object code libraries and internal data files of GCC. (10+ MiB)
"debug")) ;debug symbols of run-time libraries
(inputs (list gmp mpfr mpc libelf zlib))
@@ -216,6 +231,7 @@ (define-public gcc-4.7
(add-before 'configure 'pre-configure
(lambda* (#:key inputs outputs #:allow-other-keys)
(let ((libdir ,(libdir))
+ (static-libdir ,(static-libdir))
(libc (assoc-ref inputs "libc")))
(when libc
;; The following is not performed for `--without-headers'
@@ -257,8 +273,8 @@ (define-public gcc-4.7
;; below, make sure to update the relevant code in
;; %gcc-static package as needed.
(format #f "#define GNU_USER_TARGET_LIB_SPEC \
-\"-L~a/lib %{!static:-rpath=~a/lib %{!static-libgcc:-rpath=~a/lib -lgcc_s}} \" ~a"
- libc libc libdir suffix))
+\"-L~a/lib %{!static:-rpath=~a/lib -L~a/lib %{!static-libgcc:-rpath=~a/lib -lgcc_s}} \" ~a"
+ libc libc libdir libdir suffix))
(("#define GNU_USER_TARGET_STARTFILE_SPEC.*$" line)
(format #f "#define STANDARD_STARTFILE_PREFIX_1 \"~a/lib\"
#define STANDARD_STARTFILE_PREFIX_2 \"\"
@@ -324,19 +340,20 @@ (define-public gcc-4.7
;; and <http://bugs.gnu.org/20358>.
(substitute* "libstdc++-v3/src/Makefile.in"
(("^OPT_LDFLAGS = ")
- "OPT_LDFLAGS = -Wl,-rpath=$(libdir) "))
+ (string-append
+ "OPT_LDFLAGS = -Wl,-rpath=" libdir " ")))
- ;; Move libstdc++*-gdb.py to the "lib" output to avoid a
- ;; circularity between "out" and "lib". (Note:
+ ;; Move libstdc++*-gdb.py to the "static-lib" output to avoid a
+ ;; circularity between "out" and "shared-lib". (Note:
;; --with-python-dir is useless because it imposes $(prefix) as
;; the parent directory.)
(substitute* "libstdc++-v3/python/Makefile.in"
(("pythondir = .*$")
- (string-append "pythondir = " libdir "/share"
+ (string-append "pythondir = " static-libdir "/share"
"/gcc-$(gcc_version)/python\n")))
;; Avoid another circularity between the outputs: this #define
- ;; ends up in auto-host.h in the "lib" output, referring to
+ ;; ends up in auto-host.h in the "static-lib" output, referring to
;; "out". (This variable is used to augment cpp's search path,
;; but there's nothing useful to look for here.)
(substitute* "gcc/config.in"
@@ -917,7 +934,7 @@ (define* (custom-gcc gcc name languages
(name name)
(outputs (if separate-lib-output?
(package-outputs gcc)
- (delete "lib" (package-outputs gcc))))
+ (delete "shared-lib" (delete "static-lib" (package-outputs gcc)))))
(native-search-paths search-paths)
(properties (alist-delete 'hidden? (package-properties gcc)))
(arguments
@@ -979,7 +996,7 @@ (define-public (make-libgccjit gcc)
(package
(inherit gcc)
(name "libgccjit")
- (outputs (delete "lib" (package-outputs gcc)))
+ (outputs (delete "static-lib" (delete "shared-lib" (package-outputs gcc))))
(properties (alist-delete 'hidden? (package-properties gcc)))
(arguments
(substitute-keyword-arguments (package-arguments gcc)
diff --git a/gnu/tests/mail.trs b/gnu/tests/mail.trs
new file mode 100644
index 0000000000..e69de29bb2
--
2.38.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: Experimenting with reduced gcc-lib
2022-12-08 21:32 Experimenting with reduced gcc-lib Julien Lepiller
@ 2022-12-15 14:34 ` Ludovic Courtès
2022-12-15 16:59 ` Julien Lepiller
0 siblings, 1 reply; 4+ messages in thread
From: Ludovic Courtès @ 2022-12-15 14:34 UTC (permalink / raw)
To: Julien Lepiller; +Cc: guix-devel
Hello!
Julien Lepiller <julien@lepiller.eu> skribis:
> Before the patch we get:
>
> guix size perl
> total: 147.7 MiB
>
> After the patch:
>
> ./pre-inst-env guix size perl
> total: 114.5 MiB
Nice!
> I don't want to push that patch as is, because libstdc++.so is still in
> the gcc-static-lib output and will bring all the useless stuff with it.
> C++ programs will bring back the remaining 33.2 MiB.
It wouldn’t be worse than the current situation though?
> Another possible target is glibc, since the only useful bits in there
> are libc.so and ld-linux.so (I think?). This could reduce the closure
> of our packs by another ~40 MiB (I think our systems would still need
> most of the content of this package).
‘share/i18n’ is 16 MiB and ‘lib/iconv’ is 8 MiB. Now, separating them
without introducing loss of functionality may be tricky.
> From bff8567b0770455397d44f1ed304a67681b472c9 Mon Sep 17 00:00:00 2001
> From: Julien Lepiller <julien@lepiller.eu>
> Date: Thu, 8 Dec 2022 22:29:01 +0100
> Subject: [PATCH] gnu: gcc: Separate lib output.
>
> * gnu/packages/gcc.scm (gcc-4.7): Replace `lib' output with `static-lib'
> and `shared-lib'.
> * gnu/packages/commencement.scm (gcc-boot0): Update accordingly.
> ---
> gnu/packages/commencement.scm | 6 +++---
> gnu/packages/gcc.scm | 39 +++++++++++++++++++++++++----------
> gnu/tests/mail.trs | 0
> 3 files changed, 31 insertions(+), 14 deletions(-)
> create mode 100644 gnu/tests/mail.trs
Oops?
> (outputs '("out" ;commands, etc. (60+ MiB)
> - "lib" ;libgcc_s, libgomp, etc. (15+ MiB)
> + "shared-lib" ;libgcc_s, libgomp, etc. (5+ MiB)
> + "static-lib" ;object code libraries and internal data files of GCC. (10+ MiB)
The convention is to use “lib” for shared libraries and “static” for
static libraries.
Is libgomp in “lib”? Is it found when linking with ‘-fopenmp’?
Thanks for this work; we really need to do something about package
sizes!
Ludo’.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Experimenting with reduced gcc-lib
2022-12-15 14:34 ` Ludovic Courtès
@ 2022-12-15 16:59 ` Julien Lepiller
2022-12-19 21:14 ` Ludovic Courtès
0 siblings, 1 reply; 4+ messages in thread
From: Julien Lepiller @ 2022-12-15 16:59 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guix-devel
Le 15 décembre 2022 15:34:02 GMT+01:00, "Ludovic Courtès" <ludo@gnu.org> a écrit :
>Hello!
>
>Julien Lepiller <julien@lepiller.eu> skribis:
>
>> Before the patch we get:
>>
>> guix size perl
>> total: 147.7 MiB
>>
>> After the patch:
>>
>> ./pre-inst-env guix size perl
>> total: 114.5 MiB
>
>Nice!
>
>> I don't want to push that patch as is, because libstdc++.so is still in
>> the gcc-static-lib output and will bring all the useless stuff with it.
>> C++ programs will bring back the remaining 33.2 MiB.
>
>It wouldn’t be worse than the current situation though?
>
>> Another possible target is glibc, since the only useful bits in there
>> are libc.so and ld-linux.so (I think?). This could reduce the closure
>> of our packs by another ~40 MiB (I think our systems would still need
>> most of the content of this package).
>
>‘share/i18n’ is 16 MiB and ‘lib/iconv’ is 8 MiB. Now, separating them
>without introducing loss of functionality may be tricky.
>
>> From bff8567b0770455397d44f1ed304a67681b472c9 Mon Sep 17 00:00:00 2001
>> From: Julien Lepiller <julien@lepiller.eu>
>> Date: Thu, 8 Dec 2022 22:29:01 +0100
>> Subject: [PATCH] gnu: gcc: Separate lib output.
>>
>> * gnu/packages/gcc.scm (gcc-4.7): Replace `lib' output with `static-lib'
>> and `shared-lib'.
>> * gnu/packages/commencement.scm (gcc-boot0): Update accordingly.
>> ---
>> gnu/packages/commencement.scm | 6 +++---
>> gnu/packages/gcc.scm | 39 +++++++++++++++++++++++++----------
>> gnu/tests/mail.trs | 0
>> 3 files changed, 31 insertions(+), 14 deletions(-)
>> create mode 100644 gnu/tests/mail.trs
>
>Oops?
Oopsie :)
>
>> (outputs '("out" ;commands, etc. (60+ MiB)
>> - "lib" ;libgcc_s, libgomp, etc. (15+ MiB)
>> + "shared-lib" ;libgcc_s, libgomp, etc. (5+ MiB)
>> + "static-lib" ;object code libraries and internal data files of GCC. (10+ MiB)
>
>The convention is to use “lib” for shared libraries and “static” for
>static libraries.
But using lib introduces a --libdir argument which is used to install static libs…
>
>Is libgomp in “lib”? Is it found when linking with ‘-fopenmp’?
Well, shared-lib should be renaimed into libgcc, because it's the only library there.
Libgomp amd all other libs are in static-lib (which is not well named either ^^'). It will be found I think but not tested (it find libstdc++ in the same directory).
>
>Thanks for this work; we really need to do something about package
>sizes!
>
>Ludo’.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Experimenting with reduced gcc-lib
2022-12-15 16:59 ` Julien Lepiller
@ 2022-12-19 21:14 ` Ludovic Courtès
0 siblings, 0 replies; 4+ messages in thread
From: Ludovic Courtès @ 2022-12-19 21:14 UTC (permalink / raw)
To: Julien Lepiller; +Cc: guix-devel
Hi!
Julien Lepiller <julien@lepiller.eu> skribis:
> Le 15 décembre 2022 15:34:02 GMT+01:00, "Ludovic Courtès" <ludo@gnu.org> a écrit :
>>> (outputs '("out" ;commands, etc. (60+ MiB)
>>> - "lib" ;libgcc_s, libgomp, etc. (15+ MiB)
>>> + "shared-lib" ;libgcc_s, libgomp, etc. (5+ MiB)
>>> + "static-lib" ;object code libraries and internal data files of GCC. (10+ MiB)
>>
>>The convention is to use “lib” for shared libraries and “static” for
>>static libraries.
>
> But using lib introduces a --libdir argument which is used to install static libs…
I see. Then maybe we should override the ‘configure’ phase, or repeat
‘--libdir’ in the hope that the second one wins?
>>Is libgomp in “lib”? Is it found when linking with ‘-fopenmp’?
>
> Well, shared-lib should be renaimed into libgcc, because it's the only library there.
OK.
> Libgomp amd all other libs are in static-lib (which is not well named either ^^'). It will be found I think but not tested (it find libstdc++ in the same directory).
Heh. :-)
I think it would be great to have all the .a in a separate output.
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2022-12-19 21:15 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-12-08 21:32 Experimenting with reduced gcc-lib Julien Lepiller
2022-12-15 14:34 ` Ludovic Courtès
2022-12-15 16:59 ` Julien Lepiller
2022-12-19 21:14 ` Ludovic Courtès
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).