[bug#29181] [PATCH core-updates 0/1] Shrink Mesa

[bug#29181] [PATCH core-updates 0/1] Shrink Mesa

[Ludovic Courtès]

Hello Guix!

This patch divides the nar size of ‘mesa’ (as reported by ‘guix size’)
almost by a factor of four.  It does so by replacing hard links created
upon “make install” with symlinks.

The new post-install phase prints things like:

--8<---------------cut here---------------start------------->8---
starting phase `symlinks-instead-of-hard-links'
creating 7 symlinks to '/gnu/store/l1iwgjgp88snlzp9nafdpd0vvsqv7lsi-mesa-17.2.1/lib/dri/i915_dri.so'
creating 3 symlinks to '/gnu/store/l1iwgjgp88snlzp9nafdpd0vvsqv7lsi-mesa-17.2.1/lib/dri/i965_dri.so'
creating 1 symlinks to '/gnu/store/l1iwgjgp88snlzp9nafdpd0vvsqv7lsi-mesa-17.2.1/lib/dri/nouveau_drv_video.so'
creating 1 symlinks to '/gnu/store/l1iwgjgp88snlzp9nafdpd0vvsqv7lsi-mesa-17.2.1/lib/libXvMCnouveau.so.1.0.0'
creating 2 symlinks to '/gnu/store/l1iwgjgp88snlzp9nafdpd0vvsqv7lsi-mesa-17.2.1/lib/vdpau/libvdpau_nouveau.so.1.0.0'
phase `symlinks-instead-of-hard-links' succeeded after 0.0 seconds
--8<---------------cut here---------------end--------------->8---

The nar size does not reflect disk usage since deduplication would
recreate the hard links anyway, but it reflects how much we have to
transfer over the wire since the nar format does not represent hard
links.  A similar problem arose with Git a while back, see
<https://bugs.gnu.org/21949>.

I’d like to push to ‘core-updates’.

Thoughts?

Ludo’.

Ludovic Courtès (1):
  gnu: mesa: Use symlinks instead of hard links.

 gnu/packages/gl.scm | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

-- 
2.14.2

[bug#29181] [PATCH 1/1] gnu: mesa: Use symlinks instead of hard links.

[Ludovic Courtès]

This reduces the nar size (as shown by 'guix size') by 124 MiB, from
169 MiB to 45 MiB (almost divided by 4!).

* gnu/packages/gl.scm (mesa)[arguments]: Add #:modules.  Add
'symlinks-instead-of-hard-links' phase.
---
 gnu/packages/gl.scm | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/gnu/packages/gl.scm b/gnu/packages/gl.scm
index 5f96ab303..be0eddfdb 100644
--- a/gnu/packages/gl.scm
+++ b/gnu/packages/gl.scm
@@ -305,6 +305,10 @@ also known as DXTn or DXTC) for Mesa.")
                 "--enable-llvm"))         ; default is x86/x86_64 only
              (_
               '("--with-dri-drivers=nouveau,r200,radeon,swrast"))))
+       #:modules ((ice-9 match)
+                  (srfi srfi-1)
+                  (guix build utils)
+                  (guix build gnu-build-system))
        #:phases
        (modify-phases %standard-phases
          (add-after
@@ -339,6 +343,41 @@ also known as DXTn or DXTC) for Mesa.")
                  ;; egl_gallium support.
                  (("\"gbm_dri\\.so")
                   (string-append "\"" out "/lib/dri/gbm_dri.so")))
+               #t)))
+         (add-after 'install 'symlinks-instead-of-hard-links
+           (lambda* (#:key outputs #:allow-other-keys)
+             ;; All the drivers and gallium targets create hard links upon
+             ;; installation (search for "hardlink each megadriver instance"
+             ;; in the makefiles).  This is no good for us since we'd produce
+             ;; nars that contain several copies of these files.  Thus, turn
+             ;; them into symlinks, which saves ~124 MiB.
+             (let* ((out    (assoc-ref outputs "out"))
+                    (lib    (string-append out "/lib"))
+                    (files  (find-files lib
+                                        (lambda (file stat)
+                                          (and (string-contains file ".so")
+                                               (eq? 'regular
+                                                    (stat:type stat))))))
+                    (inodes (map (compose stat:ino stat) files)))
+               (for-each (lambda (inode)
+                           (match (filter-map (match-lambda
+                                                ((file ino)
+                                                 (and (= ino inode) file)))
+                                              (zip files inodes))
+                             ((_)
+                              #f)
+                             ((reference others ..1)
+                              (format #t "creating ~a symlinks to '~a'~%"
+                                      (length others) reference)
+                              (for-each delete-file others)
+                              (for-each (lambda (file)
+                                          (if (string=? (dirname file)
+                                                        (dirname reference))
+                                              (symlink (basename reference)
+                                                       file)
+                                              (symlink reference file)))
+                                        others))))
+                         (delete-duplicates inodes))
                #t))))))
     (home-page "https://mesa3d.org/")
     (synopsis "OpenGL implementation")
-- 
2.14.2

[bug#29181] [PATCH 1/1] gnu: mesa: Use symlinks instead of hard links.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 419 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> This reduces the nar size (as shown by 'guix size') by 124 MiB, from
> 169 MiB to 45 MiB (almost divided by 4!).

Wow, nice catch.  If I read the code correctly, it detects hard links
and replaces them with symlinks.  Could we do this unconditionally in
a gnu-build-system phase?  Are there any legitimate uses of hard links
in outputs?

That said, the patch LGTM.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#29181: [PATCH 1/1] gnu: mesa: Use symlinks instead of hard links.

[Ludovic Courtès]

Hi,

Marius Bakke <mbakke@fastmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> This reduces the nar size (as shown by 'guix size') by 124 MiB, from
>> 169 MiB to 45 MiB (almost divided by 4!).
>
> Wow, nice catch.  If I read the code correctly, it detects hard links
> and replaces them with symlinks.

Yes.

> Could we do this unconditionally in a gnu-build-system phase?  Are
> there any legitimate uses of hard links in outputs?

Good question.  There might be situations where the software relies on
having regular files (not symlinks), so I would rather do it on a
case-by-case basis than have a standard phase.  WDYT?

Besides, a phase that looks at all the files would have to be efficient
than this naïve implementation.

> That said, the patch LGTM.

Pushed as dcc00f54c619118d11982383102d2e9a1b86d080, thanks!

Ludo’.