From: Chris Marusich <cmmarusich@gmail.com>
To: Aiko Kyle <aikokyle@gmail.com>
Cc: 52943@debbugs.gnu.org
Subject: bug#52943: Cannot build guix as part of guix system reconfigure after commit 224d437fb4 on aarch64
Date: Sat, 08 Jan 2022 20:39:42 -0800 [thread overview]
Message-ID: <87o84lcyk1.fsf@gmail.com> (raw)
In-Reply-To: <CAPWcbYHjBoqd0vsLOknF7Yk86wUiC5P_ppTguv8G=fziJ=FJeQ@mail.gmail.com> (Aiko Kyle's message of "Sat, 1 Jan 2022 16:54:51 -0700")
[-- Attachment #1: Type: text/plain, Size: 19397 bytes --]
Aiko Kyle <aikokyle@gmail.com> writes:
> On the latest master I can run 'guix pull', and the latest guix seems
> to build just fine, however when I go to do 'guix system reconfigure',
> building guix fails the check phase.
Aiko has reported that the fix for 52908 did indeed fix the
"tests/guix-system.sh" problem on aarch64-linux. This means the
"service 'xorg-server' provided more than once" error has been fixed.
However, fixing that issue has revealed another. Aiko said:
> guix system reconfigure is still failing for me on aarch64 due to the
> test 'file-needed/recursive' in tests/gremlin.scm failing.
So the current status is that Aiko still can't do "guix reconfigure" on
master after running "guix pull". For this reason, I have re-opened
this bug and unmerged it from 52908, since it's not exactly the same
issue any more.
At the moment, the file-needed/recursive bug that is preventing Aiko
from running "guix reconfigure" appears to be specific to aarch64-linux.
The problem was reported on guix-devel here:
https://lists.gnu.org/archive/html/guix-devel/2022-01/msg00019.html
Pierre Langlois did some investigation and found this information,
posted in the email thread above:
> I've also been trying to fix this test but without much luck. It
> does look similar to this issue for ppc64 [0], where the `ldd/ld.so'
> beaviour isn't the same as what the gremlin guile module does. However
> the patch proposed there isn't fixing the issue for me either on
> aarch64.
>
> [0]: https://issues.guix.gnu.org/52940,
>
> Maybe we can compare notes and a solution will come up :-). So the test
> fails because 'truth', the result from `ldd', has ld-linux-aarch64.so
> listed while 'needed', from the gremlin guile module doesn't:
>
> --8<---------------cut here---------------start------------->8---
> (truth ;; result from `ldd libguile.so'
> ("/gnu/store/...-gcc-10.3.0-lib/lib/libgcc_s.so.1"
> "/gnu/store/...-glibc-2.33/lib/ld-linux-aarch64.so.1" ;; This isn't
> ;; in gremlins
> "/gnu/store/...-glibc-2.33/lib/libc.so.6"
> "/gnu/store/...-glibc-2.33/lib/libcrypt.so.1"
> "/gnu/store/...-glibc-2.33/lib/libdl.so.2"
> "/gnu/store/...-glibc-2.33/lib/libm.so.6"
> "/gnu/store/...-glibc-2.33/lib/libpthread.so.0"
> "/gnu/store/...-guile-3.0.7/lib/libguile-3.0.so.1"
> "/gnu/store/...-libffi-3.3/lib/libffi.so.7"
> "/gnu/store/...-libgc-8.0.4/lib/libgc.so.1"
> "/gnu/store/...-libunistring-0.9.10/lib/libunistring.so.2"))
>
> (needed ;; result from gremlin
> ("/gnu/store/...-gcc-10.3.0-lib/lib/libgcc_s.so.1"
> "/gnu/store/...-glibc-2.33/lib/libc.so.6"
> "/gnu/store/...-glibc-2.33/lib/libcrypt.so.1"
> "/gnu/store/...-glibc-2.33/lib/libdl.so.2"
> "/gnu/store/...-glibc-2.33/lib/libm.so.6"
> "/gnu/store/...-glibc-2.33/lib/libpthread.so.0"
> "/gnu/store/...-guile-3.0.7/lib/libguile-3.0.so.1"
> "/gnu/store/...-libffi-3.3/lib/libffi.so.7"
> "/gnu/store/...-libgc-8.0.4/lib/libgc.so.1"
> "/gnu/store/...-libunistring-0.9.10/lib/libunistring.so.2"))
> --8<---------------cut here---------------end--------------->8---
>
> Digging a bit more I started comparing x86_64 and aarch64 binaries and
> noticed that libguile.so didn't have the same dynamic section:
>
> --8<---------------cut here---------------start------------->8---
> # On aarch64:
> $ objdump -x
> /gnu/store/pqw0c33k2h8n2snpchnyvx7w617kk31s-guile-3.0.7/lib/libguile-3.0.so.1.4.0
>
>
>
> /gnu/store/pqw0c33k2h8n2snpchnyvx7w617kk31s-guile-3.0.7/lib/libguile-3.0.so.1.4.0:
> file format elf64-littleaarch64
> ...
> Dynamic Section:
>
> NEEDED libgc.so.1
>
> NEEDED libpthread.so.0
>
> NEEDED libffi.so.7
>
> NEEDED libunistring.so.2
>
> NEEDED libcrypt.so.1
>
> NEEDED libdl.so.2
>
> NEEDED libm.so.6
>
> NEEDED libgcc_s.so.1
>
> NEEDED libc.so.6
>
> SONAME libguile-3.0.so.1
> ...
>
> # On x86_64:
> $ objdump -x
> /gnu/store/3h3jn0745ngd87zp83k5smwhykxvdfgf-guile-3.0.7/lib/libguile-3.0.so.1.4.0
>
> /gnu/store/3h3jn0745ngd87zp83k5smwhykxvdfgf-guile-3.0.7/lib/libguile-3.0.so.1.4.0:
> file format elf64-x86-64
> ...
> Dynamic Section:
> NEEDED libgc.so.1
> NEEDED libpthread.so.0
> NEEDED libffi.so.7
> NEEDED libunistring.so.2
> NEEDED libcrypt.so.1
> NEEDED libdl.so.2
> NEEDED libm.so.6
> NEEDED libgcc_s.so.1
> NEEDED libc.so.6
> NEEDED ld-linux-x86-64.so.2 # ld-linux-<arch>.so is here
> SONAME libguile-3.0.so.1
> ...
> --8<---------------cut here---------------end--------------->8---
>
> On aarch64, ld-linux-<arch> is missing. I'm not sure if this is an
> issue with our aarch64 port or if that's OK. It's interesting though
> that if you run `ldd' on libguile on aarch64, the dynamic linker is
> added to the list, even though it's not in the dynamic section:
>
> --8<---------------cut here---------------start------------->8---
> # On aarch64
> $ ldd
> /gnu/store/pqw0c33k2h8n2snpchnyvx7w617kk31s-guile-3.0.7/lib/libguile-3.0.so.1.4.0
> linux-vdso.so.1 (0x0000ffff96fab000)
> libgc.so.1 =>
> /gnu/store/1gkpbfxjx2sbchjhf19yjm4a8vkir0nm-libgc-8.0.4/lib/libgc.so.1
> (0x0000ffff96d88000)
> libpthread.so.0 =>
> /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/libpthread.so.0
> (0x0000ffff96d59000)
> libffi.so.7 =>
> /gnu/store/hjirgm7pwmc2biqz6d0fc1ajr3ha4v14-libffi-3.3/lib/libffi.so.7
> (0x0000ffff96d40000)
> libunistring.so.2 =>
> /gnu/store/4k552fq1p6q73mr9ww7g5y3m77p7cfbm-libunistring-0.9.10/lib/libunistring.so.2
> (0x0000ffff96bb4000)
> libcrypt.so.1 =>
> /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/libcrypt.so.1
> (0x0000ffff96b6d000)
> libdl.so.2 =>
> /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/libdl.so.2
> (0x0000ffff96b59000)
> libm.so.6 =>
> /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/libm.so.6
> (0x0000ffff96ab0000)
> libgcc_s.so.1 =>
> /gnu/store/47snyrq6pq6896m9dysp2s5vx53m6x08-gcc-10.3.0-lib/lib/libgcc_s.so.1
> (0x0000ffff96a8b000)
> libc.so.6 =>
> /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/libc.so.6
> (0x0000ffff96917000)
>
> /gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib/ld-linux-aarch64.so.1
> (0x0000ffff96f79000)
> # ^
> --8<---------------cut here---------------end--------------->8---
>
> We could adapt the test to add the dynamic linker, emulating `ldd', but
> I'm curious if anybody deeply familiar with ELF and dynamic linking
> might have an idea what's going on.
To summarize, this seems to be the problem: On aarch64-linux, when an
ELF file's dynamic section does not contain a NEEDED entry for
ld-linux-aarch64.so.1, file-needed/recursive dutifully omits that entry,
but ldd prints it (even though it is actually missing from the file's
dynamic section). This causes the file-needed/recursive test in
tests/gremlin.scm to fail because the set of entries returned by
file-needed/recursive differs from the set of entries returned by ldd.
Although this may sound similar to 52940, it is different. Bug 52940
was an issue where, when an ELF file's dynamic section contains two
entries in RUNPATH that are different strings but refer to the same
directory (which does happen on powerpc64le-linux), it causes
file-needed/recursive to include in its result two entries that are
different strings but refer to the same file. We decided that such
redundant entries are probably benign, so we resolved 52940 by changing
the test to compare entries by file equality, rather than string
equality. However, this is not the same issue as described by Aiko and
Pierre above, so it makes sense that the fix for 52940 did not fix the
test for aarch64-linux.
So, the separate aarch64-linux problem with file-needed/recursive
persists. For comparison, on powerpc64le-linux, when an ELF file's
dynamic section does not contain a NEEDED entry for "ld64.so.2" (I
believe this is the powerpc64le-linux equivalent of
ld-linux-aarch64.so.1), both ldd and file-needed/recursive include
ld64.so.2, despite the fact that there is no actual NEEDED entry for it
in the ELF file. Here is what I see on a powerpc64le-linux system:
--8<---------------cut here---------------start------------->8---
marusich@suzaku ~/guix-master [env]$ type -P guile
/gnu/store/qr79b2m6cfdj8ar7g0psqg4hglm6djfm-profile/bin/guile
marusich@suzaku ~/guix-master [env]$ type -P ldd
/gnu/store/qr79b2m6cfdj8ar7g0psqg4hglm6djfm-profile/bin/ldd
marusich@suzaku ~/guix-master [env]$ ldd /gnu/store/qr79b2m6cfdj8ar7g0psqg4hglm6djfm-profile/bin/guile
linux-vdso64.so.1 (0x00007fff89220000)
libguile-3.0.so.1 => /gnu/store/47s31zvkhk2ixqn0z7gq6hpz7j7cn9dl-guile-3.0.7/lib/libguile-3.0.so.1 (0x00007fff89050000)
libgc.so.1 => /gnu/store/csqz3w2axyci2qm79xsj11cpfxqh7zb1-libgc-8.0.4/lib/libgc.so.1 (0x00007fff88fb0000)
libpthread.so.0 => /gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libpthread.so.0 (0x00007fff88f60000)
libffi.so.7 => /gnu/store/kmgva3hbpk1bv0gvx5s4w01n0fdvd2l9-libffi-3.3/lib/../lib/libffi.so.7 (0x00007fff88f30000)
libunistring.so.2 => /gnu/store/zf87w2xccli6yvxpplfwd82gcgm6jyrd-libunistring-0.9.10/lib/libunistring.so.2 (0x00007fff88d80000)
libcrypt.so.1 => /gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libcrypt.so.1 (0x00007fff88d30000)
libdl.so.2 => /gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libdl.so.2 (0x00007fff88d00000)
libm.so.6 => /gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libm.so.6 (0x00007fff88bb0000)
libgcc_s.so.1 => /gnu/store/l0hnwrv8ma03shjg84a03s05wmj7a0sk-gcc-10.3.0-lib/lib/libgcc_s.so.1 (0x00007fff88b70000)
libc.so.6 => /gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libc.so.6 (0x00007fff88950000)
/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/ld64.so.2 (0x00007fff89240000)
marusich@suzaku ~/guix-master [env]$ readelf -d /gnu/store/qr79b2m6cfdj8ar7g0psqg4hglm6djfm-profile/bin/guile
Dynamic section at offset 0xfc60 contains 37 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libguile-3.0.so.1]
0x0000000000000001 (NEEDED) Shared library: [libgc.so.1]
0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0]
0x0000000000000001 (NEEDED) Shared library: [libffi.so.7]
0x0000000000000001 (NEEDED) Shared library: [libunistring.so.2]
0x0000000000000001 (NEEDED) Shared library: [libcrypt.so.1]
0x0000000000000001 (NEEDED) Shared library: [libdl.so.2]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000001d (RUNPATH) Library runpath: [/gnu/store/47s31zvkhk2ixqn0z7gq6hpz7j7cn9dl-guile-3.0.7/lib:/gnu/store/csqz3w2axyci2qm79xsj11cpfxqh7zb1-libgc-8.0.4/lib:/gnu/store/kmgva3hbpk1bv0gvx5s4w01n0fdvd2l9-libffi-3.3/lib/../lib:/gnu/store/zf87w2xccli6yvxpplfwd82gcgm6jyrd-libunistring-0.9.10/lib:/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib:/gnu/store/l0hnwrv8ma03shjg84a03s05wmj7a0sk-gcc-10.3.0-lib/lib:/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/../lib:/gnu/store/l0hnwrv8ma03shjg84a03s05wmj7a0sk-gcc-10.3.0-lib/lib/gcc/powerpc64le-unknown-linux-gnu/10.3.0/../../../../lib]
0x000000000000000c (INIT) 0x100009e0
0x000000000000000d (FINI) 0x10000f30
0x0000000000000019 (INIT_ARRAY) 0x1001fc50
0x000000000000001b (INIT_ARRAYSZ) 8 (bytes)
0x000000000000001a (FINI_ARRAY) 0x1001fc58
0x000000000000001c (FINI_ARRAYSZ) 8 (bytes)
0x0000000000000004 (HASH) 0x100002a0
0x000000006ffffef5 (GNU_HASH) 0x100002f8
0x0000000000000005 (STRTAB) 0x100004a8
0x0000000000000006 (SYMTAB) 0x10000328
0x000000000000000a (STRSZ) 894 (bytes)
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000015 (DEBUG) 0x0
0x0000000000000003 (PLTGOT) 0x10020000
0x0000000000000002 (PLTRELSZ) 216 (bytes)
0x0000000000000014 (PLTREL) RELA
0x0000000000000017 (JMPREL) 0x100008e8
0x0000000070000000 (PPC64_GLINK) 0x10000eec
0x0000000070000003 (PPC64_OPT) 0x0
0x0000000000000007 (RELA) 0x10000888
0x0000000000000008 (RELASZ) 96 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x000000006ffffffe (VERNEED) 0x10000848
0x000000006fffffff (VERNEEDNUM) 2
0x000000006ffffff0 (VERSYM) 0x10000826
0x0000000000000000 (NULL) 0x0
marusich@suzaku ~/guix-master [env]$ ./pre-inst-env guix repl
GNU Guile 3.0.7
Copyright (C) 1995-2021 Free Software Foundation, Inc.
Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.
Enter `,help' for help.
scheme@(guix-user)> ,use (guix build gremlin)
scheme@(guix-user)> ,pp (file-needed/recursive "/gnu/store/qr79b2m6cfdj8ar7g0psqg4hglm6djfm-profile/bin/guile")
$1 = ("/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libc.so.6"
"/gnu/store/l0hnwrv8ma03shjg84a03s05wmj7a0sk-gcc-10.3.0-lib/lib/libgcc_s.so.1"
"/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libm.so.6"
"/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libdl.so.2"
"/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libcrypt.so.1"
"/gnu/store/zf87w2xccli6yvxpplfwd82gcgm6jyrd-libunistring-0.9.10/lib/libunistring.so.2"
"/gnu/store/kmgva3hbpk1bv0gvx5s4w01n0fdvd2l9-libffi-3.3/lib/../lib/libffi.so.7"
"/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/libpthread.so.0"
"/gnu/store/csqz3w2axyci2qm79xsj11cpfxqh7zb1-libgc-8.0.4/lib/libgc.so.1"
"/gnu/store/47s31zvkhk2ixqn0z7gq6hpz7j7cn9dl-guile-3.0.7/lib/libguile-3.0.so.1"
"/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/ld64.so.2"
"/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/../lib/libc.so.6")
$2 = ("ld64.so.2")
scheme@(guix-user)>
--8<---------------cut here---------------end--------------->8---
I don't know if it's related, but I just noticed this: it's a little
strange that in the above output (for powerpc64le-linux), ld64.so.2 is
included in the second value ($2). This is supposedly the list of .so
file names that could not be found. It's strange that ld64.so.2 shows
up in $2 because it seems to contradict the fact that it is included in
$1, which is the list of files that were found successfully.
Since Pierre shared information about the libguile shared object
specifically, I'll do the same here. On powerpc64le-linux, the
"ld64.so.2" entry is present in the dynamic section of the
/gnu/store/47s31zvkhk2ixqn0z7gq6hpz7j7cn9dl-guile-3.0.7/lib/libguile-3.0.so.1
ELF file:
--8<---------------cut here---------------start------------->8---
marusich@suzaku ~/guix-master [env]$ readelf -d /gnu/store/47s31zvkhk2ixqn0z7gq6hpz7j7cn9dl-guile-3.0.7/lib/libguile-3.0.so.1| grep -e NEEDED -e RUNPATH
0x0000000000000001 (NEEDED) Shared library: [libgc.so.1]
0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0]
0x0000000000000001 (NEEDED) Shared library: [libffi.so.7]
0x0000000000000001 (NEEDED) Shared library: [libunistring.so.2]
0x0000000000000001 (NEEDED) Shared library: [libcrypt.so.1]
0x0000000000000001 (NEEDED) Shared library: [libdl.so.2]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x0000000000000001 (NEEDED) Shared library: [ld64.so.2]
0x000000000000001d (RUNPATH) Library runpath: [/gnu/store/csqz3w2axyci2qm79xsj11cpfxqh7zb1-libgc-8.0.4/lib:/gnu/store/kmgva3hbpk1bv0gvx5s4w01n0fdvd2l9-libffi-3.3/lib/../lib:/gnu/store/zf87w2xccli6yvxpplfwd82gcgm6jyrd-libunistr
ing-0.9.10/lib:/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib:/gnu/store/l0hnwrv8ma03shjg84a03s05wmj7a0sk-gcc-10.3.0-lib/lib:/gnu/store/3jr84ajdz821y480c454pqrswxbhgzlq-glibc-2.33/lib/../lib:/gnu/store/l0hnwrv8ma03shjg84a03s05
wmj7a0sk-gcc-10.3.0-lib/lib/gcc/powerpc64le-unknown-linux-gnu/10.3.0/../../../../lib]
--8<---------------cut here---------------end--------------->8---
Maybe this information can help somehow.
It seems fishy that on aarch64-linux, there is no NEEDED entry for
ld-linux-aarch64.so.1 in the ELF files under consideration. As shown
above, a similar entry DOES exist on both x86_64-linux and
powerpc64le-linux. For this reason, it seems plausible that maybe the
missing NEEDED entry is bad. However, I don't really know enough to say
whether it's indicative of a problem with our aarch64-linux port. Is
there an aarch64-linux or ELF expert in the room who can help? :-)
It also seems fishy that, on powerpc64le-linux, file-needed/recursive
apparently resolves ld64.so.2 successfully, even though it
simultaneously includes it in the "failed to resolve" list. Confusing
as that is to me, I don't know if that's really related to the
aarch64-linux issue.
More investigation is needed...
--
Chris
PGP: https://savannah.gnu.org/people/viewgpg.php?user_id=106836
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]
next prev parent reply other threads:[~2022-01-09 4:41 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-01 23:54 bug#52943: Cannot build guix as part of guix system reconfigure after commit 224d437fb4 on aarch64 Aiko Kyle
2022-01-02 2:13 ` Leo Famulari
2022-01-02 4:11 ` Aiko Kyle
2022-01-02 4:51 ` Leo Famulari
2022-01-02 6:36 ` Aiko Kyle
2022-01-03 21:38 ` Akira Kyle
2022-01-04 1:13 ` Akira Kyle
2022-01-04 1:16 ` Akira Kyle
2022-01-09 4:39 ` Chris Marusich [this message]
2022-01-15 13:27 ` Pierre Langlois
2022-01-16 1:00 ` Chris Marusich
2022-01-17 4:46 ` Chris Marusich
2022-01-17 18:51 ` Pierre Langlois
2022-01-18 5:33 ` Vagrant Cascadian
2022-01-19 19:28 ` Vagrant Cascadian
2022-01-29 18:33 ` Leo Famulari
2022-01-29 20:54 ` Pierre Langlois
2022-01-29 21:06 ` Leo Famulari
2022-01-29 21:35 ` Pierre Langlois
2022-10-07 8:48 ` Christopher Baines
2022-10-20 14:57 ` Mathieu Othacehe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87o84lcyk1.fsf@gmail.com \
--to=cmmarusich@gmail.com \
--cc=52943@debbugs.gnu.org \
--cc=aikokyle@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).