unofficial mirror of bug-guix@gnu.org 
 help / color / mirror / code / Atom feed
From: Csepp <raingloom@riseup.net>
To: Csepp <raingloom@riseup.net>
Cc: 54944@debbugs.gnu.org, maxim.cournoyer@gmail.com
Subject: bug#54944: guix pull hangs in guix-packages-base.drv even with offloading
Date: Sun, 25 Jun 2023 21:21:21 +0200	[thread overview]
Message-ID: <874jmv2us4.fsf@riseup.net> (raw)
In-Reply-To: <87lekrx2m9.fsf@riseup.net>

[-- Attachment #1: Type: text/plain, Size: 2679 bytes --]


Csepp <raingloom@riseup.net> writes:

> Csepp <raingloom@riseup.net> writes:
>
>> Csepp <raingloom@riseup.net> writes:
>>
>>> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>>>
>>>> Hi!
>>>>
>>>> raingloom <raingloom@riseup.net> writes:
>>>>
>>>>> It's been at 67% on guix-packages-base for at least an hour now. The
>>>>> system itself is responsive and with the swap I gave it, it has more
>>>>> than enough memory. Htop shows three guile processes at the top of the
>>>>> list when sorted by CPU%, their states are S, D, D.
>>>>> Both CPUs are practically idling.
>>>>> This looks like some kind of lockup to me.
>>>>>
>>>>> Fresh install based on bare-bones example on a 32 bit netbook, but the
>>>>> install image used is the latest tagged version, since apparently there
>>>>> is no 32 bit option for edge.
>>>>>
>>>>> I also tried pulling using channel-with-substitutes, since I'm not too
>>>>> keen on locally building everything on such an old machine. Although
>>>>> Guix itself should frankly not take this long to build if we want to be
>>>>> competitive with other distros. Anyways, pulling with that in
>>>>> channels.scm gives a cert related error, so that's great, means old
>>>>> images can't easily be used for installation.
>>>>
>>>> Have you been able to reproduce this?  If so, could you share the commit
>>>> you are starting from and the CPU architecture, so that we may hopefully
>>>> reproduce too?
>>>>
>>>> Thanks,
>>>>
>>>> Maxim
>>>
>>> CPU architecture is x86, commit it happened on last time is 347733b.
>>> Other possibly relevant factors:
>>> * spinning rust storage
>>> * 1GB RAM
>>> * encrypted BTRFS root
>>> * 4GB (encrypted) swap
>>> * 128MB zswap
>>>
>>> The last was not there when I originally submitted the bug.
>>>
>>> The swap is relevant because if it's a timing issue it's very possible
>>> some part of the code assumes reads are almost instant, which is not
>>> true with swap, and delaying a read might be exposing a race condition.
>>
>> Happening again.
>> pulled to: 8320c0c
>> pulled from: 4501a50
>>
>> Same system.
>>
>> The system version is from november of last year due, because trying to
>> upgrade takes so damn long and often gets stuck on some package with no
>> substitute.
>> So... the situation is not great...
>
> The process status says sleep so it's probably hanging in a syscall?
> Maybe a kernel bug?

Happening again with offloading.
This is getting really annoying.
Offload machine is completely idle, there is a process Guile for
guix-packages-base-builder running on it, its in sleeping status.  Ran
for 17 minutes, now the time is not increasing.
I'm attaching a GDB backtrace of all the threads.

[-- Attachment #2: gdb output --]
[-- Type: text/plain, Size: 12300 bytes --]


Thread 9 (Thread 0xe4affac0 (LWP 878) "guile"):
#0  0xf7fc6579 in __kernel_vsyscall ()
#1  0xf794685e in __lll_lock_wait () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#2  0xf794d11a in pthread_mutex_lock@@GLIBC_2.0 () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#3  0xf7f1095c in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#4  0xf7f16f98 in scm_gensym () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#5  0xf5d3a8ef in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Thread 8 (Thread 0xe5497ac0 (LWP 877) "guile"):
#0  0xf7fc6579 in __kernel_vsyscall ()
#1  0xf794685e in __lll_lock_wait () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#2  0xf794d11a in pthread_mutex_lock@@GLIBC_2.0 () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#3  0xf7f1095c in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#4  0xf7f16f98 in scm_gensym () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#5  0xf5d3a8ef in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Thread 7 (Thread 0xe5c98ac0 (LWP 876) "guile"):
#0  0xf7fc6579 in __kernel_vsyscall ()
#1  0xf794685e in __lll_lock_wait () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#2  0xf794d11a in pthread_mutex_lock@@GLIBC_2.0 () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#3  0xf7f1095c in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#4  0xf7f16f98 in scm_gensym () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#5  0xf5d3a8ef in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Thread 6 (Thread 0xe6499ac0 (LWP 875) "guile"):
#0  0xf7fc6579 in __kernel_vsyscall ()
#1  0xf794685e in __lll_lock_wait () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#2  0xf794d11a in pthread_mutex_lock@@GLIBC_2.0 () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#3  0xf7f1095c in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#4  0xe32647e6 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Thread 5 (Thread 0xf5cf7ac0 (LWP 873) "guile"):
#0  0xf7fc6579 in __kernel_vsyscall ()
#1  0xf79c8237 in read () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#2  0xf7e98292 in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#3  0xf7e0a9c0 in ?? () from target:/gnu/store/5zydbv97cly5mijfmnc6vrk2148qxd1n-libgc-8.2.2/lib/libgc.so.1
#4  0xf7e0c14f in ?? () from target:/gnu/store/5zydbv97cly5mijfmnc6vrk2148qxd1n-libgc-8.2.2/lib/libgc.so.1
#5  0xf7e1599c in GC_do_blocking () from target:/gnu/store/5zydbv97cly5mijfmnc6vrk2148qxd1n-libgc-8.2.2/lib/libgc.so.1
#6  0xf7f11924 in scm_without_guile () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#7  0xf7ea0586 in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#8  0xf7e8bf2d in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#9  0xf7ea2597 in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#10 0xf7f1e1a1 in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#11 0xf7f2e9ca in scm_call_n () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#12 0xf7e8d84f in scm_call_2 () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#13 0xf7f1cc55 in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#14 0xf7f3ec20 in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#15 0xf7f1876b in scm_c_catch () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#16 0xf7e8e63d in scm_c_with_continuation_barrier () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#17 0xf7f1779a in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#18 0xf7e15925 in GC_call_with_stack_base () from target:/gnu/store/5zydbv97cly5mijfmnc6vrk2148qxd1n-libgc-8.2.2/lib/libgc.so.1
#19 0xf7f118ca in scm_with_guile () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#20 0xf7ea060f in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#21 0xf7949ee9 in start_thread () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#22 0xf79df458 in clone3 () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6

Thread 4 (Thread 0xf6552ac0 (LWP 872) "GC-marker-2"):
#0  0xf7fc6579 in __kernel_vsyscall ()
#1  0xf79d26d2 in __libc_do_syscall () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#2  0xf79464d9 in __futex_abstimed_wait_common () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#3  0xf7949474 in pthread_cond_wait@@GLIBC_2.3.2 () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#4  0xf7e10056 in ?? () from target:/gnu/store/5zydbv97cly5mijfmnc6vrk2148qxd1n-libgc-8.2.2/lib/libgc.so.1
#5  0xf7e101b9 in ?? () from target:/gnu/store/5zydbv97cly5mijfmnc6vrk2148qxd1n-libgc-8.2.2/lib/libgc.so.1
#6  0xf7949ee9 in start_thread () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#7  0xf79df458 in clone3 () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6

Thread 3 (Thread 0xf6d53ac0 (LWP 871) "GC-marker-1"):
#0  0xf7fc6579 in __kernel_vsyscall ()
#1  0xf79d26d2 in __libc_do_syscall () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#2  0xf79464d9 in __futex_abstimed_wait_common () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#3  0xf7949474 in pthread_cond_wait@@GLIBC_2.3.2 () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#4  0xf7e10056 in ?? () from target:/gnu/store/5zydbv97cly5mijfmnc6vrk2148qxd1n-libgc-8.2.2/lib/libgc.so.1
#5  0xf7e101b9 in ?? () from target:/gnu/store/5zydbv97cly5mijfmnc6vrk2148qxd1n-libgc-8.2.2/lib/libgc.so.1
#6  0xf7949ee9 in start_thread () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#7  0xf79df458 in clone3 () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6

Thread 2 (Thread 0xf7554ac0 (LWP 870) "GC-marker-0"):
#0  0xf7fc6579 in __kernel_vsyscall ()
#1  0xf79d26d2 in __libc_do_syscall () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#2  0xf79464d9 in __futex_abstimed_wait_common () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#3  0xf7949474 in pthread_cond_wait@@GLIBC_2.3.2 () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#4  0xf7e10056 in ?? () from target:/gnu/store/5zydbv97cly5mijfmnc6vrk2148qxd1n-libgc-8.2.2/lib/libgc.so.1
#5  0xf7e101b9 in ?? () from target:/gnu/store/5zydbv97cly5mijfmnc6vrk2148qxd1n-libgc-8.2.2/lib/libgc.so.1
#6  0xf7949ee9 in start_thread () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#7  0xf79df458 in clone3 () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6

Thread 1 (Thread 0xf78c6700 (LWP 869) "guile"):
#0  0xf7fc6579 in __kernel_vsyscall ()
#1  0xf79d26d2 in __libc_do_syscall () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#2  0xf79464d9 in __futex_abstimed_wait_common () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#3  0xf7949474 in pthread_cond_wait@@GLIBC_2.3.2 () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#4  0xf7f11acc in scm_pthread_cond_wait () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#5  0xf7f15538 in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#6  0xf7f17e9f in scm_timed_wait_condition_variable () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#7  0xf7ea25bd in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#8  0xf7f1e1a1 in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#9  0xf7f2e9ca in scm_call_n () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#10 0xf7e8dbfa in scm_primitive_eval () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#11 0xf7ec7d1d in scm_primitive_load () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#12 0xf7ea2597 in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#13 0xf7f1e1a1 in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#14 0xf7f2e9ca in scm_call_n () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#15 0xf7e8dbfa in scm_primitive_eval () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#16 0xf7e93d32 in scm_eval () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#17 0xf7ef95a4 in scm_shell () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#18 0xf7ea541c in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#19 0xf7e8bf2d in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#20 0xf7ea2597 in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#21 0xf7f1e1a1 in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#22 0xf7f2e9ca in scm_call_n () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#23 0xf7e8d84f in scm_call_2 () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#24 0xf7f1cc55 in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#25 0xf7f3ec20 in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#26 0xf7f1876b in scm_c_catch () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#27 0xf7e8e63d in scm_c_with_continuation_barrier () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#28 0xf7f1779a in ?? () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#29 0xf7e15925 in GC_call_with_stack_base () from target:/gnu/store/5zydbv97cly5mijfmnc6vrk2148qxd1n-libgc-8.2.2/lib/libgc.so.1
#30 0xf7f118ca in scm_with_guile () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#31 0xf7eaeb40 in scm_boot_guile () from target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/lib/libguile-3.0.so.1
#32 0x0804910c in ?? ()
#33 0xf78e9329 in __libc_start_call_main () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#34 0xf78e93ff in __libc_start_main_impl () from target:/gnu/store/0hr9jpczkcgpgqkhf4q4868xd57h5a62-glibc-2.35/lib/libc.so.6
#35 0x08049198 in ?? ()
Detaching from program: target:/gnu/store/da6ikq281d235hvb1cil2ls3iq80ni2m-guile-3.0.9/bin/guile, process 869
[Inferior 1 (process 869) detached]

[-- Attachment #3: Type: text/plain, Size: 264 bytes --]


System info:
offloading from: x86, 1 GB RAM, 4 GB swap, 2 cores, user guix commit is
8a47949, system commit is 038981e, commit being pulled is 01d5d68
offloading to:
amd64, 8 GB RAM, no swap, 4 cores
guix system commit is 9504dd2c3eef0277369acc0944f87fb4546251b1

      reply	other threads:[~2023-06-25 19:43 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-04-15  0:08 bug#54944: guix pull hangs on 32 bit raingloom
2022-06-08 20:24 ` Maxim Cournoyer
2022-12-01  0:56   ` Csepp
2023-02-21  0:36     ` Csepp
2023-02-21  0:50       ` Csepp
2023-06-25 19:21         ` Csepp [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=874jmv2us4.fsf@riseup.net \
    --to=raingloom@riseup.net \
    --cc=54944@debbugs.gnu.org \
    --cc=maxim.cournoyer@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).