From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 0HFeHtO342Ff5QAAgWs5BA (envelope-from ) for ; Sun, 16 Jan 2022 07:14:43 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id GM+dG9O342EykAAA9RJhRA (envelope-from ) for ; Sun, 16 Jan 2022 07:14:43 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 5335112F01 for ; Sun, 16 Jan 2022 07:14:42 +0100 (CET) Received: from localhost ([::1]:54078 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n8yoL-0005pZ-8W for larch@yhetil.org; Sun, 16 Jan 2022 01:14:41 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48290) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n8ymq-0005o7-C5 for bug-guix@gnu.org; Sun, 16 Jan 2022 01:13:08 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:49427) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1n8ymk-0006I4-O4 for bug-guix@gnu.org; Sun, 16 Jan 2022 01:13:07 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1n8ymk-00043x-Ie for bug-guix@gnu.org; Sun, 16 Jan 2022 01:13:02 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#53296: Intermittent segfaults when parsing (?) custom package from repo. Resent-From: elaexuotee@wilsonb.com Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 16 Jan 2022 06:13:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 53296 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 53296@debbugs.gnu.org X-Debbugs-Original-To: bug-guix@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.164231356115584 (code B ref -1); Sun, 16 Jan 2022 06:13:02 +0000 Received: (at submit) by debbugs.gnu.org; 16 Jan 2022 06:12:41 +0000 Received: from localhost ([127.0.0.1]:42330 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n8ymH-000439-Mb for submit@debbugs.gnu.org; Sun, 16 Jan 2022 01:12:41 -0500 Received: from lists.gnu.org ([209.51.188.17]:55732) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n8ymC-00042y-6y for submit@debbugs.gnu.org; Sun, 16 Jan 2022 01:12:31 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48260) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n8ymB-0005nh-SE for bug-guix@gnu.org; Sun, 16 Jan 2022 01:12:28 -0500 Received: from m42-5.mailgun.net ([69.72.42.5]:58939) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1n8ym8-0006GC-1O for bug-guix@gnu.org; Sun, 16 Jan 2022 01:12:27 -0500 DKIM-Signature: a=rsa-sha256; v=1; c=relaxed/relaxed; d=mg.wilsonb.com; q=dns/txt; s=krs; t=1642313540; h=Content-Type: MIME-Version: Message-Id: Subject: From: To: Date: Sender; bh=A7QJpo+YOEpKV0OLZ2LMeeM8npU+rqvrsA7GWPAr/ek=; b=bjuNnhZpRtcmpCkSyuNhQVL/RbsGW708y3kcS6drxUSdXegx7N/+IvVx2N05zEbCLnJP2AAs nLnzkqv2nsdNDFgKJfWd2hqbufUwEn64kSTiMSSnw1CKtvfiR3wCAm8//e87SINxY6ZOWAWI VmcYk5tfw8qTF9NuHhjXQWvQLH/0bm7/FQ6WQRqutmRy24C9I6x/S+HmOIToWO6dsUFhBeDb wmV5BHGNBDJKgdVhYYMJss4w0qrAU0+bV7WrDxfRnKcEpsaliHWxlJ5VANwjC9qrKxAKg/39 gl93WfR4/S/uZPVuQDeXqb8iVv21UNyEGFJR6CVOld8gQZ+T+rEq6w== X-Mailgun-Sending-Ip: 69.72.42.5 X-Mailgun-Sid: WyJkMDM0YyIsICJidWctZ3VpeEBnbnUub3JnIiwgIjA4NTQ3YSJd Received: from wilsonb.com (wilsonb.com [104.199.203.42]) by smtp-out-n06.prod.us-east-1.postgun.com with SMTP id 61e3b74306362278a19df4fd (version=TLS1.3, cipher=TLS_AES_128_GCM_SHA256); Sun, 16 Jan 2022 06:12:19 GMT Received: from localhost (q045100.dynamic.ppp.asahi-net.or.jp [203.181.45.100]) by wilsonb.com (Postfix) with ESMTPSA id 34AABA04CC for ; Sun, 16 Jan 2022 06:12:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wilsonb.com; s=201703; t=1642313534; bh=A7QJpo+YOEpKV0OLZ2LMeeM8npU+rqvrsA7GWPAr/ek=; h=Date:To:From:Subject:From; b=msh7FMvj0l14fdW7L0jJl34p004hPnKSpWKBZ2aAe6m7i9d8PSYSSHeCRjgLsUp9n LcA8qwO5PmuTt13FAerG+eYfgnIHpCluZ83JF9qE7braYtFyWrxsupTYLnjEoQP9BI bY6LpikXNhGMyqeY1nNKhUKjO26XCNhEfNg/wTAJIjOoR0+tYnQ+s2FB6HGV1ns2Q6 Se7jk8P5mYW6uinKSaDz+ruLsp7RTTBilp2yZBLuPizWNw4DDTbLZQjvuSonLcAs1H GFXL6zQBVdZSbXpIxPspGrEkiPambVDFsnC4ixbh05aEkgTIXwWJ7VsXmd2Unv1tUc Kc9p8AnQ41gJ42kvfP9CODWSUAkJzp6rPX+ppgSYvbcYYRizUVqyZ127Xn0DhSz5LK TeWHMkXj6ZDNTpags0kldaA1k88FX25CK2YvlyIJeAdAgpQxq0ZK2FDe7UYdq8atgT Zx/t+7I/rBWzxAmIrPz8uD8yfnoT5ub0HU9oMRepZlKAL+aK3SNYZkpEv9I1NH8vkz C3wNCmj+H1lAjFG/IX5NRZ0Tyyc4hq+k6IPmQi022e5p1Vp4LvK9Qs/WIavc7n4X2x zsBX0Tm4zAFQi/uA1Pa9E7k+hT06zBOH3Hz0xkXFX/TfC2vgrD21sBU9fbuTt3GHW7 MA2AwIdn0di8XgJ8/WK9mUZs= Date: Sun, 16 Jan 2022 15:12:08 +0900 Message-Id: <3E4C9P1M7LAZ4.3OM9BXEOU8OKP@wilsonb.com> User-Agent: mblaze/1.1 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----_=_761356131b06de4d0b2d440f_=_" Received-SPF: pass client-ip=69.72.42.5; envelope-from=bounce+7434c3.08547a-bug-guix=gnu.org@mg.wilsonb.com; helo=m42-5.mailgun.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" Reply-to: elaexuotee@wilsonb.com From: elaexuotee--- via Bug reports for GNU Guix X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1642313682; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:list-id:list-help:list-unsubscribe:list-subscribe: list-post:dkim-signature; bh=A7QJpo+YOEpKV0OLZ2LMeeM8npU+rqvrsA7GWPAr/ek=; b=r4+nMtqZwejudZUhC6ldklHu/2saAL17WHWb+I7C3tz2Voy1jnK5wwlu05RDiJt8m6wK6M ef5S5tidDGF14X/bkNDQawb/f8UUi4uHyBgMqB3Cbx9Mma2MksEjOavUN5d0H+YEyKXVrF OSZacJ3hJd4utePBviKCxBbDUucFMNEG8t095AVdPmsKIHM249bPZBxfXREOc4C0s1b4Kv RS2PpjP3uy/fRenwK45uVuJ7T1zmB3D5pC9aZaLDY+glWqJ3jMM1eP5IRbljThvkFthCah M09BbLqWfiAD7kZHqIP6Q9L67HTsSRCv3ifGDLGC5mU2CMgx6+MSMDx3TZxxXg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1642313682; a=rsa-sha256; cv=none; b=cuC1DaBQMKhMuDkjBroZ1Cmlyo52ZJt5dgKxBeSdwXQ288Pxwg0LIvEMgf1yjPpIeXC4d+ 2Liz4JXi2UTyNg0eUGMyXSpfWzTOIc6g++FzQFJMVP6BrEUoTAoUJb4lYVf+afWNgxYZ6o 6CsNVzQShi7DDUcIJhFfUQ2PB9T3aUgojPCMQtPbpCRqCIwZmxKlskwra6LJxqVzGLwYyH jQY0XdnG/BoJ5rOOwFm2DlKzB3iLtphwINFqSkVR5nI58KLppt3WKvnnB27b8adu5H0Tl2 7/y6408Wus5ujRXFS7/hN/q6MAjvGmsyLjTFbZ9PwlqeIyDZMDRXRKzyIYa8wA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=mg.wilsonb.com header.s=krs header.b=bjuNnhZp; dkim=fail ("headers rsa verify failed") header.d=wilsonb.com header.s=201703 header.b=msh7FMvj; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -5.43 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=mg.wilsonb.com header.s=krs header.b=bjuNnhZp; dkim=fail ("headers rsa verify failed") header.d=wilsonb.com header.s=201703 header.b=msh7FMvj; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 5335112F01 X-Spam-Score: -5.43 X-Migadu-Scanner: scn0.migadu.com X-TUID: 1r1wgdKLjawO This is a multipart message in MIME format. ------_=_761356131b06de4d0b2d440f_=_ Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hey Guix, Recently working on a package, I have been encounting intermittent segfault= during a build. The segfault seems to only occur when I have some error in my code that cau= ses a crash, and the segfaults tend to cluster, appearing unexpectedly for a fe= w build attempts, and then disappearing right as I think they are reproducibl= e and try to grab an strace or something. Unfortunately, that's about the ext= ent of information I have been able to gleen. The latest segfault happened with the attached package definition. Note, th= e offending code is at line 77, where I forgot to remove a docstring from a variable that used to be a procedure. Given that non-segfault runs seem to error out so early, is this better tho= ught to be an issue with Guile? FWIW, I see nothing interesting under /var/log/guix-daemon.log. However, lines like following show up in the kern= el messages ring: [318026.268095] guix[7419]: segfault at 18 ip 00007f56ef6a01a3 sp 00007fff1= 5588980 error 4 in libgc.so.1.4.3[7f56ef693000+1b000] [318026.268116] Code: 8d 2d 71 93 01 00 90 4a 8d 04 e5 00 00 00 00 48 89 04= 24 49 8b 45 00 4e 8b 3c e0 4d 85 ff 74 2a 31 ed 0f 1f 44 00 00 4d 89 fe <4= d> 8b 7f 08 49 8b 7e 10 48 f7 d7 e8 6d 35 ff ff 85 c0 0f 84 3d 01 [318029.715621] guix[7761]: segfault at 10 ip 00007f9e80b919b9 sp 00007fffd= 1b2ad20 error 4 in libgc.so.1.4.3[7f9e80b7b000+1b000] [318029.715638] Code: f7 d2 48 21 d0 48 8b 13 4c 8d 3c c5 00 00 00 00 48 8b= 04 c2 48 85 c0 74 78 48 89 ea 48 f7 d2 eb 09 48 8b 40 08 48 85 c0 74 67 <4= 8> 39 10 75 f2 44 8b 05 03 1c 04 00 49 f7 d4 4c 89 60 10 41 bc 01 [318041.537171] guix[8660]: segfault at 10 ip 00007f0d2603c9b9 sp 00007ffc7= 2e998d0 error 4 in libgc.so.1.4.3[7f0d26026000+1b000] [318041.537185] Code: f7 d2 48 21 d0 48 8b 13 4c 8d 3c c5 00 00 00 00 48 8b= 04 c2 48 85 c0 74 78 48 89 ea 48 f7 d2 eb 09 48 8b 40 08 48 85 c0 74 67 <4= 8> 39 10 75 f2 44 8b 05 03 1c 04 00 49 f7 d4 4c 89 60 10 41 bc 01 In the off chance it's helpful, below are some random machine details. Plea= se let me know if there is anything more pointed or specific I provide. $ guix system describe Generation 4 Jan 12 2022 18:59:48 (current) file name: /var/guix/profiles/system-4-link canonical file name: /gnu/store/sb01mnd31a9x2a0bznzlb2lsy91qwgk6-system label: GNU with Linux 5.15.13 bootloader: grub-efi root device: label: "root" kernel: /gnu/store/bdf2yw10jr02mhyiwm05yp2qibywqz47-linux-5.15.13/bzImage= channels: guix-bmw: repository URL: git://git@git.wilsonb.com/guix-bmw.git branch: master commit: 9fb59483371bb5d59fbd27e47baac88263410ac5 nonguix: repository URL: https://gitlab.com/nonguix/nonguix branch: master commit: 023508df4804dbd9f39cb197525f166bc259f995 guix: repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: 9a2cf2c9232e229f7bb1ab065df2cf0740f65996 configuration file: /gnu/store/pkf4vzlck0g32hkyvijmlcnp15vh8njv-configura= tion.scm $ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 43 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7 Vendor ID: AuthenticAMD Model name: AMD Ryzen 5 3500U with Radeon Vega Mobile = Gfx CPU family: 23 Model: 24 Thread(s) per core: 2 Core(s) per socket: 4 Socket(s): 1 Stepping: 1 Frequency boost: enabled CPU max MHz: 2100.0000 CPU min MHz: 1400.0000 BogoMIPS: 4191.75 Flags: fpu vme de pse tsc msr pae mce cx8 apic se= p mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxex= t fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid e= xtd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4= _2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr= 8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext pe= rfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate ssbd ibpb vmmc= all fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt= xsavec xgetbv1 xsaves clzero irperf xsaveerptr arat npt lbrv svm_lock nrip= _save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshol= d avic v_vmsave_vmload vgif overflow_recov succor smca sme sev sev_es Virtualization: AMD-V L1d cache: 128 KiB (4 instances) L1i cache: 256 KiB (4 instances) L2 cache: 2 MiB (4 instances) L3 cache: 4 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-7 Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disab= led via prctl and seccomp Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and _= _user pointer sanitization Vulnerability Spectre v2: Mitigation; Full AMD retpoline, IBPB condi= tional, STIBP disabled, RSB filling Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected $ lsmem RANGE SIZE STATE REMOVABLE BLOCK 0x0000000000000000-0x00000000bfffffff 3G online yes 0-23 0x0000000100000000-0x00000005bfffffff 19G online yes 32-183 Memory block size: 128M Total online memory: 22G Total offline memory: 0B $ lsirq IRQ TOTAL NAME RES 535178549 Rescheduling interrupts LOC 262669447 Local timer interrupts 82 125390312 PCI-MSI 2621440-edge amdgpu CAL 66644945 Function call interrupts TLB 28850164 TLB shootdowns 12 7316777 IO-APIC 12-edge i8042 77 2140601 PCI-MSI 2097156-edge iwlwifi:queue_4 9 1653508 IO-APIC 9-fasteoi acpi 73 1583984 PCI-MSI 2097152-edge iwlwifi:default_queue IWI 1102407 IRQ work interrupts 65 759824 PCI-MSI 3145728-edge ahci[0000:06:00.0] 1 532374 IO-APIC 1-edge i8042 76 253951 PCI-MSI 2097155-edge iwlwifi:queue_3 75 186427 PCI-MSI 2097154-edge iwlwifi:queue_2 74 168179 PCI-MSI 2097153-edge iwlwifi:queue_1 7 100000 IO-APIC 7-fasteoi pinctrl_amd 52 41159 PCI-MSI 524289-edge nvme0q1 58 38523 PCI-MSI 524295-edge nvme0q7 56 38206 PCI-MSI 524293-edge nvme0q5 53 36960 PCI-MSI 524290-edge nvme0q2 55 30286 PCI-MSI 524292-edge nvme0q4 54 28877 PCI-MSI 524291-edge nvme0q3 57 27421 PCI-MSI 524294-edge nvme0q6 59 27142 PCI-MSI 524296-edge nvme0q8 MCP 8512 Machine check polls 35 2317 PCI-MSI 2627584-edge xhci_hcd 81 1290 PCI-MSI 2633728-edge snd_hda_intel:card1 44 1215 PCI-MSI 2629632-edge xhci_hcd 67 338 PCI-MSI 1572864-edge rtsx_pci 80 316 PCI-MSI 2623488-edge snd_hda_intel:card0 0 34 IO-APIC 2-edge timer 78 33 PCI-MSI 2097157-edge iwlwifi:exception 34 28 PCI-MSI 524288-edge nvme0q0 8 1 IO-APIC 8-edge rtc0 25 0 PCI-MSI 18432-edge PCIe PME, aerdrv 26 0 PCI-MSI 20480-edge PCIe PME, aerdrv 27 0 PCI-MSI 22528-edge PCIe PME, aerdrv 28 0 PCI-MSI 28672-edge PCIe PME, aerdrv, pciehp 29 0 PCI-MSI 133120-edge PCIe PME 30 0 PCI-MSI 135168-edge PCIe PME 36 0 PCI-MSI 2627585-edge xhci_hcd 37 0 PCI-MSI 2627586-edge xhci_hcd 38 0 PCI-MSI 2627587-edge xhci_hcd 39 0 PCI-MSI 2627588-edge xhci_hcd 40 0 PCI-MSI 2627589-edge xhci_hcd 41 0 PCI-MSI 2627590-edge xhci_hcd 42 0 PCI-MSI 2627591-edge xhci_hcd 45 0 PCI-MSI 2629633-edge xhci_hcd 46 0 PCI-MSI 2629634-edge xhci_hcd 47 0 PCI-MSI 2629635-edge xhci_hcd 48 0 PCI-MSI 2629636-edge xhci_hcd 49 0 PCI-MSI 2629637-edge xhci_hcd 50 0 PCI-MSI 2629638-edge xhci_hcd 51 0 PCI-MSI 2629639-edge xhci_hcd 60 0 PCI-MSI 524297-edge nvme0q9 61 0 PCI-MSI 524298-edge nvme0q10 62 0 PCI-MSI 524299-edge nvme0q11 63 0 PCI-MSI 524300-edge nvme0q12 71 0 PCI-MSI 2625537-edge ccp-1 79 0 PCI-MSI 1048576-edge enp2s0 NMI 0 Non-maskable interrupts SPU 0 Spurious interrupts PMI 0 Performance monitoring interrupts RTR 0 APIC ICR read retries TRM 0 Thermal event interrupts THR 0 Threshold APIC interrupts DFR 0 Deferred Error APIC interrupts MCE 0 Machine check exceptions ERR 0=20 MIS 0=20 PIN 0 Posted-interrupt notification event NPI 0 Nested posted-interrupt event PIW 0 Posted-interrupt wakeup event ------_=_761356131b06de4d0b2d440f_=_ Content-Disposition: attachment; filename=jsoftware.scm Content-Type: text/x-patch Content-Transfer-Encoding: quoted-printable ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2022 B. Wilson ;;; ;;; This file is part of GNU Guix. ;;; ;;; GNU Guix is free software; you can redistribute it and/or modify it ;;; under the terms of the GNU General Public License as published by ;;; the Free Software Foundation; either version 3 of the License, or (at ;;; your option) any later version. ;;; ;;; GNU Guix is distributed in the hope that it will be useful, but ;;; WITHOUT ANY WARRANTY; without even the implied warranty of ;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;;; GNU General Public License for more details. ;;; ;;; You should have received a copy of the GNU General Public License ;;; along with GNU Guix. If not, see . (define-module (gnu packages jsoftware) #:use-module (guix build utils) #:use-module (guix build-system gnu) #:use-module (guix build-system trivial) #:use-module (guix git-download) #:use-module ((guix licenses) #:prefix license:) #:use-module (guix packages) #:use-module (guix utils) #:use-module (gnu packages) #:use-module (gnu packages libedit) #:use-module (gnu packages llvm) #:use-module (gnu packages maths) #:use-module (guix gexp) #:use-module (ice-9 ftw) #:use-module (ice-9 match) #:use-module (ice-9 regex) #:use-module (ice-9 rdelim) #:use-module (srfi srfi-1) #:use-module (srfi srfi-26) #:use-module (srfi srfi-71)) ;;; TODO: Make importer and packages for J addons: ;;; http://www.jsoftware.com/jal/ ;;; TODO: Package up j80x series (define (jname prefix release-type) "Return a package name for J, including RELEASE-TYPE only if not 'release= ." (match release-type ('release prefix) (_ (string-append prefix "-" (symbol->string release-type))))) ;; We want a version string where packages specifications like pkg@MAJOR wo= rk. ;; This requires that the first version part separator be dot. Subsequent ;; separators are hyphen, mirror `git-version' etc. (define* (jversion->string major #:optional minor revision commit) "Return a version string formatted like MAJOR.MINOR-REVISION-COMMIT. Onl= y MAJOR is required, and MINOR defaults to ``0'' if not supplied." (let* ((commit (and commit (string-take commit 7))) (minor (or minor "0")) (sub-parts (filter (cut (compose not eq?) #f <>) (list minor revision commit)))) (string-append major "." (string-join sub-parts "-")))) (define* (jrelease-string release-type #:optional version-minor) "Construct J release identifier string." (let ((release-type (symbol->string release-type))) (if version-minor (string-append release-type "-" version-minor) release-type))) (define* (jinfo->git-tag version-major release-type #:optional version-mino= r) "Given version parameters, construct a git tag for upstream releases." (string-append "j" version-major (jrelease-string release-type version-mi= nor))) (define ijconsole "G-exp script that detects AVX/AVX2 support at runtime and executes jcons= ole with the appropriate libj.so and profile.ijs." (with-imported-modules '((guix cpu) (guix memoization) (guix profiling) (guix sets) (srfi srfi-26)) (program-file "ijconsole" #~(begin (use-modules ((guix cpu) #:select (cpu-flags current-cpu)) ((guix sets) #:select (set-contains?)) ((srfi srfi-26) #:select (cute))) ;; Assume that this script will be installed under bin/. (define %basedir (dirname (dirname (current-filename)))) (let* ((jconsole (string-append %basedir "/libexec/j/jconsole")) (cpu-has-flag? (cute set-contains? (cpu-flags (current-cpu)) <>)) (libj (format #f "~a/lib/j/libj~a.so" %basedir (cond ((cpu-has-flag? "avx2") "-avx2") ((cpu-has-flag? "avx") "-avx") (else "")))) (jprofile (string-append %basedir "/etc/j/profile.ijs"))) (apply execl jconsole "ijconsole" "-lib" libj "-jprofile" jprof= ile (cdr (command-line)))))))) (define* (make-j #:key version revision hash tag commit (release-type 'release) (patches '()) (extra-inputs '()) (extra-envars '()) (builder "guix.gnu.org")) (let* ((version-major version-minor (if (pair? version) (car+cdr version) (values version #f)))) (package (name (jname "jsoftware-j" release-type)) (version (jversion->string version-major version-minor revision commit))= (source (origin (method git-fetch) (uri (git-reference (url "https://github.com/jsoftware/jsource") (commit (or commit tag (jinfo->git-tag version-major release-type version-minor))))) (sha256 (base32 hash)) (file-name (git-file-name name version)) (patches patches))) (build-system gnu-build-system) (native-inputs (list clang-toolchain)) (inputs (cons* `("libedit" ,libedit) `("libomp" ,libomp) `("ijconsole" ,ijconsole) ;; profilex.ijs overrides ~install and ~addons ;; directories to reside under the user-writable ~user. ;; This allows local-install of addons via pacman. TODO= : ;; Guix-ify J addons as well. `("profilex" ,(search-auxiliary-file "jsoftware/profilex.ijs")) extra-inputs)) (arguments `(#:modules (((ice-9 ftw) #:select (scandir)) ((ice-9 popen) #:select (open-pipe* close-pipe)) ((ice-9 regex) #:select (match:substring string-match)) ((ice-9 threads) #:select (parallel par-for-each)) ((srfi srfi-26) #:select (cut)) ((srfi srfi-1) #:select (fold)) ,@%gnu-build-system-modules) #:phases ;; Upstream's build system consists of ad-hoc scripts that build buil= d up ;; (very complicated) environment variables to pass to make. The bas= ic ;; build process looks like this: ;; ;; 1) Copy jsrc/jversion-x.h to jsrc/jversion.h and edit values; ;; 2) Set jplatform and j64x environment variables; ;; 3) Run make2/build_jconsole.sh and make2/build_libj.sh; ;; ;; However, upstream expects users to run J directly from the source ;; directory; they do not supply a make `install' target. Thus it ta= kes ;; some massaging to install files in FHS-style directories. (modify-phases %standard-phases ;; In particular, we have to set up ;; ;; 1) jsrc/jversion.h as in a typical build; ;; 2) jlibrary/bin/profilex.ijs to point to writable directories;= ;; 3) make2/build_*.sh to respect standard build conventions; ;; 4) jsrc/jconsole.c to fix libedit dlopen; and ;; 5) Hard coded references to addons directory. (replace 'configure (lambda* (#:key target inputs outputs #:allow-other-keys) (let* ((clang-toolchain (assoc-ref inputs "clang-toolchain")) (clang (string-append clang-toolchain "/bin/clang")) (libedit (assoc-ref inputs "libedit")) (out (assoc-ref outputs "out"))) ;; Set up build constants (copy-file "jsrc/jversion-x.h" "jsrc/jversion.h") (substitute* "jsrc/jversion.h" (("^#define jversion.*$") (format #f "#define jversion ~s\n" ,version-major)) (("^#define jtype.*$") (format #f "#define jtype ~s\n" ,(jrelease-string release-type version-minor))) (("^#define jbuilder.*$") (format #f "#define jbuilder ~s\n" ,builder))) ;; Munge the build scripts into reason: ;; 1. Short-circuit the fragile compiler detection; ;; 2. Make sure to include our CFLAGS and LFLAGS; and ;; 3. Propagate script errors to top level. (for-each (lambda (file) (with-directory-excursion "make2" (substitute* file ;; The `compiler' variable doesn't point to the actual= ;; compiler. It is just a switch to tell the build ;; scripts whether to use gcc- or clang-specific flags= =2E (("^compiler=3D.*$") "compiler=3Dclang\n") (("^LDFLAGS=3D\"" def) (string-append def "$LDFLAGS ")= ) (("^(common=3D\")(\\$USETHREAD.*)$" _ def rest) (string-append def "$CFLAGS " rest)) (("^#!.*" shebang) (string-append shebang "set -o errexit\n"))))) '("build_jconsole.sh" "build_libj.sh")) ;; The jconsole manually loads libedit with dlopen. The path= ;; must be absolute to correctly point to our input. (substitute* "jsrc/jconsole.c" (("libedit\\.so\\.[0-9]" so-file) (format #f "~a/lib/~a" libedit so-file))) ;; The ~addons/dev directory supplies tentative J-script ;; definitions of new J engine functionality. Since we point= ;; ~addons under the ~user directory, we move it under ~syste= m ;; instead, which sits as-is in the output. (with-directory-excursion "jsrc" (for-each (lambda (file) (substitute* file (("~addons/dev") "~system/dev"))) (scandir "." (lambda (f) (eq? (stat:type (stat f)) 'regular))))) ;; Implementation of 9!:14 records build time which breaks bu= ild ;; reproducibility. Note that upstream code depends on the e= xact ;; format of these strings, so we need to mimic the standard.= (substitute* "jsrc/j.c" (("__DATE__") "\"Jan 01 1970\"") (("__TIME__") "\"00:00:00\"")) ;; Upstream recommends using clang, with GCC support being ;; second-class, often resulting in build failures. (setenv "CC" clang)))) ;; The build output depends primarily on the values of the `jplatfo= rm' ;; and `j64x' environment variables. If the target is ARM, then ;; `jplatform' is "raspberry", otherwise it is `linux'. In additio= n to ;; 32- and 64- bit versions, `j64x' controlls whether AVX or AVX2 ;; variants of libj are built. ;; ;; However, build targets are not fine-grained enough to distinguis= h ;; between CPU features. Thus we build and install all variants of= ;; libj, expecting jconsole to be called with a wrapper script that= ;; detects AVX features and loads the appropriate libj at runtime. (replace 'build (lambda _ (setenv "USE_OPENMP" "1") (setenv "USE_THREAD" "1") (for-each (lambda (var-val) (apply setenv var-val)) (quote ,extra-envars)) ;; The build scripts assume that PWD is make2. (with-directory-excursion "make2" (let* ((platform ,(if (target-arm?) "raspberry" "linux")) (target-bit ,(if (target-64bit?) "64" "32")) (run (lambda* (script #:key (avx "")) (invoke "env" (string-append "jplatform=3D" platform)= (string-append "j64x=3Dj" target-bit av= x) script)))) (parallel ;; Since jconsole doesn't depend on AVX features, we just= ;; build it once. (run "./build_jconsole.sh") (run "./build_libj.sh") (if ,(target-64bit?) (parallel (run "./build_libj.sh" #:avx "avx") (run "./build_libj.sh" #:avx "avx2")))))))) ;; The test suite is expected to be run as follows for each variant= of ;; libj that we build: ;; ;; $ echo 'RUN ddall' | jconsole test/tsu.ijs ;; ;; This requires a working jconsole with accessible jlibrary files.= We ;; simply place these all under test/bin. (replace 'check (lambda* (#:key tests? #:allow-other-keys) (when tests? (let ((platform ,(if (target-arm?) "raspberry" "linux"))) (mkdir-p "test/bin") (for-each (lambda (dir) (let ((source (string-append "jlibrary/" dir)) (dest (string-append "test/bin/" dir))) (begin (mkdir-p dest) (copy-recursively source dest)))) '("system" "tools" "addons")) ;; The jlibrary/dev directory only sometimes exists, but ne= eds ;; to be copied into the ~system directory when it does. (for-each (lambda (dev-dir) (if (file-exists? dev-dir) (copy-recursively dev-dir "test/bin/system/dev"))) '("jlibrary/dev" "jlibrary/addons/dev")) (par-for-each (lambda (dir) (let* ((bin (string-append "bin/" platform)) (jbit ,(if (target-64bit?) "j64" "j32")) (jconsole (string-append bin "/" jbit "/jconsole")) (source (string-append bin "/" dir)) (dest (string-append "test/bin/" dir))) (begin (mkdir-p dest) (copy-recursively source dest) (install-file "jlibrary/bin/profile.ijs" dest) (install-file jconsole dest) (let* ((jconsole (string-append dest "/jconsole")) (tests "test/tsu.ijs") (port (open-pipe* OPEN_WRITE jconsole tests)= )) (display "RUN ddall\n" port) (when (not (zero? (status:exit-val (close-pipe port)))) (error "Some J build tests failed.")))))) (scandir (string-append "bin/" platform) (negate (cut member <> '("." ".."))))) #t)))) ;; Now that everything is built, installation is fairly ;; straightforward, following FHS conventions. The only quirk is t= hat ;; we install jconsole under /libexec to make room for the wrapper ;; replacement under /bin. (replace 'install (lambda* (#:key outputs inputs #:allow-other-keys) (let* ((platform ,(if (target-arm?) "raspberry" "linux")) (jbit ,(if (target-64bit?) "j64" "j32")) (out (assoc-ref outputs "out")) (bin (string-append out "/bin")) (etc (string-append out "/etc/j")) (lib (string-append out "/lib/j")) (libexec (string-append out "/libexec/j")) (share (string-append out "/share/j")) (system (string-append share "/system")) (dev (string-append system "/dev"))) (mkdir-p bin) (copy-file (assoc-ref inputs "ijconsole") (string-append bin "/ijconsole-" ,version-major)) (mkdir-p lib) (for-each (lambda (jarch) (let* ((jbin (string-join `("bin" ,platform ,jarch) "/"))= (javx-match (string-match "avx.*" jarch)) (javx (if (not javx-match) "" (match:substring javx-match))) (sep (if javx-match "-" "")) (source (string-append jbin "/libj.so")) (dest (format #f "~a/libj~a~a.so" lib sep javx))) (copy-file source dest))) (scandir (string-append "bin/" platform) (negate (cut member <> '("." ".."))))) (install-file (string-append "bin/" platform "/" jbit "/jcons= ole") libexec) (copy-recursively "jlibrary/system" system) (for-each (lambda (source-dev) (if (access? source-dev R_OK) (copy-recursively source-dev dev))) '("jlibrary/dev" "jlibrary/addons/dev")) (install-file "jlibrary/bin/profile.ijs" etc) (copy-file (assoc-ref inputs "profilex") (string-append etc "/profilex.ijs")))))))) (home-page "https://www.jsoftware.com/") (synopsis "Ascii-only, array programming language in the APL family") (description "J is a high-level, general-purpose programming language that is particularly suited to the mathematical, statistical, and logical analysis = of data. It is a powerful tool for developing algorithms and exploring proble= ms that are not already well understood.") (license license:gpl3+)))) (define-public jsoftware-j-901 (make-j #:version '("901" . "f") #:hash "1776021m0j1aanzwg60by83n53pw7i6afd5wplfzczwk8bywax4p" #:patches (search-patches "jsoftware-j901-f-fixes.patch"))) (define j-build-configuration-with-sleef `(#:extra-inputs (("sleef" ,sleef)) #:extra-envars (("USE_SLEEF_SRC" "0") ("LDFLAGS" "-lsleef")))) (define-public jsoftware-j-902 (apply make-j (append j-build-configuration-with-sleef `(#:version ,'("902" . "b") #:hash "0j67vgikqflwjqacsdicasvyv1k54s2c8vjgwmf0ix7l41p4xqz0")))) (define-public jsoftware-j-903 (apply make-j (append j-build-configuration-with-sleef `(#:version ,'("903" . "a") #:tag "903-release-a" #:hash "1fcfl7q7c2vj4fmnqqc8c6hwgsjm20ff93v8xxfniasss1b2fmc4")))) (define-public (jsoftware-ijconsole-symlink jpkg) "Provide bin/ijconsole symlink that points to pkg's bin/ijconsole-" (package (name "jsoftware-ijconsole") (version (package-version jpkg)) (source #f) (build-system trivial-build-system) (propagated-inputs `(("jpkg" ,jpkg))) (arguments `(#:modules ((guix build utils) (srfi srfi-26)) #:builder (begin (use-modules ((guix build utils) #:select (mkdir-p)) ((ice-9 regex) #:select (string-match)) ((ice-9 ftw) #:select (scandir)) ((srfi srfi-26) #:select (cut))) (let* ((out (assoc-ref %outputs "out")) (jpkg (assoc-ref %build-inputs "jpkg")) (ijconsole (car (scandir (string-append jpkg "/bin") (cut string-match "ijconsole-.*" <>)= ))) (source (string-append jpkg "/bin/" ijconsole)) (dest (string-append out "/bin/ijconsole"))) (mkdir-p (dirname dest)) (symlink source dest))))) (home-page (package-home-page jpkg)) (synopsis "Provide `ijconsole' symlink to default interpreter version") (description "The interpreter provided by the J package has a filename like ijconsole-, which provides support for having multiple, concurrent= versions installed. This package provides a version-agnostic `ijconsole' symlink to interpreter version indicated and build time.") (license license:gpl3+))) ------_=_761356131b06de4d0b2d440f_=_--