[bug#55431] [PATCH] guix: cpu: recognize other architectures.

[bug#55431] [PATCH] guix: cpu: recognize other architectures.

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 1533 bytes --]

Hi Guix!

The attached patch lets (guix cpu) recognize other architectures. The
code of (current-cpu) is based on the content of /proc/cpuinfo which
can be pretty different on non-intel architectures. For instance,
here's a sample from an armhf machine:

processor	: 0
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 45.47
Features	: half thumb fastmult vfp edsp thumbee neon vfpv3 tls
vfpv4 idiva idivt vfpd32 lpae evtstrm CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xc07
CPU revision	: 4

In particular, there's no flags entry, so (current-cpu) doesn't stop
until eof, and returns #f.

It's an issue because a test uses this code, for testing manifests with
--tune. If no cpu is returned, the test crashes:

In guix/transformations.scm:
   864:25  1 (_ _ _ _ ((package ad-hoc-package "gcc-toolchain")
(<E2><80><A6>) <E2><80><A6>)) In guix/cpu.scm:
     94:2  0 (cpu->gcc-architecture #f)

Since the test fails, the "guix" package doesn't build, and I can't
reconfigure on armhf or aarch64. (well armhf has other issues right
now...)

The attached patch changes the logic of the code to read all lines,
find information about the CPU even if it's an ARM CPU, and returns
always something (to prevent the crash) when it reads eof. This means
that it will return architecture information about the last CPU,
instead of the first. I don't think that's an issue because this code
is used for --tune which really only works on intel where you don't
have multiple CPUs with too different features.

WDYT?

[-- Attachment #2: 0001-guix-cpu-Recognize-other-architectures.patch --]
[-- Type: text/x-patch, Size: 3085 bytes --]

From 32ebfc93f838eba74c1f1188f7d23363f32ee94b Mon Sep 17 00:00:00 2001
From: Julien Lepiller <julien@lepiller.eu>
Date: Sun, 15 May 2022 19:01:37 +0200
Subject: [PATCH] guix: cpu: Recognize other architectures.

* guix/cpu.scm (current-cpu): Recognize aarch64.
---
 guix/cpu.scm | 35 ++++++++++++++++++++++++++---------
 1 file changed, 26 insertions(+), 9 deletions(-)

diff --git a/guix/cpu.scm b/guix/cpu.scm
index a44cd082f1..7e25fcb9cf 100644
--- a/guix/cpu.scm
+++ b/guix/cpu.scm
@@ -62,31 +62,48 @@ (define (prefix? prefix)
       (lambda (port)
         (let loop ((vendor #f)
                    (family #f)
-                   (model #f))
+                   (model #f)
+                   (flags '()))
           (match (read-line port)
             ((? eof-object?)
-             #f)
+             (cpu (utsname:machine (uname))
+                  vendor family model (list->set flags)))
             ((? (prefix? "vendor_id") str)
              (match (string-tokenize str)
                (("vendor_id" ":" vendor)
-                (loop vendor family model))))
+                (loop vendor family model flags))))
+            ((? (prefix? "CPU implementer") str)
+             (match (string-tokenize str)
+               (("CPU" "implementer" ":" vendor)
+                (loop vendor family model flags))))
             ((? (prefix? "cpu family") str)
              (match (string-tokenize str)
                (("cpu" "family" ":" family)
-                (loop vendor (string->number family) model))))
+                (loop vendor (string->number family) model flags))))
+            ((? (prefix? "CPU architecture") str)
+             (match (string-tokenize str)
+               (("CPU" "architecture:" family)
+                (loop vendor family model flags))))
             ((? (prefix? "model") str)
              (match (string-tokenize str)
                (("model" ":" model)
-                (loop vendor family (string->number model)))
+                (loop vendor family (string->number model) flags))
                (_
-                (loop vendor family model))))
+                (loop vendor family model flags))))
+            ((? (prefix? "CPU part") str)
+             (match (string-tokenize str)
+               (("CPU" "part" ":" model)
+                (loop vendor family (string->number (substring model 2) 16) flags))))
             ((? (prefix? "flags") str)
              (match (string-tokenize str)
                (("flags" ":" flags ...)
-                (cpu (utsname:machine (uname))
-                     vendor family model (list->set flags)))))
+                (loop vendor family model flags))))
+            ((? (prefix? "Features") str)
+             (match (string-tokenize str)
+               (("Features" ":" flags ...)
+                (loop vendor family model flags))))
             (_
-             (loop vendor family model))))))))
+             (loop vendor family model flags))))))))
 
 (define (cpu->gcc-architecture cpu)
   "Return the architecture name, suitable for GCC's '-march' flag, that
-- 
2.35.1

bug#55431: [PATCH] guix: cpu: recognize other architectures.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2410 bytes --]

On Sun, May 15, 2022 at 07:11:32PM +0200, Julien Lepiller wrote:
> Hi Guix!
> 
> The attached patch lets (guix cpu) recognize other architectures. The
> code of (current-cpu) is based on the content of /proc/cpuinfo which
> can be pretty different on non-intel architectures. For instance,
> here's a sample from an armhf machine:
> 
> processor	: 0
> model name	: ARMv7 Processor rev 4 (v7l)
> BogoMIPS	: 45.47
> Features	: half thumb fastmult vfp edsp thumbee neon vfpv3 tls
> vfpv4 idiva idivt vfpd32 lpae evtstrm CPU implementer	: 0x41
> CPU architecture: 7
> CPU variant	: 0x0
> CPU part	: 0xc07
> CPU revision	: 4
> 
> In particular, there's no flags entry, so (current-cpu) doesn't stop
> until eof, and returns #f.
> 
> It's an issue because a test uses this code, for testing manifests with
> --tune. If no cpu is returned, the test crashes:
> 
> In guix/transformations.scm:
>    864:25  1 (_ _ _ _ ((package ad-hoc-package "gcc-toolchain")
> (<E2><80><A6>) <E2><80><A6>)) In guix/cpu.scm:
>      94:2  0 (cpu->gcc-architecture #f)
> 
> Since the test fails, the "guix" package doesn't build, and I can't
> reconfigure on armhf or aarch64. (well armhf has other issues right
> now...)
> 
> The attached patch changes the logic of the code to read all lines,
> find information about the CPU even if it's an ARM CPU, and returns
> always something (to prevent the crash) when it reads eof. This means
> that it will return architecture information about the last CPU,
> instead of the first. I don't think that's an issue because this code
> is used for --tune which really only works on intel where you don't
> have multiple CPUs with too different features.
> 
> WDYT?

I just pushed mine without seeing yours, sorry.

I did check the gcc source code and I found the options for determining
the cpu flags for arm* processors in gcc/config/arm/arm-cpus.in. Do you
think it'd be worth it to add detection for armv7 CPUs?

Also, I'm pretty sure we can overlap armhf and aarch64 together, and
i686 and x86_64 together, and then running 32-bit code on 64-bit
processors will get a nice boost since it'll be tuned for the actual
hardware.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#55431] [PATCH] guix: cpu: recognize other architectures.

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 2849 bytes --]

Thanks for fixing that anyway :)

I was only interested in it because it made me unable to reconfigure my systems. I don't care too much about the --tune code, and I don't understand what you suggest. Do you mean to create "x86 on x86_64" as a separate architecture? Or as a --tune target?

On May 17, 2022 2:59:15 PM GMT+02:00, Efraim Flashner <efraim@flashner.co.il> wrote:
>On Sun, May 15, 2022 at 07:11:32PM +0200, Julien Lepiller wrote:
>> Hi Guix!
>> 
>> The attached patch lets (guix cpu) recognize other architectures. The
>> code of (current-cpu) is based on the content of /proc/cpuinfo which
>> can be pretty different on non-intel architectures. For instance,
>> here's a sample from an armhf machine:
>> 
>> processor	: 0
>> model name	: ARMv7 Processor rev 4 (v7l)
>> BogoMIPS	: 45.47
>> Features	: half thumb fastmult vfp edsp thumbee neon vfpv3 tls
>> vfpv4 idiva idivt vfpd32 lpae evtstrm CPU implementer	: 0x41
>> CPU architecture: 7
>> CPU variant	: 0x0
>> CPU part	: 0xc07
>> CPU revision	: 4
>> 
>> In particular, there's no flags entry, so (current-cpu) doesn't stop
>> until eof, and returns #f.
>> 
>> It's an issue because a test uses this code, for testing manifests with
>> --tune. If no cpu is returned, the test crashes:
>> 
>> In guix/transformations.scm:
>>    864:25  1 (_ _ _ _ ((package ad-hoc-package "gcc-toolchain")
>> (<E2><80><A6>) <E2><80><A6>)) In guix/cpu.scm:
>>      94:2  0 (cpu->gcc-architecture #f)
>> 
>> Since the test fails, the "guix" package doesn't build, and I can't
>> reconfigure on armhf or aarch64. (well armhf has other issues right
>> now...)
>> 
>> The attached patch changes the logic of the code to read all lines,
>> find information about the CPU even if it's an ARM CPU, and returns
>> always something (to prevent the crash) when it reads eof. This means
>> that it will return architecture information about the last CPU,
>> instead of the first. I don't think that's an issue because this code
>> is used for --tune which really only works on intel where you don't
>> have multiple CPUs with too different features.
>> 
>> WDYT?
>
>I just pushed mine without seeing yours, sorry.
>
>I did check the gcc source code and I found the options for determining
>the cpu flags for arm* processors in gcc/config/arm/arm-cpus.in. Do you
>think it'd be worth it to add detection for armv7 CPUs?
>
>Also, I'm pretty sure we can overlap armhf and aarch64 together, and
>i686 and x86_64 together, and then running 32-bit code on 64-bit
>processors will get a nice boost since it'll be tuned for the actual
>hardware.
>
>-- 
>Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
>GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
>Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: Type: text/html, Size: 3300 bytes --]

[bug#55431] [PATCH] guix: cpu: recognize other architectures.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 982 bytes --]

On Tue, May 17, 2022 at 05:55:38PM +0200, Julien Lepiller wrote:
> Thanks for fixing that anyway :)
> 
> I was only interested in it because it made me unable to reconfigure my systems. I don't care too much about the --tune code, and I don't understand what you suggest. Do you mean to create "x86 on x86_64" as a separate architecture? Or as a --tune target?
> 

As part of the autotuning process. GCC internally groups i686 and x86_64
together when determining how to tune for --march=native, and I'm pretty
sure I've added some i686 hardware for AMD when I was adding autotuning
support for AMD CPUs. Its similar for aarch64 and armhf, there are armhf
targets internally that accept armv8.X-a targets, so there's no need to
limit armhf tuning to armv7.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]