[bug#68261] [PATCH 0/2] Support 16K pages sizes with jemalloc on aarch64

[bug#68261] [PATCH 0/2] Support 16K pages sizes with jemalloc on aarch64

[Roman Scherer]

Hello Guix,

I'm running Guix system on an Apple M1 using the Asahi Linux kernel. It works,
but there is a major issue due to the fact that the Asahi Linux kernel uses a
16k page size.

There are some programs that are incompatible with this and are being
fixed. You can find more information under the "Known broken applications"
section in [1].

One of those programs that has issues is jemalloc, used by some of the most
heavy Guix packages, like rust and icecat. Running any program that uses
jemalloc crashes with the following error:

```
roman@localhost guix]$ rustc
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
terminate called without an active exception
Aborted
```

This is because jemalloc is compiled to handle 4K page sizes by default. If
jemalloc is configured to handle larger page sizes it is working. From what I
understand you can use jemalloc configured with page size X on all systems
with a page size <= X, with a small cost in performance. There is an issue
here [2] that has a more detailed discussion.

Note that this only happens when I use substitutes. If I compile jemalloc
myself on a system with 16k page size it works. Compiling the whole rust
toolchain and icecat on a Guix updates however is not really practical.

I already tried to fix this issue a while ago here [3], but unfortunatly this
did not solve it. The Guix substitutes are still not compatible on a system
with 16k page size.

It looks like some distros compile jemalloc with a larger page size. I belive
the Asahi Fedora remix is doing this and Arch Linux ARM [4].

This patch series configures jemalloc to support 16k page sizes on the aarch64
architecture. It uses the --with-lg-page switch which specifies the page size
as log2(16384)=14 for 16k pages.

To make packages using jemalloc via rust-jemalloc-sys compatible, the same is
done by specifying the JEMALLOC_SYS_WITH_LG_PAGE environment variable.

Could you please review the patch series and/or help me to get substitutes
available that are compatible with larger page sizes?

Thanks, Roman.

[1] https://asahilinux.org/2022/03/asahi-linux-alpha-release/
[2] https://github.com/jemalloc/jemalloc/issues/467
[3] https://issues.guix.gnu.org/61461
[4] https://archlinuxarm.org/packages/aarch64/jemalloc

Roman Scherer (2):
  gnu: jemalloc: Build with large page size.
  gnu: rust-jemalloc-sys: Build with large page size.

 gnu/packages/crates-io.scm | 6 +++++-
 gnu/packages/jemalloc.scm  | 2 ++
 2 files changed, 7 insertions(+), 1 deletion(-)

base-commit: ac69b423865f12310cef5662d9c303aa4b90c869
--
2.41.0

[bug#68261] [PATCH 1/2] gnu: jemalloc: Build with large page size.

[Roman Scherer]

* gnu/packages/jemalloc.scm (jemalloc): Build with large page size.

Change-Id: Ic813e7b0fe4c7ee79a1e703247abea77ad9d53f0
---
 gnu/packages/jemalloc.scm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/gnu/packages/jemalloc.scm b/gnu/packages/jemalloc.scm
index 5e7facfd5e..b5fdd39921 100644
--- a/gnu/packages/jemalloc.scm
+++ b/gnu/packages/jemalloc.scm
@@ -62,6 +62,8 @@ (define-public jemalloc
       ;; https://github.com/jemalloc/jemalloc/issues/937
       #~'("--disable-initial-exec-tls"
           #$@(match (%current-system)
+               ("aarch64-linux"
+                (list "--with-lg-page=14"))
                ("powerpc-linux"
                 (list "CPPFLAGS=-maltivec"))
                (_
-- 
2.41.0

[bug#68261] [PATCH 2/2] gnu: rust-jemalloc-sys: Build with large page size.

[Roman Scherer]

* gnu/packages/crates-io.scm (rust-jemalloc-sys): Build with large page size.

Change-Id: I91ed8450952204c1ecba19604521dd8b8ec554ec
---
 gnu/packages/crates-io.scm | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 1bfd8fb143..f8eb0c293f 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -39286,7 +39286,11 @@ (define-public rust-jemalloc-sys-0.5
                ;; https://github.com/tikv/jemallocator/issues/19
                (setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1")
                (setenv "JEMALLOC_OVERRIDE"
-                       (string-append jemalloc "/lib/libjemalloc_pic.a"))))))))
+                       (string-append jemalloc "/lib/libjemalloc_pic.a")))))
+         (add-after 'configure 'with-lg-page-jemalloc
+           (lambda _
+             (when (target-aarch64?)
+               (setenv "JEMALLOC_SYS_WITH_LG_PAGE" "14")))))))
     (native-inputs
      (list jemalloc))
     (home-page "https://github.com/tikv/jemallocator")
-- 
2.41.0

[bug#68261] [PATCH 2/2] gnu: rust-jemalloc-sys: Build with large page size.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1743 bytes --]

Since we don't carry the results of one rust package to the next and
have to rebuild everything, it would be better to add this to
(guix build cargo-build-system), in the configure phase. Then it will
apply to every package which is built using the cargo-build-system.

On Fri, Jan 05, 2024 at 01:36:42PM +0100, Roman Scherer wrote:
> * gnu/packages/crates-io.scm (rust-jemalloc-sys): Build with large page size.
> 
> Change-Id: I91ed8450952204c1ecba19604521dd8b8ec554ec
> ---
>  gnu/packages/crates-io.scm | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
> index 1bfd8fb143..f8eb0c293f 100644
> --- a/gnu/packages/crates-io.scm
> +++ b/gnu/packages/crates-io.scm
> @@ -39286,7 +39286,11 @@ (define-public rust-jemalloc-sys-0.5
>                 ;; https://github.com/tikv/jemallocator/issues/19
>                 (setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1")
>                 (setenv "JEMALLOC_OVERRIDE"
> -                       (string-append jemalloc "/lib/libjemalloc_pic.a"))))))))
> +                       (string-append jemalloc "/lib/libjemalloc_pic.a")))))
> +         (add-after 'configure 'with-lg-page-jemalloc
> +           (lambda _
> +             (when (target-aarch64?)
> +               (setenv "JEMALLOC_SYS_WITH_LG_PAGE" "14")))))))
>      (native-inputs
>       (list jemalloc))
>      (home-page "https://github.com/tikv/jemallocator")
> -- 
> 2.41.0
> 
> 
> 

-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#68261] [PATCH v2 1/2] gnu: jemalloc: Build with large page size.

[Roman Scherer]

* gnu/packages/jemalloc.scm (jemalloc): Build with large page size.

Change-Id: Ic813e7b0fe4c7ee79a1e703247abea77ad9d53f0
---
 gnu/packages/jemalloc.scm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/gnu/packages/jemalloc.scm b/gnu/packages/jemalloc.scm
index 5e7facfd5e..b5fdd39921 100644
--- a/gnu/packages/jemalloc.scm
+++ b/gnu/packages/jemalloc.scm
@@ -62,6 +62,8 @@ (define-public jemalloc
       ;; https://github.com/jemalloc/jemalloc/issues/937
       #~'("--disable-initial-exec-tls"
           #$@(match (%current-system)
+               ("aarch64-linux"
+                (list "--with-lg-page=14"))
                ("powerpc-linux"
                 (list "CPPFLAGS=-maltivec"))
                (_

base-commit: ac69b423865f12310cef5662d9c303aa4b90c869
-- 
2.41.0

[bug#68261] [PATCH v2 2/2] build/cargo-build-system: Support 16k page sizes on aarch64.

[Roman Scherer]

* guix/build/cargo-build-system.scm (configure): Support 16k page sizes on aarch64.

Change-Id: I523c192159908483577301da246d75d16b694bc8
---
 guix/build/cargo-build-system.scm | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index ffb2ec898e..c5c2299a4c 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -119,7 +119,7 @@ (define* (check-for-pregenerated-files #:rest _)
       (error "Possible pre-generated files found:" pregenerated-files))))
 
 (define* (configure #:key inputs
-                    target
+                    target system
                     (vendor-dir "guix-vendor")
                     #:allow-other-keys)
   "Vendor Cargo.toml dependencies as guix inputs."
@@ -178,6 +178,10 @@ (define* (configure #:key inputs
     ;; Prevent targeting the build machine.
     (setenv "CRATE_CC_NO_DEFAULTS" "1"))
 
+  ;; Support 16k kernel page sizes on aarch64 with jemalloc.
+  (when (string-prefix? "aarch64" (or target system))
+    (setenv "JEMALLOC_SYS_WITH_LG_PAGE" "14"))
+
   ;; Configure cargo to actually use this new directory with all the crates.
   (setenv "CARGO_HOME" (string-append (getcwd) "/.cargo"))
   (mkdir-p ".cargo")
-- 
2.41.0

[bug#68261] [PATCH 2/2] gnu: rust-jemalloc-sys: Build with large page size.

[Roman Scherer]

[-- Attachment #1: Type: text/plain, Size: 1841 bytes --]


Hi Efraim,

thanks for the review! I moved the setting of the
JEMALLOC_SYS_WITH_LG_PAGE now to the cargo build system, as you
suggested, and submitted a v2 of the patch series.

Can you have another look, please?

Thanks, Roman.

Efraim Flashner <efraim@flashner.co.il> writes:

> [[PGP Signed Part:Undecided]]
> Since we don't carry the results of one rust package to the next and
> have to rebuild everything, it would be better to add this to
> (guix build cargo-build-system), in the configure phase. Then it will
> apply to every package which is built using the cargo-build-system.
>
> On Fri, Jan 05, 2024 at 01:36:42PM +0100, Roman Scherer wrote:
>> * gnu/packages/crates-io.scm (rust-jemalloc-sys): Build with large page size.
>>
>> Change-Id: I91ed8450952204c1ecba19604521dd8b8ec554ec
>> ---
>>  gnu/packages/crates-io.scm | 6 +++++-
>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
>> index 1bfd8fb143..f8eb0c293f 100644
>> --- a/gnu/packages/crates-io.scm
>> +++ b/gnu/packages/crates-io.scm
>> @@ -39286,7 +39286,11 @@ (define-public rust-jemalloc-sys-0.5
>>                 ;; https://github.com/tikv/jemallocator/issues/19
>>                 (setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1")
>>                 (setenv "JEMALLOC_OVERRIDE"
>> -                       (string-append jemalloc "/lib/libjemalloc_pic.a"))))))))
>> +                       (string-append jemalloc "/lib/libjemalloc_pic.a")))))
>> +         (add-after 'configure 'with-lg-page-jemalloc
>> +           (lambda _
>> +             (when (target-aarch64?)
>> +               (setenv "JEMALLOC_SYS_WITH_LG_PAGE" "14")))))))
>>      (native-inputs
>>       (list jemalloc))
>>      (home-page "https://github.com/tikv/jemallocator")
>> --
>> 2.41.0
>>
>>
>>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 528 bytes --]