Merging core-updates?

Merging core-updates?

[Julien Lepiller]

Hi Guix!

As discussed at Guix Days before Fosdem, we haven't merged core-updates
in a very long time. I'd volunteer to lead this effort, but I don't
know what steps I should follow. Do we have some documentation about
that?

Re: Merging core-updates?

[Andreas Enge]

Hello,

Am Sun, Feb 12, 2023 at 10:05:40AM +0100 schrieb Julien Lepiller:
> As discussed at Guix Days before Fosdem, we haven't merged core-updates
> in a very long time. I'd volunteer to lead this effort, but I don't
> know what steps I should follow. Do we have some documentation about
> that?

I volunteer to follow your lead, but also have no clue what is actually
expected.

With Chris's help, I scheduled an evaluation on the build coordinator,
which has not finished yet, so there is not yet anything to see at
   http://data.qa.guix.gnu.org/gnu/store/bjyfp7bhb9j25sw62nrkqjiivnzjxija-mpc-1.3.1.drv
But if you click down the dependency tree, some of them have been built.
And the first one already failed:
   http://data.qa.guix.gnu.org/gnu/store/0p0zs4gv8vg7rny25ki1szv4c7yk9zzb-patch-mesboot-2.5.9.drv
With a strange error:
starting phase `build'
make: stat:Makefile: sterror: unknown error
make: *** No targets specified and no makefile found.  Stop.
error: in phase 'build': uncaught exception:
srfi-34 #<condition &invoke-error [program: "make" arguments: () exit-status: 2 term-signal: #f stop-signal: #f] 1439cc0> 
phase `build' failed after 0.0 seconds
command "make" failed with status 2
Having failed three times in a row, this does not look like a transient error.

I also reconfigured bayfront to do more builds itself.

Actually I am wondering whether the first step of killing these untamed
non-feature branches would not be to build and merge staging? It is based
on master, supposed to contain only medium sized changes, but which I
suspect end up being a world-rebuilding cluster of changes.

Andreas

Re: Merging core-updates?

[Julien Lepiller]

Le Sun, 12 Feb 2023 12:06:14 +0100,
Andreas Enge <andreas@enge.fr> a écrit :

> Hello,
> 
> Am Sun, Feb 12, 2023 at 10:05:40AM +0100 schrieb Julien Lepiller:
> > As discussed at Guix Days before Fosdem, we haven't merged
> > core-updates in a very long time. I'd volunteer to lead this
> > effort, but I don't know what steps I should follow. Do we have
> > some documentation about that?  
> 
> I volunteer to follow your lead, but also have no clue what is
> actually expected.
> 
> With Chris's help, I scheduled an evaluation on the build coordinator,
> which has not finished yet, so there is not yet anything to see at
>    http://data.qa.guix.gnu.org/gnu/store/bjyfp7bhb9j25sw62nrkqjiivnzjxija-mpc-1.3.1.drv
> But if you click down the dependency tree, some of them have been
> built. And the first one already failed:
>    http://data.qa.guix.gnu.org/gnu/store/0p0zs4gv8vg7rny25ki1szv4c7yk9zzb-patch-mesboot-2.5.9.drv
> With a strange error:
> starting phase `build'
> make: stat:Makefile: sterror: unknown error
> make: *** No targets specified and no makefile found.  Stop.
> error: in phase 'build': uncaught exception:
> srfi-34 #<condition &invoke-error [program: "make" arguments: ()
> exit-status: 2 term-signal: #f stop-signal: #f] 1439cc0> phase
> `build' failed after 0.0 seconds command "make" failed with status 2
> Having failed three times in a row, this does not look like a
> transient error.
> 
> I also reconfigured bayfront to do more builds itself.
> 
> Actually I am wondering whether the first step of killing these
> untamed non-feature branches would not be to build and merge staging?
> It is based on master, supposed to contain only medium sized changes,
> but which I suspect end up being a world-rebuilding cluster of
> changes.
> 
> Andreas
> 

I just tried to build mpc on my machine, from core-updates. I get the
same derivation as the one shown on the data service, and it built
fine. Maybe there's something wrong with the machines behind bordeaux?
I probably got substitutes from berlin, for gcc and friends (though I
built gmp, mpfr and mpc).

Re: Merging core-updates?

[Julien Lepiller]

Le Sun, 12 Feb 2023 12:52:51 +0100,
Julien Lepiller <julien@lepiller.eu> a écrit :

> Le Sun, 12 Feb 2023 12:06:14 +0100,
> Andreas Enge <andreas@enge.fr> a écrit :
> 
> I just tried to build mpc on my machine, from core-updates. I get the
> same derivation as the one shown on the data service, and it built
> fine. Maybe there's something wrong with the machines behind bordeaux?
> I probably got substitutes from berlin, for gcc and friends (though I
> built gmp, mpfr and mpc).
> 

And I was able to rebuild (with --check) patch-mesboot. The error looks
a lot like https://issues.guix.gnu.org/49985. We should fix that indeed
:)

Re: Merging core-updates?

[Leo Famulari]

On Sun, Feb 12, 2023 at 10:05:40AM +0100, Julien Lepiller wrote:
> As discussed at Guix Days before Fosdem, we haven't merged core-updates
> in a very long time. I'd volunteer to lead this effort, but I don't
> know what steps I should follow. Do we have some documentation about
> that?

The process could be considered "free form".

Basically, you can try to reconfigure your system on core-updates. It
will fail, many build failures will have to be fixed, and eventually it
will work.

Then I would run the Guix test suite and the system tests, and also
try to make use of QA.

It's likely that different architectures will have some specific
problems to be fixed.

Re: Merging core-updates?

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1218 bytes --]


Julien Lepiller <julien@lepiller.eu> writes:

> Le Sun, 12 Feb 2023 12:52:51 +0100,
> Julien Lepiller <julien@lepiller.eu> a écrit :
>
>> Le Sun, 12 Feb 2023 12:06:14 +0100,
>> Andreas Enge <andreas@enge.fr> a écrit :
>> 
>> I just tried to build mpc on my machine, from core-updates. I get the
>> same derivation as the one shown on the data service, and it built
>> fine. Maybe there's something wrong with the machines behind bordeaux?
>> I probably got substitutes from berlin, for gcc and friends (though I
>> built gmp, mpfr and mpc).
>
> And I was able to rebuild (with --check) patch-mesboot. The error looks
> a lot like https://issues.guix.gnu.org/49985. We should fix that indeed
> :)

It could be something "wrong" with a build machine, although my
suspicion for a while is that various things are broken/don't work on
btrfs, which milano-guix-1 (the most important x86_64-linux) build
machine uses.

I've filed a few issues that probably relate to this:

https://issues.guix.gnu.org/53415
https://issues.guix.gnu.org/53416
https://issues.guix.gnu.org/60202

There are other build machines that don't use btrfs, so it should be
possible to get these things built anyway.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: Merging core-updates?

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 603 bytes --]


Julien Lepiller <julien@lepiller.eu> writes:

> As discussed at Guix Days before Fosdem, we haven't merged core-updates
> in a very long time. I'd volunteer to lead this effort, but I don't
> know what steps I should follow. Do we have some documentation about
> that?

I can try and help with this, at least in terms of helping get
bordeaux.guix.gnu.org substitute availability to a good level.

I'd also like to get the kind of comparison that the qa-frontpage can do
for patches working for branches, but that'll probably take a bit of
work in the data service/qa-frontpage to get working smoothly.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: Merging core-updates?

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 981 bytes --]

Hi everyone,

Andreas Enge <andreas@enge.fr> writes:

> I volunteer to follow your lead, but also have no clue what is actually
> expected.

I would also like to give a hand!

> [...]
> Actually I am wondering whether the first step of killing these untamed
> non-feature branches would not be to build and merge staging? It is based
> on master, supposed to contain only medium sized changes, but which I
> suspect end up being a world-rebuilding cluster of changes.

You're right, for this to smoothly transition into the "feature branch
workflow" (if that's actually what we want to do) I guess we also need
to lay out a plan first, and prepare for the post-c.u-merge
world. Getting rid of staging first could be an easier exercise,
followed by c-u. I was planning on taking your notes of the Guix days
and opening a discussion about how we could concretely transition to
this new workflow, I can maybe do that this evening.

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: Merging core-updates?

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 681 bytes --]

On Sun, Feb 12, 2023 at 10:05:40AM +0100, Julien Lepiller wrote:
> Hi Guix!
> 
> As discussed at Guix Days before Fosdem, we haven't merged core-updates
> in a very long time. I'd volunteer to lead this effort, but I don't
> know what steps I should follow. Do we have some documentation about
> that?

I think we normally have a 2 week last-chance window to get all sorts of
last minute packages bumped and then we freeze it and try to build
"everything".

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Merging core-updates?

[Andreas Enge]

Am Sun, Feb 12, 2023 at 12:58:06PM +0100 schrieb Julien Lepiller:
> And I was able to rebuild (with --check) patch-mesboot. The error looks
> a lot like https://issues.guix.gnu.org/49985. We should fix that indeed
> :)

Ah indeed, that looks like deal breaking; maybe someone from MES can have
a look?

What is the magic incantation with double "@@" to build this package?
Ah, here we go, for reference to self:
   guix build -e '(@@ (gnu packages commencement) patch-mesboot)'

Andreas

Re: Merging core-updates?

[Kaelyn]

Hi,

------- Original Message -------
On Sunday, February 12th, 2023 at 5:08 PM, Andreas Enge <andreas@enge.fr> wrote:

>
>
> Am Sun, Feb 12, 2023 at 12:58:06PM +0100 schrieb Julien Lepiller:
>
> > And I was able to rebuild (with --check) patch-mesboot. The error looks
> > a lot like https://issues.guix.gnu.org/49985. We should fix that indeed
> > :)
>
>
> Ah indeed, that looks like deal breaking; maybe someone from MES can have
> a look?
>
> What is the magic incantation with double "@@" to build this package?
> Ah, here we go, for reference to self:
> guix build -e '(@@ (gnu packages commencement) patch-mesboot)'
>
> Andreas

While not directly related to the patch-mesboot error, I want to mention that there is also https://issues.guix.gnu.org/58719 blocking i686 builds on core-updates (and x86_64 builds of certain packages like wine64, which has i686 dependencies) since the update to glibc 2.35.

It may also need assistance from the MES folks to fix, since the error message is about an undefined symbol in glibc-mesboot's libpthread.so.0:

make[2]: Entering directory '/tmp/guix-build-file-5.44.drv-0/file-5.44/magic'
../src/file -C -m magic
/tmp/guix-build-file-5.44.drv-0/file-5.44/src/.libs/file: symbol lookup error: /gnu/store/s4yd6ibxsh5q1j9ipygb9vpjj4g00wc9-glibc-mesboot-2.16.0/lib/libpthread.so.0: undefined symbol: h_errno, version GLIBC_PRIVATE
make[2]: *** [Makefile:863: magic.mgc] Error 127

Cheers,
Kaelyn


P.S. I also have https://issues.guix.gnu.org/59453 open for few months to fix the Vulkan layer paths in mesa on core-updates, which I'd really like to see land before a core-updates merge happens (it had originally been part of a mesa version update patch I sent in back in October, but which had been obsoleted by a newer patch release of mesa landing in core-updates; mesa in core-updates is also a few releases behind at this point, but that's a separate matter).

Re: Merging core-updates?

[Janneke Nieuwenhuizen]

Andreas Enge writes:

> Am Sun, Feb 12, 2023 at 12:58:06PM +0100 schrieb Julien Lepiller:
>> And I was able to rebuild (with --check) patch-mesboot. The error looks
>> a lot like https://issues.guix.gnu.org/49985. We should fix that indeed
>> :)
>
> Ah indeed, that looks like deal breaking; maybe someone from MES can have
> a look?

I have backported

    DRAFT lib: stat: Use SYS_stat64 for 32bit platforms.wip

to `wip'

    https://git.savannah.gnu.org/cgit/mes.git/commit/?id=05d10877410b48d0a4f8960fd62190d70329eea7

that was slated to become Mes 0.25, after adding initial risc-v support
and some thorough testing.  If this works, maybe we should postpone
initial risc-v support to 0.26.

> What is the magic incantation with double "@@" to build this package?
> Ah, here we go, for reference to self:
>    guix build -e '(@@ (gnu packages commencement) patch-mesboot)'

Thanks.  It would be nice, however, if there was a way for me to
reproduce this bug.

Greetings,
Janneke

-- 
Janneke Nieuwenhuizen <janneke@gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

Re: Merging core-updates?

[John Kehayias]

Hi guix-devel!

On Sun, Feb 12, 2023 at 03:49 PM, Josselin Poiret wrote:

> Hi everyone,
>
> Andreas Enge <andreas@enge.fr> writes:
>
>> I volunteer to follow your lead, but also have no clue what is actually
>> expected.
>
> I would also like to give a hand!
>

Count me in as well!

I only did some spot fixes the last round, but at least I have some familiarity with what happened. I don't have any particular expertise but I'm happy to help coordinate overall, test, and generally do some cat herding.

And now that I have commit access I hope I can really help move things through with review and pushing ready patches. Unfortunately had some other stuff come up for the last few months since I got access, but I should have more time starting in a week or so.

On that front, I think looking through relevant core-updates patches (the Mesa ones in particular, for me) is a good first step for review and I'll try building on my end to see how far I get.

>> [...]
>> Actually I am wondering whether the first step of killing these untamed
>> non-feature branches would not be to build and merge staging? It is based
>> on master, supposed to contain only medium sized changes, but which I
>> suspect end up being a world-rebuilding cluster of changes.
>
> You're right, for this to smoothly transition into the "feature branch
> workflow" (if that's actually what we want to do) I guess we also need
> to lay out a plan first, and prepare for the post-c.u-merge
> world. Getting rid of staging first could be an easier exercise,
> followed by c-u. I was planning on taking your notes of the Guix days
> and opening a discussion about how we could concretely transition to
> this new workflow, I can maybe do that this evening.
>

I need to catch up on the thread about feature branches and I'm looking forward to that as well. It has something that I have long wanted and will gladly help in that transition and doing what I can to make that go smoothly.

John

Re: Merging core-updates?

[zimoun]

Hi,

On Sun, 12 Feb 2023 at 10:05, Julien Lepiller <julien@lepiller.eu> wrote:

> As discussed at Guix Days before Fosdem, we haven't merged core-updates
> in a very long time. I'd volunteer to lead this effort, but I don't
> know what steps I should follow. Do we have some documentation about
> that?

Maybe a start could be to fix: <https://ci.guix.gnu.org/eval/125433/dashboard>

Well, it could be helpful is Berlin or Bordeaux could build some
manifest of core-updates (not necessary the whole core-updates).  And
then, once the manifest builds, we could add some packages and repeat.

It would avoid that we all build the same things; worse, that each of us
burn many CPU just for knowing it fails.

Chris, Mathieu?  What do you think?


Cheers,
simon

Architecture support [was: Re: Merging core-updates?]

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1081 bytes --]

On Mon, Feb 13, 2023 at 10:43:17AM +0100, zimoun wrote:
> Hi,
> 
> Well, it could be helpful is Berlin or Bordeaux could build some
> manifest of core-updates (not necessary the whole core-updates).  And
> then, once the manifest builds, we could add some packages and repeat.
> 
> It would avoid that we all build the same things; worse, that each of us
> burn many CPU just for knowing it fails.
> 
> Chris, Mathieu?  What do you think?
> 

At least locally I try to build out to hello, and then to mesa.

Currently I believe only x86_64 is building to hello. I'm not against
downgrading file to an earlier version if it'll get i686 to get to
hello.

i686 is getting stuck at an unknown file.
Aarch64 and armhf are getting stuck at gcc-cross-boot0.
Riscv64 is getting stuck at gcc-final.
I haven't tested powerpc64le (or powerpc or mips64el).

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Merging core-updates?

[Janneke Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 1069 bytes --]

Andreas Enge writes:

> Am Sun, Feb 12, 2023 at 12:58:06PM +0100 schrieb Julien Lepiller:
>> And I was able to rebuild (with --check) patch-mesboot. The error looks
>> a lot like https://issues.guix.gnu.org/49985. We should fix that indeed
>> :)
>
> Ah indeed, that looks like deal breaking; maybe someone from MES can have
> a look?
>
> What is the magic incantation with double "@@" to build this package?
> Ah, here we go, for reference to self:
>    guix build -e '(@@ (gnu packages commencement) patch-mesboot)'

To use stat64 and friends on 32bit, I created the attached patch for GNU
Mes and hope to create a 0.24.2 release from

    https://gitlab.com/janneke/mes/-/tree/wip-stat64

Also, I have update my core-updates branch with preliminary 0.24.2 mes
and mes-boot packages here

    https://gitlab.com/janneke/guix/-/tree/core-updates

Just hoping this is also a fix for

    https://debbugs.gnu.org/41264
    https://debbugs.gnu.org/49985

as I haven't been able to reproduce this bug.  IWBN if someone could
shed more light on that...

Greetings,
Janneke


[-- Attachment #2: 0001-lib-stat-Use-SYS_stat64-for-32bit-platforms.patch --]
[-- Type: text/x-patch, Size: 59393 bytes --]

From bc1fa57851d360abb161c54dce5339ad9d7af7aa Mon Sep 17 00:00:00 2001
From: "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org>
Date: Sat, 29 Oct 2022 13:17:58 +0200
Subject: [PATCH] lib: stat: Use SYS_stat64 for 32bit platforms.

This fixes <https://debbugs.gnu.org/41264>.

* include/linux/arm/syscall.h (SYS_stat64, SYS_lstat64,
SYS_fstat64)[__SIZEOF_LONG_LONG__ == 8]:
New defines.
(SYS_stat, SYS_lstat, SYS_fstat)[__SIZEOF_LONG_LONG__ == 8]: Redefine them.
* include/linux/x86/syscall.h (SYS_stat64, SYS_lstat64,
SYS_fstat64)[__SIZEOF_LONG_LONG__ == 8]:
New defines.
(SYS_stat, SYS_lstat, SYS_fstat)[__SIZEOF_LONG_LONG__ == 8]: Redefine them.
* include/sys/stat.h (struct stat): Move definition to...
* include/linux/arm/kernel-stat.h,
include/linux/m2/kernel-stat.h,
include/linux/x86/kernel-stat.h,
include/linux/x86_64/kernel-stat.h: These new files.
* include/gnu/x86/kernel-stat.h: New file.
* configure (main): Copy <srcdest>include/<kernel>/<arch>/*.h to
include/.
* configure.sh: Likewise.
* .gitignore: Ignore them.  Add copyright header.
* build-aux/GNUmakefile.in (X86_ARCH_HEADERS, ARCH_HEADERS): New
variables.
(build): Use them.
(include/arch/%.h, arch-dir): New targets.
* build-aux/bootstrap.sh.in (AM_CPPFLAGS): Replace
<srcdest>include/<kernel>/<cpu> with built ../include.
* build-aux/build.sh.in (AM_CPPFLAGS): Likewise.
* build-aux/install.sh.in: Also install built include.
* include/m2/types.h: New file.
* kaem.run: Use it.
* simple.sh: Copy kernel-stat.h, syscall.h for kernel/cpu to
include/arch.
---
 .gitignore                         |  19 ++++
 build-aux/GNUmakefile.in           |  11 ++-
 build-aux/bootstrap.sh.in          |   4 +-
 build-aux/build.sh.in              |  11 ++-
 build-aux/install.sh.in            |   3 +-
 configure                          |   7 ++
 configure.sh                       |   4 +
 include/gnu/x86/kernel-stat.h      |  25 ++++++
 include/linux/arm/kernel-stat.h    |  79 +++++++++++++++++
 include/linux/arm/syscall.h        |  19 ++++
 include/linux/m2/kernel-stat.h     |  47 ++++++++++
 include/linux/x86/kernel-stat.h    |  79 +++++++++++++++++
 include/linux/x86/syscall.h        |  20 ++++-
 include/linux/x86_64/kernel-stat.h |  51 +++++++++++
 include/m2/types.h                 | 138 +++++++++++++++++++++++++++++
 include/sys/stat.h                 |  51 +----------
 kaem.run                           |   3 +
 lib/linux/_getcwd.c                |   4 +-
 lib/linux/_open3.c                 |   2 +-
 lib/linux/_read.c                  |   4 +-
 lib/linux/access.c                 |   4 +-
 lib/linux/brk.c                    |   4 +-
 lib/linux/chdir.c                  |   4 +-
 lib/linux/chmod.c                  |   4 +-
 lib/linux/clock_gettime.c          |   4 +-
 lib/linux/close.c                  |   4 +-
 lib/linux/dup.c                    |   4 +-
 lib/linux/dup2.c                   |   4 +-
 lib/linux/execve.c                 |   4 +-
 lib/linux/fcntl.c                  |   4 +-
 lib/linux/fork.c                   |   4 +-
 lib/linux/fstat.c                  |   4 +-
 lib/linux/fsync.c                  |   4 +-
 lib/linux/getdents.c               |   4 +-
 lib/linux/getegid.c                |   4 +-
 lib/linux/geteuid.c                |   4 +-
 lib/linux/getgid.c                 |   4 +-
 lib/linux/getpid.c                 |   4 +-
 lib/linux/getppid.c                |   4 +-
 lib/linux/getrusage.c              |   4 +-
 lib/linux/gettimeofday.c           |   4 +-
 lib/linux/getuid.c                 |   4 +-
 lib/linux/ioctl.c                  |   2 +-
 lib/linux/ioctl3.c                 |   2 +-
 lib/linux/kill.c                   |   4 +-
 lib/linux/link.c                   |   4 +-
 lib/linux/lseek.c                  |   2 +-
 lib/linux/lstat.c                  |   4 +-
 lib/linux/mkdir.c                  |   4 +-
 lib/linux/mknod.c                  |   4 +-
 lib/linux/nanosleep.c              |   4 +-
 lib/linux/pipe.c                   |   4 +-
 lib/linux/read.c                   |   2 +-
 lib/linux/readlink.c               |   4 +-
 lib/linux/rename.c                 |   4 +-
 lib/linux/rmdir.c                  |   4 +-
 lib/linux/setgid.c                 |   4 +-
 lib/linux/settimer.c               |   4 +-
 lib/linux/setuid.c                 |   4 +-
 lib/linux/signal.c                 |   4 +-
 lib/linux/sigprogmask.c            |   4 +-
 lib/linux/stat.c                   |   4 +-
 lib/linux/symlink.c                |   4 +-
 lib/linux/time.c                   |   2 +-
 lib/linux/unlink.c                 |   4 +-
 lib/linux/waitpid.c                |   2 +-
 lib/m2/execve.c                    |   4 +-
 lib/m2/open.c                      |   2 +
 lib/m2/time.c                      |   2 +-
 lib/m2/waitpid.c                   |   2 +-
 simple.sh                          |  12 ++-
 71 files changed, 617 insertions(+), 158 deletions(-)
 create mode 100644 include/gnu/x86/kernel-stat.h
 create mode 100644 include/linux/arm/kernel-stat.h
 create mode 100644 include/linux/m2/kernel-stat.h
 create mode 100644 include/linux/x86/kernel-stat.h
 create mode 100644 include/linux/x86_64/kernel-stat.h
 create mode 100644 include/m2/types.h

diff --git a/.gitignore b/.gitignore
index 58cb2afb..72ba34d8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,21 @@
+# GNU Mes --- Maxwell Equations of Software
+# Copyright © 2016,2017,2019,2020,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+#
+# This file is part of GNU Mes.
+#
+# GNU Mes is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Mes is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Mes.  If not, see <http://www.gnu.org/licenses/>.
+
 *-
 *~
 .#*
@@ -117,6 +135,7 @@
 /doc/images/gcc-mesboot-graph.pdf
 /doc/web/
 /config.sh
+/include/arch
 /include/mes/config.h
 /gcc-lib
 /mescc-lib
diff --git a/build-aux/GNUmakefile.in b/build-aux/GNUmakefile.in
index d74e9f99..c3304192 100644
--- a/build-aux/GNUmakefile.in
+++ b/build-aux/GNUmakefile.in
@@ -85,17 +85,26 @@ PHONY_TARGETS:=\
 
 .PHONY: $(PHONY_TARGETS)
 
+X86_ARCH_HEADERS = $(wildcard $(scrdest)include/linux/x86/*.h)
+ARCH_HEADERS = $(X86_ARCH_HEADERS:$(srcdest)include/linux/x86/%=include/arch/%)
+
 default: all
 
 all: doc
 
 doc: build
 
-build:
+build: | $(ARCH_HEADERS)
 	$(SHELL) build.sh
 
 src/${program_prefix}mes: build
 
+include/arch/%.h: $(srcdest)include/$(mes_kernel)/$(mes_cpu)/%.h | arch-dir
+	cp -f $< $@
+
+arch-dir:
+	mkdir -p include/arch
+
 clean:
 	rm -f *.o *.s bin/mes bin/mes-gcc bin/mes-mescc
 	rm -f mes.{aux,cp,cps,fn,info,log,tmp,toc,vr,vrs}
diff --git a/build-aux/bootstrap.sh.in b/build-aux/bootstrap.sh.in
index b2d56c50..a2451d02 100644
--- a/build-aux/bootstrap.sh.in
+++ b/build-aux/bootstrap.sh.in
@@ -50,7 +50,7 @@ srcdest=../${srcdest}
 ln -sf ${srcdest}mes .
 ln -sf ${srcdest}module .
 ln -sf ${srcdest}src .
-AM_CPPFLAGS="-D HAVE_CONFIG_H=1 -I ${srcdest}include -I ${srcdest}include/$mes_kernel/$mes_cpu"
+AM_CPPFLAGS="-D HAVE_CONFIG_H=1 -I ${srcdest}include -I ../include -I include"
 AM_CFLAGS="-L ${srcdest}lib"
 
 mkdir -p $mes_cpu-mes
@@ -108,7 +108,7 @@ $AR crD $mes_cpu-mes/libc+tcc.a $objects
 
 cd ..
 srcdest=
-CPPFLAGS="-D HAVE_CONFIG_H=1 -I ${srcdest}include -I ${srcdest}include/$mes_kernel/$mes_cpu"
+AM_CPPFLAGS="-D HAVE_CONFIG_H=1 -I ${srcdest}include -I ../include -I include"
 AM_CFLAGS="-L ${srcdest}lib"
 
 objects=
diff --git a/build-aux/build.sh.in b/build-aux/build.sh.in
index f4a6a264..3f356e10 100644
--- a/build-aux/build.sh.in
+++ b/build-aux/build.sh.in
@@ -66,9 +66,8 @@ fi
         AM_CPPFLAGS="
 -D HAVE_CONFIG_H=1
 -I ${srcdest}lib
--I include
 -I ${srcdest}include
--I ${srcdest}include/$mes_kernel/$mes_cpu
+-I ../include
 "
         if test $mes_kernel = gnu; then
             AM_CPPFLAGS="$AM_CPPFLAGS
@@ -93,9 +92,9 @@ fi
     AM_CPPFLAGS="
 -D HAVE_CONFIG_H=1
 -I ${srcdest}lib
--I include
 -I ${srcdest}include
--I ${srcdest}include/$mes_kernel/$mes_cpu
+-I ../include
+-I include
 "
     if test "$compiler" != bootstrap; then
         ${SHELL} ${srcdest}build-aux/build-mes.sh
@@ -137,9 +136,9 @@ fi
     AM_CPPFLAGS="
 -D HAVE_CONFIG_H=1
 -I ${srcdest}lib
--I include
 -I ${srcdest}include
--I ${srcdest}include/$mes_kernel/$mes_cpu
+-I ../include
+-I include
 "
     compiler=mescc
     AR=${MESAR-"${srcdest}pre-inst-env mesar"}
diff --git a/build-aux/install.sh.in b/build-aux/install.sh.in
index 340faf5f..f3a69229 100644
--- a/build-aux/install.sh.in
+++ b/build-aux/install.sh.in
@@ -1,7 +1,7 @@
 #! @SHELL@
 
 # GNU Mes --- Maxwell Equations of Software
-# Copyright © 2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+# Copyright © 2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
 #
 # This file is part of GNU Mes.
 #
@@ -99,6 +99,7 @@ mkdir -p $DESTDIR$includedir
 mkdir -p $DESTDIR$libdir
 mkdir -p $DESTDIR$pkgdatadir
 tar -cf- -C ${srcdir}/include . | tar -${v}xf- -C $DESTDIR$includedir
+tar -cf- -C include . | tar -${v}xf- -C $DESTDIR$includedir
 tar -cf- -C ${srcdir}/lib $mes_cpu-mes | tar -${v}xf- -C $DESTDIR$libdir
 tar -cf- -C ${srcdir}/lib $mes_kernel/$mes_cpu-mes | tar -${v}xf- -C $DESTDIR$libdir
 if test -z "$srcdest"; then
diff --git a/configure b/configure
index b02e75bf..98467094 100755
--- a/configure
+++ b/configure
@@ -707,6 +707,13 @@ See \"Porting GNU Mes\" in the manual, or try --with-courage\n" mes-system)
 #define MES_VERSION \"" VERSION "\"
 ")))))
         (substitute (string-append srcdest "build-aux/config.make.in") pairs #:target ".config.make"))
+      (let ((arch-dir (string-append srcdest "include/" mes-kernel "/" mes-cpu)))
+        (define (copy-header file-name)
+          (system* "cp" "-f" "-v"
+                   (string-append arch-dir "/" file-name)
+                   (string-append "include/arch/" file-name)))
+        (system* "mkdir" "-p" "include/arch")
+        (for-each copy-header '("kernel-stat.h" "syscall.h")))
 
       (let ((make (and=> (file-name "make" deps) basename)))
         (display (string-append "
diff --git a/configure.sh b/configure.sh
index 51d99b6f..f182530e 100755
--- a/configure.sh
+++ b/configure.sh
@@ -271,6 +271,10 @@ cat >> include/mes/config.h <<EOF
 #define MES_VERSION "$VERSION"
 EOF
 
+mkdir -p include/arch
+cp -f -v ${srcdest}include/${mes_kernel}/${mes_cpu}/kernel-stat.h include/arch
+cp -f -v ${srcdest}include/${mes_kernel}/${mes_cpu}/syscall.h include/arch
+
 cat <<EOF
 GNU Mes is configured for
    compiler:   $compiler
diff --git a/include/gnu/x86/kernel-stat.h b/include/gnu/x86/kernel-stat.h
new file mode 100644
index 00000000..7b49efe9
--- /dev/null
+++ b/include/gnu/x86/kernel-stat.h
@@ -0,0 +1,25 @@
+/* -*-comment-start: "//";comment-end:""-*-
+ * GNU Mes --- Maxwell Equations of Software
+ * Copyright © 2017,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ *
+ * This file is part of GNU Mes.
+ *
+ * GNU Mes is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 3 of the License, or (at
+ * your option) any later version.
+ *
+ * GNU Mes is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with GNU Mes.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __MES_GNU_X86_KERNEL_STAT_H
+#define __MES_GNU_X86_KERNEL_STAT_H 1
+
+#include <arch/syscall.h>
+
+#endif // __MES_GNU_X86_KERNEL_STAT_H
diff --git a/include/linux/arm/kernel-stat.h b/include/linux/arm/kernel-stat.h
new file mode 100644
index 00000000..79dc48ec
--- /dev/null
+++ b/include/linux/arm/kernel-stat.h
@@ -0,0 +1,79 @@
+/* -*-comment-start: "//";comment-end:""-*-
+ * GNU Mes --- Maxwell Equations of Software
+ * Copyright © 2017,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ *
+ * This file is part of GNU Mes.
+ *
+ * GNU Mes is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 3 of the License, or (at
+ * your option) any later version.
+ *
+ * GNU Mes is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with GNU Mes.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __MES_LINUX_ARM_KERNEL_STAT_H
+#define __MES_LINUX_ARM_KERNEL_STAT_H 1
+
+// https://github.com/torvalds/linux/blob/master/arch/arm/include/uapi/asm/stat.h
+
+#include <arch/syscall.h>
+
+#if __SIZEOF_LONG_LONG__ != 8
+
+// *INDENT-OFF*
+struct stat
+{
+  unsigned long  st_dev;
+  unsigned long  st_ino;
+  unsigned short st_mode;
+  unsigned short st_nlink;
+  unsigned short st_uid;
+  unsigned short st_gid;
+  unsigned long  st_rdev;
+  unsigned long  st_size;
+  unsigned long  st_blksize;
+  unsigned long  st_blocks;
+  unsigned long  st_atime;
+  unsigned long  st_atime_usec;
+  unsigned long  st_mtime;
+  unsigned long  st_mtime_usec;
+  unsigned long  st_ctime;
+  unsigned long  st_ctime_usec;
+  unsigned long  __pad0;
+  unsigned long  __pad1;
+};
+
+#else // __SIZEOF_LONG_LONG__ == 8
+
+struct stat
+{
+  unsigned long long st_dev;
+  unsigned char __pad0[4];
+  unsigned long __st_ino;
+  unsigned int st_mode;
+  unsigned int st_nlink;
+  unsigned long st_uid;
+  unsigned long st_gid;
+  unsigned long long st_rdev;
+  unsigned char __pad3[4];
+  long long st_size;
+  unsigned long st_blksize;
+  unsigned long long st_blocks;
+  unsigned long st_atime;
+  unsigned long st_atime_nsec;
+  unsigned long st_mtime;
+  unsigned int st_mtime_nsec;
+  unsigned long st_ctime;
+  unsigned long st_ctime_nsec;
+  unsigned long long st_ino;
+};
+
+#endif // __SIZEOF_LONG_LONG__ == 8
+
+#endif // __MES_LINUX_ARM_KERNEL_STAT_H
diff --git a/include/linux/arm/syscall.h b/include/linux/arm/syscall.h
index b04ff039..ca9f1f97 100644
--- a/include/linux/arm/syscall.h
+++ b/include/linux/arm/syscall.h
@@ -109,4 +109,23 @@
 #define SYS_readlink  0x55
 #define SYS_mknod     0x0e
 
+#if __SIZEOF_LONG_LONG__ == 8
+
+#define SYS_stat64     0xc3
+#define SYS_lstat64    0xc4
+#define SYS_fstat64    0xc5
+#define SYS_fcntl64    0xdd
+#define SYS_getdents64 0xdc
+
+#undef SYS_stat
+#define SYS_stat SYS_stat64
+
+#undef SYS_lstat
+#define SYS_lstat SYS_lstat64
+
+#undef SYS_fstat
+#define SYS_fstat SYS_fstat64
+
+#endif  // __SIZEOF_LONG_LONG__ == 8
+
 #endif /* __MES_LINUX_ARM_SYSCALL_H */
diff --git a/include/linux/m2/kernel-stat.h b/include/linux/m2/kernel-stat.h
new file mode 100644
index 00000000..c446bdbc
--- /dev/null
+++ b/include/linux/m2/kernel-stat.h
@@ -0,0 +1,47 @@
+/* -*-comment-start: "//";comment-end:""-*-
+ * GNU Mes --- Maxwell Equations of Software
+ * Copyright © 2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ *
+ * This file is part of GNU Mes.
+ *
+ * GNU Mes is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 3 of the License, or (at
+ * your option) any later version.
+ *
+ * GNU Mes is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with GNU Mes.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __LINUX_M2_KERNEL_STAT_H
+#define __LINUX_M2_KERNEL_STAT_H
+
+/* https://github.com/torvalds/linux/blob/master/arch/x86/include/uapi/asm/stat.h */
+
+/* *INDENT-OFF* */
+struct stat
+{
+  unsigned st_dev;
+  unsigned st_ino;
+  char st_mode[2];
+  char st_nlink[2];
+  char st_uid[2];
+  char st_gid[2];
+  unsigned st_rdev;
+  unsigned st_size;
+  unsigned st_blksize;
+  unsigned st_blocks;
+  unsigned st_atime;
+  unsigned st_atime_usec;
+  unsigned st_mtime;
+  unsigned st_mtime_usec;
+  unsigned st_ctime;
+  unsigned st_ctime_usec;
+  unsigned __pad0;
+  unsigned __pad1;
+};
+#endif /* __LINUX_M2_KERNEL_STAT_H */
diff --git a/include/linux/x86/kernel-stat.h b/include/linux/x86/kernel-stat.h
new file mode 100644
index 00000000..997fadc7
--- /dev/null
+++ b/include/linux/x86/kernel-stat.h
@@ -0,0 +1,79 @@
+/* -*-comment-start: "//";comment-end:""-*-
+ * GNU Mes --- Maxwell Equations of Software
+ * Copyright © 2017,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ *
+ * This file is part of GNU Mes.
+ *
+ * GNU Mes is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 3 of the License, or (at
+ * your option) any later version.
+ *
+ * GNU Mes is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with GNU Mes.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __MES_LINUX_X86_KERNEL_STAT_H
+#define __MES_LINUX_X86_KERNEL_STAT_H 1
+
+// https://github.com/torvalds/linux/blob/master/arch/x86/include/uapi/asm/stat.h
+
+#include <arch/syscall.h>
+
+#if __SIZEOF_LONG_LONG__ != 8
+
+// *INDENT-OFF*
+struct stat
+{
+  unsigned long  st_dev;
+  unsigned long  st_ino;
+  unsigned short st_mode;
+  unsigned short st_nlink;
+  unsigned short st_uid;
+  unsigned short st_gid;
+  unsigned long  st_rdev;
+  unsigned long  st_size;
+  unsigned long  st_blksize;
+  unsigned long  st_blocks;
+  unsigned long  st_atime;
+  unsigned long  st_atime_usec;
+  unsigned long  st_mtime;
+  unsigned long  st_mtime_usec;
+  unsigned long  st_ctime;
+  unsigned long  st_ctime_usec;
+  unsigned long  __pad0;
+  unsigned long  __pad1;
+};
+
+#else // __SIZEOF_LONG_LONG__ == 8
+
+struct stat
+{
+  unsigned long long st_dev;
+  unsigned char __pad0[4];
+  unsigned long __st_ino;
+  unsigned int st_mode;
+  unsigned int st_nlink;
+  unsigned long st_uid;
+  unsigned long st_gid;
+  unsigned long long st_rdev;
+  unsigned char __pad3[4];
+  long long st_size;
+  unsigned long st_blksize;
+  unsigned long long st_blocks;
+  unsigned long st_atime;
+  unsigned long st_atime_nsec;
+  unsigned long st_mtime;
+  unsigned int st_mtime_nsec;
+  unsigned long st_ctime;
+  unsigned long st_ctime_nsec;
+  unsigned long long st_ino;
+};
+
+#endif // __SIZEOF_LONG_LONG__ == 8
+
+#endif // __MES_LINUX_X86_KERNEL_STAT_H
diff --git a/include/linux/x86/syscall.h b/include/linux/x86/syscall.h
index 01c46a35..d849c175 100644
--- a/include/linux/x86/syscall.h
+++ b/include/linux/x86/syscall.h
@@ -75,7 +75,6 @@
 #define SYS_stat   0x6a
 
 /* libc+gnu */
-
 #define SYS_chdir     0x0c
 #define SYS_link      0x09
 #define SYS_getpid    0x14
@@ -112,4 +111,23 @@
 #define SYS_readlink  0x55
 #define SYS_mknod     0x0e
 
+#if __SIZEOF_LONG_LONG__ == 8
+
+#define SYS_stat64     0xc3
+#define SYS_lstat64    0xc4
+#define SYS_fstat64    0xc5
+#define SYS_fcntl64    0xdd
+#define SYS_getdents64 0xdc
+
+#undef SYS_stat
+#define SYS_stat SYS_stat64
+
+#undef SYS_lstat
+#define SYS_lstat SYS_lstat64
+
+#undef SYS_fstat
+#define SYS_fstat SYS_fstat64
+
+#endif  // __SIZEOF_LONG_LONG__ == 8
+
 #endif /* __MES_LINUX_X86_SYSCALL_H */
diff --git a/include/linux/x86_64/kernel-stat.h b/include/linux/x86_64/kernel-stat.h
new file mode 100644
index 00000000..fdb15946
--- /dev/null
+++ b/include/linux/x86_64/kernel-stat.h
@@ -0,0 +1,51 @@
+/* -*-comment-start: "//";comment-end:""-*-
+ * GNU Mes --- Maxwell Equations of Software
+ * Copyright © 2017,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ *
+ * This file is part of GNU Mes.
+ *
+ * GNU Mes is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 3 of the License, or (at
+ * your option) any later version.
+ *
+ * GNU Mes is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with GNU Mes.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __MES_LINUX_X86_64_KERNEL_STAT_H
+#define __MES_LINUX_X86_64_KERNEL_STAT_H 1
+
+// https://github.com/torvalds/linux/blob/master/arch/x86/include/uapi/asm/stat.h
+
+// *INDENT-OFF*
+struct stat
+{
+  unsigned long	st_dev;
+  unsigned long	st_ino;
+  unsigned long	st_nlink;
+  unsigned int	st_mode;
+  unsigned int	st_uid;
+  unsigned int	st_gid;
+  unsigned int	__pad0;
+  unsigned long	st_rdev;
+  unsigned long	st_size;
+  unsigned long	st_atime;
+  unsigned long	st_atime_nsec;
+  unsigned long	st_mtime;
+  unsigned long	st_mtime_nsec;
+  unsigned long	st_ctime;
+  unsigned long	st_ctime_nsec;
+  unsigned long	st_blksize;
+  long		st_blocks;
+  unsigned long	__pad1;
+  unsigned long	__pad2;
+  unsigned long	__pad3;
+  unsigned long	__pad4;
+};
+
+#endif // __MES_LINUX_X86_64_KERNEL_STAT_H
diff --git a/include/m2/types.h b/include/m2/types.h
new file mode 100644
index 00000000..c5547f5c
--- /dev/null
+++ b/include/m2/types.h
@@ -0,0 +1,138 @@
+/* -*-comment-start: "//";comment-end:""-*-
+ * GNU Mes --- Maxwell Equations of Software
+ * Copyright © 2017,2022,2023 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ *
+ * This file is part of GNU Mes.
+ *
+ * GNU Mes is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 3 of the License, or (at
+ * your option) any later version.
+ *
+ * GNU Mes is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with GNU Mes.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __M2_TYPES_H
+#define __M2_TYPES_H 1
+
+/*
+#ifndef __MES_CLOCK_T
+#define __MES_CLOCK_T
+#undef clock_t
+typedef long clock_t;
+#endif
+*/
+
+#ifndef __MES_DEV_T
+#define __MES_DEV_T
+#undef dev_t
+typedef long dev_t;
+#endif
+
+/*
+#if !defined (__MES_FILE_T) && ! defined (_FILE_T)
+#define __MES_FILE_T
+#define _FILE_T
+typedef long FILE;
+#endif
+*/
+
+#ifndef __MES_GID_T
+#define __MES_GID_T
+#undef gid_t
+typedef unsigned gid_t;
+#endif
+
+#ifndef __MES_INO_T
+#define __MES_INO_T
+#undef ino_t
+typedef unsigned ino_t;
+#endif
+
+#if __SIZEOF_LONG_LONG__ == 8
+#ifndef __MES_INO64_T
+#define __MES_INO64_T
+#undef ino64_t
+typedef unsigned ino64_t;
+#endif
+#endif // __SIZEOF_LONG_LONG__ == 8
+
+#if !defined (__MES_INTPTR_T) && !defined (__intptr_t_defined)
+#define __MES_INTPTR_T
+#define __intptr_t_defined
+#undef intptr_t
+typedef long intptr_t;
+#undef uintptr_t
+typedef unsigned uintptr_t;
+#endif
+
+#ifndef __MES_OFF_T
+#define __MES_OFF_T
+#undef off_t
+typedef long off_t;
+#endif
+
+#if __SIZEOF_LONG_LONG__ == 8
+#ifndef __MES_OFF64_T
+#define __MES_OFF64_T
+#undef off64_t
+typedef unsigned off64_t;
+#endif
+#endif // __SIZEOF_LONG_LONG__ == 8
+
+#ifndef __MES_PID_T
+#define __MES_PID_T
+#undef pid_t
+typedef int pid_t;
+#endif
+
+#ifndef __PTRDIFF_T
+#define __PTRDIFF_T
+#ifndef __MES_PTRDIFF_T
+#define __MES_PTRDIFF_T
+#undef ptrdiff_t
+typedef long ptrdiff_t;
+#endif
+#endif
+
+#ifndef __MES_SIGVAL_T
+#define __MES_SIGVAL_T
+#undef clock_t
+typedef long sigval_t;
+#endif
+
+#ifndef __SIZE_T
+#define __SIZE_T
+#ifndef __MES_SIZE_T
+#define __MES_SIZE_T
+typedef unsigned size_t;
+#endif
+#endif
+
+#ifndef __MES_SSIZE_T
+#define __MES_SSIZE_T
+#undef ssize_t
+typedef long ssize_t;
+#endif
+
+#ifndef __MES_UID_T
+#define __MES_UID_T
+#undef uid_t
+typedef unsigned uid_t;
+#endif
+
+#ifndef __WCHAR_T
+#define __WCHAR_T
+#ifndef __MES_WCHAR_T
+#define __MES_WCHAR_T
+#undef wchar_t
+typedef int wchar_t;
+#endif
+#endif
+
+#endif // __M2_TYPES_H
diff --git a/include/sys/stat.h b/include/sys/stat.h
index 0aefa286..fbcee2f1 100644
--- a/include/sys/stat.h
+++ b/include/sys/stat.h
@@ -19,7 +19,7 @@
  * along with GNU Mes.  If not, see <http://www.gnu.org/licenses/>.
  */
 #ifndef __MES_SYS_STAT_H
-#define __MES_SYS_STAT_H 1lei
+#define __MES_SYS_STAT_H 1
 
 #if SYSTEM_LIBC
 #undef __MES_SYS_STAT_H
@@ -29,60 +29,13 @@
 
 #include <time.h>
 #include <sys/types.h>
+#include <arch/kernel-stat.h>
 
 #ifndef __MES_MODE_T
 #define __MES_MODE_T
 typedef int mode_t;
 #endif
 
-// *INDENT-OFF*
-#if __i386__ || __arm__
-struct stat
-{
-  unsigned long  st_dev;
-  unsigned long  st_ino;
-  unsigned short st_mode;
-  unsigned short st_nlink;
-  unsigned short st_uid;
-  unsigned short st_gid;
-  unsigned long  st_rdev;
-  long           st_size; /* Linux: unsigned long; glibc: off_t (i.e. signed) */
-  unsigned long  st_blksize;
-  unsigned long  st_blocks;
-  time_t         st_atime; /* Linux: unsigned long; glibc: time_t */
-  unsigned long  st_atime_usec;
-  time_t         st_mtime; /* Linux: unsigned long; glibc: time_t */
-  unsigned long  st_mtime_usec;
-  time_t         st_ctime; /* Linux: unsigned long; glibc: time_t */
-  unsigned long  st_ctime_usec;
-  unsigned long  __foo0;
-  unsigned long  __foo1;
-};
-#elif __x86_64__
-struct stat
-{
-  unsigned long  st_dev;
-  unsigned long  st_ino;
-  unsigned int   st_mode;
-  unsigned int   st_nlink;
-  unsigned int   st_uid;
-  unsigned int   st_gid;
-  unsigned long  st_rdev;
-  long           st_size;
-  unsigned long  st_blksize;
-  unsigned long  st_blocks;
-  time_t         st_atime;
-  unsigned long  st_atime_usec;
-  time_t         st_mtime;
-  unsigned long  st_mtime_usec;
-  time_t         st_ctime;
-  unsigned long  st_ctime_usec;
-  unsigned long  __foo0;
-  unsigned long  __foo1;
-};
-#endif
-// *INDENT-ON*
-
 int chmod (char const *file_name, mode_t mode);
 int fstat (int filedes, struct stat *buf);
 int mkdir (char const *file_name, mode_t mode);
diff --git a/kaem.run b/kaem.run
index 3c78e39a..1d8df90b 100644
--- a/kaem.run
+++ b/kaem.run
@@ -53,6 +53,7 @@ M2-Planet                                       \
     -f lib/mes/fdputc.c                         \
     -f lib/mes/eputc.c                          \
                                                 \
+    -f include/m2/types.h                       \
     -f include/mes/mes.h                        \
     -f include/mes/builtins.h                   \
     -f include/mes/constants.h                  \
@@ -81,6 +82,8 @@ M2-Planet                                       \
     -f lib/mes/fdungetc.c                       \
     -f lib/posix/setenv.c                       \
     -f lib/linux/access.c                       \
+    -f include/linux/m2/kernel-stat.h           \
+    -f include/sys/stat.h                       \
     -f lib/m2/chmod.c                           \
     -f lib/linux/ioctl3.c                       \
     -f lib/m2/isatty.c                          \
diff --git a/lib/linux/_getcwd.c b/lib/linux/_getcwd.c
index 85153cb1..10eb270e 100644
--- a/lib/linux/_getcwd.c
+++ b/lib/linux/_getcwd.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2016,2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -20,7 +20,7 @@
 
 #include <mes/lib.h>
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 
 char *
 _getcwd (char *buffer, size_t size)
diff --git a/lib/linux/_open3.c b/lib/linux/_open3.c
index 3072f9bf..cfad689a 100644
--- a/lib/linux/_open3.c
+++ b/lib/linux/_open3.c
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <mes/lib.h>
 #include <fcntl.h>
 
diff --git a/lib/linux/_read.c b/lib/linux/_read.c
index 6c5ff42f..c5a34abc 100644
--- a/lib/linux/_read.c
+++ b/lib/linux/_read.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2016,2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <mes/lib.h>
 #include <fcntl.h>
 
diff --git a/lib/linux/access.c b/lib/linux/access.c
index 805e8f01..ceb18595 100644
--- a/lib/linux/access.c
+++ b/lib/linux/access.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2016,2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -20,7 +20,7 @@
 
 #include <mes/lib.h>
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 
 int
 access (char const *file_name, int how)
diff --git a/lib/linux/brk.c b/lib/linux/brk.c
index 586bd7e1..3517850c 100644
--- a/lib/linux/brk.c
+++ b/lib/linux/brk.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2016,2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -20,7 +20,7 @@
 
 #include <mes/lib.h>
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 
 long
 brk (void *addr)
diff --git a/lib/linux/chdir.c b/lib/linux/chdir.c
index ada7feec..aed98378 100644
--- a/lib/linux/chdir.c
+++ b/lib/linux/chdir.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 
 int
 chdir (char const *file_name)
diff --git a/lib/linux/chmod.c b/lib/linux/chmod.c
index 1feabec5..a6d74a43 100644
--- a/lib/linux/chmod.c
+++ b/lib/linux/chmod.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2016,2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -20,7 +20,7 @@
 
 #include <mes/lib.h>
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <sys/stat.h>
 
 int
diff --git a/lib/linux/clock_gettime.c b/lib/linux/clock_gettime.c
index 051dba33..63753ef8 100644
--- a/lib/linux/clock_gettime.c
+++ b/lib/linux/clock_gettime.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -20,7 +20,7 @@
 
 #include <mes/lib.h>
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <time.h>
 
 int
diff --git a/lib/linux/close.c b/lib/linux/close.c
index e25e8318..490b4881 100644
--- a/lib/linux/close.c
+++ b/lib/linux/close.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2016,2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <mes/lib.h>
 
 int
diff --git a/lib/linux/dup.c b/lib/linux/dup.c
index 9237ef97..b954d3de 100644
--- a/lib/linux/dup.c
+++ b/lib/linux/dup.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2016,2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 
 int
 dup (int old)
diff --git a/lib/linux/dup2.c b/lib/linux/dup2.c
index ef996273..9a0752e3 100644
--- a/lib/linux/dup2.c
+++ b/lib/linux/dup2.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2016,2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 
 int
 dup2 (int old, int new)
diff --git a/lib/linux/execve.c b/lib/linux/execve.c
index 51c8b4ff..7cb02e20 100644
--- a/lib/linux/execve.c
+++ b/lib/linux/execve.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2016,2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 
 int
 execve (char const *file_name, char *const argv[], char *const env[])
diff --git a/lib/linux/fcntl.c b/lib/linux/fcntl.c
index e10e7348..85a167ae 100644
--- a/lib/linux/fcntl.c
+++ b/lib/linux/fcntl.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <stdarg.h>
 
 int
diff --git a/lib/linux/fork.c b/lib/linux/fork.c
index 1f5c25fc..d07050bf 100644
--- a/lib/linux/fork.c
+++ b/lib/linux/fork.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2016,2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 
 int
 fork ()
diff --git a/lib/linux/fstat.c b/lib/linux/fstat.c
index 19d3f6a9..f4965776 100644
--- a/lib/linux/fstat.c
+++ b/lib/linux/fstat.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <sys/stat.h>
 
 int
diff --git a/lib/linux/fsync.c b/lib/linux/fsync.c
index 0eef6db4..ba75088d 100644
--- a/lib/linux/fsync.c
+++ b/lib/linux/fsync.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2016,2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 
 int
 fsync (int filedes)
diff --git a/lib/linux/getdents.c b/lib/linux/getdents.c
index 5ebafa45..03717710 100644
--- a/lib/linux/getdents.c
+++ b/lib/linux/getdents.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <sys/types.h>
 
 int
diff --git a/lib/linux/getegid.c b/lib/linux/getegid.c
index 5ad2f2c6..2fb8098f 100644
--- a/lib/linux/getegid.c
+++ b/lib/linux/getegid.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <unistd.h>
 
 gid_t
diff --git a/lib/linux/geteuid.c b/lib/linux/geteuid.c
index 4fcf9fd1..62d2da47 100644
--- a/lib/linux/geteuid.c
+++ b/lib/linux/geteuid.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <unistd.h>
 
 uid_t
diff --git a/lib/linux/getgid.c b/lib/linux/getgid.c
index 4402b528..48fd3579 100644
--- a/lib/linux/getgid.c
+++ b/lib/linux/getgid.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <unistd.h>
 
 gid_t
diff --git a/lib/linux/getpid.c b/lib/linux/getpid.c
index 9cab47ae..73cb74b6 100644
--- a/lib/linux/getpid.c
+++ b/lib/linux/getpid.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <unistd.h>
 
 pid_t
diff --git a/lib/linux/getppid.c b/lib/linux/getppid.c
index 7eea4539..49d472ba 100644
--- a/lib/linux/getppid.c
+++ b/lib/linux/getppid.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <unistd.h>
 
 pid_t
diff --git a/lib/linux/getrusage.c b/lib/linux/getrusage.c
index 2a789949..174d4c0b 100644
--- a/lib/linux/getrusage.c
+++ b/lib/linux/getrusage.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <sys/resource.h>
 
 int
diff --git a/lib/linux/gettimeofday.c b/lib/linux/gettimeofday.c
index 495f059f..ed4e336f 100644
--- a/lib/linux/gettimeofday.c
+++ b/lib/linux/gettimeofday.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -20,7 +20,7 @@
 
 #include <mes/lib.h>
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <sys/time.h>
 
 int
diff --git a/lib/linux/getuid.c b/lib/linux/getuid.c
index e6edd257..d5ca3a50 100644
--- a/lib/linux/getuid.c
+++ b/lib/linux/getuid.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <unistd.h>
 
 uid_t
diff --git a/lib/linux/ioctl.c b/lib/linux/ioctl.c
index 0e6e14ac..27547c68 100644
--- a/lib/linux/ioctl.c
+++ b/lib/linux/ioctl.c
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <stdarg.h>
 #include <sys/ioctl.h>
 
diff --git a/lib/linux/ioctl3.c b/lib/linux/ioctl3.c
index 3f759d06..7990b8b7 100644
--- a/lib/linux/ioctl3.c
+++ b/lib/linux/ioctl3.c
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <mes/lib.h>
 
 int
diff --git a/lib/linux/kill.c b/lib/linux/kill.c
index 4298a9db..f04424fb 100644
--- a/lib/linux/kill.c
+++ b/lib/linux/kill.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <unistd.h>
 
 int
diff --git a/lib/linux/link.c b/lib/linux/link.c
index cf8dec32..e2d66912 100644
--- a/lib/linux/link.c
+++ b/lib/linux/link.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 
 int
 link (char const *old_name, char const *new_name)
diff --git a/lib/linux/lseek.c b/lib/linux/lseek.c
index f71af59f..c72a75cf 100644
--- a/lib/linux/lseek.c
+++ b/lib/linux/lseek.c
@@ -20,7 +20,7 @@
 
 #include <mes/lib.h>
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <stdio.h>
 #include <sys/types.h>
 
diff --git a/lib/linux/lstat.c b/lib/linux/lstat.c
index 039de0e1..feebc6cb 100644
--- a/lib/linux/lstat.c
+++ b/lib/linux/lstat.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <sys/stat.h>
 
 int
diff --git a/lib/linux/mkdir.c b/lib/linux/mkdir.c
index 53188888..59319329 100644
--- a/lib/linux/mkdir.c
+++ b/lib/linux/mkdir.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <sys/stat.h>
 
 int
diff --git a/lib/linux/mknod.c b/lib/linux/mknod.c
index 8339f7a6..24a9b0c7 100644
--- a/lib/linux/mknod.c
+++ b/lib/linux/mknod.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <sys/stat.h>
 
 int
diff --git a/lib/linux/nanosleep.c b/lib/linux/nanosleep.c
index bc838a4f..a5e2a044 100644
--- a/lib/linux/nanosleep.c
+++ b/lib/linux/nanosleep.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <time.h>
 #include <sys/time.h>
 
diff --git a/lib/linux/pipe.c b/lib/linux/pipe.c
index 0ed4c23e..f6b3689a 100644
--- a/lib/linux/pipe.c
+++ b/lib/linux/pipe.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <unistd.h>
 
 int
diff --git a/lib/linux/read.c b/lib/linux/read.c
index efd25744..d91f81b2 100644
--- a/lib/linux/read.c
+++ b/lib/linux/read.c
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <mes/lib.h>
 #include <fcntl.h>
 
diff --git a/lib/linux/readlink.c b/lib/linux/readlink.c
index 9990b50f..96443273 100644
--- a/lib/linux/readlink.c
+++ b/lib/linux/readlink.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <sys/stat.h>
 
 ssize_t
diff --git a/lib/linux/rename.c b/lib/linux/rename.c
index 492c734d..762c3093 100644
--- a/lib/linux/rename.c
+++ b/lib/linux/rename.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <unistd.h>
 
 int
diff --git a/lib/linux/rmdir.c b/lib/linux/rmdir.c
index 7c096832..5d4c0f49 100644
--- a/lib/linux/rmdir.c
+++ b/lib/linux/rmdir.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2016,2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 
 int
 rmdir (char const *file_name)
diff --git a/lib/linux/setgid.c b/lib/linux/setgid.c
index 5512c622..13399c2e 100644
--- a/lib/linux/setgid.c
+++ b/lib/linux/setgid.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <unistd.h>
 
 int
diff --git a/lib/linux/settimer.c b/lib/linux/settimer.c
index a66240f1..7247c8a0 100644
--- a/lib/linux/settimer.c
+++ b/lib/linux/settimer.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <sys/time.h>
 #include <unistd.h>
 
diff --git a/lib/linux/setuid.c b/lib/linux/setuid.c
index 5157dcae..d5e2ae76 100644
--- a/lib/linux/setuid.c
+++ b/lib/linux/setuid.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <unistd.h>
 
 int
diff --git a/lib/linux/signal.c b/lib/linux/signal.c
index 11174be9..23cf106d 100644
--- a/lib/linux/signal.c
+++ b/lib/linux/signal.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <unistd.h>
 #include <signal.h>
 
diff --git a/lib/linux/sigprogmask.c b/lib/linux/sigprogmask.c
index c0326a28..4b0bb8eb 100644
--- a/lib/linux/sigprogmask.c
+++ b/lib/linux/sigprogmask.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <signal.h>
 #include <unistd.h>
 
diff --git a/lib/linux/stat.c b/lib/linux/stat.c
index d8f4465b..df0022aa 100644
--- a/lib/linux/stat.c
+++ b/lib/linux/stat.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2016,2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <sys/stat.h>
 
 int
diff --git a/lib/linux/symlink.c b/lib/linux/symlink.c
index 53f99fb7..4e4084d2 100644
--- a/lib/linux/symlink.c
+++ b/lib/linux/symlink.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <unistd.h>
 
 int
diff --git a/lib/linux/time.c b/lib/linux/time.c
index f4931970..53c92052 100644
--- a/lib/linux/time.c
+++ b/lib/linux/time.c
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <time.h>
 #include <sys/time.h>
 #include <stdlib.h>
diff --git a/lib/linux/unlink.c b/lib/linux/unlink.c
index 03713e64..9f204b5f 100644
--- a/lib/linux/unlink.c
+++ b/lib/linux/unlink.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2016,2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -20,7 +20,7 @@
 
 #include <mes/lib.h>
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 
 int
 unlink (char const *file_name)
diff --git a/lib/linux/waitpid.c b/lib/linux/waitpid.c
index 693e3dfa..bb89c692 100644
--- a/lib/linux/waitpid.c
+++ b/lib/linux/waitpid.c
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <sys/types.h>
 
 pid_t
diff --git a/lib/m2/execve.c b/lib/m2/execve.c
index 7fe7c9ba..1f078c3d 100644
--- a/lib/m2/execve.c
+++ b/lib/m2/execve.c
@@ -1,6 +1,6 @@
 /* -*-comment-start: "//";comment-end:""-*-
  * GNU Mes --- Maxwell Equations of Software
- * Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+ * Copyright © 2016,2017,2018,2019,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
  *
  * This file is part of GNU Mes.
  *
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 
 int
 execve (char const *file_name, char **argv, char **env)
diff --git a/lib/m2/open.c b/lib/m2/open.c
index 3d3fe4dc..ee5513e7 100644
--- a/lib/m2/open.c
+++ b/lib/m2/open.c
@@ -18,6 +18,8 @@
  * along with GNU Mes.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <linux/syscall.h>
+#include <arch/syscall.h>
 #include <mes/lib.h>
 #include <fcntl.h>
 #include <stdarg.h>
diff --git a/lib/m2/time.c b/lib/m2/time.c
index c589de85..7f43cdad 100644
--- a/lib/m2/time.c
+++ b/lib/m2/time.c
@@ -19,7 +19,7 @@
  */
 
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <time.h>
 
 long
diff --git a/lib/m2/waitpid.c b/lib/m2/waitpid.c
index a3d98d1d..5cd18d98 100644
--- a/lib/m2/waitpid.c
+++ b/lib/m2/waitpid.c
@@ -20,7 +20,7 @@
 
 #include <mes/lib.h>
 #include <linux/syscall.h>
-#include <syscall.h>
+#include <arch/syscall.h>
 #include <sys/types.h>
 
 int
diff --git a/simple.sh b/simple.sh
index ce0ec375..fc5ac371 100755
--- a/simple.sh
+++ b/simple.sh
@@ -1,7 +1,7 @@
 #! /bin/sh
 
 # GNU Mes --- Maxwell Equations of Software
-# Copyright © 2019,2020,2022 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+# Copyright © 2019,2020,2022,2023 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
 #
 # This file is part of GNU Mes.
 #
@@ -36,7 +36,9 @@ cat > include/mes/config.h <<EOF
 EOF
 
 ## Build ##
-gcc -g -D HAVE_CONFIG_H=1 -I include            \
+gcc -g -D HAVE_CONFIG_H=1                       \
+    -I include                                  \
+    -I include/$mes_kernel/$mes_cpu             \
     -o out-system-libc/mes                      \
                                                 \
     lib/mes/eputs.c                             \
@@ -163,11 +165,15 @@ cat > include/mes/config.h <<EOF
 #define MES_VERSION "git"
 EOF
 
+mkdir -p include/arch
+cp -f include/$mes_kernel/$mes_cpu/kernel-stat.h include/arch
+cp -f include/$mes_kernel/$mes_cpu/syscall.h include/arch
+
 ## Build ##
 compiler=gcc     # not configurable
 $CC -g -D HAVE_CONFIG_H=1                               \
     -I include -I include/$mes_kernel/$mes_cpu          \
-       -nostdinc -nostdlib                              \
+    -nostdinc -nostdlib                                 \
     -fno-builtin -fno-stack-protector                   \
     -o out-mes/mes                                      \
                                                         \
-- 
2.38.1


[-- Attachment #3: Type: text/plain, Size: 164 bytes --]


-- 
Janneke Nieuwenhuizen <janneke@gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

Re: Architecture support

[Andreas Enge]

Am Mon, Feb 13, 2023 at 12:56:45PM +0200 schrieb Efraim Flashner:
> Aarch64 and armhf are getting stuck at gcc-cross-boot0.

Hm, it looks like I went further: A bit earlier there was a directory
/tmp/guix-build-gcc-cross-boot0, and now it is gone and replaced by
/tmp/guix-build-glibc-intermediate.

Maybe a transient failure? Lack of memory? But I also have only 4GB.

Andreas

Re: Merging core-updates?

[Andreas Enge]

Am Mon, Feb 13, 2023 at 12:34:56PM +0100 schrieb Janneke Nieuwenhuizen:
> To use stat64 and friends on 32bit, I created the attached patch for GNU
> Mes and hope to create a 0.24.2 release from
>     https://gitlab.com/janneke/mes/-/tree/wip-stat64

Thanks a lot, Janneke!

> Just hoping this is also a fix for
>     https://debbugs.gnu.org/41264
>     https://debbugs.gnu.org/49985
> as I haven't been able to reproduce this bug.  IWBN if someone could
> shed more light on that...

The patch-mes package also builds on my laptop with an SSD, so I do not
know how to reproduce it. If someone else does, that would be great.

Andreas

Re: Merging core-updates?

[Katherine Cox-Buday]

Efraim Flashner <efraim@flashner.co.il> writes:

> On Sun, Feb 12, 2023 at 10:05:40AM +0100, Julien Lepiller wrote:
>> Hi Guix!
>> 
>> As discussed at Guix Days before Fosdem, we haven't merged core-updates
>> in a very long time. I'd volunteer to lead this effort, but I don't
>> know what steps I should follow. Do we have some documentation about
>> that?
>
> I think we normally have a 2 week last-chance window to get all sorts of
> last minute packages bumped and then we freeze it and try to build
> "everything".


On that note, could we possibly give Mesa one final bump? The latest
version is 22.3.5, and the version in core-updates is 22.2.4.

I also hope to be able to test, but my schedule tends to run away from
me.

-- 
Katherine

Re: Merging core-updates?

[John Kehayias]

Hi Katherine et al,

On Mon, Feb 13, 2023 at 09:40 AM, Katherine Cox-Buday wrote:

> Efraim Flashner <efraim@flashner.co.il> writes:
>
>> On Sun, Feb 12, 2023 at 10:05:40AM +0100, Julien Lepiller wrote:
>>> Hi Guix!
>>>
>>> As discussed at Guix Days before Fosdem, we haven't merged core-updates
>>> in a very long time. I'd volunteer to lead this effort, but I don't
>>> know what steps I should follow. Do we have some documentation about
>>> that?
>>
>> I think we normally have a 2 week last-chance window to get all sorts of
>> last minute packages bumped and then we freeze it and try to build
>> "everything".
>
>
> On that note, could we possibly give Mesa one final bump? The latest
> version is 22.3.5, and the version in core-updates is 22.2.4.
>
> I also hope to be able to test, but my schedule tends to run away from
> me.

Yes, I would like to get Mesa up to date, we have the smaller LLVM closure now and the pending patches for some Vulkan paths/packages. I would like to help review/test/push those but I'm in the middle of a move so if anyone wants to beat me to it in the next week or so, please do :) But yes, those are all on my radar and certainly a preferred activity to more packing...

John

Re: Merging core-updates?

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2161 bytes --]

On Sun, Feb 12, 2023 at 06:29:04PM +0000, Kaelyn wrote:
> Hi,
> 
> ------- Original Message -------
> On Sunday, February 12th, 2023 at 5:08 PM, Andreas Enge <andreas@enge.fr> wrote:
> 
> >
> >
> > Am Sun, Feb 12, 2023 at 12:58:06PM +0100 schrieb Julien Lepiller:
> >
> > > And I was able to rebuild (with --check) patch-mesboot. The error looks
> > > a lot like https://issues.guix.gnu.org/49985. We should fix that indeed
> > > :)
> >
> >
> > Ah indeed, that looks like deal breaking; maybe someone from MES can have
> > a look?
> >
> > What is the magic incantation with double "@@" to build this package?
> > Ah, here we go, for reference to self:
> > guix build -e '(@@ (gnu packages commencement) patch-mesboot)'
> >
> > Andreas
> 
> While not directly related to the patch-mesboot error, I want to mention that there is also https://issues.guix.gnu.org/58719 blocking i686 builds on core-updates (and x86_64 builds of certain packages like wine64, which has i686 dependencies) since the update to glibc 2.35.
> 
> It may also need assistance from the MES folks to fix, since the error message is about an undefined symbol in glibc-mesboot's libpthread.so.0:
> 
> make[2]: Entering directory '/tmp/guix-build-file-5.44.drv-0/file-5.44/magic'
> ../src/file -C -m magic
> /tmp/guix-build-file-5.44.drv-0/file-5.44/src/.libs/file: symbol lookup error: /gnu/store/s4yd6ibxsh5q1j9ipygb9vpjj4g00wc9-glibc-mesboot-2.16.0/lib/libpthread.so.0: undefined symbol: h_errno, version GLIBC_PRIVATE
> make[2]: *** [Makefile:863: magic.mgc] Error 127
> 
> Cheers,
> Kaelyn

I think I found where this is coming from. %boot3-inputs added
ld-wrapper-boot3 but didn't remove ld-wrapper-0, which pulled in
glibc-mesboot. I'm testing out removing ld-wrapper-0 from %boot3-inputs
to see if that's enough to make that final file build. Hopefully it'll
also fix the final tar for i686, which I found was also failing for me.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Merging core-updates?

[Andreas Enge]

Hello,

Am Sun, Feb 12, 2023 at 05:51:47PM +0200 schrieb Efraim Flashner:
> I think we normally have a 2 week last-chance window to get all sorts of
> last minute packages bumped and then we freeze it and try to build
> "everything".

I am a bit hesitant to let more breakages in :)  It looks like we have
reached the Debian trap: releases/core-update merges happen so infrequently
that people become desperate to get their favourite update in, which causes
build failures, which delay the release cycle, da capo ad infinitum.

Personally I would prefer to freeze now, then to let other big changes
(mesa for instance) happen in a feature branch; if all goes well, it could
be merged a week or two later, if it breaks things, it will anyway take
the time it needs to fix them.

Andreas

Re: Merging core-updates?

[Andreas Enge]

Hello all,

here are my first important failures when trying to go beyond hello and mpc
(aka the easy C programs):

ocaml-4.0.9 fails with
gcc -c -O2 -fno-strict-aliasing -fwrapv -Wall -fno-tree-vrp -g -D_FILE_OFFSET_BITS=64 -D_REENTRANT -DCAML_NAME_SPACE  -DOCAML_STDLIB_DIR='"/gnu/store/mzzak7bp6dpngq04hphlx58wzp62755m-ocaml-4.09.0/lib/ocaml"'  -DNATIVE_CODE -DTARGET_amd64 -DMODEL_default -DSYS_linux   -o signals_nat_n.o signals_nat.c
signals_nat.c:185:13: error: variably modified ‘sig_alt_stack’ at file scope
  185 | static char sig_alt_stack[SIGSTKSZ];
      |             ^~~~~~~~~~~~~
make[3]: *** [Makefile:343: signals_nat_n.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[3]: Leaving directory '/tmp/guix-build-ocaml-4.09.0.drv-0/ocaml-4.09.0/runtime'
make[2]: *** [Makefile:1004: makeruntimeopt] Error 2
make[2]: Leaving directory '/tmp/guix-build-ocaml-4.09.0.drv-0/ocaml-4.09.0'
make[1]: *** [Makefile:417: opt.opt] Error 2
make[1]: Leaving directory '/tmp/guix-build-ocaml-4.09.0.drv-0/ocaml-4.09.0'
make: *** [Makefile:468: world.opt] Error 2
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("-j" "4" "world.opt") exit-status: 2 term-signal: #f stop-signal: #f>
phase `build' failed after 45.7 seconds
command "make" "-j" "4" "world.opt" failed with status 2

openjdk-19.0.1-checkout fails:
`File make/modules/java.desktop/lib/Awt2dLibraries.gmk is read-only; trying to patch anyway
patching file make/modules/java.desktop/lib/Awt2dLibraries.gmk
Hunk #1 succeeded at 214 (offset -3 lines).
File src/java.desktop/unix/native/libawt_xawt/xawt/XlibWrapper.c is read-only; trying to patch anyway
patching file src/java.desktop/unix/native/libawt_xawt/xawt/XlibWrapper.c
Hunk #1 succeeded at 44 (offset 4 lines).
Hunk #2 succeeded at 978 (offset 4 lines).
File test/jdk/java/awt/JAWT/Makefile.unix is read-only; trying to patch anyway
patching file test/jdk/java/awt/JAWT/Makefile.unix
File test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c is read-only; trying to patch anyway
patching file test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c
Hunk #1 FAILED at 67.
1 out of 1 hunk FAILED -- saving rejects to file test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c.rej
source is at 'openjdk-19.0.1-checkout'
applying '/gnu/store/g6sgv9nipmvsxbmimlgsmhdcdvcssmhf-openjdk-15-xcursor-no-dynamic.patch'...
applying '/gnu/store/ms15ddy4ac2kmiyy82l0spnjj5b6h6lb-openjdk-10-hotspot-stack-size.patch'...
Backtrace:
           5 (primitive-load "/gnu/store/kpgzrwyczkxymc54lnkld0nmrd0…")
In ice-9/eval.scm:
    619:8  4 (_ #(#(#<directory (guile-user) 7ffff5fd1c80> "ope…") #))
In ice-9/boot-9.scm:
    142:2  3 (dynamic-wind #<procedure 7ffff3f4f500 at ice-9/eval.s…> …)
In ice-9/eval.scm:
    619:8  2 (_ #(#(#<directory (guile-user) 7ffff5fd1c80>)))
In srfi/srfi-1.scm:
    634:9  1 (for-each #<procedure apply-patch (a)> _)
In guix/build/utils.scm:
    812:6  0 (invoke "/gnu/store/qy964im7g1w77yzd97rahl0r7v8rkv0g-p…" …)

guix/build/utils.scm:812:6: In procedure invoke:
ERROR:
  1. &invoke-error:
      program: "/gnu/store/qy964im7g1w77yzd97rahl0r7v8rkv0g-patch-2.7.6/bin/patch"
      arguments: ("--force" "--no-backup-if-mismatch" "-p1" "--input" "/gnu/store/ms15ddy4ac2kmiyy82l0spnjj5b6h6lb-openjdk-10-hotspot-stack-size.patch")
      exit-status: 1
      term-signal: #f
      stop-signal: #f
Just a permission error on files to be patched, or a problem with the patch itself?

Andreas

Re: Merging core-updates?

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi

On Mon, Feb 13, 2023 at 12:22 PM Andreas Enge <andreas@enge.fr> wrote:
>
> It looks like we have reached the Debian trap

How about applying selectively those patches from core-updates that do
not break anything? It would split the difference.

Future changes waiting in Debbugs could join the remainder, which
needs more work. Later, the new branch could go through the same
procedure, i.e. any breakage would be caught.

The new process would be gradual and allow some commits to advance
into the project history (also known as the 'master' branch) within a
predicable time frame.

Thanks everyone for your hard work!

Kind regards
Felix Lechner

Re: Merging core-updates?

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 4061 bytes --]

On Mon, Feb 13, 2023 at 09:35:32PM +0100, Andreas Enge wrote:
> Hello all,
> 
> here are my first important failures when trying to go beyond hello and mpc
> (aka the easy C programs):
> 
> ocaml-4.0.9 fails with
> gcc -c -O2 -fno-strict-aliasing -fwrapv -Wall -fno-tree-vrp -g -D_FILE_OFFSET_BITS=64 -D_REENTRANT -DCAML_NAME_SPACE  -DOCAML_STDLIB_DIR='"/gnu/store/mzzak7bp6dpngq04hphlx58wzp62755m-ocaml-4.09.0/lib/ocaml"'  -DNATIVE_CODE -DTARGET_amd64 -DMODEL_default -DSYS_linux   -o signals_nat_n.o signals_nat.c
> signals_nat.c:185:13: error: variably modified ‘sig_alt_stack’ at file scope
>   185 | static char sig_alt_stack[SIGSTKSZ];
>       |             ^~~~~~~~~~~~~
> make[3]: *** [Makefile:343: signals_nat_n.o] Error 1
> make[3]: *** Waiting for unfinished jobs....
> make[3]: Leaving directory '/tmp/guix-build-ocaml-4.09.0.drv-0/ocaml-4.09.0/runtime'
> make[2]: *** [Makefile:1004: makeruntimeopt] Error 2
> make[2]: Leaving directory '/tmp/guix-build-ocaml-4.09.0.drv-0/ocaml-4.09.0'
> make[1]: *** [Makefile:417: opt.opt] Error 2
> make[1]: Leaving directory '/tmp/guix-build-ocaml-4.09.0.drv-0/ocaml-4.09.0'
> make: *** [Makefile:468: world.opt] Error 2
> error: in phase 'build': uncaught exception:
> %exception #<&invoke-error program: "make" arguments: ("-j" "4" "world.opt") exit-status: 2 term-signal: #f stop-signal: #f>
> phase `build' failed after 45.7 seconds
> command "make" "-j" "4" "world.opt" failed with status 2
> 
> openjdk-19.0.1-checkout fails:
> `File make/modules/java.desktop/lib/Awt2dLibraries.gmk is read-only; trying to patch anyway
> patching file make/modules/java.desktop/lib/Awt2dLibraries.gmk
> Hunk #1 succeeded at 214 (offset -3 lines).
> File src/java.desktop/unix/native/libawt_xawt/xawt/XlibWrapper.c is read-only; trying to patch anyway
> patching file src/java.desktop/unix/native/libawt_xawt/xawt/XlibWrapper.c
> Hunk #1 succeeded at 44 (offset 4 lines).
> Hunk #2 succeeded at 978 (offset 4 lines).
> File test/jdk/java/awt/JAWT/Makefile.unix is read-only; trying to patch anyway
> patching file test/jdk/java/awt/JAWT/Makefile.unix
> File test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c is read-only; trying to patch anyway
> patching file test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c
> Hunk #1 FAILED at 67.
> 1 out of 1 hunk FAILED -- saving rejects to file test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c.rej
> source is at 'openjdk-19.0.1-checkout'
> applying '/gnu/store/g6sgv9nipmvsxbmimlgsmhdcdvcssmhf-openjdk-15-xcursor-no-dynamic.patch'...
> applying '/gnu/store/ms15ddy4ac2kmiyy82l0spnjj5b6h6lb-openjdk-10-hotspot-stack-size.patch'...
> Backtrace:
>            5 (primitive-load "/gnu/store/kpgzrwyczkxymc54lnkld0nmrd0…")
> In ice-9/eval.scm:
>     619:8  4 (_ #(#(#<directory (guile-user) 7ffff5fd1c80> "ope…") #))
> In ice-9/boot-9.scm:
>     142:2  3 (dynamic-wind #<procedure 7ffff3f4f500 at ice-9/eval.s…> …)
> In ice-9/eval.scm:
>     619:8  2 (_ #(#(#<directory (guile-user) 7ffff5fd1c80>)))
> In srfi/srfi-1.scm:
>     634:9  1 (for-each #<procedure apply-patch (a)> _)
> In guix/build/utils.scm:
>     812:6  0 (invoke "/gnu/store/qy964im7g1w77yzd97rahl0r7v8rkv0g-p…" …)
> 
> guix/build/utils.scm:812:6: In procedure invoke:
> ERROR:
>   1. &invoke-error:
>       program: "/gnu/store/qy964im7g1w77yzd97rahl0r7v8rkv0g-patch-2.7.6/bin/patch"
>       arguments: ("--force" "--no-backup-if-mismatch" "-p1" "--input" "/gnu/store/ms15ddy4ac2kmiyy82l0spnjj5b6h6lb-openjdk-10-hotspot-stack-size.patch")
>       exit-status: 1
>       term-signal: #f
>       stop-signal: #f
> Just a permission error on files to be patched, or a problem with the patch itself?
> 
> Andreas

Looks like you might be able to drop openjdk-10-hotspot-stack-size.patch
from openjdk-19.0.1.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Merging core-updates?

[Kaelyn]

------- Original Message -------
On Monday, February 13th, 2023 at 8:04 PM, Efraim Flashner <efraim@flashner.co.il> wrote:
> 
> 
> On Sun, Feb 12, 2023 at 06:29:04PM +0000, Kaelyn wrote:
> 
> > Hi,
> > 
> > ------- Original Message -------
> > On Sunday, February 12th, 2023 at 5:08 PM, Andreas Enge andreas@enge.fr wrote:
> > 
> > > Am Sun, Feb 12, 2023 at 12:58:06PM +0100 schrieb Julien Lepiller:
> > > 
> > > > And I was able to rebuild (with --check) patch-mesboot. The error looks
> > > > a lot like https://issues.guix.gnu.org/49985. We should fix that indeed
> > > > :)
> > > 
> > > Ah indeed, that looks like deal breaking; maybe someone from MES can have
> > > a look?
> > > 
> > > What is the magic incantation with double "@@" to build this package?
> > > Ah, here we go, for reference to self:
> > > guix build -e '(@@ (gnu packages commencement) patch-mesboot)'
> > > 
> > > Andreas
> > 
> > While not directly related to the patch-mesboot error, I want to mention that there is also https://issues.guix.gnu.org/58719 blocking i686 builds on core-updates (and x86_64 builds of certain packages like wine64, which has i686 dependencies) since the update to glibc 2.35.
> > 
> > It may also need assistance from the MES folks to fix, since the error message is about an undefined symbol in glibc-mesboot's libpthread.so.0:
> > 
> > make[2]: Entering directory '/tmp/guix-build-file-5.44.drv-0/file-5.44/magic'
> > ../src/file -C -m magic
> > /tmp/guix-build-file-5.44.drv-0/file-5.44/src/.libs/file: symbol lookup error: /gnu/store/s4yd6ibxsh5q1j9ipygb9vpjj4g00wc9-glibc-mesboot-2.16.0/lib/libpthread.so.0: undefined symbol: h_errno, version GLIBC_PRIVATE
> > make[2]: *** [Makefile:863: magic.mgc] Error 127
> > 
> > Cheers,
> > Kaelyn
> 
> 
> I think I found where this is coming from. %boot3-inputs added
> ld-wrapper-boot3 but didn't remove ld-wrapper-0, which pulled in
> glibc-mesboot. I'm testing out removing ld-wrapper-0 from %boot3-inputs
> to see if that's enough to make that final file build. Hopefully it'll
> also fix the final tar for i686, which I found was also failing for me.

Interesting! Since my last email, I was able to fix the issue with file by adding "--disable-xzlib" to the file package in gnu/packages/commencement.scm (after discovering it when noticing "--disable-bzlib" was being passed to the configure script), but hadn't sent in a patch yet because I hit a subsequent test failure while building tar. I thought to disable xz support because I traced the source of the glibc-mesboot libpthread.so in the error message to xz-mesboot being detected by the configure script and linked in even though file itself was being linked against a newer glibc and had no explicit dependencies. (I think the error after upgrading to glibc 2.35 from 2.33 was an abi compatibility between the newer glibc and the old pthread being pulled in via xz-mesboot.)

Cheers,
Kaelyn

> 
> --
> Efraim Flashner efraim@flashner.co.il אפרים פלשנר
> 
> GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
> Confidentiality cannot be guaranteed on emails sent or received unencrypted

Re: Merging core-updates?

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 3599 bytes --]

On Mon, Feb 13, 2023 at 09:36:17PM +0000, Kaelyn wrote:
> ------- Original Message -------
> On Monday, February 13th, 2023 at 8:04 PM, Efraim Flashner <efraim@flashner.co.il> wrote:
> > 
> > 
> > On Sun, Feb 12, 2023 at 06:29:04PM +0000, Kaelyn wrote:
> > 
> > > Hi,
> > > 
> > > ------- Original Message -------
> > > On Sunday, February 12th, 2023 at 5:08 PM, Andreas Enge andreas@enge.fr wrote:
> > > 
> > > > Am Sun, Feb 12, 2023 at 12:58:06PM +0100 schrieb Julien Lepiller:
> > > > 
> > > > > And I was able to rebuild (with --check) patch-mesboot. The error looks
> > > > > a lot like https://issues.guix.gnu.org/49985. We should fix that indeed
> > > > > :)
> > > > 
> > > > Ah indeed, that looks like deal breaking; maybe someone from MES can have
> > > > a look?
> > > > 
> > > > What is the magic incantation with double "@@" to build this package?
> > > > Ah, here we go, for reference to self:
> > > > guix build -e '(@@ (gnu packages commencement) patch-mesboot)'
> > > > 
> > > > Andreas
> > > 
> > > While not directly related to the patch-mesboot error, I want to mention that there is also https://issues.guix.gnu.org/58719 blocking i686 builds on core-updates (and x86_64 builds of certain packages like wine64, which has i686 dependencies) since the update to glibc 2.35.
> > > 
> > > It may also need assistance from the MES folks to fix, since the error message is about an undefined symbol in glibc-mesboot's libpthread.so.0:
> > > 
> > > make[2]: Entering directory '/tmp/guix-build-file-5.44.drv-0/file-5.44/magic'
> > > ../src/file -C -m magic
> > > /tmp/guix-build-file-5.44.drv-0/file-5.44/src/.libs/file: symbol lookup error: /gnu/store/s4yd6ibxsh5q1j9ipygb9vpjj4g00wc9-glibc-mesboot-2.16.0/lib/libpthread.so.0: undefined symbol: h_errno, version GLIBC_PRIVATE
> > > make[2]: *** [Makefile:863: magic.mgc] Error 127
> > > 
> > > Cheers,
> > > Kaelyn
> > 
> > 
> > I think I found where this is coming from. %boot3-inputs added
> > ld-wrapper-boot3 but didn't remove ld-wrapper-0, which pulled in
> > glibc-mesboot. I'm testing out removing ld-wrapper-0 from %boot3-inputs
> > to see if that's enough to make that final file build. Hopefully it'll
> > also fix the final tar for i686, which I found was also failing for me.
> 
> Interesting! Since my last email, I was able to fix the issue with file by adding "--disable-xzlib" to the file package in gnu/packages/commencement.scm (after discovering it when noticing "--disable-bzlib" was being passed to the configure script), but hadn't sent in a patch yet because I hit a subsequent test failure while building tar. I thought to disable xz support because I traced the source of the glibc-mesboot libpthread.so in the error message to xz-mesboot being detected by the configure script and linked in even though file itself was being linked against a newer glibc and had no explicit dependencies. (I think the error after upgrading to glibc 2.35 from 2.33 was an abi compatibility between the newer glibc and the old pthread being pulled in via xz-mesboot.)
> 
> Cheers,
> Kaelyn

I ended up going a different route and moving xz from the finalize
packages to an actual xz-final and replacing xz-bootstrap/xz-mesboot in
%boot6-inputs.

I also tracked down the issue in tar and adjusted the testsuite so it
shouldn't be a problem on 32-bit systems.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Architecture support [was: Re: Merging core-updates?]

[Andreas Enge]

Hello,

Am Mon, Feb 13, 2023 at 12:56:45PM +0200 schrieb Efraim Flashner:
> Aarch64 and armhf are getting stuck at gcc-cross-boot0.

I got to hello on my aarch64, which is very encouraging! I will have to
try again with your latest changes to core-updates, but am rather optimistic.

Andreas

Re: Architecture support [was: Re: Merging core-updates?]

[Julien Lepiller]

Could we get berlin to evaluate a small set of core packages (mpc, hello, …)? Are the changes intended to fix the issue with bordeaux's machines? Is it configured to build core-updates?

Le 14 février 2023 17:30:32 GMT+01:00, Andreas Enge <andreas@enge.fr> a écrit :
>Hello,
>
>Am Mon, Feb 13, 2023 at 12:56:45PM +0200 schrieb Efraim Flashner:
>> Aarch64 and armhf are getting stuck at gcc-cross-boot0.
>
>I got to hello on my aarch64, which is very encouraging! I will have to
>try again with your latest changes to core-updates, but am rather optimistic.
>
>Andreas
>
>

Re: Merging core-updates?

[Andreas Enge]

Am Mon, Feb 13, 2023 at 11:31:15PM +0200 schrieb Efraim Flashner:
> Looks like you might be able to drop openjdk-10-hotspot-stack-size.patch
> from openjdk-19.0.1.

Maybe. What is strange is that we have the same openjdk package on master,
apparently with the patch. I will give it a try nevertheless.

Andreas

Re: Architecture support [was: Re: Merging core-updates?]

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 844 bytes --]

On Tue, Feb 14, 2023 at 05:30:32PM +0100, Andreas Enge wrote:
> Hello,
> 
> Am Mon, Feb 13, 2023 at 12:56:45PM +0200 schrieb Efraim Flashner:
> > Aarch64 and armhf are getting stuck at gcc-cross-boot0.
> 
> I got to hello on my aarch64, which is very encouraging! I will have to
> try again with your latest changes to core-updates, but am rather optimistic.

It might just be my machines then. My pinebook pro doesn't have a lot of
space available for building and my pine64 only has 2GB of ram.

Also I made a typo in the tar fix, so I'll push a fix for that after I
confirm that it actually skips the broken test.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Merging core-updates?

[Kaelyn]

------- Original Message -------
On Tuesday, February 14th, 2023 at 2:50 PM, Efraim Flashner <efraim@flashner.co.il> wrote:
> 
[snip] 
> 
> I ended up going a different route and moving xz from the finalize
> packages to an actual xz-final and replacing xz-bootstrap/xz-mesboot in
> %boot6-inputs.
> 
> I also tracked down the issue in tar and adjusted the testsuite so it
> shouldn't be a problem on 32-bit systems.

I just tried building wine-minimal from core-updates at commit da7d615629, and I think you may have be missing a "!" with the new flag for the tar test suite (i.e. it needs to be "-k '!tricky time stamps'"). Based on the build log, right now only the failing test is being run:

## ------------- ##
## Test results. ##
## ------------- ##

ERROR: 1 test was run,
1 failed unexpectedly.

## ------------------------ ##
## Summary of the failures. ##
## ------------------------ ##
Failed tests:
GNU tar 1.34 test suite test groups:

 NUM: FILE-NAME:LINE     TEST-GROUP-NAME
      KEYWORDS

 151: time01.at:20       time: tricky time stamps
      time time01

Cheers,
Kaelyn

> 
> --
> Efraim Flashner efraim@flashner.co.il אפרים פלשנר
> 
> GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
> Confidentiality cannot be guaranteed on emails sent or received unencrypted

Re: Merging core-updates?

[Kaelyn]

------- Original Message -------
On Tuesday, February 14th, 2023 at 8:29 PM, Kaelyn <kaelyn.alexi@protonmail.com> wrote:
> 
> ------- Original Message -------
> On Tuesday, February 14th, 2023 at 2:50 PM, Efraim Flashner efraim@flashner.co.il wrote:
> 
> [snip]
> 
> > I ended up going a different route and moving xz from the finalize
> > packages to an actual xz-final and replacing xz-bootstrap/xz-mesboot in
> > %boot6-inputs.
> > 
> > I also tracked down the issue in tar and adjusted the testsuite so it
> > shouldn't be a problem on 32-bit systems.
> 
> 
> I just tried building wine-minimal from core-updates at commit da7d615629, and I think you may have be missing a "!" with the new flag for the tar test suite (i.e. it needs to be "-k '!tricky time stamps'"). Based on the build log, right now only the failing test is being run:
> 
> ## ------------- ##
> ## Test results. ##
> ## ------------- ##
> 
> ERROR: 1 test was run,
> 1 failed unexpectedly.
> 
> ## ------------------------ ##
> ## Summary of the failures. ##
> ## ------------------------ ##
> Failed tests:
> GNU tar 1.34 test suite test groups:
> 
> NUM: FILE-NAME:LINE TEST-GROUP-NAME
> KEYWORDS
> 
> 151: time01.at:20 time: tricky time stamps
> time time01

I wanted to give a quick update: once I fixed the test exclusion for tar, I was successful in building wine-minimal on core-updates (which is 32-bit package on x86_64 as well).

> Cheers,
> Kaelyn
> 
> > --
> > Efraim Flashner efraim@flashner.co.il אפרים פלשנר
> > 
> > GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
> > Confidentiality cannot be guaranteed on emails sent or received unencrypted

Re: Merging core-updates?

[Janneke Nieuwenhuizen]

Andreas Enge writes:

Hello,

> Am Mon, Feb 13, 2023 at 12:34:56PM +0100 schrieb Janneke Nieuwenhuizen:
>> To use stat64 and friends on 32bit, I created the attached patch for GNU
>> Mes and hope to create a 0.24.2 release from
>>     https://gitlab.com/janneke/mes/-/tree/wip-stat64
>
> Thanks a lot, Janneke!
>
>> Just hoping this is also a fix for
>>     https://debbugs.gnu.org/41264
>>     https://debbugs.gnu.org/49985
>> as I haven't been able to reproduce this bug.  IWBN if someone could
>> shed more light on that...
>
> The patch-mes package also builds on my laptop with an SSD, so I do not
> know how to reproduce it. If someone else does, that would be great.

Thanks a lot for testing!

I have released 0.24.2 and updated mes-boot on core-updates as
    b928e38bd333e6186727fe5c5e94b85d157b79d6

Let's hope this fixes these bugs.

Greetings,
Janneke

-- 
Janneke Nieuwenhuizen <janneke@gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

Re: Architecture support [was: Re: Merging core-updates?]

[Andreas Enge]

Am Tue, Feb 14, 2023 at 10:10:27PM +0200 schrieb Efraim Flashner:
> > I got to hello on my aarch64, which is very encouraging! I will have to
> > try again with your latest changes to core-updates, but am rather optimistic.
> 
> Also I made a typo in the tar fix, so I'll push a fix for that after I
> confirm that it actually skips the broken test.

With the typo fixed, I get to hello also on armhf (built on aarch64).
Great!

Andreas

Re: Architecture support

[Andreas Enge]

Am Tue, Feb 14, 2023 at 05:40:36PM +0100 schrieb Julien Lepiller:
> Could we get berlin to evaluate a small set of core packages (mpc, hello, …)? Are the changes intended to fix the issue with bordeaux's machines? Is it configured to build core-updates?

The MES changes were meant to fix bug #41264, which was not bordeaux
specific. I had launched by hand a core-updates evaluation a few days ago
there, but am hesitant to do so again, given that the machines are also
quite busy with QA work for other patches.

It might be better to try on berlin, where the machines are mostly idle.
However, the most recent core-updates evaluation failed (as well as all
previous ones):
   https://ci.guix.gnu.org/eval/194090
with
   https://ci.guix.gnu.org/eval/194090/log/raw
substitute: ␛[Kupdating substitutes from 'http://141.80.167.131'...   0.0%
substitute: ␛[Kcould not fetch http://141.80.167.131/9s8c9jc055nlspssczi5qh2p48d8ka4q.narinfo 404
substitute: updating substitutes from 'http://141.80.167.131'... 100.0%
building of `/gnu/store/95as6p67634qd68yah9zswsmbl0hmbjs-gcc-cross-boot0-11.3.0.drv' timed out after 21600 seconds

It looks like there is a problem with offloading, it would be nice if
someone with knowledge of cuirass and/or berlin could have a look.

Andreas

Re: Merging core-updates?

[Andreas Enge]

Am Tue, Feb 14, 2023 at 07:27:10PM +0100 schrieb Andreas Enge:
> Am Mon, Feb 13, 2023 at 11:31:15PM +0200 schrieb Efraim Flashner:
> > Looks like you might be able to drop openjdk-10-hotspot-stack-size.patch
> > from openjdk-19.0.1.
> Maybe. What is strange is that we have the same openjdk package on master,
> apparently with the patch. I will give it a try nevertheless.

Actually the patch has already been applied to openjdk13, if I am not
mistaken. So I do not understand how the source could be built in master
then, while the exact same code (?!) fails on core-updates...

I am trying to build openjdk13 without the patch as follows:
(define-public openjdk13
  (make-openjdk openjdk12 "13.0.13"
                "0pxf4dlig61k0pg7amg4mi919hzam7nzwckry01avgq1wj8ambji"
   (source (origin
             (method git-fetch)
             (uri (git-reference
                    (url "https://github.com/openjdk/jdk13u/")
                    (commit "jdk-13.0.13-ga")))
             (file-name (git-file-name name version))
             (sha256 #f)))))
(with the hash to be determined later).

This fails with
Initialized empty Git repository in /gnu/store/dyxp7njz9k20mar2h2cp8f2g438vigm2-openjdk-13.0.13-checkout/.git/
fatal: unable to access 'https://github.com/openjdk/jdk13u/': Could not resolve host: github.com

Do you know why? I am at a total loss as to what is happening...

Is there a way to keep the origin determined by make-openjdk and to just
empty its patches field?

Andreas

Re: Openjdk (was: Merging core-updates?)

[Andreas Enge]

Am Wed, Feb 15, 2023 at 07:51:56PM +0100 schrieb Andreas Enge:
> Actually the patch has already been applied to openjdk13, if I am not
> mistaken. So I do not understand how the source could be built in master
> then, while the exact same code (?!) fails on core-updates...

Well, there is a somewhat hidden difference.
core-updates introduces
                        "openjdk-10-hotspot-pointer-comparison.patch"
                        "openjdk-10-hotspot-stack-size.patch"
to openjdk10.

openjdk11 is a package of its own without the patch.

openjdk12 uses the newly defined make-openjdk to start from openjdk11,
overwriting the source together with openjdk-10-hotspot-stack-size.patch
in core-updates, and without the patch in master. (And it uses an obscure
tarball instead of a git checkout - why?)

openjdk13 has the same code in core-updates and master:
(define-public openjdk13
  (make-openjdk openjdk12 "13.0.13"
                "0pxf4dlig61k0pg7amg4mi919hzam7nzwckry01avgq1wj8ambji"))
So in core-updates it inherits the patch from openjdk12, in master
it does not (I think). And then I suppose it passes the patch on to all
its descendants.

The following seems to work and create source for openjdk13 and later:
(define-public openjdk13
  (make-openjdk openjdk12 "13.0.13"
                "0pxf4dlig61k0pg7amg4mi919hzam7nzwckry01avgq1wj8ambji"
  (source (origin
            (inherit (package-source base))
            (patches '())))))

Okay to push if I manage to build current openjdk with it?

Is it necessary to keep all these version of openjdk and to bootstrap
version n with version n-1?

Andreas

Re: Openjdk (was: Merging core-updates?)

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2355 bytes --]

On Wed, Feb 15, 2023 at 08:19:08PM +0100, Andreas Enge wrote:
> Am Wed, Feb 15, 2023 at 07:51:56PM +0100 schrieb Andreas Enge:
> > Actually the patch has already been applied to openjdk13, if I am not
> > mistaken. So I do not understand how the source could be built in master
> > then, while the exact same code (?!) fails on core-updates...
> 
> Well, there is a somewhat hidden difference.
> core-updates introduces
>                         "openjdk-10-hotspot-pointer-comparison.patch"
>                         "openjdk-10-hotspot-stack-size.patch"
> to openjdk10.
> 
> openjdk11 is a package of its own without the patch.
> 
> openjdk12 uses the newly defined make-openjdk to start from openjdk11,
> overwriting the source together with openjdk-10-hotspot-stack-size.patch
> in core-updates, and without the patch in master. (And it uses an obscure
> tarball instead of a git checkout - why?)
> 
> openjdk13 has the same code in core-updates and master:
> (define-public openjdk13
>   (make-openjdk openjdk12 "13.0.13"
>                 "0pxf4dlig61k0pg7amg4mi919hzam7nzwckry01avgq1wj8ambji"))
> So in core-updates it inherits the patch from openjdk12, in master
> it does not (I think). And then I suppose it passes the patch on to all
> its descendants.

It's definitely possible that the master->core-updates merge messed with
the package definitions and the inheritance and I didn't notice it.

> The following seems to work and create source for openjdk13 and later:
> (define-public openjdk13
>   (make-openjdk openjdk12 "13.0.13"
>                 "0pxf4dlig61k0pg7amg4mi919hzam7nzwckry01avgq1wj8ambji"
>   (source (origin
>             (inherit (package-source base))
>             (patches '())))))
> 
> Okay to push if I manage to build current openjdk with it?

Yeah, that's probably fine.

> Is it necessary to keep all these version of openjdk and to bootstrap
> version n with version n-1?

Probably? I assume if you can cut some out that'd be ok. I'm pretty sure
openjdk-11 and openjdk-17 are considered LTS by upstream so it would
make sense to keep those specifically.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Openjdk (was: Merging core-updates?)

[Julien Lepiller]

Le 16 février 2023 12:03:35 GMT+01:00, Efraim Flashner <efraim@flashner.co.il> a écrit :
>On Wed, Feb 15, 2023 at 08:19:08PM +0100, Andreas Enge wrote:
>
>> Is it necessary to keep all these version of openjdk and to bootstrap
>> version n with version n-1?
>
>Probably? I assume if you can cut some out that'd be ok. I'm pretty sure
>openjdk-11 and openjdk-17 are considered LTS by upstream so it would
>make sense to keep those specifically.

Upstream ensures openjdk N+1 can be built by openjdk N, but not necessarily with N-1. We can try :)

Re: Merging core-updates?

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 1214 bytes --]



On 15-02-2023 19:51, Andreas Enge wrote:
> I am trying to build openjdk13 without the patch as follows:
> (define-public openjdk13
>    (make-openjdk openjdk12 "13.0.13"
>                  "0pxf4dlig61k0pg7amg4mi919hzam7nzwckry01avgq1wj8ambji"
>     (source (origin
>               (method git-fetch)
>               (uri (git-reference
>                      (url"https://github.com/openjdk/jdk13u/")
>                      (commit "jdk-13.0.13-ga")))
>               (file-name (git-file-name name version))
>               (sha256 #f)))))
> (with the hash to be determined later).
> 
> This fails with
> Initialized empty Git repository in/gnu/store/dyxp7njz9k20mar2h2cp8f2g438vigm2-openjdk-13.0.13-checkout/.git/
> fatal: unable to access 'https://github.com/openjdk/jdk13u/': Could not resolve host: github.com
> 
> Do you know why? I am at a total loss as to what is happening...

You didn't write the hash.  As the hash is unknown, it would be 
irreproducible for the Guix daemon to grant the build process access to 
the network, so the Guix daemon doesn't.

You'll need to enter a hash (possibly a bogus one that the daemon can 
complain about later).

Greetings,
Maxime.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: Merging core-updates?

[Andreas Enge]

Am Wed, Feb 15, 2023 at 09:39:39AM +0100 schrieb Janneke Nieuwenhuizen:
> I have released 0.24.2 and updated mes-boot on core-updates as
> Let's hope this fixes these bugs.

With your latest patch, I have successfully bootstrapped core-updates
on x86_64 up to hello and mpc. Thanks a lot!

Andreas

bug#49985: Merging core-updates?

[Janneke Nieuwenhuizen]

Andreas Enge writes:

> Am Wed, Feb 15, 2023 at 09:39:39AM +0100 schrieb Janneke Nieuwenhuizen:
>> I have released 0.24.2 and updated mes-boot on core-updates as
>> Let's hope this fixes these bugs.
>
> With your latest patch, I have successfully bootstrapped core-updates
> on x86_64 up to hello and mpc. Thanks a lot!

Great, thanks so much for checking!  Are you using any of tmpfs or btrfs
on /tmp?

Greetings,
janneke

-- 
Janneke Nieuwenhuizen <janneke@gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

Re: Merging core-updates?

[Andreas Enge]

Am Thu, Feb 16, 2023 at 04:03:15PM +0100 schrieb Janneke Nieuwenhuizen:
> Great, thanks so much for checking!  Are you using any of tmpfs or btrfs
> on /tmp?

No, it is all on SSD, so we probably cannot conclude for the bugs,
unfortunately.

Andreas

Re: Merging core-updates?

[Julien Lepiller]

I haven't tried the patch, but before it, I was already able to build mpc for x86_64 on a SSD with btrfs.

Le 16 février 2023 16:03:15 GMT+01:00, Janneke Nieuwenhuizen <janneke@gnu.org> a écrit :
>Andreas Enge writes:
>
>> Am Wed, Feb 15, 2023 at 09:39:39AM +0100 schrieb Janneke Nieuwenhuizen:
>>> I have released 0.24.2 and updated mes-boot on core-updates as
>>> Let's hope this fixes these bugs.
>>
>> With your latest patch, I have successfully bootstrapped core-updates
>> on x86_64 up to hello and mpc. Thanks a lot!
>
>Great, thanks so much for checking!  Are you using any of tmpfs or btrfs
>on /tmp?
>
>Greetings,
>janneke
>

Re: Merging core-updates?

[Andreas Enge]

Am Thu, Feb 16, 2023 at 12:41:08PM +0100 schrieb Maxime Devos:
> You didn't write the hash.  As the hash is unknown, it would be
> irreproducible for the Guix daemon to grant the build process access to the
> network, so the Guix daemon doesn't.
> You'll need to enter a hash (possibly a bogus one that the daemon can
> complain about later).

Interesting, thanks for the explanation!

Andreas

Re: Merging core-updates?

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 389 bytes --]

Hi everyone, 

I'd love to help with the core-updates merge, but I don't have a beefy
machine right now and would love to avoid building all the bootstrap
locally.  The evaluations on CI seem to keep failing, with no info
available [1].  Do we have more info about them/know what's making them
fail?

[1] https://ci.guix.gnu.org/jobset/core-updates

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: Openjdk (was: Merging core-updates?)

[Andreas Enge]

Hello,

Am Thu, Feb 16, 2023 at 01:03:35PM +0200 schrieb Efraim Flashner:
> > Okay to push if I manage to build current openjdk with it?
> Yeah, that's probably fine.

after a day, I got past the point of mesa in core-updates on x86_64.
Which in itself is an encouraging milestone.

Andreas

Re: Openjdk (was: Merging core-updates?)

[Andreas Enge]

> The following seems to work and create source for openjdk13 and later:
> (define-public openjdk13
>   (make-openjdk openjdk12 "13.0.13"
>                 "0pxf4dlig61k0pg7amg4mi919hzam7nzwckry01avgq1wj8ambji"
>   (source (origin
>             (inherit (package-source base))
>             (patches '())))))

Hm, I compiled up to openjdk@13, @14 fails with the message below.
This is strange. It looks as if the patch that has become obsolete,
because integrated into the source @13, is needed again @14!
And then @15, the patch is again already integrated into the source code.
Very weird!

Giving it a try now.

Andreas


Creating CDS archive for jdk image
In file included from /tmp/guix-build-openjdk-14.0.2.drv-0/source/test/hotspot/jtreg/vmTestbase/vm/mlvm/indy/func/jvmti/stepBreakPopReturn/libstepBreakPopReturn.cpp:31:
/tmp/guix-build-openjdk-14.0.2.drv-0/source/test/hotspot/jtreg/vmTestbase/vm/mlvm/share/mlvmJvmtiUtils.cpp: In function 'MethodName* getMethodName(jvmtiEnv*, jmethodID)':
/tmp/guix-build-openjdk-14.0.2.drv-0/source/test/hotspot/jtreg/vmTestbase/vm/mlvm/share/mlvmJvmtiUtils.cpp:100:12: warning: 'char* strncpy(char*, const char*, size_t)' specified bound 256 equals destination size [-Wstringop-truncation]
  100 |     strncpy(mn->classSig, szSignature, sizeof(mn->classSig));
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /tmp/guix-build-openjdk-14.0.2.drv-0/source/test/hotspot/jtreg/vmTestbase/vm/mlvm/indy/func/jvmti/share/libIndyRedefineClass.cpp:31:
/tmp/guix-build-openjdk-14.0.2.drv-0/source/test/hotspot/jtreg/vmTestbase/vm/mlvm/share/mlvmJvmtiUtils.cpp: In function 'MethodName* getMethodName(jvmtiEnv*, jmethodID)':
/tmp/guix-build-openjdk-14.0.2.drv-0/source/test/hotspot/jtreg/vmTestbase/vm/mlvm/share/mlvmJvmtiUtils.cpp:100:12: warning: 'char* strncpy(char*, const char*, size_t)' specified bound 256 equals destination size [-Wstringop-truncation]
  100 |     strncpy(mn->classSig, szSignature, sizeof(mn->classSig));
      |     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/guix-build-openjdk-14.0.2.drv-0/source/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c: In function 'set_signal_handler':
/tmp/guix-build-openjdk-14.0.2.drv-0/source/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c:71:15: error: storage size of 'altstack' isn't constant
   71 |   static char altstack[SIGSTKSZ];
      |               ^~~~~~~~
make[3]: *** [JtregNativeHotspot.gmk:1532: /tmp/guix-build-openjdk-14.0.2.drv-0/source/build/linux-x86_64-server-release/support/test/hotspot/jtreg/native/support/exeinvoke/exeinvoke.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [make/Main.gmk:550: build-test-hotspot-jtreg-native] Error 2

ERROR: Build failed for target 'all' in configuration 'linux-x86_64-server-release' (exit code 2)

=== Output from failing command(s) repeated here ===
* For target support_test_hotspot_jtreg_native_support_exeinvoke_exeinvoke.o:
/tmp/guix-build-openjdk-14.0.2.drv-0/source/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c: In function 'set_signal_handler':
/tmp/guix-build-openjdk-14.0.2.drv-0/source/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c:71:15: error: storage size of 'altstack' isn't constant
   71 |   static char altstack[SIGSTKSZ];
      |               ^~~~~~~~

* All command lines available in /tmp/guix-build-openjdk-14.0.2.drv-0/source/build/linux-x86_64-server-release/make-support/failure-logs.
=== End of repeated output ===

No indication of failed target found.
Hint: Try searching the build log for '] Error'.
Hint: See doc/building.html#troubleshooting for assistance.

make[1]: *** [/tmp/guix-build-openjdk-14.0.2.drv-0/source/make/Init.gmk:312: main] Error 2
make: *** [/tmp/guix-build-openjdk-14.0.2.drv-0/source/make/Init.gmk:186: all] Error 2
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("all" "JOBS=4") exit-status: 2 term-signal: #f stop-signal: #f>
phase `build' failed after 1605.0 seconds
command "make" "all" "JOBS=4" failed with status 2
builder for `/gnu/store/3i4i2kh5zbw2l3rdp7gx41ykxgnbfr1x-openjdk-14.0.2.drv' failed with exit code 1
build of /gnu/store/3i4i2kh5zbw2l3rdp7gx41ykxgnbfr1x-openjdk-14.0.2.drv failed
View build log at '/var/log/guix/drvs/3i/4i2kh5zbw2l3rdp7gx41ykxgnbfr1x-openjdk-14.0.2.drv.gz'.

Re: Openjdk (was: Merging core-updates?)

[Andreas Enge]

Am Fri, Feb 17, 2023 at 03:49:19PM +0100 schrieb Andreas Enge:
> Hm, I compiled up to openjdk@13, @14 fails with the message below.
> This is strange. It looks as if the patch that has become obsolete,
> because integrated into the source @13, is needed again @14!
> And then @15, the patch is again already integrated into the source code.
> Very weird!

Things become totally crazy.
So far I have compiled @13 without the patch, @14 with the patch,
@15 without the patch, and @16 seems to need it again. Is this an
odd/even thing? Are there spring and autumn versions of openjdk with
different release teams?

Well, in @17, @18 and @19 the patch seems to be definitely integrated
into the source code.

Andreas

Re: Architecture support [was: Re: Merging core-updates?]

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 555 bytes --]


Julien Lepiller <julien@lepiller.eu> writes:

> Could we get berlin to evaluate a small set of core packages (mpc,
> hello, …)? Are the changes intended to fix the issue with bordeaux's
> machines? Is it configured to build core-updates?

There's some rudimentary support for building packages from branches in
the qa-frontpage, but that currently just submits builds for all
packages.

I'll keep an eye on the core-updates state on data.qa.guix.gnu.org and
submit some builds for core things manually when the derivations are
available.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: Openjdk (was: Merging core-updates?)

[Kaelyn]

Hi,

------- Original Message -------
On Friday, February 17th, 2023 at 4:28 PM, Andreas Enge <andreas@enge.fr> wrote:

> Am Fri, Feb 17, 2023 at 03:49:19PM +0100 schrieb Andreas Enge:
> 
> > Hm, I compiled up to openjdk@13, @14 fails with the message below.
> > This is strange. It looks as if the patch that has become obsolete,
> > because integrated into the source @13, is needed again @14!
> > And then @15, the patch is again already integrated into the source code.
> > Very weird!
> 
> 
> Things become totally crazy.
> So far I have compiled @13 without the patch, @14 with the patch,
> @15 without the patch, and @16 seems to need it again. Is this an
> odd/even thing? Are there spring and autumn versions of openjdk with
> different release teams?

Hi,

I don't know much about openjdk or its development process, but I had one possible thought about the pattern of patch application. Was it a patch that was applied or backported to existing openjdk releases, and are @14 and @16 the latest releases of those series? At least to me as a naive outside observer, the patch being needed for some of the older release branches but not others sounds like a patch applied to multiple existing release branches, with openjdk@14 and @16 being at older point releases from prior to the patch being applied upstream.

HTH! (And I apologize for the noise if not!)

Cheers,
Kaelyn
 
> Well, in @17, @18 and @19 the patch seems to be definitely integrated
> into the source code.
> 
> Andreas

Re: Openjdk (was: Merging core-updates?)

[Andreas Enge]

Am Fri, Feb 17, 2023 at 05:27:02PM +0000 schrieb Kaelyn:
> I don't know much about openjdk or its development process, but I had one possible thought about the pattern of patch application. Was it a patch that was applied or backported to existing openjdk releases, and are @14 and @16 the latest releases of those series? At least to me as a naive outside observer, the patch being needed for some of the older release branches but not others sounds like a patch applied to multiple existing release branches, with openjdk@14 and @16 being at older point releases from prior to the patch being applied upstream.

Ah, right, this sounds like a good explanation, thanks, Kaelyn!

I think we should consider core-updates frozen for the moment, but someone
updating these intermediate openjdk versions in the future should pay
attention to it.

In the meantime, I have pushed a convoluted patch that adds and removes
the patch by dancing back and forth.

Andreas

Ocaml (was: Merging core-updates?)

[Andreas Enge]

Hello,

Am Mon, Feb 13, 2023 at 09:35:32PM +0100 schrieb Andreas Enge:
> ocaml-4.0.9 fails with
> gcc -c -O2 -fno-strict-aliasing -fwrapv -Wall -fno-tree-vrp -g -D_FILE_OFFSET_BITS=64 -D_REENTRANT -DCAML_NAME_SPACE  -DOCAML_STDLIB_DIR='"/gnu/store/mzzak7bp6dpngq04hphlx58wzp62755m-ocaml-4.09.0/lib/ocaml"'  -DNATIVE_CODE -DTARGET_amd64 -DMODEL_default -DSYS_linux   -o signals_nat_n.o signals_nat.c
> signals_nat.c:185:13: error: variably modified ‘sig_alt_stack’ at file scope
>   185 | static char sig_alt_stack[SIGSTKSZ];
>       |             ^~~~~~~~~~~~~
> make[3]: *** [Makefile:343: signals_nat_n.o] Error 1

this looks exactly like the line that posed problems for openjdk.
Maybe there is a patch? Could someone familiar with ocaml have a look?

Andreas

Re: Openjdk (was: Merging core-updates?)

[Andreas Enge]

Am Thu, Feb 16, 2023 at 12:38:23PM +0100 schrieb Julien Lepiller:
> Upstream ensures openjdk N+1 can be built by openjdk N, but not necessarily with N-1. We can try :)

And fail :)

configure: Potential Boot JDK found at /gnu/store/lqfppbbxhq503hfy2xf3djivqv3s8df4-openjdk-17.0.5-jdk is incorrect JDK version (openjdk version "17.0.5" 2022-10-18 OpenJDK Runtime Environment (build 17.0.5+0-adhoc..source) OpenJDK 64-Bit Server VM (build 17.0.5+0-adhoc..source, mixed mode, sharing)); ignoring
configure: (Your Boot JDK version must be one of: 18 19)
configure: Could not find a valid Boot JDK. OpenJDK distributions are available at http://jdk.java.net/.
configure: This might be fixed by explicitly setting --with-boot-jdk
configure: error: Cannot continue

So @19 does not bootstrap with @17. Neither does @17 with @15. Dommage!

Andreas

Re: Ocaml (was: Merging core-updates?)

[Andreas Enge]

Am Sat, Feb 18, 2023 at 12:03:31PM +0100 schrieb Andreas Enge:
> this looks exactly like the line that posed problems for openjdk.
> Maybe there is a patch? Could someone familiar with ocaml have a look?

It looks like this:
   https://github.com/ocaml/ocaml/pull/10266

Andreas

Re: Ocaml (was: Merging core-updates?)

[Julien Lepiller]

ocaml-4.14 and ocaml-5 don't have this issue. I just pushed a fix for
ocaml-4.09. I'll also have to fix ocaml-4.07 since it fails to build as
early as camlboot. It'll take a while to test. Thanks for pointing me
to that patch!

Le Sat, 18 Feb 2023 12:38:55 +0100,
Andreas Enge <andreas@enge.fr> a écrit :

> Am Sat, Feb 18, 2023 at 12:03:31PM +0100 schrieb Andreas Enge:
> > this looks exactly like the line that posed problems for openjdk.
> > Maybe there is a patch? Could someone familiar with ocaml have a
> > look?  
> 
> It looks like this:
>    https://github.com/ocaml/ocaml/pull/10266
> 
> Andreas
> 

Python (was: Merging core-updates?)

[Andreas Enge]

Hello,

I am having problems with at least two python packages in core-updates:

***   File "/gnu/store/5i3yqwaqd8mayl2vr9lmrihxwv8203b1-python-pycrypto-2.6.1/lib/python3.10/site-packages/Crypto/Util/number.py", line 139
    value |= 2L ** (N-1)                # Ensure high bit is set
             ^
SyntaxError: invalid decimal literal
error: in phase 'install': uncaught exception:
%exception #<&invoke-error program: "python" arguments: ("-m" "compileall" "--invalidation-mode=unchecked-hash" "/gnu/store/5i3yqwaqd8mayl2vr9lmrihxwv8203b1-python-pycrypto-2.6.1") exit-status: 1 term-signal: #f stop-signal: #f>
phase `install' failed after 0.5 seconds
command "python" "-m" "compileall" "--invalidation-mode=unchecked-hash" "/gnu/store/5i3yqwaqd8mayl2vr9lmrihxwv8203b1-python-pycrypto-2.6.1" failed with status 1


starting phase `sanity-check'
validating 'attrdict' /gnu/store/lvy1fmmf1dsr3fjw82zal2aaisf3d47k-python-attrdict-2.0.1/lib/python3.10/site-pac
kages
...checking requirements: OK
...trying to load module attrdict: ERROR:
Traceback (most recent call last):
  File "/gnu/store/35ix1m6m8a5s21j02ajhdyqxb2xkshfb-sanity-check.py", line 69, in <module>
    importlib.import_module(name)
  File "/gnu/store/blals34ar25fiifvm17m2b504waxzys0-python-3.10.7/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/gnu/store/lvy1fmmf1dsr3fjw82zal2aaisf3d47k-python-attrdict-2.0.1/lib/python3.10/site-packages/attrdict/__init__.py", line 5, in <module>
    from attrdict.mapping import AttrMap
  File "/gnu/store/lvy1fmmf1dsr3fjw82zal2aaisf3d47k-python-attrdict-2.0.1/lib/python3.10/site-packages/attrdict/mapping.py", line 4, in <module>
    from collections import Mapping
ImportError: cannot import name 'Mapping' from 'collections' (/gnu/store/blals34ar25fiifvm17m2b504waxzys0-python-3.10.7/lib/python3.10/collections/__init__.py)
error: in phase 'sanity-check': uncaught exception:
%exception #<&invoke-error program: "python" arguments: ("/gnu/store/35ix1m6m8a5s21j02ajhdyqxb2xkshfb-sanity-check.py" "/gnu/store/lvy1fmmf1dsr3fjw82zal2aaisf3d47k-python-attrdict-2.0.1/lib/python3.10/site-packages") exit-status: 1 term-signal: #f stop-signal: #f>
phase `sanity-check' failed after 0.2 seconds
command "python" "/gnu/store/35ix1m6m8a5s21j02ajhdyqxb2xkshfb-sanity-check.py" "/gnu/store/lvy1fmmf1dsr3fjw82zal2aaisf3d47k-python-attrdict-2.0.1/lib/python3.10/site-packages" failed with status 1
for python-attrdict.

Both are at their latest version from Pypi.

Have there been some incompatible changes in Python 3.10? Should we revert
the Python update or try to backport patches? (I have no idea about Python,
and probably need it only for calibre.)

Andreas


PS: On the second issue: The latest commit is this:
    v2.0.1  2019/02/01 -- Haven't used or looked at this in years so updating tests to the current version of python and then marking it inactive.
    This would rather make me thing we should drop it, but here we go:
    Building the following 160 packages would ensure 366 dependent packages are rebuilt: kicad@6.0.10 ...

    There is something called attrdict3:
    https://pypi.org/project/attrdict3/
    at the same version +0.0.1; maybe we should use this?


PPS: On the first issue, the homepage says:
   PyCrypto 2.x is unmaintained, obsolete, and contains security vulnerabilities.
   Please choose one of the following alternatives:
   Cryptography
     Recommended for new applications.
     Newer API with fewer gotchas.
     API docs
     GitHub
     PyPI
   PyCryptodome
     Recommended for existing software that depends on PyCrypto.
     Fork of PyCrypto. Most applications should run unmodified.
     API docs
     GitHub
     PyPI

   Then we have:
      Building the following 6 packages would ensure 9 dependent packages are rebuilt: python-miio@0.5.11 ledger-agent@0.9.0 electrum@4.3.2 eolie@0.9.101 jrnl@1.9.7 poezio@0.13.2
   We already have python-pycryptodome and python-pycryptodomex.
   Maybe we should try rebuilding the 9 dependent packages with one of them?
   Do the specialists have a preference as to which one to use?
   Both have a similar number of dependents currently.

Re: Python (was: Merging core-updates?)

[Andreas Enge]

Am Sun, Feb 19, 2023 at 12:02:15PM +0100 schrieb Andreas Enge:
> PPS: On the first issue, the homepage says:
>    PyCrypto 2.x is unmaintained, obsolete, and contains security vulnerabilities.
>       Building the following 6 packages would ensure 9 dependent packages are rebuilt: python-miio@0.5.11 ledger-agent@0.9.0 electrum@4.3.2 eolie@0.9.101 jrnl@1.9.7 poezio@0.13.2

I am looking at these packages. One of them, ledger-agent, dates from 2017
and has seen 25 releases in the meantime. I can of course try to update it
(in main? core-updates?), but I am also wondering whether we have a
deprecation policy. This feels like a package nobody is interested in, and
it is demotivating to spend time fixing it... (Well, it is entirely possible
that flocks of users are still clinging on to a perfectly working old
version, but well!)

Andreas

Re: Python (was: Merging core-updates?)

[Andreas Enge]

Am Sun, Feb 19, 2023 at 12:15:59PM +0100 schrieb Andreas Enge:
> I am looking at these packages. One of them, ledger-agent, dates from 2017
> and has seen 25 releases in the meantime.

Well, maybe, maybe not. The version in Pypi has not changed,
but it is somehow in the same git repository as trezor-agent,
and I do not totally understand how these are related. Taking
back my rant and acknowledging my ignorance.

Andreas

Re: Python (was: Merging core-updates?)

[Andreas Enge]

And another one: python-ecdsa

I tried to update it from 0.17.0 to 0.18.0, but it still fails its tests
with this message:
src/ecdsa/test_jacobi.py:393: TypeError
=============================== warnings summary ===============================
src/ecdsa/test_der.py::TestEncodeBitstring::test_implicit_unused_bits
src/ecdsa/test_der.py::TestEncodeBitstring::test_new_call_convention
src/ecdsa/test_der.py::TestRemoveBitstring::test_implicit_unexpected_unused
src/ecdsa/test_der.py::TestRemoveBitstring::test_new_call_convention
  /gnu/store/blals34ar25fiifvm17m2b504waxzys0-python-3.10.7/lib/python3.10/unittest/case.py:549: PytestRemovedIn8Warning: Passing None has been deprecated.
  See https://docs.pytest.org/en/latest/how-to/capture-warnings.html#additional-use-cases-of-warnings-in-tests for alternatives in common use cases.
    method()

Andreas

Re: Python (was: Merging core-updates?)

[Lars-Dominik Braun]

Hi Andreas,

> ***   File "/gnu/store/5i3yqwaqd8mayl2vr9lmrihxwv8203b1-python-pycrypto-2.6.1/lib/python3.10/site-packages/Crypto/Util/number.py", line 139
>     value |= 2L ** (N-1)                # Ensure high bit is set
>              ^
> SyntaxError: invalid decimal literal
> error: in phase 'install': uncaught exception:
> %exception #<&invoke-error program: "python" arguments: ("-m" "compileall" "--invalidation-mode=unchecked-hash" "/gnu/store/5i3yqwaqd8mayl2vr9lmrihxwv8203b1-python-pycrypto-2.6.1") exit-status: 1 term-signal: #f stop-signal: #f>
> phase `install' failed after 0.5 seconds
> command "python" "-m" "compileall" "--invalidation-mode=unchecked-hash" "/gnu/store/5i3yqwaqd8mayl2vr9lmrihxwv8203b1-python-pycrypto-2.6.1" failed with status 1
this particular line looks different with Python 3.9, since the package
is built with an automated Python 2 to Python 3 converter, which does
not seems to work correctly on 3.10 (build_py_2to3 in setup.py). Not
sure why though. Given the warning on their homepage it’s probably
safe to drop the package.

>     from collections import Mapping
> ImportError: cannot import name 'Mapping' from 'collections' (/gnu/store/blals34ar25fiifvm17m2b504waxzys0-python-3.10.7/lib/python3.10/collections/__init__.py)
This is trivial to fix and should be

	from collections.abc import Mapping

which seems to be the only change in attrdict3, see https://github.com/pirofti/AttrDict3/commit/f6678b627b469c9aeddca2a9e4ba4e1ee9e3ccbb

Cheers,
Lars

Re: Python (was: Merging core-updates?)

[Andreas Enge]

Am Sun, Feb 19, 2023 at 12:02:15PM +0100 schrieb Andreas Enge:
>    Then we have:
>       Building the following 6 packages would ensure 9 dependent packages are rebuilt: python-miio@0.5.11 ledger-agent@0.9.0 electrum@4.3.2 eolie@0.9.101 jrnl@1.9.7 poezio@0.13.2

Concerning poezio, it depends on python-potr (and is its only dependent),
which in turn depends on python-pycrypto.

Concerning python-potr, I am a bit at a loss. There is
   https://github.com/python-otr/pure-python-otr
with their latest release 1.0.2 in 2018 and a big bold comment
"This software is experimental and potentially insecure. Do not rely on it".

Pypi has this:
   https://pypi.org/project/python-otr/
which I suppose is a different project.

Would it make sense to remove python-potr and poezio?
I am not confident with crypto libraries that call themselves insecure...

Andreas

Re: Python (was: Merging core-updates?)

[Andreas Enge]

Hello Lars,

thanks for having a look!

Am Sun, Feb 19, 2023 at 12:47:46PM +0100 schrieb Lars-Dominik Braun:
> > command "python" "-m" "compileall" "--invalidation-mode=unchecked-hash" "/gnu/store/5i3yqwaqd8mayl2vr9lmrihxwv8203b1-python-pycrypto-2.6.1" failed with status 1
> this particular line looks different with Python 3.9, since the package
> is built with an automated Python 2 to Python 3 converter, which does
> not seems to work correctly on 3.10 (build_py_2to3 in setup.py). Not
> sure why though. Given the warning on their homepage it’s probably
> safe to drop the package.

Except that we have to decide what to do about its dependents...

> >     from collections import Mapping
> > ImportError: cannot import name 'Mapping' from 'collections' (/gnu/store/blals34ar25fiifvm17m2b504waxzys0-python-3.10.7/lib/python3.10/collections/__init__.py)
> This is trivial to fix and should be
> 	from collections.abc import Mapping
> which seems to be the only change in attrdict3, see https://github.com/pirofti/AttrDict3/commit/f6678b627b469c9aeddca2a9e4ba4e1ee9e3ccbb

Great, I will replace the package then.

Andreas

Re: Python (was: Merging core-updates?)

[Attila Lendvai]

> but it is somehow in the same git repository as trezor-agent,
> and I do not totally understand how these are related. Taking
> back my rant and acknowledging my ignorance.

weirdly enough, upstream uses one git repo for multiple projects, and uses prefixed tag names for them.

FYI, there's this long-pending patchset to update the trezor-agent (something i can test myself):

https://issues.guix.gnu.org/58437#4

it's been pending so long, maybe it should be updated again.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“Hurt people hurt people. That's how pain patterns gets passed on, generation after generation after generation. Break the chain today. Meet anger with sympathy, contempt with compassion, cruelty with kindness. Greet grimaces with smiles. Forgive and forget about finding fault. Love is the weapon of the future.”
	— Yehuda Berg

Re: Python (was: Merging core-updates?)

[Lars-Dominik Braun]

Hi,

> Except that we have to decide what to do about its dependents...
upgrade or drop if not possible. pycryptodome does not provide an entirely
compatible interface (see https://www.pycryptodome.org/src/vs_pycrypto),
so we cannot simply switch existing packages from pycrypto to pycryptdome
without manual testing and (possibly) patching.

eolie upstream looks dead, same with jrnl. The rest seems to be alive
without any references to python-pycrypto. So these should be upgradable
and then we can drop python-pycrypto.

Lars

Re: Python (was: Merging core-updates?)

[Andreas Enge]

Am Sun, Feb 19, 2023 at 12:57:07PM +0100 schrieb Andreas Enge:
> > which seems to be the only change in attrdict3, see https://github.com/pirofti/AttrDict3/commit/f6678b627b469c9aeddca2a9e4ba4e1ee9e3ccbb
> Great, I will replace the package then.

Done. Interestingly enough, there was only one dependent: python-wxpython;
which has three dependents, of which python-matplotlib, and from there it
propagates everywhere...

Andreas

Re: Python

[Andreas Enge]

Am Sun, Feb 19, 2023 at 04:50:37PM +0100 schrieb Lars-Dominik Braun:
> eolie upstream looks dead, same with jrnl.

How do we drop them from the distribution? Do we need to prepare a NEWS
item?

> The rest seems to be alive
> without any references to python-pycrypto. So these should be upgradable
> and then we can drop python-pycrypto.

I found python-miio, which depends on python-android-backup, which depends
on python-pycrypto.

The first one looks alive with a new release last July, the middle one
looks dead.

Looking at
   https://github.com/rytilahti/python-miio/blob/master/pyproject.toml
android-backup appears to be an optional dependency.

Are you okay with dropping python-android-backup from Guix?

Andreas

Re: Python

[Andreas Enge]

And yet another one: python-testtool

Tests running...
======================================================================
FAIL: testtools.tests.test_testresult.TestNonAsciiResults.test_syntax_error
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/guix-build-python-testtools-2.5.0.drv-0/testtools-2.5.0/testtools/tests/test_testresult.py", line 2675, in test_syntax_error
    self.assertIn(self._as_output(
  File "/tmp/guix-build-python-testtools-2.5.0.drv-0/testtools-2.5.0/testtools/testcase.py", line 399, in assertIn
    self.assertThat(haystack, Contains(needle), message)
  File "/tmp/guix-build-python-testtools-2.5.0.drv-0/testtools-2.5.0/testtools/testcase.py", line 480, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: '  File "<string>", line 1\n    f(a, b c)\n         ^\nSyntaxError: ' not in 'Tests running...\n======================================================================\nERROR: test_syntax_error.Test.runTest\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File "/tmp/guix-build-python-testtools-2.5.0.drv-0/TestNonAsciiResults3xz34fmg/test_syntax_error.py", line 6, in runTest\n    exec (\'f(a, b c)\')\n  File "<string>", line 1\n    f(a, b c)\n         ^^^\nSyntaxError: invalid syntax. Perhaps you forgot a comma?\n\nRan 1 test in 0.001s\nFAILED (failures=1)\n'
======================================================================
FAIL: testtools.tests.test_testresult.TestNonAsciiResultsWithUnittest.test_syntax_error
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/guix-build-python-testtools-2.5.0.drv-0/testtools-2.5.0/testtools/tests/test_testresult.py", line 2675, in test_syntax_error
    self.assertIn(self._as_output(
  File "/tmp/guix-build-python-testtools-2.5.0.drv-0/testtools-2.5.0/testtools/testcase.py", line 399, in assertIn
    self.assertThat(haystack, Contains(needle), message)
  File "/tmp/guix-build-python-testtools-2.5.0.drv-0/testtools-2.5.0/testtools/testcase.py", line 480, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: '  File "<string>", line 1\n    f(a, b c)\n         ^\nSyntaxError: ' not in 'E\n======================================================================\nERROR: runTest (test_syntax_error.Test)\ntest_syntax_error.Test.runTest\n----------------------------------------------------------------------\ntesttools.testresult.real._StringException: Traceback (most recent call last):\n  File "/tmp/guix-build-python-testtools-2.5.0.drv-0/TestNonAsciiResultsWithUnittest7v17h5v5/test_syntax_error.py", line 6, in runTest\n    exec (\'f(a, b c)\')\n  File "<string>", line 1\n    f(a, b c)\n         ^^^\nSyntaxError: invalid syntax. Perhaps you forgot a comma?\n\n\n----------------------------------------------------------------------\nRan 1 test in 0.000s\n\nFAILED (errors=1)\n'

Ran 2627 tests in 1.178s
FAILED (failures=2)
error: in phase 'check': uncaught exception:
%exception #<&invoke-error program: "python" arguments: ("-m" "testtools.run" "testtools.tests.test_suite") exit-status: 1 term-signal: #f stop-signal: #f>
phase `check' failed after 2.2 seconds
command "python" "-m" "testtools.run" "testtools.tests.test_suite" failed with status 1
builder for `/gnu/store/gj4g2wcf8gam52rl1ndjrypiya6pa6r0-python-testtools-2.5.0.drv' failed with exit code 1
build of /gnu/store/gj4g2wcf8gam52rl1ndjrypiya6pa6r0-python-testtools-2.5.0.drv failed
View build log at '/var/log/guix/drvs/gj/4g2wcf8gam52rl1ndjrypiya6pa6r0-python-testtools-2.5.0.drv.gz'.

What an entangled mess these languages are that are not C!
Again, this is apparently the latest version of the package.

Andreas

Re: Python (was: Merging core-updates?)

[Andreas Enge]

Am Sun, Feb 19, 2023 at 04:50:37PM +0100 schrieb Lars-Dominik Braun:
> The rest seems to be alive
> without any references to python-pycrypto. So these should be upgradable
> and then we can drop python-pycrypto.

I more or less got rid of one of them: python-ledgerblue.
I have updated it from 0.1.16 of 2016 (!) to 0.1.44 of last month.
The package builds, but the tests fail. I did not find an intermediate
commit that would not depend on python-pycrypto, but pass its tests.
(Well, I did not check each and every of them either.)

I pushed nevertheless, since the situation is not worse than before.
Maybe someone more knowledgeable could have a look and see whether the
tests can be fixed or should be disabled. Here is the error message:
running build_ext
usage: -c [-h] [--targetId TARGETID] [--rootPrivateKey ROOTPRIVATEKEY]
          [--apdu] [--deployLegacy]
-c: error: unrecognized arguments: test
error: in phase 'check': uncaught exception:
%exception #<&invoke-error program: "python" arguments: ("-c" "import setuptools, tokenize;__file__='setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\\r\\n', '\\n');f.close();exec(compile(code, __file__, 'exec'))" "test") exit-status: 2 term-signal: #f stop-signal: #f>
phase `check' failed after 1.2 seconds
command "python" "-c" "import setuptools, tokenize;__file__='setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\\r\\n', '\\n');f.close();exec(compile(code, __file__, 'exec'))" "test" failed with status 2
builder for `/gnu/store/9kfks35xhr6abgkmpmy0la2m2nrwg6i1-python-ledgerblue-0.1.44.drv' failed with exit code 1
build of /gnu/store/9kfks35xhr6abgkmpmy0la2m2nrwg6i1-python-ledgerblue-0.1.44.drv failed

Andreas

Re: Python (was: Merging core-updates?)

[Andreas Enge]

There is poezio, which has a new release (0.14), with a license change to
gpl3+. I updated python-slixmpp, a dependency of poezio, but this is not
enough: The newest poezio still depends on python-potr, which in turn depends
on python-pycrypto.

Andreas

Re: Python (was: Merging core-updates?)

[Andreas Enge]

Hello Ricardo,

python-graphviz does not pass its tests any more in core-updates, and
I can trace it back to your commit 3d388fe3d0475f2e991ae061cc1364529a97af42.
Adding python-mock back to native-inputs fixes it.

Or maybe python-pytest-mock should have python-mock as propagated input?
It calls itself a "Thin-wrapper around the mock package for easier use
with py.test", but does not even have python-mock as any kind of input.

Thanks for your help,

Andreas

Re: Architecture support

[Andreas Enge]

Hello,

Am Fri, Feb 17, 2023 at 04:49:33PM +0000 schrieb Christopher Baines:
> I'll keep an eye on the core-updates state on data.qa.guix.gnu.org and
> submit some builds for core things manually when the derivations are
> available.

the bootstrapping of core-updates seems to be in place, so maybe you could
launch "hello" so that people can work on issues further down (concerning
particular languages and packages, for instance) without having to do the
bootstrap for themselves?

Andreas

Re: Python (was: Merging core-updates?)

[Kaelyn]

------- Original Message -------
On Sunday, February 19th, 2023 at 10:08 PM, Andreas Enge <andreas@enge.fr> wrote:


> 
> There is poezio, which has a new release (0.14), with a license change to
> gpl3+. I updated python-slixmpp, a dependency of poezio, but this is not
> enough: The newest poezio still depends on python-potr, which in turn depends
> on python-pycrypto.

It was mentioned recently that python-pycryptodome is / should be a drop-in replacement for python-pycrypto (it is also says that in the package description); perhaps replace the python-pycrypto input with python-pycryptodome for python-potr, with a snippet to change the pycrypto dependency to pycryptodome in python-potr's setup.py? After taking a peek at the poezio and python-potr git repos, the main alternative I can see to patching the dependency is to remove python-potr from poezio's inputs since python-potr is listed as an optional dependency in poezio's setup.py (for its OTR plugin).

Cheers,
Kaelyn

> 
> Andreas

Re: Architecture support

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 928 bytes --]


Andreas Enge <andreas@enge.fr> writes:

> Hello,
>
> Am Fri, Feb 17, 2023 at 04:49:33PM +0000 schrieb Christopher Baines:
>> I'll keep an eye on the core-updates state on data.qa.guix.gnu.org and
>> submit some builds for core things manually when the derivations are
>> available.
>
> the bootstrapping of core-updates seems to be in place, so maybe you could
> launch "hello" so that people can work on issues further down (concerning
> particular languages and packages, for instance) without having to do the
> bootstrap for themselves?

I submitted builds for hello for several architectures a couple of days
ago [1], it seems to have built for aarch64-linux but got stuck on
x86_64-linux. I'm guessing this could be system/filesystem related, so
I've submitted some more builds to try and find out.

1: http://data.qa.guix.gnu.org/revision/3b57f25f55c52c97428106de285d3cf2746554dc/package/hello/2.12.1?locale=en_US.UTF-8

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: Ocaml (was: Merging core-updates?)

[Simon Tournier]

Hi Julien,

On dim., 19 févr. 2023 at 10:15, Julien Lepiller <julien@lepiller.eu> wrote:
> ocaml-4.14 and ocaml-5 don't have this issue. I just pushed a fix for
> ocaml-4.09. I'll also have to fix ocaml-4.07 since it fails to build as
> early as camlboot. It'll take a while to test. Thanks for pointing me
> to that patch!

Do we still keep 4.07 and 4.09?  And all the package variants?

Cheers,
simon

Re: Merging core-updates?

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1825 bytes --]


Christopher Baines <mail@cbaines.net> writes:

> [[PGP Signed Part:Undecided]]
>
> Julien Lepiller <julien@lepiller.eu> writes:
>
>> Le Sun, 12 Feb 2023 12:52:51 +0100,
>> Julien Lepiller <julien@lepiller.eu> a écrit :
>>
>>> Le Sun, 12 Feb 2023 12:06:14 +0100,
>>> Andreas Enge <andreas@enge.fr> a écrit :
>>> 
>>> I just tried to build mpc on my machine, from core-updates. I get the
>>> same derivation as the one shown on the data service, and it built
>>> fine. Maybe there's something wrong with the machines behind bordeaux?
>>> I probably got substitutes from berlin, for gcc and friends (though I
>>> built gmp, mpfr and mpc).
>>
>> And I was able to rebuild (with --check) patch-mesboot. The error looks
>> a lot like https://issues.guix.gnu.org/49985. We should fix that indeed
>> :)
>
> It could be something "wrong" with a build machine, although my
> suspicion for a while is that various things are broken/don't work on
> btrfs, which milano-guix-1 (the most important x86_64-linux) build
> machine uses.
>
> I've filed a few issues that probably relate to this:
>
> https://issues.guix.gnu.org/53415
> https://issues.guix.gnu.org/53416
> https://issues.guix.gnu.org/60202
>
> There are other build machines that don't use btrfs, so it should be
> possible to get these things built anyway.

Following up on this, there are some failed builds for patch-mesboot [1]
for the latest revision of core-updates [2].

1: https://data.qa.guix.gnu.org/gnu/store/3nidliz9rdxwvvl46i0afvwkfx4yly74-patch-mesboot-2.5.9.drv
2: https://data.qa.guix.gnu.org/revision/3b57f25f55c52c97428106de285d3cf2746554dc

I've tried this manually on milano-guix-1 (which uses btrfs) and I get
the same failure, so this seems to be specific to the
machine/configuration.

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: Ocaml (was: Merging core-updates?)

[Julien Lepiller]

We can probably get rid of most 4.07 variants but I would keep the bootstrap (in the hope we can bootstrap later versions from it). 4.09 is still used for unison, and I think Andreas uses it a lot :)

I'd say remove all leaf 4.07 packages that are libraries only.

Le 20 février 2023 11:35:16 GMT+01:00, Simon Tournier <zimon.toutoune@gmail.com> a écrit :
>Hi Julien,
>
>On dim., 19 févr. 2023 at 10:15, Julien Lepiller <julien@lepiller.eu> wrote:
>> ocaml-4.14 and ocaml-5 don't have this issue. I just pushed a fix for
>> ocaml-4.09. I'll also have to fix ocaml-4.07 since it fails to build as
>> early as camlboot. It'll take a while to test. Thanks for pointing me
>> to that patch!
>
>Do we still keep 4.07 and 4.09?  And all the package variants?
>
>Cheers,
>simon

Re: Ocaml

[Andreas Enge]

Am Mon, Feb 20, 2023 at 12:16:17PM +0100 schrieb Julien Lepiller:
> We can probably get rid of most 4.07 variants but I would keep the bootstrap (in the hope we can bootstrap later versions from it). 4.09 is still used for unison, and I think Andreas uses it a lot :)
> I'd say remove all leaf 4.07 packages that are libraries only.

My impression is that all 4.07 packages apart from ocaml@4.07 itself are
libraries; "guix refresh -l ocaml@4.07" returns only packages of the form
ocaml4.07-xxx. So by recursively removing leaf libraries, nothing will
be left over except for the bootstrap.

Unison has a newer release 2.53, with the following NEWS:
* OCaml >= 4.08 is required to build unison.
* unison can be built with (unreleased) OCaml 5.
So this could be tackled (after the core-updates merge, I would say).

Then there is usync depending on unison. It is unmaintained with the last
commit 5 years ago, so can maybe be dropped. Well, it is a simple scsh
script, so will probably continue to work by calling the unison and rsync
binaries.

Andreas

Re: Python (was: Merging core-updates?)

[Andreas Enge]

Am Sun, Feb 19, 2023 at 10:59:35PM +0000 schrieb Kaelyn:
> It was mentioned recently that python-pycryptodome is / should be a drop-in replacement for python-pycrypto (it is also says that in the package description);

Apparently it is not, as Lars wrote. And in any case, it does require some
patching: I tried to compile python-potr with either of python-pycryptodome
and python-pycryptodomex, and it fails in the check phase, where it tries
to download pycrypto via pip.

> perhaps replace the python-pycrypto input with python-pycryptodome for python-potr, with a snippet to change the pycrypto dependency to pycryptodome in python-potr's setup.py?

Indeed this would be an alternative; but then here, I would still argue
that it is not a "drop-in replacement" for python-potr (in C, one could
imagine a separate project creating a library with the same soname).

> After taking a peek at the poezio and python-potr git repos, the main alternative I can see to patching the dependency is to remove python-potr from poezio's inputs since python-potr is listed as an optional dependency in poezio's setup.py (for its OTR plugin).

But without python-potr, the tests fail... So it may be optional, but not
for the tests.

I took the liberty to update poezio while keeping the python-potr
dependency, as it does not worsen the situation, and could be argued to
improve it.

Andreas

Re: Python

[Andreas Enge]

Am Sun, Feb 19, 2023 at 03:10:03PM +0000 schrieb Attila Lendvai:
> weirdly enough, upstream uses one git repo for multiple projects, and uses prefixed tag names for them.
> FYI, there's this long-pending patchset to update the trezor-agent (something i can test myself):
> https://issues.guix.gnu.org/58437#4
> it's been pending so long, maybe it should be updated again.

Feel free, but I would say this is independent of core-updates.

Well, except that python-trezor-agent does not build in core-updates,
but this is due to its dependencies python-ecdsa (already reported in
a different message), and, as I just saw, python-configargparse:

======================================================================
FAIL: testBasicCase2 (tests.test_configargparse.TestBasicUseCases)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/guix-build-python-configargparse-1.2.3.drv-0/ConfigArgParse-1.2.3/tests/test_configargparse.py", line 250, in testBasicCase2
    self.assertRegex(self.format_help(),
AssertionError: Regex didn't match: 'usage: .* \\[-h\\] --genome GENOME \\[-v\\] -g MY_CFG_FILE\n?\\s+\\[-d DBSNP\\]\\s+\\[-f FRMT\\]\\s+vcf \\[vcf ...\\]\n\n(.+\\s+)(.+\\s+)(.+\\s+)(.+\\s+)(.+\\s+)(.+\\s+)(.+\\s+)(.+\\s+)(.+\\s+)positional arguments:\n  vcf \\s+ Variant file\\(s\\)\n\noptional arguments:\n  -h, --help \\s+ show this help message and exit\n  --genome GENOME \\s+ Path to genome file\n  -v\n  -g MY_CFG_FILE, --my-cfg-file MY_CFG_FILE\n  -d DBSNP, --dbsnp DBSNP\\s+\\[env var: DBSNP_PATH\\]\n  -f FRMT, --format FRMT\\s+\\[env var: OUTPUT_FORMAT\\]\n' not found in "usage: setup.py [-h] --genome GENOME [-v] -g MY_CFG_FILE [-d DBSNP] [-f FRMT]\n                vcf [vcf ...]\n\nArgs that start with '--' (eg. --genome) can also be set in a config file\n(/etc/settings.ini or /home/jeff/.user_settings or /tmp/guix-build-python-\nconfigargparse-1.2.3.drv-0/tmprdqc6hnh or specified via -g). Config file\nsyntax allows: key=value, flag=true, stuff=[a,b,c] (for details, see syntax at\nhttps://goo.gl/R74nmi). If an arg is specified in more than one place, then\ncommandline values override environment variables which override config file\nvalues which override defaults.\n\npositional arguments:\n  vcf                   Variant file(s)\n\noptions:\n  -h, --help            show this help message and exit\n  --genome GENOME       Path to genome file\n  -v\n  -g MY_CFG_FILE, --my-cfg-file MY_CFG_FILE\n  -d DBSNP, --dbsnp DBSNP\n                        [env var: DBSNP_PATH]\n  -f FRMT, --format FRMT\n                        [env var: OUTPUT_FORMAT]\n"

and many more.

It would be nice if someone actually using Python could sort out
the mess...

Andreas

Re: Python

[Andreas Enge]

Am Sun, Feb 19, 2023 at 09:31:44PM +0100 schrieb Andreas Enge:
> And yet another one: python-testtool
> FAIL: testtools.tests.test_testresult.TestNonAsciiResults.test_syntax_error
> FAIL: testtools.tests.test_testresult.TestNonAsciiResultsWithUnittest.test_syntax_error

This is reported upstream here:
   https://github.com/testing-cabal/testtools/issues/333
No patch yet.

Andreas

Re: Python (was: Merging core-updates?)

[Andreas Enge]

Am Sun, Feb 19, 2023 at 12:30:42PM +0100 schrieb Andreas Enge:
> And another one: python-ecdsa

This just built. Strange, but I will not complain!

Andreas

Re: Python (was: Merging core-updates?)

[Andreas Enge]

Hello,

Am Sun, Feb 19, 2023 at 11:24:44PM +0100 schrieb Andreas Enge:
> python-graphviz does not pass its tests any more in core-updates, and
> I can trace it back to your commit 3d388fe3d0475f2e991ae061cc1364529a97af42.
> Adding python-mock back to native-inputs fixes it.

I opted for this fix and could compile python-graphviz; this enables me to
test the build of icecat now.

Andreas

Re: Merging core-updates?

[Ludovic Courtès]

Hi!

Leo Famulari <leo@famulari.name> skribis:

> On Sun, Feb 12, 2023 at 10:05:40AM +0100, Julien Lepiller wrote:
>> As discussed at Guix Days before Fosdem, we haven't merged core-updates
>> in a very long time. I'd volunteer to lead this effort, but I don't
>> know what steps I should follow. Do we have some documentation about
>> that?
>
> The process could be considered "free form".
>
> Basically, you can try to reconfigure your system on core-updates. It
> will fail, many build failures will have to be fixed, and eventually it
> will work.

Before you get to that point, there are probably more basic tests to do,
as people pointed out.

Overall, “leading the effort” IMO will be largely about (1) monitoring
the status, and (2) sending weekly (?) status updates to bug the
relevant people, with the understanding that it’ll take a few weeks
before the dust settles down.

Ludo’.

Re: Icecat (was: Python)

[Andreas Enge]

Am Tue, Feb 21, 2023 at 05:58:26PM +0100 schrieb Andreas Enge:
> I opted for this fix and could compile python-graphviz; this enables me to
> test the build of icecat now.

Good news for once: I could build icecat from core-updates on x86_64!

Andreas

Re: Python

[Andreas Enge]

In my goal of building calibre on core-updates, I encounter many build failures.
The Python package graph seems to be extremely entangled, with dependencies
on specific input package and even Python versions.

The only approach I could imagine is updating to newer versions, and this
solved some of the build failures (but not all of them). I have pushed a
few commits to core-updates. Please feel free to revert them if this is not
the good approach.

It would be nice if the Python team could take over, and more people joined
the team :)

Andreas

Re: Python

[Andreas Enge]

Another very problematic python package:
   python-sgmllib3k

It fails its tests in core-updates.

Unmaintained since 2010, the author states that they do not wish to maintain it.

Some dependencies:
Building the following 6 packages would ensure 8 dependent packages are rebuilt: emacs-calibredb@2.12.0 limnoria@2019.11.22 rss2email@3.14 quodlibet@4.5.0 python-woob@3.0 gfeeds@1.0.3

There is calibre before (behind?) emacs-calibredb. It would be good to see
if newer versions of the dependents can do without, or if otherwise they
could be dropped.

Andreas

Re: Python

[Andreas Enge]

Yet another python failure: python-pathlib

import pathlib
  File "/tmp/guix-build-python-pathlib-1.0.1.drv-0/pathlib-1.0.1/pathlib.py", line 10, in <module>
    from collections import Sequence
ImportError: cannot import name 'Sequence' from 'collections' (/gnu/store/blals34ar25fiifvm17m2b504waxzys0-python-3.10.7/lib/python3.10/collections/__init__.py)
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "python" arguments: ("-c" "import setuptools, tokenize;__file__='setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\\r\\n', '\\n');f.close();exec(compile(code, __file__, 'exec'))" "build") exit-status: 1 term-signal: #f stop-signal: #f>
phase `build' failed after 0.2 seconds
command "python" "-c" "import setuptools, tokenize;__file__='setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\\r\\n', '\\n');f.close();exec(compile(code, __file__, 'exec'))" "build" failed with status 1
note: keeping build directory `/tmp/guix-build-python-pathlib-1.0.1.drv-0'
builder for `/gnu/store/3j0y64p8cp8kcgmmpxmnhjqglvagi6g7-python-pathlib-1.0.1.drv' failed with exit code 1
build of /gnu/store/3j0y64p8cp8kcgmmpxmnhjqglvagi6g7-python-pathlib-1.0.1.drv failed
View build log at '/var/log/guix/drvs/3j/0y64p8cp8kcgmmpxmnhjqglvagi6g7-python-pathlib-1.0.1.drv.gz'.

Materially, it does not look very difficult. It is probably enough to replace
   from collections import Sequence
by
   from collections.abc import Sequence

Metaphysically, this is a package with its latest release in 2014 (!),
the vcs page has disappeared, but it has many dependencies:

Building the following 87 packages would ensure 209 dependent packages are rebuilt: instantmusic@1.0-1.300891d conda@22.9.0 python-nb-clean@2.1.0 python-pytest-perf@0.12.0 fdroidserver@1.1.9 python-regions@0.7 python-photutils@1.6.0 python-poppy@1.0.3 python-biom-format@2.1.12 python-ipython-documentation@8.2.0 python-iml@0.6.2 python-slurm-magic@0.0-0.73dd1a2 python-pytest-exploratory@0.5 python-enoslib@8.0.1 pgcli@3.2.0 autokey@0.95.10 python-pyfuse3@3.2.1 python-telethon@1.17.5 emacs-calibredb@2.12.0 python-astroquery@0.4.6 vorta@0.8.7 cura@4.13.1 komikku@1.9.0 jrnl@1.9.7 python-harmony@0.7.1 caja-extensions@1.24.1 gourmet@0.17.4-0.8af29c8 python-swiftclient@4.0.1 dbxfs@1.0.63 linuxdcpp@1.1.0 ikiwiki@3.20200202.3 breezy@3.2.2 vembrane@0.13.2 python-nanopb@0.4.6.4 python-pynixutil@0.5.0 openconnect-sso@0.8.0 qtile@0.18.1 electrum@4.3.2 pantalaimon@0.10.5 weechat-matrix@0.3.0 snakemake@6.15.5 snakemake@7.7.0 python-trio-websocket@0.9.2 python-commonroad-route-planner@2022.3 python-sunpy@4.1.1 trytond-currency-rs@6.2.0 trytond-stock-package-shipping-mygls@6.2.1 trytond-stock-package-shipping-dpd@6.2.3 python-mailman-hyperkitty@1.2.0 python-falcon-cors@1.1.7 python-ajsonrpc@1.2.0 python-ipdb@0.13.9 python-hicexplorer@3.7.2 xeus@2.4.1 python-bash-kernel@0.7.2 r-torch@0.9.0 python-cleanlab@2.2.0 python-pari-jupyter@1.4.1 orange@3.32.0 python-ipython-cluster-helper@0.6.4 python-numpy-documentation@1.23.2 rfcat@1.9.6 ruby-iruby@0.3 nanosv@1.2.4 flair@1.6.4 tombo@1.5.1 archivebox@0.6.2 r-millefy@0.1.9-beta python-jupytext@1.14.1 python-ikarus@0.0.2 python-sparqlkernel@1.3.0 python-nbdime@3.1.1 nikola@8.2.2 pigx-sars-cov-2@0.0.8 pigx@0.0.3 python-pytest-check-links@0.3.0 python-multivelo@0.1.2 python-plotly@5.6.0 python-ipympl@0.9.1 python-ipydatawidgets@4.2.0 python-matplotlib-documentation@3.5.2 scregseg@0.1.1 r-irkernel@1.3.1 abjad-ext-ipython@3.3 guix-jupyter@0.2.2 python-jupyter-sphinx@0.3.2 python-cellbender@0.2.2

Why do so many maintained projects depend on an unmaintained library?

Andreas

Re: Python

[Andreas Enge]

Am Fri, Feb 24, 2023 at 05:47:33PM +0100 schrieb Andreas Enge:
> Yet another python failure: python-pathlib

Maybe this is our fault; Debian has a pathlib2 package. Us too.
So we should try to see whether we can replace the dependency python-pathlib
by python-pathlib2.

Work for another day.

Andreas

Re: Python

[Lars-Dominik Braun]

Hi Andreas,

sorry, I can’t quite keep up with the Python issues on core-updates
right now :(

> Yet another python failure: python-pathlib
this is a backport of Python’s built-in pathlib library. It should be
dropped as a dependency for all of these packages, since our Python is >=
3.4 – the version pathlib was introduced into the standard library. And
then drop the package entirely.

Lars

Re: Python

[Andreas Enge]

Hello Lars,

Am Fri, Feb 24, 2023 at 07:08:44PM +0100 schrieb Lars-Dominik Braun:
> sorry, I can’t quite keep up with the Python issues on core-updates
> right now :(

thanks for your reply, this is very helpful, as I am more than insecure
when it comes to python packaging!

> > Yet another python failure: python-pathlib
> this is a backport of Python’s built-in pathlib library. It should be
> dropped as a dependency for all of these packages, since our Python is >=
> 3.4 – the version pathlib was introduced into the standard library. And
> then drop the package entirely.

Good, thanks for the info.

What generally makes me hesitant is that we are down to manual dependency
resolution without having a good overview:
I need python-json-spec, which so far had pathlib as an input.
When I drop pathlib, the package nevertheless fails to build due to its
own Collection.abc issue. So I tried to update it to the latest version.

This version requires python-importlib-metadata; not its latest version 6,
but something at least 5 and less than 6. We were still at 4.something.
So I have just updated it to 5.2.0, the latest version 5 from last December.
This gives me python-json-spec, so I am one step closer to calibre, which
I am interested in.

But python-importlib-metadata has 892 dependents (among which interesting
looking ones, such as freecad and gnome-terminal). I wonder whether I am not
breaking 891 dependents by enabling, maybe, the one that I am interested in.

Anyway, I am operating from the assumption that updating to a newer version
is always good, and that packages that do not build anymore will have to
be updated or patched themselves. Hopefully this is a reasonable assumption
in the python world... If not, please feel free to revert my changes and to
propose a different solution.

Andreas

Re: Python

[Lars-Dominik Braun]

Hi Andreas,

> This version requires python-importlib-metadata; not its latest version 6,
> but something at least 5 and less than 6. We were still at 4.something.
> So I have just updated it to 5.2.0, the latest version 5 from last December.
> This gives me python-json-spec, so I am one step closer to calibre, which
> I am interested in.
note that importlib-metadata is – again – part of the standard
library, as the table on [1] points out. So if we would ship Python 3.12,
we would not need it. Bumping it to version 5.2 seems like the correct
approach right now, since we’re at 3.10.7 on core-updates (as far as I see).

> But python-importlib-metadata has 892 dependents (among which interesting
> looking ones, such as freecad and gnome-terminal). I wonder whether I am not
> breaking 891 dependents by enabling, maybe, the one that I am interested in.
Besides the 'sanity-check phase, which checks for the most common issues,
the only thing we can do is rebuild and run tests. Since Python is a
dynamic language that still does not guarantee that it will run, but
it’s all we can realisically do -.-

Cheers,
Lars

[1] https://pypi.org/project/importlib-metadata/

Re: Python

[Andreas Enge]

Am Sat, Feb 25, 2023 at 04:45:39PM +0100 schrieb Lars-Dominik Braun:
> note that importlib-metadata is – again – part of the standard
> library, as the table on [1] points out. So if we would ship Python 3.12,
> we would not need it. Bumping it to version 5.2 seems like the correct
> approach right now, since we’re at 3.10.7 on core-updates (as far as I see).

Very interesting! I am just not used to these very dynamically changing
languages. After a bit of discussion, we recently decided with my coauthor
to assume the C99 standard in our library instead of C89 :)

Right now I am left with a number of test failures that look real and cannot
easily be solved by an upgrade (either because we are already on the latest
version or because the tests still fail): python-sgmllib3k, python-typeguard
and python-coveralls. See messages below.

Andreas


python-coveralls:
=================================== FAILURES ===================================
___________________ ReporterTest.test_reporter_with_branches ___________________

self = <tests.api.reporter_test.ReporterTest testMethod=test_reporter_with_branches>

    def test_reporter_with_branches(self):
        subprocess.call(['coverage', 'run', '--branch', '--omit=**/.tox/*',
                         'runtests.py'], cwd=EXAMPLE_DIR)
        results = Coveralls(repo_token='xxx').get_coverage()
        assert len(results) == 2

        # Branches are expressed as four values each in a flat list
        assert not len(results[0]['branches']) % 4
        assert not len(results[1]['branches']) % 4

>       assert_coverage(results[0], {
            'source': ('def hello():\n'
                       '    print(\'world\')\n\n\n'
                       'class Foo:\n'
                       '    """ Bar """\n\n\n'
                       'def baz():\n'
                       '    print(\'this is not tested\')\n\n'
                       'def branch(cond1, cond2):\n'
                       '    if cond1:\n'
                       '        print(\'condition tested both ways\')\n'
                       '    if cond2:\n'
                       '        print(\'condition not tested both ways\')\n'),
            'name': 'project.py',
            'branches': [13, 0, 14, 1, 13, 0, 15, 1, 15, 0, 16, 1, 15, 0, 12,
                         0],
            'coverage': [1, 1, None, None, 1, None, None,
                         None, 1, 0, None, 1, 1, 1, 1, 1]})

/tmp/guix-build-python-coveralls-3.2.0.drv-0/source/tests/api/reporter_test.py:182:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

actual = {'branches': [5, 0, 6, 1, 5, 0, ...], 'coverage': [1, 1, None, None, 1, None, ...], 'name': 'project.py', 'source': 'd...1:\n        print(\'condition tested both ways\')\n    if cond2:\n        print(\'condition not tested both ways\')\n'}
expected = {'branches': [13, 0, 14, 1, 13, 0, ...], 'coverage': [1, 1, None, None, 1, None, ...], 'name': 'project.py', 'source':...1:\n        print(\'condition tested both ways\')\n    if cond2:\n        print(\'condition not tested both ways\')\n'}

    def assert_coverage(actual, expected):
        assert actual['source'].strip() == expected['source'].strip()
        assert actual['name'] == expected['name']
        assert actual['coverage'] == expected['coverage']
>       assert actual.get('branches') == expected.get('branches')
E       assert [5, 0, 6, 1, 5, 0, 9, 1, 13, 0, 14, 1, 13, 0, 15, 1, 15, 0, 16, 1, 15, 0, 12, 0] == [13, 0, 14, 1, 13, 0, 15, 1, 15, 0, 16, 1, 15, 0, 12, 0]
E         At index 0 diff: 5 != 13
E         Left contains 8 more items, first extra item: 15
E         Full diff:
E         - [13, 0, 14, 1, 13, 0, 15, 1, 15, 0, 16, 1, 15, 0, 12, 0]
E         + [5, 0, 6, 1, 5, 0, 9, 1, 13, 0, 14, 1, 13, 0, 15, 1, 15, 0, 16, 1, 15, 0, 12, 0]
E         ?  ++++++++++++++++++++++++

/tmp/guix-build-python-coveralls-3.2.0.drv-0/source/tests/api/reporter_test.py:17: AssertionError
----------------------------- Captured stdout call -----------------------------
world
condition not tested both ways
condition tested both ways
condition not tested both ways
=========================== short test summary info ============================
FAILED tests/api/reporter_test.py::ReporterTest::test_reporter_with_branches
=================== 1 failed, 51 passed, 1 skipped in 2.51s ====================
error: in phase 'check': uncaught exception:
%exception #<&invoke-error program: "pytest" arguments: ("-vv") exit-status: 1 term-signal: #f stop-signal: #f>
phase `check' failed after 2.8 seconds
command "pytest" "-vv" failed with status 1
builder for `/gnu/store/2f7hab84ljj1k284w61zn14cbapzfdnf-python-coveralls-3.2.0.drv' failed with exit code 1
build of /gnu/store/2f7hab84ljj1k284w61zn14cbapzfdnf-python-coveralls-3.2.0.drv failed
View build log at '/var/log/guix/drvs/2f/7hab84ljj1k284w61zn14cbapzfdnf-python-coveralls-3.2.0.drv.gz'.


python-sgmllib3k:
======================================================================
FAIL: test_declaration_junk_chars (test_sgmllib.SGMLParserTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/guix-build-python-sgmllib3k-1.0.0-1.7999646.drv-0/source/test_sgmllib.py", line 310, in test_declaration_junk_chars
    self.check_parse_error("<!DOCTYPE foo $ >")
  File "/tmp/guix-build-python-sgmllib3k-1.0.0-1.7999646.drv-0/source/test_sgmllib.py", line 127, in check_parse_error
    parser.feed(source)
  File "/tmp/guix-build-python-sgmllib3k-1.0.0-1.7999646.drv-0/source/sgmllib.py", line 98, in feed
    self.goahead(0)
  File "/tmp/guix-build-python-sgmllib3k-1.0.0-1.7999646.drv-0/source/sgmllib.py", line 168, in goahead
    k = self.parse_declaration(i)
  File "/gnu/store/blals34ar25fiifvm17m2b504waxzys0-python-3.10.7/lib/python3.10/_markupbase.py", line 134, in parse_declaration
    raise AssertionError("unexpected %r char in declaration" % rawdata[j])
AssertionError: unexpected '$' char in declaration

----------------------------------------------------------------------
Ran 23 tests in 0.008s

FAILED (failures=1)
Test failed: <unittest.runner.TextTestResult run=23 errors=0 failures=1>
error: Test failed: <unittest.runner.TextTestResult run=23 errors=0 failures=1>
error: in phase 'check': uncaught exception:
%exception #<&invoke-error program: "python" arguments: ("-c" "import setuptools, tokenize;__file__='setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\\r\\n', '\\n');f.close();exec(compile(code, __file__, 'exec'))" "test") exit-status: 1 term-signal: #f stop-signal: #f> 
phase `check' failed after 0.3 seconds
command "python" "-c" "import setuptools, tokenize;__file__='setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\\r\\n', '\\n');f.close();exec(compile(code, __file__, 'exec'))" "test" failed with status 1
builder for `/gnu/store/cp8sgcp3c1599dzjwmfms38jz3iddyk1-python-sgmllib3k-1.0.0-1.7999646.drv' failed with exit code 1
build of /gnu/store/cp8sgcp3c1599dzjwmfms38jz3iddyk1-python-sgmllib3k-1.0.0-1.7999646.drv failed
View build log at '/var/log/guix/drvs/cp/8sgcp3c1599dzjwmfms38jz3iddyk1-python-sgmllib3k-1.0.0-1.7999646.drv.gz'.


python-typeguard:
=================================== FAILURES ===================================
___________________ TestTypeChecked.test_typed_dict[correct] ___________________
tests/test_typeguard.py:1343: in test_typed_dict
    foo(value)
/gnu/store/qmli9ql0wplg1hyacy07kvgaqbpfa11z-python-typeguard-2.13.3/lib/python3.10/site-packages/typeguard/__init__.py:1032: in wrapper
    check_argument_types(memo)
/gnu/store/qmli9ql0wplg1hyacy07kvgaqbpfa11z-python-typeguard-2.13.3/lib/python3.10/site-packages/typeguard/__init__.py:875: in check_argument_types
    raise TypeError(*exc.args) from None
E   TypeError: TypedDict does not support instance and class checks
__________________ TestTypeChecked.test_typed_dict[missing_x] __________________
tests/test_typeguard.py:1341: in test_typed_dict
    pytest.raises(TypeError, foo, value).match(error_re)
E   AssertionError: Regex pattern 'required key\\(s\\) \\("x"\\) missing from argument "arg"' does not match 'TypedDict does not support instance and class checks'.
___________________ TestTypeChecked.test_typed_dict[wrong_y] ___________________
tests/test_typeguard.py:1341: in test_typed_dict
    pytest.raises(TypeError, foo, value).match(error_re)
E   AssertionError: Regex pattern 'type of dict item "y" for argument "arg" must be str; got int instead' does not match 'TypedDict does not support instance and class checks'.
_______________ TestTypeChecked.test_typed_dict[missing_y_error] _______________
tests/test_typeguard.py:1341: in test_typed_dict
    pytest.raises(TypeError, foo, value).match(error_re)
E   AssertionError: Regex pattern 'required key\\(s\\) \\("y"\\) missing from argument "arg"' does not match 'TypedDict does not support instance and class checks'.
________________ TestTypeChecked.test_typed_dict[missing_y_ok] _________________
tests/test_typeguard.py:1343: in test_typed_dict
    foo(value)
/gnu/store/qmli9ql0wplg1hyacy07kvgaqbpfa11z-python-typeguard-2.13.3/lib/python3.10/site-packages/typeguard/__init__.py:1032: in wrapper
    check_argument_types(memo)
/gnu/store/qmli9ql0wplg1hyacy07kvgaqbpfa11z-python-typeguard-2.13.3/lib/python3.10/site-packages/typeguard/__init__.py:875: in check_argument_types
    raise TypeError(*exc.args) from None
E   TypeError: TypedDict does not support instance and class checks
___________________ TestTypeChecked.test_typed_dict[wrong_x] ___________________
tests/test_typeguard.py:1341: in test_typed_dict
    pytest.raises(TypeError, foo, value).match(error_re)
E   AssertionError: Regex pattern 'type of dict item "x" for argument "arg" must be int; got str instead' does not match 'TypedDict does not support instance and class checks'.
_________________ TestTypeChecked.test_typed_dict[unknown_key] _________________
tests/test_typeguard.py:1341: in test_typed_dict
    pytest.raises(TypeError, foo, value).match(error_re)
E   AssertionError: Regex pattern 'extra key\\(s\\) \\("foo"\\) in argument "arg"' does not match 'TypedDict does not support instance and class checks'.
=============================== warnings summary ===============================
tests/test_typeguard.py::TestTypeChecker::test_generator
tests/test_typeguard.py::TestTypeChecker::test_exception
tests/test_typeguard_py36.py::TestTypeChecker::test_async_generator[generator]
tests/test_typeguard_py36.py::TestTypeChecker::test_async_generator[iterable]
tests/test_typeguard_py36.py::TestTypeChecker::test_async_generator[iterator]
  /gnu/store/fgbf94ab19n90rsz6h7vrvqyw17w9kqa-python-pytest-7.1.3/lib/python3.10/site-packages/_pytest/python.py:192: PytestRemovedIn8Warning: Passing None has been deprecated.
  See https://docs.pytest.org/en/latest/how-to/capture-warnings.html#additional-use-cases-of-warnings-in-tests for alternatives in common use cases.
    result = testfunction(**testargs)

tests/test_typeguard_py36.py::TestTypeChecker::test_callable
  /gnu/store/qmli9ql0wplg1hyacy07kvgaqbpfa11z-python-typeguard-2.13.3/lib/python3.10/site-packages/typeguard/__init__.py:1027: UserWarning: no code associated -- not typechecking test_typeguard_py36.<test_typeguard_py36.TestTypeChecker.test_callable.<locals>.command object at 0x7ffff5d4a800>
    warn('no code associated -- not typechecking {}'.format(function_name(func)))

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========== 7 failed, 233 passed, 3 deselected, 6 warnings in 2.13s ============
error: in phase 'check': uncaught exception:
%exception #<&invoke-error program: "pytest" arguments: ("-vv" "-k" "not usefixtures and not test_cached_module") exit-status: 1 term-signal: #f stop-signal: #f> 
phase `check' failed after 2.5 seconds
command "pytest" "-vv" "-k" "not usefixtures and not test_cached_module" failed with status 1
builder for `/gnu/store/l9gb4nadq9c43zn82wqbxmwgib7bkqqs-python-typeguard-2.13.3.drv' failed with exit code 1
build of /gnu/store/l9gb4nadq9c43zn82wqbxmwgib7bkqqs-python-typeguard-2.13.3.drv failed
View build log at '/var/log/guix/drvs/l9/gb4nadq9c43zn82wqbxmwgib7bkqqs-python-typeguard-2.13.3.drv.gz'.

Re: Python

[Lars-Dominik Braun]

Hi,

> Right now I am left with a number of test failures that look real and cannot
> easily be solved by an upgrade (either because we are already on the latest
> version or because the tests still fail): python-sgmllib3k, python-typeguard
> and python-coveralls. See messages below.
I don’t know for sure why any of these packages’ tests fail. typeguard
looks like it expects specific strings from Python or one of its libraries
– safe to ignore. sgmllib3k looks pretty dead upstream. Perhaps it’s
not even needed any more? Updates to Python packages (via `guix refresh`)
do not update dependencies and thus the list of inputs/native-inputs
are most likely outdated.

Well, there are reasons no-one is updating the Python ecosystem
regularly…

Cheers,
Lars

Re: Python

[Andreas Enge]

Am Sun, Feb 19, 2023 at 09:27:23PM +0100 schrieb Andreas Enge:
> Am Sun, Feb 19, 2023 at 04:50:37PM +0100 schrieb Lars-Dominik Braun:
> > eolie upstream looks dead, same with jrnl.

Removed.

> I found python-miio, which depends on python-android-backup, which depends
> on python-pycrypto.
> The first one looks alive with a new release last July, the middle one
> looks dead.
> Looking at
>    https://github.com/rytilahti/python-miio/blob/master/pyproject.toml
> android-backup appears to be an optional dependency.

python-miio updated, python-android-backup removed.

I also removed python-potr and python-pycrypto;
which naturally led me to removing python-potr from the inputs of poezio.

OTR is marked as a plugin of poezio, but apparently needed for its tests
to pass. I disabled the tests for the time being. The egg information also
mentions potr as a requirement. I could not test the package, since upon
startup it complains about unicode and locales.

If someone else could have a look, that would be good. If potr is indeed
a requirement, we should remove poezio also: We should not ship
unmaintained cryptography software.

Andreas

Re: Python

[Andreas Enge]

Am Sat, Feb 25, 2023 at 05:56:59PM +0100 schrieb Lars-Dominik Braun:
> Well, there are reasons no-one is updating the Python ecosystem
> regularly…

;-)

> I don’t know for sure why any of these packages’ tests fail. typeguard
> looks like it expects specific strings from Python or one of its libraries
> – safe to ignore.

So you suggest to add a #:test? #f?

> sgmllib3k looks pretty dead upstream. Perhaps it’s
> not even needed any more? Updates to Python packages (via `guix refresh`)
> do not update dependencies and thus the list of inputs/native-inputs
> are most likely outdated.

Indeed, dead upstream. It has two dependents:
quodlibet in the music module. There it is given as an input, but I do not
see it in the pyproject.toml file, so it may indeed not be needed.

It is also a propagated input of python-feedparser, which is an input to
quodlibet and a (small) number of other packages:
Building the following 6 packages would ensure 8 dependent packages are rebuilt: emacs-calibredb@2.12.0 limnoria@2019.11.22 rss2email@3.14 quodlibet@4.5.0 python-woob@3.0 gfeeds@1.0.3
We are at version 6.0.8 (from June 2021), but I also tried the latest
release 6.0.10 (from May 2022). Even in its git repository, the
pyproject.toml has this:
[tool.poetry.dependencies]
python = ">=3.7"
sgmllib3k = "^1.0.0"

And indeed, it does not build without.

One option would be to disable the tests of sgmllib3k; then it would be
nice if we could build any of the binaries depending on it to have a real
test case. But they all seem to have lots of other dependencies (for
instance python-cheetah for quodlibet, which also fails).

Andreas

Re: Python

[Andreas Enge]

Python-cheetah is misnamed, it should be called python-cheetah3; the first
one is the python@2 version, both exist on pypi. Is it okay to rename it?
It looks like moderate work:

$ guix refresh -l python-cheetah
Die folgenden 6 Pakete zu erstellen, würde zur Folge haben, dass 9 abhängige Pakete neu erstellt werden: quodlibet@4.5.0 gr-dsd@1.0.0-0.f9b9936 gr-satellites@4.6.0 gqrx@2.15.9 urh@2.9.4 gnss-sdr@0.0.17

Andreas

Re: Python

[Andreas Enge]

Am Sat, Feb 25, 2023 at 07:06:31PM +0100 schrieb Andreas Enge:
> Python-cheetah is misnamed, it should be called python-cheetah3; the first
> one is the python@2 version, both exist on pypi. Is it okay to rename it?
> It looks like moderate work:
> 
> $ guix refresh -l python-cheetah
> Die folgenden 6 Pakete zu erstellen, würde zur Folge haben, dass 9 abhängige Pakete neu erstellt werden: quodlibet@4.5.0 gr-dsd@1.0.0-0.f9b9936 gr-satellites@4.6.0 gqrx@2.15.9 urh@2.9.4 gnss-sdr@0.0.17

Actually, upstream has changed its pypi name!
   https://pypi.org/project/CT3/

Andreas

Re: Python

[Andreas Enge]

Am Sat, Feb 25, 2023 at 07:00:30PM +0100 schrieb Andreas Enge:
> It is also a propagated input of python-feedparser

And this fails its tests after disabling the tests of python-sgmllib3k,
even after updating to its latest version 6.0.10, like below.

Then why I was at it, I also disabled the tests of python-feedparser.

But then quodlibet fails its tests...

Andreas


======================================================================
FAIL: test_000629 (__main__.TestStrictParser)
./tests/wellformed/sanitize/xml_declaration_unexpected_character.xml: xml declaration unexpected character
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/guix-build-python-feedparser-6.0.10.drv-0/feedparser-6.0.10/tests/runtests.py", line 912, in fn
    self.fail_unless_eval(xmlfile, eval_string)
  File "/tmp/guix-build-python-feedparser-6.0.10.drv-0/feedparser-6.0.10/tests/runtests.py", line 173, in fail_unless_eval
    env = feedparser.parse(xmlfile)
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/api.py", line 263, in parse
    saxparser.parse(source)
  File "/gnu/store/blals34ar25fiifvm17m2b504waxzys0-python-3.10.7/lib/python3.10/xml/sax/expatreader.py", line 111, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "/gnu/store/blals34ar25fiifvm17m2b504waxzys0-python-3.10.7/lib/python3.10/xml/sax/xmlreader.py", line 125, in parse
    self.feed(buffer)
  File "/gnu/store/blals34ar25fiifvm17m2b504waxzys0-python-3.10.7/lib/python3.10/xml/sax/expatreader.py", line 217, in feed
    self._parser.Parse(data, isFinal)
  File "./Modules/pyexpat.c", line 468, in EndElement
  File "/gnu/store/blals34ar25fiifvm17m2b504waxzys0-python-3.10.7/lib/python3.10/xml/sax/expatreader.py", line 381, in end_element_ns
    self._cont_handler.endElementNS(pair, None)
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/parsers/strict.py", line 124, in endElementNS
    self.unknown_endtag(localname)
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/mixin.py", line 321, in unknown_endtag
    method()
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/namespaces/_base.py", line 385, in _end_title
    value = self.pop_content('title')
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/mixin.py", line 629, in pop_content
    value = self.pop(tag)
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/mixin.py", line 543, in pop
    output = resolve_relative_uris(output, self.baseuri, self.encoding, self.contentparams.get('type', 'text/html'))
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/urls.py", line 154, in resolve_relative_uris
    p.feed(html_source)
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/html.py", line 156, in feed
    super(_BaseHTMLProcessor, self).feed(data)
  File "/gnu/store/map94l87lb8fxb6q0szwg02sq0r3s74w-python-sgmllib3k-1.0.0-1.7999646/lib/python3.10/site-packages/sgmllib.py", line 98, in feed
    self.goahead(0)
  File "/gnu/store/map94l87lb8fxb6q0szwg02sq0r3s74w-python-sgmllib3k-1.0.0-1.7999646/lib/python3.10/site-packages/sgmllib.py", line 168, in goahead
    k = self.parse_declaration(i)
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/html.py", line 351, in parse_declaration
    return sgmllib.SGMLParser.parse_declaration(self, i)
  File "/gnu/store/blals34ar25fiifvm17m2b504waxzys0-python-3.10.7/lib/python3.10/_markupbase.py", line 134, in parse_declaration
    raise AssertionError("unexpected %r char in declaration" % rawdata[j])
AssertionError: unexpected '~' char in declaration

======================================================================
FAIL: test_000629 (__main__.TestLooseParser)
./tests/wellformed/sanitize/xml_declaration_unexpected_character.xml: xml declaration unexpected character
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/gnu/store/map94l87lb8fxb6q0szwg02sq0r3s74w-python-sgmllib3k-1.0.0-1.7999646/lib/python3.10/site-packages/sgmllib.py", line 352, in finish_endtag
    method = getattr(self, 'end_' + tag)
AttributeError: 'LooseFeedParser' object has no attribute 'end_title'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/guix-build-python-feedparser-6.0.10.drv-0/feedparser-6.0.10/tests/runtests.py", line 912, in fn
    self.fail_unless_eval(xmlfile, eval_string)
  File "/tmp/guix-build-python-feedparser-6.0.10.drv-0/feedparser-6.0.10/tests/runtests.py", line 173, in fail_unless_eval
    env = feedparser.parse(xmlfile)
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/api.py", line 272, in parse
    feedparser.feed(data.decode('utf-8', 'replace'))
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/html.py", line 156, in feed
    super(_BaseHTMLProcessor, self).feed(data)
  File "/gnu/store/map94l87lb8fxb6q0szwg02sq0r3s74w-python-sgmllib3k-1.0.0-1.7999646/lib/python3.10/site-packages/sgmllib.py", line 98, in feed
    self.goahead(0)
  File "/gnu/store/map94l87lb8fxb6q0szwg02sq0r3s74w-python-sgmllib3k-1.0.0-1.7999646/lib/python3.10/site-packages/sgmllib.py", line 137, in goahead
    k = self.parse_endtag(i)
  File "/gnu/store/map94l87lb8fxb6q0szwg02sq0r3s74w-python-sgmllib3k-1.0.0-1.7999646/lib/python3.10/site-packages/sgmllib.py", line 314, in parse_endtag
    self.finish_endtag(tag)
  File "/gnu/store/map94l87lb8fxb6q0szwg02sq0r3s74w-python-sgmllib3k-1.0.0-1.7999646/lib/python3.10/site-packages/sgmllib.py", line 354, in finish_endtag
    self.unknown_endtag(tag)
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/mixin.py", line 321, in unknown_endtag
    method()
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/namespaces/_base.py", line 385, in _end_title
    value = self.pop_content('title')
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/mixin.py", line 629, in pop_content
    value = self.pop(tag)
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/mixin.py", line 543, in pop
    output = resolve_relative_uris(output, self.baseuri, self.encoding, self.contentparams.get('type', 'text/html'))
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/urls.py", line 154, in resolve_relative_uris
    p.feed(html_source)
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/html.py", line 156, in feed
    super(_BaseHTMLProcessor, self).feed(data)
  File "/gnu/store/map94l87lb8fxb6q0szwg02sq0r3s74w-python-sgmllib3k-1.0.0-1.7999646/lib/python3.10/site-packages/sgmllib.py", line 98, in feed
    self.goahead(0)
  File "/gnu/store/map94l87lb8fxb6q0szwg02sq0r3s74w-python-sgmllib3k-1.0.0-1.7999646/lib/python3.10/site-packages/sgmllib.py", line 168, in goahead
    k = self.parse_declaration(i)
  File "/gnu/store/ipizx6yzh9ndb58xp0fhfmdgikw160d9-python-feedparser-6.0.10/lib/python3.10/site-packages/feedparser/html.py", line 351, in parse_declaration
    return sgmllib.SGMLParser.parse_declaration(self, i)
  File "/gnu/store/blals34ar25fiifvm17m2b504waxzys0-python-3.10.7/lib/python3.10/_markupbase.py", line 134, in parse_declaration
    raise AssertionError("unexpected %r char in declaration" % rawdata[j])
AssertionError: unexpected '~' char in declaration

----------------------------------------------------------------------
Ran 4335 tests in 3.675s

FAILED (failures=2)
error: in phase 'check': uncaught exception:
%exception #<&invoke-error program: "python" arguments: ("tests/runtests.py") exit-status: 1 term-signal: #f stop-signal: #f>
phase `check' failed after 4.0 seconds
command "python" "tests/runtests.py" failed with status 1
builder for `/gnu/store/xdmf42yd69nhbrwvjs5fhqz2m2774xc7-python-feedparser-6.0.10.drv' failed with exit code 1
build of /gnu/store/xdmf42yd69nhbrwvjs5fhqz2m2774xc7-python-feedparser-6.0.10.drv failed
View build log at '/var/log/guix/drvs/xd/mf42yd69nhbrwvjs5fhqz2m2774xc7-python-feedparser-6.0.10.drv.gz'.

Re: Python

[Andreas Enge]

Am Sat, Feb 25, 2023 at 07:15:24PM +0100 schrieb Andreas Enge:
> Actually, upstream has changed its pypi name!
>    https://pypi.org/project/CT3/

I updated it to its latest version under its current name python-cheetah,
but would suggest to rename it to python-ct3. What do you think?

Andreas

Re: Python

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 848 bytes --]

On Fri, Feb 24, 2023 at 07:08:44PM +0100, Lars-Dominik Braun wrote:
> Hi Andreas,
> 
> sorry, I can’t quite keep up with the Python issues on core-updates
> right now :(
> 
> > Yet another python failure: python-pathlib
> this is a backport of Python’s built-in pathlib library. It should be
> dropped as a dependency for all of these packages, since our Python is >=
> 3.4 – the version pathlib was introduced into the standard library. And
> then drop the package entirely.

Do we have a list of packages in the python importer that can be removed
from inputs? Like already exists for hackage (and maybe others)?

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Python

[Lars-Dominik Braun]

Hi,

> Do we have a list of packages in the python importer that can be removed
> from inputs? Like already exists for hackage (and maybe others)?
I’m not aware of any list like that and to compile it we’d probably
have to build all python-* packages and check whether any of their
installed modules collides with anything the python package itself
provides (where collision does not necessarily mean file collision,
but package namespace collision in the Python world).

Cheers,
Lars

Re: Python

[Lars-Dominik Braun]

Hi,

> I updated it to its latest version under its current name python-cheetah,
> but would suggest to rename it to python-ct3. What do you think?

I don’t think we should follow PyPi’s names strictly. python-cheetah3
makes much more sense than python-ct3. That’s what upstream-name is for.

Lars

Re: Python

[Andreas Enge]

Am Mon, Feb 27, 2023 at 08:14:54PM +0100 schrieb Lars-Dominik Braun:
> I don’t think we should follow PyPi’s names strictly. python-cheetah3
> makes much more sense than python-ct3. That’s what upstream-name is for.

Okay, I am fine with doing nothing :)

Andreas

Re: Merging core-updates?

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1176 bytes --]


Christopher Baines <mail@cbaines.net> writes:

> Julien Lepiller <julien@lepiller.eu> writes:
>
>> As discussed at Guix Days before Fosdem, we haven't merged core-updates
>> in a very long time. I'd volunteer to lead this effort, but I don't
>> know what steps I should follow. Do we have some documentation about
>> that?
>
> I can try and help with this, at least in terms of helping get
> bordeaux.guix.gnu.org substitute availability to a good level.
>
> I'd also like to get the kind of comparison that the qa-frontpage can do
> for patches working for branches, but that'll probably take a bit of
> work in the data service/qa-frontpage to get working smoothly.

I've now addressed some performance issues with data.qa.guix.gnu.org,
which means that the comparisons for core-updates (and branches like it)
should now work, even if they're still a bit slow.

This has enabled qa.guix.gnu.org to submit builds to the bordeaux build
farm, they're still being slowly submitted but things have already
started to happen for aarch64-linux at least.

Once there's enough build data to work with, I'll see if I can start
getting some information to show up on qa.guix.gnu.org.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: Merging core-updates? OFF TOPIC PRAISE

[Joshua Branson]

Christopher Baines <mail@cbaines.net> writes:

> Christopher Baines <mail@cbaines.net> writes:
>
>> Julien Lepiller <julien@lepiller.eu> writes:
>>
>>> As discussed at Guix Days before Fosdem, we haven't merged core-updates
>>> in a very long time. I'd volunteer to lead this effort, but I don't
>>> know what steps I should follow. Do we have some documentation about
>>> that?
>>
>> I can try and help with this, at least in terms of helping get
>> bordeaux.guix.gnu.org substitute availability to a good level.
>>
>> I'd also like to get the kind of comparison that the qa-frontpage can do
>> for patches working for branches, but that'll probably take a bit of
>> work in the data service/qa-frontpage to get working smoothly.
>
> I've now addressed some performance issues with data.qa.guix.gnu.org,
> which means that the comparisons for core-updates (and branches like it)
> should now work, even if they're still a bit slow.
>
> This has enabled qa.guix.gnu.org to submit builds to the bordeaux build
> farm, they're still being slowly submitted but things have already
> started to happen for aarch64-linux at least.
>
> Once there's enough build data to work with, I'll see if I can start
> getting some information to show up on qa.guix.gnu.org.

Christopher thanks a million for your work on bordeaux and
qa.guix.gnu.org!  It's awesome that guix is moving towards having a QA
assurance that new patches won't break testsuitse!

Thanks again!

Joshua

Re: Python

[Andreas Enge]

Am Sat, Feb 25, 2023 at 07:00:30PM +0100 schrieb Andreas Enge:
> > sgmllib3k looks pretty dead upstream. Perhaps it’s
> > not even needed any more? Updates to Python packages (via `guix refresh`)
> > do not update dependencies and thus the list of inputs/native-inputs
> > are most likely outdated.
> It is also a propagated input of python-feedparser

There is a bug report for feedparser:
   https://github.com/kurtmckee/feedparser/issues/328
Unfortunately it was immediately closed with a link to an alternative
project by the feedparser author, but which has seen its latest release
in January 2022. This is not very encouraging...

Andreas

Re: Python

[Andreas Enge]

Am Sat, Mar 11, 2023 at 12:20:25PM +0100 schrieb Andreas Enge:
> There is a bug report for feedparser:
>    https://github.com/kurtmckee/feedparser/issues/328
> Unfortunately it was immediately closed with a link to an alternative
> project by the feedparser author, but which has seen its latest release
> in January 2022. This is not very encouraging...

I have added a comment to the issue (hoping that it will be seen although
the issue is closed) and have taken the liberty to upgrade the package
to its latest version. According to the NEWS file very little changed.
I had some hope that we might just patch out the use of sgmllib3k, but
it looks rather central and is apparently used to parse not only broken
XML, but any XML/HTML:
./feedparser/html.py:64:class _BaseHTMLProcessor(sgmllib.SGMLParser, object):

Andreas

Re: Python

[Andreas Enge]

Hello,

Am Sat, Feb 25, 2023 at 05:56:59PM +0100 schrieb Lars-Dominik Braun:
> > Right now I am left with a number of test failures that look real and cannot
> > easily be solved by an upgrade (either because we are already on the latest
> > version or because the tests still fail): python-sgmllib3k, python-typeguard
> > and python-coveralls. See messages below.
> I don’t know for sure why any of these packages’ tests fail. typeguard
> looks like it expects specific strings from Python or one of its libraries
> – safe to ignore.

would you mind having a look how we should proceed? Fix tests or disable
specific ones that you deem safe to ignore?

> sgmllib3k looks pretty dead upstream. Perhaps it’s
> not even needed any more? Updates to Python packages (via `guix refresh`)
> do not update dependencies and thus the list of inputs/native-inputs
> are most likely outdated.

This one is used for python-feedparser, used for calibre and quodlibet.
The feedparser author is not enclined to work on it:
   https://github.com/kurtmckee/feedparser/issues/328
I would suggest to try compiling python-sgmllib3k (and potentially
python-feedparser) without the tests, and see whether calibre still works.
But for this, we first need to get all other inputs of calibre into working
shape.

Andreas

Re: Python

[Lars-Dominik Braun]

Hello Andreas,

> This one is used for python-feedparser, used for calibre and quodlibet.
> The feedparser author is not enclined to work on it:
>    https://github.com/kurtmckee/feedparser/issues/328
> I would suggest to try compiling python-sgmllib3k (and potentially
> python-feedparser) without the tests, and see whether calibre still works.
> But for this, we first need to get all other inputs of calibre into working
> shape.
I fixed both python-sgmllib3k and python-feedparser. Disabling tests
would not do much in this case, since the packages really *were* broken
by a Python upstream change.

f9bff8614be32f9c2c1a54da80eaed1365b49be3 gnu: python-feedparser: Add Python >=3.9 compatibility.
cfccd6fe5ae00d7e81cd755be55d51ff3bf17186 gnu: python-sgmllib3k: Add Python >=3.9 compatibility.

Cheers,
Lars

Re: Python

[Andreas Enge]

Hello Lars,

Am Sat, Mar 18, 2023 at 09:59:37AM +0100 schrieb Lars-Dominik Braun:
> I fixed both python-sgmllib3k and python-feedparser. Disabling tests
> would not do much in this case, since the packages really *were* broken
> by a Python upstream change.

excellent! It contradicts the feedparser author's statement that everything
works well, so if you have the courage, maybe you could report the issue
upstream.

Thanks, more things that work!

Andreas

Re: Python

[Lars-Dominik Braun]

Hi again,

> > > Right now I am left with a number of test failures that look real and cannot
> > > easily be solved by an upgrade (either because we are already on the latest
> > > version or because the tests still fail): python-sgmllib3k, python-typeguard
> > > and python-coveralls. See messages below.
> > I don’t know for sure why any of these packages’ tests fail. typeguard
> > looks like it expects specific strings from Python or one of its libraries
> > – safe to ignore.
> 
> would you mind having a look how we should proceed? Fix tests or disable
> specific ones that you deem safe to ignore?

python-typeguard and python-coveralls are functional again:

d9a31cd34eb9f63460d74d3cfff0190f964f48f4 gnu: python-coveralls: Disable failing test and relax version requirements.
a7c96167ae8748a8b76fabaf74a997094cdf7fc5 gnu: python-typeguard: Python 3.10 compatibility.

Anything else that needs work?

Cheers,
Lars

Re: Python

[Lars-Dominik Braun]

Hello Andreas,

> excellent! It contradicts the feedparser author's statement that everything
> works well, so if you have the courage, maybe you could report the issue
> upstream.

I just saw it was fixed on the develop branch already, but there’s no
new release:
https://github.com/kurtmckee/feedparser/commit/c55bd8ad37db89bd219783bc514d600c9523ed38

Cheers,
Lars

Re: Python

[Andreas Enge]

Am Sat, Mar 18, 2023 at 11:02:57AM +0100 schrieb Lars-Dominik Braun:
> I just saw it was fixed on the develop branch already, but there’s no
> new release:
> https://github.com/kurtmckee/feedparser/commit/c55bd8ad37db89bd219783bc514d600c9523ed38

This dates from 2021, but strangely is not part of any of the 8 releases
made till then.

Andreas

PyQt in core-updates

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 1747 bytes --]

Hello,

the attached patch seems to be necessary to start configuring python-pyqt
in core-updates.

But then the configure step still fails. The error message of "guix build"
is not helpful, but running the configure step in the result of "guix build -K"
shows the following:
...
The interpreter used by pyuic5 is
/gnu/store/82nin1sk01l31p5vpnz9c2ki76qka9b0-python-wrapper-3.10.7/bin/python.
Generating the C++ source for the QtCore module...
Error: Unable to create the C++ code.

A search seems to indicate that sip does not do its job, but the exact same
error with this PyQt version does not show up.

The error message comes from the function
def generate_sip_module_code
in configure.py, which creates a command that is run with run_command.
Precisely, the command is
/gnu/store/zd8nrmc0207l90vscnf7dlswd6z16aas-python-sip-5.5.0/bin/sip5 -w -n PyQt5.sip -t WS_X11 -t Qt_5_15_0 -f -P -o -y QtCore.pyi -c /tmp/guix-build-python-pyqt-5.15.8.drv-2/PyQt5-5.15.8/QtCore -I sip -I /tmp/guix-build-python-pyqt-5.15.8.drv-2/PyQt5-5.15.8/sip /tmp/guix-build-python-pyqt-5.15.8.drv-2/PyQt5-5.15.8/sip/QtCore/QtCoremod.sip

If I run it by hand, it prints
.sip5-real: /tmp/guix-build-python-pyqt-5.15.8.drv-1/PyQt5-5.15.8/sip/QtCore/QtCoremod.sip:23: syntax error
This line 23 is very simple:
%Module(name=PyQt5.QtCore, call_super_init=True, default_VirtualErrorHandler=PyQt5, keyword_arguments="Optional", use_limited_api=True, py_ssize_t_clean=True)

And anyway, it is part of the source code...

There is a version 5.15.9 of PyQt (interestingly enough, not related to a
corresponding Qt version), with NEWS stating "Bug fixes", but what I have
written above is still valid.

At this point, I have to give up for lack of knowledge about sip.

Andreas


[-- Attachment #2: 0001-gnu-python-pyqt-Set-variable-in-configure-script.patch --]
[-- Type: text/plain, Size: 2518 bytes --]

From 82cb67b324e9ef63bb105caae5b778467aedf894 Mon Sep 17 00:00:00 2001
From: Andreas Enge <andreas@enge.fr>
Date: Sat, 18 Mar 2023 15:48:48 +0100
Subject: [PATCH] gnu: python-pyqt: Set variable in configure script.

* gnu/packages/patches/pyqt-minimum-sip-version.patch: New file.
* gnu/packages/qt.scm (python-pyqt): Add patch.
* gnu/local.mk (dist_patch_DATA): Register patch.
---
 gnu/local.mk                                       |  1 +
 .../patches/pyqt-minimum-sip-version.patch         | 14 ++++++++++++++
 gnu/packages/qt.scm                                |  3 ++-
 3 files changed, 17 insertions(+), 1 deletion(-)
 create mode 100644 gnu/packages/patches/pyqt-minimum-sip-version.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 43b7e4ceb3..d81f8ee0f8 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1707,6 +1707,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/pybugz-stty.patch			\
   %D%/packages/patches/pygpgme-disable-problematic-tests.patch  \
   %D%/packages/patches/pyqt-configure.patch			\
+  %D%/packages/patches/pyqt-minimum-sip-version.patch           \
   %D%/packages/patches/python-2-deterministic-build-info.patch	\
   %D%/packages/patches/python-2.7-adjust-tests.patch		\
   %D%/packages/patches/python-2.7-expat-compat.patch		\
diff --git a/gnu/packages/patches/pyqt-minimum-sip-version.patch b/gnu/packages/patches/pyqt-minimum-sip-version.patch
new file mode 100644
index 0000000000..abcb4113e8
--- /dev/null
+++ b/gnu/packages/patches/pyqt-minimum-sip-version.patch
@@ -0,0 +1,14 @@
+Set the MinimumSipVersion to something, namely the currently available one.
+
+diff -u PyQt5-5.15.8.alt/configure.py PyQt5-5.15.8/configure.py
+--- PyQt5-5.15.8.alt/configure.py
++++ PyQt5-5.15.8/configure.py
+@@ -29,7 +29,7 @@ import sys
+ 
+ # Initialise the constants.
+ PYQT_VERSION_STR = "5.15.8"
+-SIP_MIN_VERSION = '@MinimumSipVersion@'
++SIP_MIN_VERSION = "5.5.0"
+ 
+ 
+ class ModuleMetadata:
diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
index 22a33341bf..45d99c241c 100644
--- a/gnu/packages/qt.scm
+++ b/gnu/packages/qt.scm
@@ -3294,7 +3294,8 @@ (define-public python-pyqt
         (sha256
          (base32
           "0q53xn1ax2kpfqwxkasby787ryq5a21chmw1p345cp0kp7py71dw"))
-        (patches (search-patches "pyqt-configure.patch"))))
+        (patches (search-patches "pyqt-configure.patch"
+                                 "pyqt-minimum-sip-version.patch"))))
     (build-system gnu-build-system)
     (native-inputs
      (list qtbase-5)) ; for qmake
-- 
2.39.2

Re: Python

[Andreas Enge]

Am Sat, Mar 18, 2023 at 10:43:17AM +0100 schrieb Lars-Dominik Braun:
> Anything else that needs work?

Well, there is pyqt as written elsewhere, but this is not only python.

I just updated python-ipython to the first version that passes all tests
without any change, in the hope that this would have the least impact on
its dependents. Without luck, now python-trio fails. Updating this to the
latest version 0.22 does not help. I copy-paste the error message below.
If you could have a look, that would definitely be welcome.

Andreas


starting phase `check'
============================= test session starts ==============================
platform linux -- Python 3.10.7, pytest-7.1.3, pluggy-0.13.1 -- /gnu/store/82nin1sk01l31p5vpnz9c2ki76qka9b0-python-wrapper-3.10.7/bin/python
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/tmp/guix-build-python-trio-0.20.0.drv-0/trio-0.20.0/.hypothesis/examples')
rootdir: /tmp/guix-build-python-trio-0.20.0.drv-0/trio-0.20.0, configfile: pyproject.toml
plugins: hypothesis-6.54.5, xdist-2.5.0, cov-3.0.0, forked-1.4.0
gw0 I / gw1 I / gw2 I / gw3 I
[gw0] linux Python 3.10.7 cwd: /tmp/guix-build-python-trio-0.20.0.drv-0/trio-0.20.0
[gw1] linux Python 3.10.7 cwd: /tmp/guix-build-python-trio-0.20.0.drv-0/trio-0.20.0
[gw2] linux Python 3.10.7 cwd: /tmp/guix-build-python-trio-0.20.0.drv-0/trio-0.20.0
[gw3] linux Python 3.10.7 cwd: /tmp/guix-build-python-trio-0.20.0.drv-0/trio-0.20.0
[gw0] Python 3.10.7 (main, Jan  1 1970, 00:00:01) [GCC 11.3.0]
[gw1] Python 3.10.7 (main, Jan  1 1970, 00:00:01) [GCC 11.3.0]
[gw2] Python 3.10.7 (main, Jan  1 1970, 00:00:01) [GCC 11.3.0]
[gw3] Python 3.10.7 (main, Jan  1 1970, 00:00:01) [GCC 11.3.0]
gw0 [0] / gw1 [0] / gw2 [0] / gw3 [0]

scheduling tests via LoadScheduling

==================================== ERRORS ====================================
________________________ ERROR collecting test session _________________________
/gnu/store/i0d555a5fd7isi606aqqmbp5zgy9jh6p-python-3.10.7/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:992: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:992: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:992: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
<frozen importlib._bootstrap_external>:883: in exec_module
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
trio/__init__.py:18: in <module>
    from ._core import (
trio/_core/__init__.py:20: in <module>
    from ._multierror import MultiError
trio/_core/_multierror.py:511: in <module>
    warnings.warn(
E   RuntimeWarning: You seem to already have a custom sys.excepthook handler installed. I'll skip installing Trio's custom handler, but this means MultiErrors will not show full tracebacks.
________________________ ERROR collecting test session _________________________
/gnu/store/i0d555a5fd7isi606aqqmbp5zgy9jh6p-python-3.10.7/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:992: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:992: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:992: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
<frozen importlib._bootstrap_external>:883: in exec_module
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
trio/__init__.py:18: in <module>
    from ._core import (
trio/_core/__init__.py:20: in <module>
    from ._multierror import MultiError
trio/_core/_multierror.py:511: in <module>
    warnings.warn(
E   RuntimeWarning: You seem to already have a custom sys.excepthook handler installed. I'll skip installing Trio's custom handler, but this means MultiErrors will not show full tracebacks.
________________________ ERROR collecting test session _________________________
/gnu/store/i0d555a5fd7isi606aqqmbp5zgy9jh6p-python-3.10.7/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:992: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:992: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:992: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
<frozen importlib._bootstrap_external>:883: in exec_module
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
trio/__init__.py:18: in <module>
    from ._core import (
trio/_core/__init__.py:20: in <module>
    from ._multierror import MultiError
trio/_core/_multierror.py:511: in <module>
    warnings.warn(
E   RuntimeWarning: You seem to already have a custom sys.excepthook handler installed. I'll skip installing Trio's custom handler, but this means MultiErrors will not show full tracebacks.
________________________ ERROR collecting test session _________________________
/gnu/store/i0d555a5fd7isi606aqqmbp5zgy9jh6p-python-3.10.7/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:992: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:992: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:992: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
<frozen importlib._bootstrap_external>:883: in exec_module
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
trio/__init__.py:18: in <module>
    from ._core import (
trio/_core/__init__.py:20: in <module>
    from ._multierror import MultiError
trio/_core/_multierror.py:511: in <module>
    warnings.warn(
E   RuntimeWarning: You seem to already have a custom sys.excepthook handler installed. I'll skip installing Trio's custom handler, but this means MultiErrors will not show full tracebacks.
=========================== short test summary info ============================
ERROR  - RuntimeWarning: You seem to already have a custom sys.excepthook han...
ERROR  - RuntimeWarning: You seem to already have a custom sys.excepthook han...
ERROR  - RuntimeWarning: You seem to already have a custom sys.excepthook han...
ERROR  - RuntimeWarning: You seem to already have a custom sys.excepthook han...
============================== 4 errors in 2.94s ===============================
error: in phase 'check': uncaught exception:
%exception #<&invoke-error program: "pytest" arguments: ("-vv" "-n" "4" "-k" "not test_ki_protection_works and not test_guest_mode_ki and not test_run_in_trio_thread_ki and not test_simple_cancel_scope_usage_doesnt_create_cyclic_garbage and not test_nursery_cancel_doesnt_create_cyclic_garbage and not test_cancel_scope_exit_doesnt_create_cyclic_garbage and not test_locals_destroyed_promptly_on_cancel and not test_ipython_exc_handler and not test_for_leaking_fds and not test_ki_self and not test_ki_wakes_us_up and not test_getnameinfo and not test_SocketType_resolve and not test_getprotobyname and not test_static_tool_sees_all_symbols") exit-status: 1 term-signal: #f stop-signal: #f>
phase `check' failed after 3.4 seconds
command "pytest" "-vv" "-n" "4" "-k" "not test_ki_protection_works and not test_guest_mode_ki and not test_run_in_trio_thread_ki and not test_simple_cancel_scope_usage_doesnt_create_cyclic_garbage and not test_nursery_cancel_doesnt_create_cyclic_garbage and not test_cancel_scope_exit_doesnt_create_cyclic_garbage and not test_locals_destroyed_promptly_on_cancel and not test_ipython_exc_handler and not test_for_leaking_fds and not test_ki_self and not test_ki_wakes_us_up and not test_getnameinfo and not test_SocketType_resolve and not test_getprotobyname and not test_static_tool_sees_all_symbols" failed with status 1
builder for `/gnu/store/21qmmyrnvwjh7254zr2byfg3l5mwd0rh-python-trio-0.20.0.drv' failed with exit code 1
build of /gnu/store/21qmmyrnvwjh7254zr2byfg3l5mwd0rh-python-trio-0.20.0.drv failed
View build log at '/var/log/guix/drvs/21/qmmyrnvwjh7254zr2byfg3l5mwd0rh-python-trio-0.20.0.drv.gz'.

Re: Python

[Lars-Dominik Braun]

Hi,

> Well, there is pyqt as written elsewhere, but this is not only python.

I looked at it and it seems that python-sip and python-pyqt-builder need a
version bump and we should maybe switch to pyproject-build-system. However
python-sip does not expose any switches to specify a different path for
.sip file includes. That requires custom patches for python-sip.
python-pyqt builds, but everything else is work in progress…

Lars

Re: Python

[Lars-Dominik Braun]

Hello Andreas,

> I looked at it and it seems that python-sip and python-pyqt-builder need a
> version bump and we should maybe switch to pyproject-build-system. However
> python-sip does not expose any switches to specify a different path for
> .sip file includes. That requires custom patches for python-sip.
> python-pyqt builds, but everything else is work in progress…

python-pyqt and python-pyqtwebengine should build on core-updates
now. I’ve tested qutebrowser, which works. Other packages depending
on python-pyqt fail due to unrelated problems. See

d9dc32b8716e5213fa127587dce64a6bbe5daee8 gnu: python-pyqtwebengine: Update to 5.15.9.
628eefa3695fad74f8fb8daca1f243b1684fe9f8 gnu: python-pyqt: Update to 5.15.9.
9b7e8365e228407c4f6f7071f6b36046be5e20b7 gnu: python-pyqt-builder: Update to 1.14.1.
eebb0bfd68a863f6ef6c18e20daad7aaa6d3ba2a gnu: python-sip: Update to 6.7.7.

Cheers,
Lars

Re: Python

[Andreas Enge]

Hello Lars,

Am Thu, Mar 30, 2023 at 11:57:53AM +0200 schrieb Lars-Dominik Braun:
> python-pyqt and python-pyqtwebengine should build on core-updates
> now. I’ve tested qutebrowser, which works.

splendid, thanks a lot for this major piece of work!

> Other packages depending on python-pyqt fail due to unrelated problems.

Okay, we can see this one by one now.

Cheers,

Andreas

Re: Python

[Lars-Dominik Braun]

Hi again,

> I just updated python-ipython to the first version that passes all tests
> without any change, in the hope that this would have the least impact on
> its dependents. Without luck, now python-trio fails. Updating this to the
> latest version 0.22 does not help. I copy-paste the error message below.
> If you could have a look, that would definitely be welcome.

this one’s also fixed, see

cf26ee1f99f84f817f96269541073846d546026b gnu: python-trio: Run pytest on tests directory only.

Lars

Re: Python

[Andreas Enge]

Am Thu, Mar 30, 2023 at 01:05:43PM +0200 schrieb Lars-Dominik Braun:
> > its dependents. Without luck, now python-trio fails.
> this one’s also fixed, see
> cf26ee1f99f84f817f96269541073846d546026b gnu: python-trio: Run pytest on tests directory only.

Thanks for putting in all this energy!

Andreas

PyQt in core-updates

[Andreas Enge]

Hello,

Am Thu, Mar 30, 2023 at 11:57:53AM +0200 schrieb Lars-Dominik Braun:
> python-pyqt and python-pyqtwebengine should build on core-updates
> now. I’ve tested qutebrowser, which works. Other packages depending
> on python-pyqt fail due to unrelated problems.

I have just fixed calibre. It failed to build because .sip fails are
now in a subdirectory /lib/python3.10/site-packages/PyQt5/bindings
instead of /share/sip (or maybe before, they were in both directories).

Just as a potential hint for other packages.

Andreas

Re: PyQt in core-updates

[Lars-Dominik Braun]

Hi Andreas,

> I have just fixed calibre. It failed to build because .sip fails are
> now in a subdirectory /lib/python3.10/site-packages/PyQt5/bindings
> instead of /share/sip (or maybe before, they were in both directories).

no, it was definitely in /share/sip before, but the pyproject-based
build system does not expose any option to move it there :( If anyone
can figure out how to move it back *and* successfully compile pyqt,
please, change it back to /share/sip.

Lars

Re: PyQt in core-updates

[Andreas Enge]

Am Tue, Apr 04, 2023 at 09:55:37AM +0200 schrieb Lars-Dominik Braun:
> > I have just fixed calibre. It failed to build because .sip fails are
> > now in a subdirectory /lib/python3.10/site-packages/PyQt5/bindings
> > instead of /share/sip (or maybe before, they were in both directories).
> no, it was definitely in /share/sip before, but the pyproject-based
> build system does not expose any option to move it there :( If anyone
> can figure out how to move it back *and* successfully compile pyqt,
> please, change it back to /share/sip.

Well, if we want to have files somewhere else, we could always add a phase
and move them from one place to the other. I wonder how other distributions
do it.

Andreas