Core-updates, the last metres

Core-updates, the last metres

[Andreas Enge]

Hello,

yesterday I updated my system to core-updates. Since I am writing this
message now, you can deduce that it succeeded. Well, there is no reason
you should care, but it could encourage you to do the same. :)

I used commands like "./pre-inst-env guix package/system ...", but this
resulted in strange behaviour; I suppose I may have ended up with a
mixture between old and new guix.
I think the safest approach is to use "guix pull --commit=..." with
a commit from core-updates, or probably just
   guix pull --commit=core-updates
Then I would start by updating the system, followed by the user profiles.

Please tell us if there is a show-stopper for the merge!

We already received a first report:
python-yubikey-manager does not build. It should be repaired very shortly
after the merge (the fix is there, but would cause too many rebuilds now);
so if you rely on it, maybe delay updating your system, or hold this one
package back in your profile ("guix package --do-not-upgrade ... -u").

As for the architectures, x86_64 looks good; i686 and powerpc64le look
okay (or at least not much worse than on master; R does not work on
powerpc64le, and should also be fixed shortly after the merge).
For aarch64 it is difficult to say, since the build farm has trouble
keeping up. We brought back a few machines, but they are churning through
the backlog.

Andreas

Re: Core-updates, the last metres

[John Kehayias]

Dear Andreas and Guix,

On Sun, Apr 23, 2023 at 09:30 AM, Andreas Enge wrote:

> Hello,
>
> yesterday I updated my system to core-updates. Since I am writing this
> message now, you can deduce that it succeeded. Well, there is no reason
> you should care, but it could encourage you to do the same. :)
>

As did I! (I'm on x86_64.) And boldly late at night and everything
went smoothly (caveat/tip below). I did build all my profiles and
system beforehand so my reconfigure was just some new minor builds and
activating.

Big thanks again to you (Andreas) for really pushing this all through.
I hope our move to more feature branches will make such a job less
onerous or maybe even extinct in the near future.

> I used commands like "./pre-inst-env guix package/system ...", but this
> resulted in strange behaviour; I suppose I may have ended up with a
> mixture between old and new guix.
> I think the safest approach is to use "guix pull --commit=..." with
> a commit from core-updates, or probably just
>    guix pull --commit=core-updates
> Then I would start by updating the system, followed by the user profiles.
>

I agree here; I reconfigured my system (I switched my channels.scm
over to core-updates or similar for all my channels) but forgot to
update all my user profiles. I think this is what lead to not dropping
back to lightdm after I exited my WM, though the system booted fine. I
updated my profiles and rebooted (had to use some magic keys or I was
too impatient again) and all was good.

So, I also suggest updating everything all together before doing a
restart. I think last time I had some weird stuff as well, likely
because of the deep changes from moving to core-updates.

One noticeable difference was some fonts, perhaps a different
rendering or substitutes for some graphical programs? My terminal
emulator, kitty, also decided to pick some different default font, but
looked fine when I manually specified a font. Perhaps a difference in
some default font picking somewhere in an update? I also did a manual
fc-cache -fv for good measure.

> Please tell us if there is a show-stopper for the merge!
>

None I'm aware of, other than a few others with the same weirdness
when partially upgrading it seems.

> We already received a first report:
> python-yubikey-manager does not build. It should be repaired very shortly
> after the merge (the fix is there, but would cause too many rebuilds now);
> so if you rely on it, maybe delay updating your system, or hold this one
> package back in your profile ("guix package --do-not-upgrade ... -u").
>

I have a patch set almost ready for this. I need to do some minor
polishing and try to get some ordering here. It is one of those cases
where an update needed by one package causes others to need updates
and so on. It isn't too bad, about 20 mostly trivial patches. A few
need 10s to 100s of rebuilds, and at least one (python-filelock) will
need 3000 packages rebuilt. This will get us some overdue updates for
the affected packages.

(If you use a yubikey as a gpg smartcard everything should be fine, as
that's what I'm doing. I only use python-yubikey-manager for one time
codes or card setup. Just be aware before you update, or use another
computer/device for codes if need be.)

Likely we'll want to tack on any other core python packages in need of
updates, so please feel free to let me know of those or send patches.
I'll send a bug number here once I submit the series and we can make a
quick feature branch just after the core-updates merge. We do want to
keep this quick and not let it become another huge endevor, but I'm
happy to incorporate what we can.

> As for the architectures, x86_64 looks good; i686 and powerpc64le look
> okay (or at least not much worse than on master; R does not work on
> powerpc64le, and should also be fixed shortly after the merge).
> For aarch64 it is difficult to say, since the build farm has trouble
> keeping up. We brought back a few machines, but they are churning through
> the backlog.
>
> Andreas

If things continue looking good, are we planning to see the merge in
the next few days? Any other more leaf packages anyone has noticed
needs a fix someone should look at?

Thanks everyone for your great work here!

John

Re: Core-updates, the last metres

[Andreas Enge]

Hello John,

thanks for your report, and the patch work!

Am Sun, Apr 23, 2023 at 06:51:27PM +0000 schrieb John Kehayias:
> If things continue looking good, are we planning to see the merge in
> the next few days?

Yes, the plan is to merge on Tuesday.

Andreas

Re: Core-updates, the last metres

[Guillaume Le Vaillant]

[-- Attachment #1: Type: text/plain, Size: 575 bytes --]

John Kehayias <john.kehayias@protonmail.com> skribis:

> If things continue looking good, are we planning to see the merge in
> the next few days? Any other more leaf packages anyone has noticed
> needs a fix someone should look at?

Hi,

Here are a few leaf packages that don't build because of some
failing dependencies:
 - blender is blocked by opencolorio
 - electrum is blocked by python-ledgerblue
 - freecad is blocked by python-pyside-2

I'm not sure if I'll be able to look at them before the merge, so if
someone has the time to try and fix them, please go for it.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

Re: Core-updates, the last metres

[John Kehayias]

Hi Guillaume,

On Sun, Apr 23, 2023 at 08:00 PM, Guillaume Le Vaillant wrote:

> John Kehayias <john.kehayias@protonmail.com> skribis:
>
>> If things continue looking good, are we planning to see the merge in
>> the next few days? Any other more leaf packages anyone has noticed
>> needs a fix someone should look at?
>
> Hi,
>
> Here are a few leaf packages that don't build because of some
> failing dependencies:
>  - blender is blocked by opencolorio

Not sure what is wrong here (error for a null pointer...) and it is
quite out of date. Looks like it'll need more than a quick fix to
update it, though admittedly I didn't try.

>  - electrum is blocked by python-ledgerblue

python-ledgerblue doesn't have tests. Looking at an old build log it
would run tests, but there was nothing. Something has changed so this
errors now. Anyway, disabled, it built, as did electrum locally.
Pushed as bacee2da69dc3fdc20c384bed479d7596d9234a9.

>  - freecad is blocked by python-pyside-2
>

Didn't look much here yet, looks like part of the Python QT bindings
ecosystem. Maybe someone familiar with this can see a quick fix.

> I'm not sure if I'll be able to look at them before the merge, so if
> someone has the time to try and fix them, please go for it.
>

I picked off the easy one, thanks in advance to whoever gets the
tougher ones done!

python-ledgerblue as input for electrum? (was: Core-updates, the last metres)

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 733 bytes --]

On 2023-04-23, Guillaume Le Vaillant wrote:
> Here are a few leaf packages that don't build because of some
> failing dependencies:
>  - blender is blocked by opencolorio
>  - electrum is blocked by python-ledgerblue

Hrm this is kind of an aisde, but python-ledgerblue is a plugin for
electrum to support specific hardware, which makes me wonder why it is
in one of the inputs for electrum at all...

There is at least one other, python-btchip-python; electron-cash has
similar inputs, python-keepkey and python-trezor.

My understanding is optional plugins in general should not be in inputs
for packages; people should add the plugins to their profile or
manifest, etc. for the hardware they plan to use...


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: python-ledgerblue as input for electrum? (was: Core-updates, the last metres)

[John Kehayias]

Hi,

On Sun, Apr 23, 2023 at 04:32 PM, Vagrant Cascadian wrote:

> On 2023-04-23, Guillaume Le Vaillant wrote:
>> Here are a few leaf packages that don't build because of some
>> failing dependencies:
>>  - blender is blocked by opencolorio
>>  - electrum is blocked by python-ledgerblue
>
> Hrm this is kind of an aisde, but python-ledgerblue is a plugin for
> electrum to support specific hardware, which makes me wonder why it is
> in one of the inputs for electrum at all...
>
> There is at least one other, python-btchip-python; electron-cash has
> similar inputs, python-keepkey and python-trezor.
>
> My understanding is optional plugins in general should not be in inputs
> for packages; people should add the plugins to their profile or
> manifest, etc. for the hardware they plan to use...
>

Right, that's a good point. I had no idea about these packages before
trying to fix this one, but what you point out suggests a
refactoring/restructuring needed.

Alas, python-ecdsa (needed for electrum) is a transient failure I
think, as it built locally.

John

Re: Core-updates, the last metres

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 1258 bytes --]


-- 

John Kehayias <john.kehayias@protonmail.com> writes:

> Hi Guillaume,
>
> On Sun, Apr 23, 2023 at 08:00 PM, Guillaume Le Vaillant wrote:
>> Here are a few leaf packages that don't build because of some
>> failing dependencies:
>>  - blender is blocked by opencolorio
>
> Not sure what is wrong here (error for a null pointer...) and it is
> quite out of date. Looks like it'll need more than a quick fix to
> update it, though admittedly I didn't try.

Apparently fixed now!

>> [...]
>>  - freecad is blocked by python-pyside-2
>>
>
> Didn't look much here yet, looks like part of the Python QT bindings
> ecosystem. Maybe someone familiar with this can see a quick fix.

I thought it would be an easy fix: Ludo fixed python-shiboken-2
previously but the fix is erroneous, see the following patch.  However,
I'm still getting a build error that boils down to the following:

--8<---------------cut here---------------start------------->8---
import importlib
importlib.machinery
--8<---------------cut here---------------end--------------->8---

errors out with python-minimal@3.10, but not with python@3.10!  I don't
really understand how this could happen, does anyone have an idea?

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

[PATCH] gnu: python-shiboken-2: Do not rely on _Py_Mangle being available.

[Josselin Poiret]

* gnu/packages/patches/python-shiboken-2-compat.patch: Fix the patch according
to upstream.
---
 .../patches/python-shiboken-2-compat.patch    | 29 +++++++++----------
 1 file changed, 14 insertions(+), 15 deletions(-)

diff --git a/gnu/packages/patches/python-shiboken-2-compat.patch b/gnu/packages/patches/python-shiboken-2-compat.patch
index 0c7de64395..48aa74173b 100644
--- a/gnu/packages/patches/python-shiboken-2-compat.patch
+++ b/gnu/packages/patches/python-shiboken-2-compat.patch
@@ -1,16 +1,15 @@
-Provide a declaration that's missing in Python 3.10.
+Backport a fix from 5.15.9, mentioned in [1].
 
---- pyside-setup-opensource-src-5.15.8/sources/shiboken2/libshiboken/sbkpython.h	2023-04-19 23:14:11.415337435 +0200
-+++ pyside-setup-opensource-src-5.15.8/sources/shiboken2/libshiboken/sbkpython.h	2023-04-19 23:18:09.055605606 +0200
-@@ -108,6 +108,11 @@ extern "C" {
- #if PY_MAJOR_VERSION >= 3
-     #define IS_PY3K
- 
-+extern "C" {
-+  /* In Python 3.10 this declaration is not provided by <Python.h>.  */
-+  extern PyObject* _Py_Mangle(PyObject *p, PyObject *name);
-+}
-+
-     #define PyInt_Type PyLong_Type
-     #define PyInt_Check PyLong_Check
-     #define PyInt_CheckExact PyLong_CheckExact
+[1] https://bugreports.qt.io/browse/PYSIDE-2176
+
+--- pyside-setup-opensource-src-5.15.8/sources/shiboken2/libshiboken/pep384impl.cpp	2023-04-24 11:05:48.770934499 +0200
++++ pyside-setup-opensource-src-5.15.8/sources/shiboken2/libshiboken/pep384impl.cpp	2023-04-24 11:04:51.566943936 +0200
+@@ -751,7 +751,7 @@
+ #endif // IS_PY2
+     Shiboken::AutoDecRef privateobj(PyObject_GetAttr(
+         reinterpret_cast<PyObject *>(Py_TYPE(self)), Shiboken::PyMagicName::name()));
+-#ifndef Py_LIMITED_API
++#ifdef IS_PY2
+     return _Py_Mangle(privateobj, name);
+ #else
+     // PYSIDE-1436: _Py_Mangle is no longer exposed; implement it always.

base-commit: 827df9d1dde4f4a06e1789ec17cf0586602aa121
-- 
2.39.2

Re: [PATCH] gnu: python-shiboken-2: Do not rely on _Py_Mangle being available.

[Andreas Enge]

Am Mon, Apr 24, 2023 at 12:01:51PM +0200 schrieb Josselin Poiret:
> * gnu/packages/patches/python-shiboken-2-compat.patch: Fix the patch according
> to upstream.

Pushed!

Andreas