* Thursday 18th: ‘core-updates-frozen’ sprint! @ 2021-11-17 11:46 Ludovic Courtès 2021-11-17 12:06 ` Ricardo Wurmus 2021-11-19 15:01 ` Ludovic Courtès 0 siblings, 2 replies; 10+ messages in thread From: Ludovic Courtès @ 2021-11-17 11:46 UTC (permalink / raw) To: Guix Devel Hello Guix! I hereby declare tomorrow, Thursday Nov. 18th, day of the on-line ‘core-updates-frozen’ sprint! The idea is that you join #guix on IRC, you try to build your system or profile from ‘core-updates-frozen’, and you report (and fix!) any issues you may have, in the warmth of a collective of friendly human beings all sharing the same goal and helping each other. Nov. 23th will mark Guix’s 9th birthday. Wouldn’t it be great to have that branch finally merged in ‘master’ by then? See you on Thursday! :-) Ludo’. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Thursday 18th: ‘core-updates-frozen’ sprint! 2021-11-17 11:46 Thursday 18th: ‘core-updates-frozen’ sprint! Ludovic Courtès @ 2021-11-17 12:06 ` Ricardo Wurmus 2021-11-19 15:01 ` Ludovic Courtès 1 sibling, 0 replies; 10+ messages in thread From: Ricardo Wurmus @ 2021-11-17 12:06 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel Ludovic Courtès <ludo@gnu.org> writes: > I hereby declare tomorrow, Thursday Nov. 18th, day of the > on-line > ‘core-updates-frozen’ sprint! > > The idea is that you join #guix on IRC, you try to build your > system or > profile from ‘core-updates-frozen’, and you report (and fix!) > any issues > you may have, in the warmth of a collective of friendly human > beings all > sharing the same goal and helping each other. I’ll be there! To give everyone a head start: we have some annoying errors: 1) /gnu/store/aw65rwl2c50ckwghafv0p98hpg4wqvxm-vigra-1.11.1-0.9b514fa.drv has a bunch of test failures; this blocks libreoffice. Looks like this and was caused by the most recent boost upgrade: --8<---------------cut here---------------start------------->8--- Boost.Python.ArgumentError: Python argument types in vigra.vigranumpycore.constructArrayFromAxistags(type, tuple, numpy.dtype[float32], AxisTags, bool) did not match C++ signature: constructArrayFromAxistags(boost::python::api::object, vigra::ArrayVector<long, std::allocator<long> >, NPY_TYPES, vigra::AxisTags, bool) --8<---------------cut here---------------end--------------->8--- 2) python-notebook has failing tests. Same for a more recent version (6.4.3). The failing tests *all* relate to deleting files with send2trash. Upgrading send2trash does not help. These tests also fail outside the build container environment. The errors look like this: --8<---------------cut here---------------start------------->8--- notebook-6.4.3/notebook/services/contents/manager.py:279: in delete self.delete_file(path) notebook-6.4.3/notebook/services/contents/filemanager.py:533: in delete_file send2trash(os_path) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ path = '/tmp/guix-build-python-notebook-6.4.3.drv-0/tmpkwgdpuk_/Untitled.ipynb' def send2trash(path): try: f = Gio.File.new_for_path(path) f.trash(cancellable=None) except GObject.GError as e: if e.code == Gio.IOErrorEnum.NOT_SUPPORTED: # We get here if we can't create a trash directory on the same # device. I don't know if other errors can result in NOT_SUPPORTED. raise TrashPermissionError('') > raise OSError(e.message) E OSError: Error trashing file /tmp/guix-build-python-notebook-6.4.3.drv-0/tmpkwgdpuk_/Untitled.ipynb: No such file or directory --8<---------------cut here---------------end--------------->8--- 3) Building Gnome systems fails because of a package conflict. Commit 781f475bbac4e73848f68cb9f420a7283ec17c16 is the direct cause; different variants of at-spi2-atk end up in the same profile. > Nov. 23th will mark Guix’s 9th birthday. Wouldn’t it be great > to have > that branch finally merged in ‘master’ by then? Yes! -- Ricardo ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Thursday 18th: ‘core-updates-frozen’ sprint! 2021-11-17 11:46 Thursday 18th: ‘core-updates-frozen’ sprint! Ludovic Courtès 2021-11-17 12:06 ` Ricardo Wurmus @ 2021-11-19 15:01 ` Ludovic Courtès 2021-11-19 23:43 ` Ricardo Wurmus 1 sibling, 1 reply; 10+ messages in thread From: Ludovic Courtès @ 2021-11-19 15:01 UTC (permalink / raw) To: Guix Devel Hi there! Ludovic Courtès <ludo@gnu.org> skribis: > I hereby declare tomorrow, Thursday Nov. 18th, day of the on-line > ‘core-updates-frozen’ sprint! I’d like to congratulate ourselves and call it a success! A dedicated group worked all day long anywhere-on-earth and pushed 79 commits fixing a variety of things—thumbs up! We’re almost done but we have yet to get past the finish line. Please give it a spin either by testing your system: sudo guix time-machine --branch=core-updates-frozen -- \ system reconfigure /…/config.scm or by upgrading your profile: guix time-machine --branch=core-updates-frozen -- upgrade Expect rebuilds as things in the GNOME/Freedesktop stack are still receiving fixes, or wait until ci.guix has caught up. The issue Ricardo mentioned about ‘python-jupyter’ has yet to be fixed too, and there’s probably more. If you email bug-guix or guix-patches with “core-updates-frozen” in the subject and a clear description of what the problem/fix is, your friendly committer collective will be sure to fast-track your patch (right? :-)). > Nov. 23th will mark Guix’s 9th birthday. Wouldn’t it be great to have > that branch finally merged in ‘master’ by then? Sounds doable, let’s keep up the good work! Ludo’. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Thursday 18th: ‘core-updates-frozen’ sprint! 2021-11-19 15:01 ` Ludovic Courtès @ 2021-11-19 23:43 ` Ricardo Wurmus 2021-11-22 13:19 ` Ludovic Courtès 0 siblings, 1 reply; 10+ messages in thread From: Ricardo Wurmus @ 2021-11-19 23:43 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel Ludovic Courtès <ludo@gnu.org> writes: > … there’s probably more. I noticed that we’re still using PYTHONPATH in a bunch of places, even though we should now be using GUIX_PYTHONPATH. I’ve been slowly hacking away at the remaining offenders. That said, it’s amazing how much progress we made in so little time! I’ll continue to prioritize core-updates-frozen for the next few days. >> Nov. 23th will mark Guix’s 9th birthday. Wouldn’t it be great >> to have >> that branch finally merged in ‘master’ by then? > > Sounds doable, let’s keep up the good work! I hope we can make it! -- Ricardo ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Thursday 18th: ‘core-updates-frozen’ sprint! 2021-11-19 23:43 ` Ricardo Wurmus @ 2021-11-22 13:19 ` Ludovic Courtès 2021-11-22 13:36 ` zimoun 0 siblings, 1 reply; 10+ messages in thread From: Ludovic Courtès @ 2021-11-22 13:19 UTC (permalink / raw) To: Ricardo Wurmus; +Cc: guix-devel Hi! Ricardo Wurmus <rekado@elephly.net> skribis: > Ludovic Courtès <ludo@gnu.org> writes: > >> … there’s probably more. > > I noticed that we’re still using PYTHONPATH in a bunch of places, even > though we should now be using GUIX_PYTHONPATH. I’ve been slowly > hacking away at the remaining offenders. I see you fixed a bunch of them, yay! > That said, it’s amazing how much progress we made in so little time! > I’ll continue to prioritize core-updates-frozen for the next few days. Yes, let’s do that. I’ve fixed things I’ll probably never ever use, like trytond. A call to everyone though: make sure your favorite packages work because you can’t necessarily count on others to test it and fix it. :-) I’ve noticed Python packages still failing so if you rely on Python, better be safe than sorry. >>> Nov. 23th will mark Guix’s 9th birthday. Wouldn’t it be great to >>> have >>> that branch finally merged in ‘master’ by then? >> >> Sounds doable, let’s keep up the good work! > > I hope we can make it! Yup, let’s see! Ludo’. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Thursday 18th: ‘core-updates-frozen’ sprint! 2021-11-22 13:19 ` Ludovic Courtès @ 2021-11-22 13:36 ` zimoun 2021-11-22 20:13 ` Maxim Cournoyer 0 siblings, 1 reply; 10+ messages in thread From: zimoun @ 2021-11-22 13:36 UTC (permalink / raw) To: Ludovic Courtès, Efraim Flashner; +Cc: Guix Devel Hi Ludo, On Mon, 22 Nov 2021 at 14:21, Ludovic Courtès <ludo@gnu.org> wrote: > >> … there’s probably more. Julia is annoying, Because the test suite sometimes passes, sometimes not. Well, I am not even sure it is the same part that fails. I notice that "guix build julia --without-tests=julia" then "guix build julia" tends to works. :-) Any idea to debug this? Cheers, simon ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Thursday 18th: ‘core-updates-frozen’ sprint! 2021-11-22 13:36 ` zimoun @ 2021-11-22 20:13 ` Maxim Cournoyer 2021-11-24 11:07 ` zimoun 2021-11-25 2:13 ` ‘core-updates-frozen’: julia status zimoun 0 siblings, 2 replies; 10+ messages in thread From: Maxim Cournoyer @ 2021-11-22 20:13 UTC (permalink / raw) To: zimoun; +Cc: Guix Devel Hello, zimoun <zimon.toutoune@gmail.com> writes: > Hi Ludo, > > On Mon, 22 Nov 2021 at 14:21, Ludovic Courtès <ludo@gnu.org> wrote: > >> >> … there’s probably more. > > Julia is annoying, Because the test suite sometimes passes, sometimes > not. Well, I am not even sure it is the same part that fails. I > notice that "guix build julia --without-tests=julia" then "guix build > julia" tends to works. :-) > > Any idea to debug this? No idea how to debug this, but when faced with nondeterministic tests, here's how I proceed: 1. Report them upstream. 2. Disable the problematic tests. You may need to disable a whole test suite of similarly faulted tests, if they all behave the same. Thanks, Maxim ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Thursday 18th: ‘core-updates-frozen’ sprint! 2021-11-22 20:13 ` Maxim Cournoyer @ 2021-11-24 11:07 ` zimoun 2021-11-25 2:13 ` ‘core-updates-frozen’: julia status zimoun 1 sibling, 0 replies; 10+ messages in thread From: zimoun @ 2021-11-24 11:07 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: Guix Devel Hi, On Mon, 22 Nov 2021 at 15:13, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote: > 2. Disable the problematic tests. You may need to disable a whole test suite of > similarly faulted tests, if they all behave the same. I am becoming crazy… For instance this test randomly* fails: @test mul!(C, vf, transpose(vf), 2, 3) == 2vf*vf' .+ 3C0 <https://github.com/JuliaLang/julia/blob/v1.6.3/stdlib/LinearAlgebra/test/matmul.jl#L155> Therefore, I add, --8<---------------cut here---------------start------------->8--- (substitute* "stdlib/LinearAlgebra/test/matmul.jl" (("@test mul\\!\\(C, vf, transpose\\(vf\\), 2, 3\\) == 2vf\\*vf' \\.+ 3C0") "@test_broken @test mul!(C, vf, transpose(vf), 2, 3) == 2vf*vf' .+ 3C0") (substitute* "test/math.jl" (("@test isinf\\(log1p\\(-one\\(T\\)\\)\\)") "@test_broken isinf(log1p(-one(T)))")))) --8<---------------cut here---------------end--------------->8--- So far, so good. Then I get: --8<---------------cut here---------------start------------->8--- Error During Test at /tmp/guix-build-julia-1.6.3.drv-0/julia-1.6.3/test/math.jl:278 Unexpected Pass Expression: isinf(log1p(-(one(T)))) Got correct result, please change to @test if no longer broken. --8<---------------cut here---------------end--------------->8--- and thus the build is reported as failed. Argh!! Ok, because these 2 tests are random, I replace the snippet above by: --8<---------------cut here---------------start------------->8--- (substitute* "stdlib/LinearAlgebra/test/matmul.jl" (("@test mul\\!\\(C, vf, transpose\\(vf\\), 2, 3\\) == 2vf\\*vf' \\.+ 3C0") "") (substitute* "test/math.jl" (("@test isinf\\(log1p\\(-one\\(T\\)\\)\\)") "")))) --8<---------------cut here---------------end--------------->8--- And now, something magical happens, I get this error with parenthesis: --8<---------------cut here---------------start------------->8--- Test Failed at /tmp/guix-build-julia-1.6.3.drv-0/julia-1.6.3/usr/share/julia/stdlib/v1.6/LinearAlgebra/test/matmul.jl:155 Expression: mul!(C, vf, transpose(vf), 2, 3) == (2vf) * vf' .+ 3C0 --8<---------------cut here---------------end--------------->8--- What?! And AFAIU, it is not in the test suite. Do I miss something with ’substitute*’? Cheers, simon *randomly: because I do not know why, nothing is random with computer. ;-) PS: Note that current Julia master contains this test: @test mul!(C, vf, transpose(vf), 2, 3) ≈ 2vf*vf' .+ 3C0 https://github.com/JuliaLang/julia/blob/master/stdlib/LinearAlgebra/test/matmul.jl#L155 I am trying to ’substitute*’ with that. Then I give up for now. ^ permalink raw reply [flat|nested] 10+ messages in thread
* ‘core-updates-frozen’: julia status 2021-11-22 20:13 ` Maxim Cournoyer 2021-11-24 11:07 ` zimoun @ 2021-11-25 2:13 ` zimoun 2021-11-25 23:49 ` ‘core-updates-frozen’: julia status update zimoun 1 sibling, 1 reply; 10+ messages in thread From: zimoun @ 2021-11-25 2:13 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: Guix Devel Hi, On Mon, 22 Nov 2021 at 15:13, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote: > zimoun <zimon.toutoune@gmail.com> writes: >> Julia is annoying, Because the test suite sometimes passes, sometimes >> not. Well, I am not even sure it is the same part that fails. I >> notice that "guix build julia --without-tests=julia" then "guix build >> julia" tends to works. :-) >> >> Any idea to debug this? > > No idea how to debug this, but when faced with nondeterministic tests, > here's how I proceed: > > 1. Report them upstream. > 2. Disable the problematic tests. You may need to disable a whole test suite of > similarly faulted tests, if they all behave the same. After this hacking day, Julia builds deterministically* on core-updates-frozen. Along the way, now the test suite runs in parallel which reduces a lot the feedback loop. :-) And a variant of the current patch of julia source is upstream PR. *deterministically: at least on 2 machines. ;-) One or two failing tests randomly failing should be investigated and probably reported upstream. The test suite of julia-* packages supports also (more or less) some parallelism. From my understanding, the limitation comes from the script 'test/runtests.jl' from each package; each implements more or less some parallelism. Now, the environment variable JULIA_CPU_THREADS is set but Julia does not always honor it as its doc specifies – see the patch fixing the parallelism of Julia test suite ;-). Therefore, the test suite of each package is launched with '--procs=(parallel-job-count)' and Julia exploits at best the available workers. I am currently rebuilding all the julia-* packages. It is high probable some are now broken. But it appears to me not a blocker for the merge. On a side note, Julia is not currently reproducible [1] and 'julia-build-system' neither [2]. Therefore, probable non-reproducible build introduced by parallelism is currently not the point, IMHO. Thanks Maxim for the help and the good vibes to tackle that. Cheers, simon 1: <http://issues.guix.gnu.org/issue/22304> 2: <http://issues.guix.gnu.org/issue/47354> ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: ‘core-updates-frozen’: julia status update 2021-11-25 2:13 ` ‘core-updates-frozen’: julia status zimoun @ 2021-11-25 23:49 ` zimoun 0 siblings, 0 replies; 10+ messages in thread From: zimoun @ 2021-11-25 23:49 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: Guix Devel Hi, On Thu, 25 Nov 2021 at 03:13, zimoun <zimon.toutoune@gmail.com> wrote: > I am currently rebuilding all the julia-* packages. It is high probable > some are now broken. But it appears to me not a blocker for the > merge. All is fixed in patch#52117 [1]. After this hacking session, 1. The package julia uses parallelism to build, i.e., it requires much much less time to build; something like 5x speedup on Berlin. 2. The julia-build-system runs the tests with some degree of parallelism. The speedup depends on each ’test/runtests.jl’ script provided by the package julia-* itself. The strong issues to tackle are: a) bug#22304: build julia package with reproducible b) bug#47354: build julia-* packages reproducible 1: <https://issues.guix.gnu.org/52117> 2: <http://issues.guix.gnu.org/issue/22304> 3: <http://issues.guix.gnu.org/issue/47354> Cheers, simon ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2021-11-25 23:54 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-11-17 11:46 Thursday 18th: ‘core-updates-frozen’ sprint! Ludovic Courtès 2021-11-17 12:06 ` Ricardo Wurmus 2021-11-19 15:01 ` Ludovic Courtès 2021-11-19 23:43 ` Ricardo Wurmus 2021-11-22 13:19 ` Ludovic Courtès 2021-11-22 13:36 ` zimoun 2021-11-22 20:13 ` Maxim Cournoyer 2021-11-24 11:07 ` zimoun 2021-11-25 2:13 ` ‘core-updates-frozen’: julia status zimoun 2021-11-25 23:49 ` ‘core-updates-frozen’: julia status update zimoun
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).