antioxidant update: librsvg builds, and other things (core-updates)

antioxidant update: librsvg builds, and other things (core-updates)

[Maxime Devos]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 3940 bytes --]

Some updates:

  * core-updates is now targeted instead of master (because the librsvg
    on core-updates has a less complicated build system and hence easier
    to support with antioxidant)
  * librsvg now builds. It's about 3 times larger than the
    cargo-build-system librsvg (at least, the librsvg from master,
    didn't compare against core-updates yet because there was no
    substitute available), so perhaps some compilation flags need to be
    changed.
  * According to ci.guix.gnu.org, antioxidation is at 90%, but some of
    the 'failures' are because of cancelled builds in dependencies, the
    real number should be higher. Maybe 'Restart all builds' would be
    correct that, though I don't know if that wouldn't restart too much
    builds (including builds that succeeded).
  * Running tests is supported, #:tests? and #:parallel-tests? is
    respected, there is an option to skip some tests (using
    #:test-options).  As antioxidant doesn't do #:skip-build? #t, I
    expect more packages to be tested with antioxidant than with
    cargo-build-system.

    There are some test failure, things like "tries accessing the
    network" are simply skipped. Sometimes it is assuming incorrect
    things about the build environment such assuming that stderr is a
    terminal, those are simply skipped. Sometimes test files are removed
    from the tarballs uploaded at crates.io, this appears to be
    intentional, there is a project to remove all such 'bloat',
    disregarding the value of tests -- for those packages we will have
    to switch to git checkouts, but for now I simply disabled tests there.

    I've also found some time bombs (e.g. rust-rustls appears to have a
    certificate expiration problem), there tests are disabled too.

    There were also some genuine test failures, some of them have been
    reported upstream but not all of them yet.
  * antioxidant now has some defenses against bundling -- it detects if
    the feature "bundle", "vendor" or "vendored" is implicitly enabled
    and if so, bails out unless they were explicitly added to the
    package definition.
  * I don't know if I mentioned it previously, but antioxidant now
    supports some 'unstable' rust code even when using 'stable' Rust
    compilers, by setting RUSTC_BOOTSTRAP=1. It's 'not supposed to be
    used', but if it's good enough for the Rust compiler, then it should
    be good enough for other software too, I'd think.

Some questions:

  * Some Rust crates have 'examples' and 'benchmarks' that can be
    compiled and installed. I could teach antioxidant to compile and
    install them, though it appears to provide very little value at cost
    of more compile time and a greater closure size (more binaries, more
    dependencies and wasn't there some bug with grafts whose fix causes
    outputs to be substituted that ended up unused? Though hopefully
    someone could figure out a better fix ...)
  * Due to how regularised the Rust build system is, it's feasible to
    compile tests even when cross-compiling (*), so cross-compiled could
    run the cross-compiled tests on the system they are cross-compiling
    for after the cross-compilation to verify their cross-compiled software.

    Currently, tests are only compiled and installed when #:tests? #true
    which is only the case when compiling natively(*), but with some
    work I could separate building tests from running tests and build
    tests by default when cross-compiling. Downside: tests have to be
    installed in an output, which increases the closure size.

    (Currently tests are installed in an output, but I could change that
    to the working directory).

(*) antioxidant doesn't support cross-compilation yet, but in the past 
I've tried out cross-compiling the standard library and it seemed to 
work, though it wasn't completed.


[-- Attachment #1.1.1.2: Type: text/html, Size: 4613 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: antioxidant update: librsvg builds, and other things (core-updates)

[Liliana Marie Prikler]

Am Samstag, dem 27.08.2022 um 12:54 +0200 schrieb Maxime Devos:
> Some questions:
>  * Some Rust crates have 'examples' and 'benchmarks' that can be
> compiled and installed. I could teach antioxidant to compile and
> install them, though it appears to provide very little value at cost
> of more compile time and a greater closure size (more binaries, more
> dependencies and wasn't there some bug with grafts whose fix causes
> outputs to be substituted that ended up unused? Though hopefully
> someone could figure out a better fix ...)
I think you could try building those examples as a separate output or
package.

>  * Due to how regularised the Rust build system is, it's feasible to
> compile tests even when cross-compiling (*), so cross-compiled could
> run the cross-compiled tests on the system they are cross-compiling
> for after the cross-compilation to verify their cross-compiled
> software.
How exactly does this work without emulating the system in question?

Re: antioxidant update: librsvg builds, and other things (core-updates)

[Maxime Devos]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 747 bytes --]

On 27-08-2022 21:54, Liliana Marie Prikler wrote:

>>   * Due to how regularised the Rust build system is, it's feasible to
>> compile tests even when cross-compiling (*), so cross-compiled could
>> run the cross-compiled tests on the system they are cross-compiling
>> for after the cross-compilation to verify their cross-compiled
>> software.
> How exactly does this work without emulating the system in question?

It works by not performing any work except compilation -- Guix' 
responsibility would only be to cross-compile and install the tests 
(_not_ running them), you are supposed to install the cross-compiled 
thing (including tests) on the target system and run the tests on the 
target system.

Greetings,
Maxime


[-- Attachment #1.1.1.2: Type: text/html, Size: 1225 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: antioxidant update: librsvg builds, and other things (core-updates)

[Liliana Marie Prikler]

Am Samstag, dem 27.08.2022 um 22:01 +0200 schrieb Maxime Devos:
> On 27-08-2022 21:54, Liliana Marie Prikler wrote:
> > > * Due to how regularised the Rust build system is, it's feasible
> > > to compile tests even when cross-compiling (*), so cross-compiled
> > > could run the cross-compiled tests on the system they are
> > > cross-compiling for after the cross-compilation to verify their
> > > cross-compiled software.
> > How exactly does this work without emulating the system in
> > question?
> It works by not performing any work except compilation -- Guix'
> responsibility would only be to cross-compile and install the tests
> (_not_ running them), you are supposed to install the cross-compiled
> thing (including tests) on the target system and run the tests on the
> target system.
This doesn't strike me as a rust-specific setup, though.  In principle,
you should be able to do the same with a C/C++ program, but most of the
time "make check" implies both building and running the tests.

Cheers

Re: antioxidant update: librsvg builds, and other things (core-updates)

[Maxime Devos]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 1859 bytes --]


On 28-08-2022 00:04, Liliana Marie Prikler wrote:
> Am Samstag, dem 27.08.2022 um 22:01 +0200 schrieb Maxime Devos:
>> On 27-08-2022 21:54, Liliana Marie Prikler wrote:
>>>> * Due to how regularised the Rust build system is, it's feasible
>>>> to compile tests even when cross-compiling (*), so cross-compiled
>>>> could run the cross-compiled tests on the system they are
>>>> cross-compiling for after the cross-compilation to verify their
>>>> cross-compiled software.
>>> How exactly does this work without emulating the system in
>>> question?
>> It works by not performing any work except compilation -- Guix'
>> responsibility would only be to cross-compile and install the tests
>> (_not_  running them), you are supposed to install the cross-compiled
>> thing (including tests) on the target system and run the tests on the
>> target system.
> This doesn't strike me as a rust-specific setup, though.  In principle,
> you should be able to do the same with a C/C++ program, but most of the
> time "make check" implies both building and running the tests.

The difference between Rust and many C/C++ setups here, is that with 
Rust it's trivial to only compile and install the tests without running 
them (currently, antioxidant-build-system compiles+installs and runs the 
tests in separate phases), whereas in case of C/C++, there usually isn't 
a convenient '"make install-the-tests-without-running-them" target, 
rather building and running the tests is combined in a single "make 
check" as you note.

As I've written previously:

> Due to how regularised the Rust build system is, it's feasible
> to compile tests even when cross-compiling (*)
but that does not appear to be the case for C/C++, as you've noted with 
your comments about "make check" both building and running the tests.

Greetings,
Maxime.

[-- Attachment #1.1.1.2: Type: text/html, Size: 3189 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: antioxidant update: librsvg builds, and other things (core-updates)

[Hartmut Goebel]

[-- Attachment #1: Type: text/plain, Size: 430 bytes --]


Hi Maxmine,

great news, thanks for the update and for working on antioxidant.


> Some questions:
>
>   * Some Rust crates have 'examples' and 'benchmarks' that can be
>     compiled and installed.
>
I support skipping these, as there is little value.


-- 
Regards
Hartmut Goebel

| Hartmut Goebel          |h.goebel@crazy-compilers.com                |
|www.crazy-compilers.com  | compilers which you thought are impossible |

[-- Attachment #2: Type: text/html, Size: 1205 bytes --]