Re: [RFC]: Skipping rust crate tests by default

[Efraim Flashner <efraim@flashner.co.il>]

[-- Attachment #1: Type: text/plain, Size: 3566 bytes --]

On Mon, Oct 16, 2023 at 11:47:53AM -0400, Maxim Cournoyer wrote:
> Hi Efraim,
> 
> Efraim Flashner <efraim@flashner.co.il> writes:
> 
> > Currently for for rust crates we build the crates, run the tests, and
> > then in %output we only have the license files and a repackaged version
> > of the source.
> >
> > The build system goes:
> > unpack source
> > unpack crates
> > patch shebangs
> > patch checksums of the crates
> > 'build
> > 'package
> > 'check
> > 'install
> >
> > 'install is clear, it does whatever the install command is.
> >
> > 'package repacks the source crate, after we've done any changes to it in
> > the snippet and later if we've gone and patched paths to binaries or
> > libraries. In theory this is useful with using these crates in a
> > GUIX_ENVIRONMENT
> >
> > 'check runs the test suite, which fairly often seems to need some
> > massaging to skip the odd test which fails or to try to skip the doc
> > tests, which fail far too often.
> 
> Why do the doc tests often fail?  Is something wrong on our side?  As
> rust aims to be fully deterministic, unless we stray too far away from
> the specified input versions, it seems these failures should not happen
> or be reported upstream.

Addressing not just the doc tests but some of the failing tests in
general:

Often it is because it is an older version of a library, released for an
older version of rust. Over time rust has been 'tightening' some of
their allowed code patterns, meaning that while in the past tests might
have passed with a warning, they now fail.

There are some packages which haven't honored rust's semver guarantee as
strongly as they might have, and those have resulted in some libraries
failing when we upgrade some of the more heavily depended upon
libraries, like serde.

It's possible that we've packaged yanked versions of crates, the
importer isn't as careful about that as it could be. I have a patch in
the bugtracker to not import yanked crates.

> > 'build sounds like it just builds the package. The first thing it does
> > it makes sure that all the necessary crates are included in the build
> > environment.
> >
> > IMO the 'build phase is the most important one, it's the one that lets
> > us know if all the cargo-inputs and cargo-development-inputs are
> > correct. We don't care if rust-rand-0.6 or rust-nb-connect-1 builds, we
> > only care that it has the correct inputs so that when we pull it in for
> > an actual binary or library everything builds correctly.
> >
> > I propose changing the cargo-build-system to have '#:tests? #f' by
> > default and then enable them for packages which have a "clear output".
> > It will keep the benefits of knowing we have the correct inputs without
> > worrying about test errors we don't care about. If it fails to build
> > during its own 'build phase that's actually worth looking into. It will
> > also cut down the amount of time the CI spends building unneeded rust
> > crates, and lets us see which ones are actually broken.
> 
> It seems useful to me to have some assurance that each crate packaged in
> Guix passes its test suite, so I'm reticent to have a #:tests? #f by
> default, while I sympathize with the work that is it given the sheer
> amount of Rust crates.
> 
> -- 
> Thanks,
> Maxim
> 

-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]