Update to source-highlight seems to have broken rust

Update to source-highlight seems to have broken rust

[Kaelyn]

Hi Efraim and guix-devel,

Based on my local cuirass instance and some spot testing this morning, it appears the change to source-highlight in commit 367bc2d198 has broken the build of rust 1.73.
* `guix refresh -l source-highlight` reports modifying the package would require a large number of rebuilds ("Building the following 6129 packages would ensure 9069 dependent packages are rebuilt").
* I am seeing a consistent test failure for rust starting with that commit.
* `guix weather rust@1.73` reports 100% availability for both ci.guix.gnu.org and bordeaux.guix.gnu.org at commit fbeae77ae6 but 0% availability at commit 367bc2d198.

The consistently-reported failure is:

---- metadata::cargo_metadata_non_utf8 stdout ----
thread 'metadata::cargo_metadata_non_utf8' panicked at crates/cargo-test-support/src/paths.rs:155:33:
failed to mkdir_p /tmp/guix-build-rust-1.73.0.drv-0/rustc-1.73.0-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/tmp/cit/t1684/foo/�/./src: Invalid or incomplete multibyte or wide character (os error 84)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    metadata::cargo_metadata_non_utf8

test result: FAILED. 2801 passed; 1 failed; 198 ignored; 0 measured; 0 filtered out; finished in 92.81s

error: test failed, to rerun pass `--test testsuite`


I don't exactly understand how wrapping scripts in source-highlight ended up breaking the above rust test, nor do I use rust, so I wanted to bring it to folks' attention since the failure appears to affect a great many packages (I noticed it in an ffmpeg build failure on my local cuirass instance).

Cheers,
Kaelyn

Re: Update to source-highlight seems to have broken rust

[John Kehayias]

Hi Kaeylyn,


On Monday, January 29th, 2024 at 12:26 PM, Kaelyn <kaelyn.alexi@protonmail.com> wrote:

> 
> 
> Hi Efraim and guix-devel,
> 
> Based on my local cuirass instance and some spot testing this morning, it appears the change to source-highlight in commit 367bc2d198 has broken the build of rust 1.73.
> * `guix refresh -l source-highlight` reports modifying the package would require a large number of rebuilds ("Building the following 6129 packages would ensure 9069 dependent packages are rebuilt").
> * I am seeing a consistent test failure for rust starting with that commit.
> * `guix weather rust@1.73` reports 100% availability for both ci.guix.gnu.org and bordeaux.guix.gnu.org at commit fbeae77ae6 but 0% availability at commit 367bc2d198.
> 

This was reverted in 0fb160a5e8b0b800426489c0a2ad387c6934fba so maybe you were just unlucky in what commit you were at?

I can't comment on the details of the change or rust, but hopefully just pulling to a newer commit will go back to good coverage for you as well.

Hope that helps!
John

> The consistently-reported failure is:
> 
> ---- metadata::cargo_metadata_non_utf8 stdout ----
> thread 'metadata::cargo_metadata_non_utf8' panicked at crates/cargo-test-support/src/paths.rs:155:33:
> failed to mkdir_p /tmp/guix-build-rust-1.73.0.drv-0/rustc-1.73.0-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/tmp/cit/t1684/foo/�/./src: Invalid or incomplete multibyte or wide character (os error 84)
> note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
> 
> 
> failures:
> metadata::cargo_metadata_non_utf8
> 
> test result: FAILED. 2801 passed; 1 failed; 198 ignored; 0 measured; 0 filtered out; finished in 92.81s
> 
> error: test failed, to rerun pass `--test testsuite`
> 
> 
> I don't exactly understand how wrapping scripts in source-highlight ended up breaking the above rust test, nor do I use rust, so I wanted to bring it to folks' attention since the failure appears to affect a great many packages (I noticed it in an ffmpeg build failure on my local cuirass instance).
> 
> Cheers,
> Kaelyn

Re: Update to source-highlight seems to have broken rust

[Kaelyn]

On Monday, January 29th, 2024 at 9:49 AM, John Kehayias <john.kehayias@protonmail.com> wrote:

> 
> 
> Hi Kaeylyn,
> 
> 
> On Monday, January 29th, 2024 at 12:26 PM, Kaelyn kaelyn.alexi@protonmail.com wrote:
> 
> > Hi Efraim and guix-devel,
> > 
> > Based on my local cuirass instance and some spot testing this morning, it appears the change to source-highlight in commit 367bc2d198 has broken the build of rust 1.73.
> > * `guix refresh -l source-highlight` reports modifying the package would require a large number of rebuilds ("Building the following 6129 packages would ensure 9069 dependent packages are rebuilt").
> > * I am seeing a consistent test failure for rust starting with that commit.
> > * `guix weather rust@1.73` reports 100% availability for both ci.guix.gnu.org and bordeaux.guix.gnu.org at commit fbeae77ae6 but 0% availability at commit 367bc2d198.
> 
> 
> This was reverted in 0fb160a5e8b0b800426489c0a2ad387c6934fba so maybe you were just unlucky in what commit you were at?
> 
> I can't comment on the details of the change or rust, but hopefully just pulling to a newer commit will go back to good coverage for you as well.
> 
> Hope that helps!

That does help, thank you! I think I missed the revert because of a minor misbehavior? of cuirass (so me getting unlucky about the commit in a different way!).

I have a local cuirass server set up to build the packages in a local channel, along with a job configured to build the packages in my Guix Home configs (currently using a manually generated manifest file that is checked in as part of the local channel)--but I don't have notifications set up and I only occasionally check on its status. This morning I happened to notice that wine64-staging was reported as having been failing for a while, because of ffmpeg-6.1.1 failing to build due to the rust failure (i.e. cuirass reported ffmpeg as the root failed build, and looking at the error log was what revealed that it was actually rust that failed). I then mistakenly assumed the issue was still present because the list of evaluations and evaluation dashboard reported wine64-staging as still failing up through the evaluation for commit 2c5293a from a few hours prior to my email. I'm guessing the underlying issue for the rebuild not happening for the revert commit is tied to dependency propagation triggering rebuilds (as a likely-related example, I find it odd that while cuirass showed the wine64-staging rebuild failed because its dependency ffmpeg failed to build, but claimed ffmpeg was the failed package when it was a unit test in the rust build that actually failed).

Thank you again, John, for your response about the commit having already been reverted!

Cheers,
Kaelyn

> John
> 
> > The consistently-reported failure is:
> > 
> > ---- metadata::cargo_metadata_non_utf8 stdout ----
> > thread 'metadata::cargo_metadata_non_utf8' panicked at crates/cargo-test-support/src/paths.rs:155:33:
> > failed to mkdir_p /tmp/guix-build-rust-1.73.0.drv-0/rustc-1.73.0-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/tmp/cit/t1684/foo/�/./src: Invalid or incomplete multibyte or wide character (os error 84)
> > note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
> > 
> > failures:
> > metadata::cargo_metadata_non_utf8
> > 
> > test result: FAILED. 2801 passed; 1 failed; 198 ignored; 0 measured; 0 filtered out; finished in 92.81s
> > 
> > error: test failed, to rerun pass `--test testsuite`
> > 
> > I don't exactly understand how wrapping scripts in source-highlight ended up breaking the above rust test, nor do I use rust, so I wanted to bring it to folks' attention since the failure appears to affect a great many packages (I noticed it in an ffmpeg build failure on my local cuirass instance).
> > 
> > Cheers,
> > Kaelyn