bug#64369: Request for merging "mesa-updates" branch

bug#64369: Request for merging "mesa-updates" branch

[John Kehayias via Bug reports for GNU Guix]

Hello all,

This is a request to merge the recently created "mesa-updates" branch. Currently there are just 2 patches on there, fixing/updating mesa only. The main thing to see is how substitute building goes in case anything breaks, but I'm hoping there isn't anything caused by this update.

I believe the "ruby-team" and "tex-team-next" [1] are ahead in the queue, not sure the timing of where those are. In addition/alternatively, would it make sense to have this branch as a separate build job on Cuirass directly as the "kernel-updates" branch? This would need a build roughly every month or so when mesa puts out a new update, we check for breakages, and then merge to master with substitutes available already.

I wasn't sure if this needs formal blockers in debbugs for the other branch merge requests, let me know!

Thanks,
John

[0] <https://issues.guix.gnu.org/63713>

[1] <https://issues.guix.gnu.org/64324>

[bug#64369] Request for merging "mesa-updates" branch

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1476 bytes --]


John Kehayias via Bug reports for GNU Guix <bug-guix@gnu.org> writes:

> This is a request to merge the recently created "mesa-updates"
> branch. Currently there are just 2 patches on there, fixing/updating
> mesa only. The main thing to see is how substitute building goes in
> case anything breaks, but I'm hoping there isn't anything caused by
> this update.
>
> I believe the "ruby-team" and "tex-team-next" [1] are ahead in the
> queue, not sure the timing of where those are. In
> addition/alternatively, would it make sense to have this branch as a
> separate build job on Cuirass directly as the "kernel-updates" branch?
> This would need a build roughly every month or so when mesa puts out a
> new update, we check for breakages, and then merge to master with
> substitutes available already.

I think ruby-team should be merged in the next few days. There's quite a
few changes in tex-team-next so that might take a little longer.

QA should start building the branch automatically when ruby-team is
merged, and I've created a specification on ci.guix.gnu.org for
mesa-updates now.

> I wasn't sure if this needs formal blockers in debbugs for the other
> branch merge requests, let me know!

I've gone ahead and reassigned this issue to guix-patches, rather than
the guix package. It's a very minor distinction, but I think
guix-patches is the right place for this issue.

As for the blocking things, I don't think that's necessary at the
moment.

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

[bug#64369] Request for merging "mesa-updates" branch

[John Kehayias via Guix-patches via]

Hi Chris,

On Sat, Jul 01, 2023 at 11:36 AM, Christopher Baines wrote:
>
> John Kehayias via Bug reports for GNU Guix <bug-guix@gnu.org> writes:
>
>> This is a request to merge the recently created "mesa-updates"
>> branch. Currently there are just 2 patches on there, fixing/updating
>> mesa only. The main thing to see is how substitute building goes in
>> case anything breaks, but I'm hoping there isn't anything caused by
>> this update.
>>
>> I believe the "ruby-team" and "tex-team-next" [1] are ahead in the
>> queue, not sure the timing of where those are. In
>> addition/alternatively, would it make sense to have this branch as a
>> separate build job on Cuirass directly as the "kernel-updates" branch?
>> This would need a build roughly every month or so when mesa puts out a
>> new update, we check for breakages, and then merge to master with
>> substitutes available already.
>
> I think ruby-team should be merged in the next few days. There's quite a
> few changes in tex-team-next so that might take a little longer.
>
> QA should start building the branch automatically when ruby-team is
> merged, and I've created a specification on ci.guix.gnu.org for
> mesa-updates now.
>

Thanks! I've seen it build and looks like good coverage on x86_64 and
i686 at least in raw weather number, though I'm not sure on the QA
branch comparison page it is showing me the correct info. For example,
this shows nothing as broken or still working:
<https://data.qa.guix.gnu.org/compare/package-derivations?base_commit=2b25bc03a11e1c6a473bbb000c35e94233120346&target_commit=7013921be6f9f2de49d5806a93eaee1d95cac1c6&system=x86_64-linux&target=none&build_change=still-working&after_name=&limit_results=40>

Or is that because Bordeaux hasn't built the branch?

So I'm not sure if anything much broke, but at least a good number of
substitutes are built for x86_64/i686.

>> I wasn't sure if this needs formal blockers in debbugs for the other
>> branch merge requests, let me know!
>
> I've gone ahead and reassigned this issue to guix-patches, rather than
> the guix package. It's a very minor distinction, but I think
> guix-patches is the right place for this issue.
>
> As for the blocking things, I don't think that's necessary at the
> moment.
>

Sounds good, thanks!

John

[bug#64369] Request for merging "mesa-updates" branch

[John Kehayias via Guix-patches via]

Hi,

On Thu, Jul 20, 2023 at 06:58 PM, 宋文武 wrote:

>>
>>     gnu: mesa: Enable zstd compression for shader cache.
>>
>>     * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib.
>>     [arguments]: Add '-Dzstd=enabled' to configure-flags.
>
> Pushed to the mesa-updates branch, merging progress tracked in
> https://issues.guix.gnu.org/64369.
>
> Thanks.

I just saw this as the message only went to the bug number. I would not
have merged this yet without checking about build status after the big
texlive updates. I was hoping to avoid a full rebuild so we could merge
to master with substitutes already, after checking post-texlive and the
graft of mesa on master.

However, doesn't seem like Cuirass has picked up this change and hasn't
built yet. Any idea why?

My plan right now was going to be to merge master into mesa-updates, see
what the build status looks like, and then decide from there. If there
were not too many rebuilds (say less than 1000) from the updates on
master, we should be able to go ahead soon. And then start on the next
round of patches/updates for mesa-updates.

On the other hand, if there were lots of rebuilds there are other
patches waiting that should be used also (libdrm, libva if I remember)
as well as ungrafting the change on master. So, preventing waiting and
doing another big rebuild with the changes that have been waiting since
the first build of this branch.

What do people think? My thought right now was to revert this change
(since it rebuilds all of mesa dependents), merge master in, and check
the status. But I'm confused why Cuirass wasn't building..

John

[bug#64369] Request for merging "mesa-updates" branch

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 2314 bytes --]


John Kehayias <john.kehayias@protonmail.com> writes:

> Hi,
>
> On Thu, Jul 20, 2023 at 06:58 PM, 宋文武 wrote:
>
>>>
>>>     gnu: mesa: Enable zstd compression for shader cache.
>>>
>>>     * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib.
>>>     [arguments]: Add '-Dzstd=enabled' to configure-flags.
>>
>> Pushed to the mesa-updates branch, merging progress tracked in
>> https://issues.guix.gnu.org/64369.
>>
>> Thanks.
>
> I just saw this as the message only went to the bug number. I would not
> have merged this yet without checking about build status after the big
> texlive updates. I was hoping to avoid a full rebuild so we could merge
> to master with substitutes already, after checking post-texlive and the
> graft of mesa on master.
>
> However, doesn't seem like Cuirass has picked up this change and hasn't
> built yet. Any idea why?

Cuirass at ci.guix.gnu.org seems to be having problems noticing new
revisions. At least it's about a day behind for master branch revisions.

> My plan right now was going to be to merge master into mesa-updates, see
> what the build status looks like, and then decide from there. If there
> were not too many rebuilds (say less than 1000) from the updates on
> master, we should be able to go ahead soon. And then start on the next
> round of patches/updates for mesa-updates.
>
> On the other hand, if there were lots of rebuilds there are other
> patches waiting that should be used also (libdrm, libva if I remember)
> as well as ungrafting the change on master. So, preventing waiting and
> doing another big rebuild with the changes that have been waiting since
> the first build of this branch.
>
> What do people think? My thought right now was to revert this change
> (since it rebuilds all of mesa dependents), merge master in, and check
> the status. But I'm confused why Cuirass wasn't building.

The mesa package wasn't affected by tex-team-next, so unless there are
other inputs to most/all off the packages to which mesa is an input,
there shouldn't be many rebuilds. I'd still rebase or merge master in to
the branch and see what QA says though.

Whether you want to revert this latest commit and try and merge sooner,
or wait for things to be built and merge later is up to you though.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

[bug#64369] Request for merging "mesa-updates" branch

[John Kehayias via Guix-patches via]

Hi Chris,

On Thu, Jul 20, 2023 at 05:03 PM, Christopher Baines wrote:

>
> John Kehayias <john.kehayias@protonmail.com> writes:
>> However, doesn't seem like Cuirass has picked up this change and hasn't
>> built yet. Any idea why?
>
> Cuirass at ci.guix.gnu.org seems to be having problems noticing new
> revisions. At least it's about a day behind for master branch revisions.
>

Okay, thanks for the info. So either way will let things build through
the weekend probably.

>> My plan right now was going to be to merge master into mesa-updates, see
>> what the build status looks like, and then decide from there. If there
>> were not too many rebuilds (say less than 1000) from the updates on
>> master, we should be able to go ahead soon. And then start on the next
>> round of patches/updates for mesa-updates.
>>
>> On the other hand, if there were lots of rebuilds there are other
>> patches waiting that should be used also (libdrm, libva if I remember)
>> as well as ungrafting the change on master. So, preventing waiting and
>> doing another big rebuild with the changes that have been waiting since
>> the first build of this branch.
>>
>> What do people think? My thought right now was to revert this change
>> (since it rebuilds all of mesa dependents), merge master in, and check
>> the status. But I'm confused why Cuirass wasn't building.
>
> The mesa package wasn't affected by tex-team-next, so unless there are
> other inputs to most/all off the packages to which mesa is an input,
> there shouldn't be many rebuilds. I'd still rebase or merge master in to
> the branch and see what QA says though.
>

Right, wasn't sure what the number of rebuilds the texlive updates
caused beyond tex-related stuff, so I didn't want to assume.

Are we able to rebase and then force push on savannah? I thought not,
or is that just for the master branch? If I can force push (I suppose
I could just try later), I would leave off 64738 (zstd in mesa) and
rebase on master. Otherwise I guess revert and merge master to
mesa-updates.

> Whether you want to revert this latest commit and try and merge sooner,
> or wait for things to be built and merge later is up to you though.
>

Then either way see what rebuilding looks like. If there isn't much
I'd opt for going sooner while we have substitutes already and then
gather these other patches for the next merge. Likely waiting for a
new mesa update which is probably due very soon. But if things will be
rebuilding a lot anyway, I might as well get all these other related
changes in to just build once.

Thanks Chris, I'll see what to do and keep an eye on the builds.

John

[bug#64369] Request for merging "mesa-updates" branch

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 466 bytes --]


John Kehayias <john.kehayias@protonmail.com> writes:

> Are we able to rebase and then force push on savannah? I thought not,
> or is that just for the master branch? If I can force push (I suppose
> I could just try later), I would leave off 64738 (zstd in mesa) and
> rebase on master. Otherwise I guess revert and merge master to
> mesa-updates.

Not directly, but you can delete the branch and then just push as if it
is a new branch to effectively force push.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

[bug#64369] Request for merging "mesa-updates" branch

[John Kehayias via Guix-patches via]

On Thu, Jul 20, 2023 at 06:04 PM, Christopher Baines wrote:

>
> John Kehayias <john.kehayias@protonmail.com> writes:
>
>> Are we able to rebase and then force push on savannah? I thought not,
>> or is that just for the master branch? If I can force push (I suppose
>> I could just try later), I would leave off 64738 (zstd in mesa) and
>> rebase on master. Otherwise I guess revert and merge master to
>> mesa-updates.
>
> Not directly, but you can delete the branch and then just push as if it
> is a new branch to effectively force push.
>

Gotcha. That's what I did to rebase on master as of now. Let's see
what happens with the build and go from there.

Thanks!

[bug#64369] Request for merging "mesa-updates" branch

[John Kehayias via Guix-patches via]

On Thu, Jul 20, 2023 at 03:23 PM, John Kehayias wrote:

> On Thu, Jul 20, 2023 at 06:04 PM, Christopher Baines wrote:
>
>>
>> John Kehayias <john.kehayias@protonmail.com> writes:
>>
>>> Are we able to rebase and then force push on savannah? I thought not,
>>> or is that just for the master branch? If I can force push (I suppose
>>> I could just try later), I would leave off 64738 (zstd in mesa) and
>>> rebase on master. Otherwise I guess revert and merge master to
>>> mesa-updates.
>>
>> Not directly, but you can delete the branch and then just push as if it
>> is a new branch to effectively force push.
>>
>
> Gotcha. That's what I did to rebase on master as of now. Let's see
> what happens with the build and go from there.
>
> Thanks!

I'm not sure what is going on with the CI and QA.
<https://ci.guix.gnu.org/> hasn't shown any new evaluations in several
days (the workers were stuck at some point too, but nckx gave it a
poke and now I don't see any builds left). But when I check with guix
weather I do get substitutes. Any ideas what the actual status is?

Thanks,
John

[bug#64369] Request for merging "mesa-updates" branch

[John Kehayias via Guix-patches via]

Hello,

On Mon, Jul 24, 2023 at 03:14 PM, John Kehayias wrote:

> I'm not sure what is going on with the CI and QA.
> <https://ci.guix.gnu.org/> hasn't shown any new evaluations in several
> days (the workers were stuck at some point too, but nckx gave it a
> poke and now I don't see any builds left). But when I check with guix
> weather I do get substitutes. Any ideas what the actual status is?
>

Things got unstuck recently at ci.guix.gnu.org was churning away. I saw
some failed builds due to what looks like some build farm download
issue, e.g. <https://ci.guix.gnu.org/build/1671444/details> which build
fine locally. I think this is what blocked a bunch of KDE-related
packages as well. A bunch of java packages also failed but the ones I
tried also built locally (some had long build logs but I didn't see the
cause).

So I think x86 is okay, while e.g. aarch64 is slowly chipping away.

If that's okay then I'll cherry pick the commits to master today or
tomorrow.

Thanks for your help Chris!

bug#64369: Request for merging "mesa-updates" branch

[John Kehayias via Guix-patches via]

On Fri, Jul 28, 2023 at 03:29 PM, John Kehayias wrote:

> Hello,
>
> On Mon, Jul 24, 2023 at 03:14 PM, John Kehayias wrote:
>
>> I'm not sure what is going on with the CI and QA.
>> <https://ci.guix.gnu.org/> hasn't shown any new evaluations in several
>> days (the workers were stuck at some point too, but nckx gave it a
>> poke and now I don't see any builds left). But when I check with guix
>> weather I do get substitutes. Any ideas what the actual status is?
>>
>
> Things got unstuck recently at ci.guix.gnu.org was churning away. I saw
> some failed builds due to what looks like some build farm download
> issue, e.g. <https://ci.guix.gnu.org/build/1671444/details> which build
> fine locally. I think this is what blocked a bunch of KDE-related
> packages as well. A bunch of java packages also failed but the ones I
> tried also built locally (some had long build logs but I didn't see the
> cause).
>
> So I think x86 is okay, while e.g. aarch64 is slowly chipping away.
>
> If that's okay then I'll cherry pick the commits to master today or
> tomorrow.
>
> Thanks for your help Chris!

Pushed to master with 4f0ce65b74a3d28bf6ecbe4c15052dd0de22b284 the final
commit from that branch.

Thanks!

(I'll delete the remote branch for now, until needed again.)