unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Should commits rather be buildable or small
@ 2023-12-08  8:42 Saku Laesvuori
  2023-12-08 11:41 ` Tomas Volf
                   ` (2 more replies)
  0 siblings, 3 replies; 19+ messages in thread
From: Saku Laesvuori @ 2023-12-08  8:42 UTC (permalink / raw)
  To: guix-devel

[-- Attachment #1: Type: text/plain, Size: 561 bytes --]

Hi,

I'm planning on refreshing Guix's haskell packages as my fix for
https://issues.guix.gnu.org/66347 requires rebuilding all of them
anyway. Should I try to keep commits small with only one update per
commit (which is more work but managable if I don't care about the
commits being buildable) or should I try to keep them buildable (i.e.
update everything in one commit)? It is quite certain that most of them
will not build after updating ghc or a subset of their dependencies, so
making many small commits would cause nearly all of them to be
unbuildable.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2023-12-08  8:42 Should commits rather be buildable or small Saku Laesvuori
@ 2023-12-08 11:41 ` Tomas Volf
  2023-12-08 12:05 ` Lars-Dominik Braun
  2023-12-08 15:44 ` Liliana Marie Prikler
  2 siblings, 0 replies; 19+ messages in thread
From: Tomas Volf @ 2023-12-08 11:41 UTC (permalink / raw)
  To: Saku Laesvuori; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 910 bytes --]

On 2023-12-08 10:42:07 +0200, Saku Laesvuori wrote:
> Hi,
>
> I'm planning on refreshing Guix's haskell packages as my fix for
> https://issues.guix.gnu.org/66347 requires rebuilding all of them
> anyway. Should I try to keep commits small with only one update per
> commit (which is more work but managable if I don't care about the
> commits being buildable) or should I try to keep them buildable (i.e.
> update everything in one commit)? It is quite certain that most of them
> will not build after updating ghc or a subset of their dependencies, so
> making many small commits would cause nearly all of them to be
> unbuildable.

If for not other reason then to make git history bisectable, I think each commit
should be buildable.  But this is just a opinion from the peanut gallery.

Tomas

--
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2023-12-08  8:42 Should commits rather be buildable or small Saku Laesvuori
  2023-12-08 11:41 ` Tomas Volf
@ 2023-12-08 12:05 ` Lars-Dominik Braun
  2023-12-08 16:35   ` Felix Lechner via Development of GNU Guix and the GNU System distribution.
  2023-12-08 15:44 ` Liliana Marie Prikler
  2 siblings, 1 reply; 19+ messages in thread
From: Lars-Dominik Braun @ 2023-12-08 12:05 UTC (permalink / raw)
  To: Saku Laesvuori; +Cc: guix-devel

Hi,

> I'm planning on refreshing Guix's haskell packages as my fix for
> https://issues.guix.gnu.org/66347 requires rebuilding all of them
> anyway. Should I try to keep commits small with only one update per
> commit (which is more work but managable if I don't care about the
> commits being buildable) or should I try to keep them buildable (i.e.
> update everything in one commit)?

so far I’ve been updating Haskell packages in bulk in a
single commit. See 49a320aaa6fb4c20d6b30c56c35a8c7ffceed822 or
b97f549b14402421fcfb360ddd4cff7de93b9af0 for example. I also used custom
scripts last time, because `guix refresh` was not sufficient to update
all fields required (arguments, inputs, …). This is hopefully different
this time.

Cheers,
Lars


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2023-12-08  8:42 Should commits rather be buildable or small Saku Laesvuori
  2023-12-08 11:41 ` Tomas Volf
  2023-12-08 12:05 ` Lars-Dominik Braun
@ 2023-12-08 15:44 ` Liliana Marie Prikler
  2023-12-10 15:28   ` Saku Laesvuori
  2 siblings, 1 reply; 19+ messages in thread
From: Liliana Marie Prikler @ 2023-12-08 15:44 UTC (permalink / raw)
  To: Saku Laesvuori, guix-devel

Hi Saku,

Am Freitag, dem 08.12.2023 um 10:42 +0200 schrieb Saku Laesvuori:
> Hi,
> 
> I'm planning on refreshing Guix's haskell packages as my fix for
> https://issues.guix.gnu.org/66347 requires rebuilding all of them
> anyway. Should I try to keep commits small with only one update per
> commit (which is more work but managable if I don't care about the
> commits being buildable) or should I try to keep them buildable (i.e.
> update everything in one commit)? It is quite certain that most of
> them will not build after updating ghc or a subset of their
> dependencies, so making many small commits would cause nearly all of
> them to be unbuildable.
Define "buildable" and "unbuildable".  Depending on the context, it may
be fine or even required to break dependant packages for a short while
and update them along a longer series.  However, in each commit at
least the package touched in that commit ought to build.

Cheers


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2023-12-08 12:05 ` Lars-Dominik Braun
@ 2023-12-08 16:35   ` Felix Lechner via Development of GNU Guix and the GNU System distribution.
  0 siblings, 0 replies; 19+ messages in thread
From: Felix Lechner via Development of GNU Guix and the GNU System distribution. @ 2023-12-08 16:35 UTC (permalink / raw)
  To: Lars-Dominik Braun, Saku Laesvuori; +Cc: guix-devel

Hi,

On Fri, Dec 08 2023, Lars-Dominik Braun wrote:

> I’ve been updating Haskell packages in bulk in a single commit.

Making logically related commits was always my custom before arriving at
Guix. I still do so in all of my own projects.

The preference at Guix, however, is to split even mundane changes into
the tiniest parts possible---each affecting ideally only a single file.

The custom seems to be driven by concerns for the GNU ChangeLog, which
is generated automatically from the Git commit messages.

Kind regards
Felix


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2023-12-08 15:44 ` Liliana Marie Prikler
@ 2023-12-10 15:28   ` Saku Laesvuori
  2023-12-10 15:50     ` Liliana Marie Prikler
  2023-12-10 17:02     ` Attila Lendvai
  0 siblings, 2 replies; 19+ messages in thread
From: Saku Laesvuori @ 2023-12-10 15:28 UTC (permalink / raw)
  To: Liliana Marie Prikler; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 1981 bytes --]

> Hi Saku,
> 
> Am Freitag, dem 08.12.2023 um 10:42 +0200 schrieb Saku Laesvuori:
> > Hi,
> > 
> > I'm planning on refreshing Guix's haskell packages as my fix for
> > https://issues.guix.gnu.org/66347 requires rebuilding all of them
> > anyway. Should I try to keep commits small with only one update per
> > commit (which is more work but managable if I don't care about the
> > commits being buildable) or should I try to keep them buildable (i.e.
> > update everything in one commit)? It is quite certain that most of
> > them will not build after updating ghc or a subset of their
> > dependencies, so making many small commits would cause nearly all of
> > them to be unbuildable.
>
> Define "buildable" and "unbuildable".

I used these definitions: a buildable commit does not have build
failures (or at least no new ones). An unbuildable commit introduces new build failures (in
this case a lot of them).

Buildable commits are safe spots to land on with time-machine in the
sense that the packages defined in them can be used. I expect it would
be very painful to try jumping to past commits with time-machine if a
large portion of the commits in Guix were unbuildable.

> Depending on the context, it may be fine or even required to break
> dependant packages for a short while and update them along a longer
> series.

I guess "required" here means that in some cases Guix's policy is to
prefer small commits over buildable commits (with the previous
definition). I at least don't see any technical reasons why it would be
required. The question then becomes whether that policy applies in this
case.

> However, in each commit at least the package touched in that
> commit ought to build.

This should, of course, be theoretically possible with at least one
update order but I don't know how would I discover that order (more
efficiently than by trial and error. I don't want to try ~800² different
combinations).

- Saku

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2023-12-10 15:28   ` Saku Laesvuori
@ 2023-12-10 15:50     ` Liliana Marie Prikler
  2023-12-10 17:02     ` Attila Lendvai
  1 sibling, 0 replies; 19+ messages in thread
From: Liliana Marie Prikler @ 2023-12-10 15:50 UTC (permalink / raw)
  To: Saku Laesvuori; +Cc: guix-devel

Am Sonntag, dem 10.12.2023 um 17:28 +0200 schrieb Saku Laesvuori:
> > Hi Saku,
> > 
> > Am Freitag, dem 08.12.2023 um 10:42 +0200 schrieb Saku Laesvuori:
> > > Hi,
> > > 
> > > I'm planning on refreshing Guix's haskell packages as my fix for
> > > https://issues.guix.gnu.org/66347 requires rebuilding all of them
> > > anyway. Should I try to keep commits small with only one update
> > > per commit (which is more work but managable if I don't care
> > > about the commits being buildable) or should I try to keep them
> > > buildable (i.e. update everything in one commit)? It is quite
> > > certain that most of them will not build after updating ghc or a
> > > subset of their dependencies, so making many small commits would
> > > cause nearly all of them to be unbuildable.
> > 
> > Define "buildable" and "unbuildable".
> 
> I used these definitions: a buildable commit does not have build
> failures (or at least no new ones). An unbuildable commit introduces
> new build failures (in this case a lot of them).
> 
> Buildable commits are safe spots to land on with time-machine in the
> sense that the packages defined in them can be used. I expect it
> would be very painful to try jumping to past commits with time-
> machine if a large portion of the commits in Guix were unbuildable.
Yeah, it's not really good commit etiquette to drop a bunch of world-
breaking builds on top of master.  We mostly use feature branches for
larger changes.  OTOH, if it rebuilds less than 300 packages, it really
is your call – the number of breakages is limited in that case too :)

> > Depending on the context, it may be fine or even required to break
> > dependant packages for a short while and update them along a longer
> > series.
> 
> I guess "required" here means that in some cases Guix's policy is to
> prefer small commits over buildable commits (with the previous
> definition). I at least don't see any technical reasons why it would
> be required. The question then becomes whether that policy applies in
> this case.
This is typically allowed when branching off, as few people will time-
machine into an intermediate commit on an off-master branch safe for
debugging some very arcane failures.  

> > However, in each commit at least the package touched in that
> > commit ought to build.
> 
> This should, of course, be theoretically possible with at least one
> update order but I don't know how would I discover that order (more
> efficiently than by trial and error. I don't want to try ~800²
> different combinations).
A reasonable way is to plan according to guix graph.  Tackle the low-
level nodes first and stack the high-level nodes on top.  If two
packages share immediate dependencies, but neither relies on the other,
then any order is fine between these two.

Cheers


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2023-12-10 15:28   ` Saku Laesvuori
  2023-12-10 15:50     ` Liliana Marie Prikler
@ 2023-12-10 17:02     ` Attila Lendvai
  2023-12-10 17:50       ` Ricardo Wurmus
  1 sibling, 1 reply; 19+ messages in thread
From: Attila Lendvai @ 2023-12-10 17:02 UTC (permalink / raw)
  To: Saku Laesvuori; +Cc: Liliana Marie Prikler, guix-devel

> > Define "buildable" and "unbuildable".
> 
> 
> I used these definitions: a buildable commit does not have build
> failures (or at least no new ones). An unbuildable commit introduces
> new build failures (in this case a lot of them).
> 
> Buildable commits are safe spots to land on with time-machine in the
> sense that the packages defined in them can be used. I expect it would
> be very painful to try jumping to past commits with time-machine if a
> large portion of the commits in Guix were unbuildable.

[...]

> I guess "required" here means that in some cases Guix's policy is to
> prefer small commits over buildable commits (with the previous
> definition). I at least don't see any technical reasons why it would be
> required. The question then becomes whether that policy applies in this
> case.


FWIW, this commit policy has always bothered me as a newcomer to Guix. pretty much everywhere else it's a major offence against your colleagues to commit something that breaks the build in any way.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“I will probably be asked why I don't cite the author's name? Because my philosophy teacher taught me that it sometimes jeopardizes the effects of the quote.”
	— Author's name withheld.



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2023-12-10 17:02     ` Attila Lendvai
@ 2023-12-10 17:50       ` Ricardo Wurmus
  2023-12-10 23:20         ` Attila Lendvai
  0 siblings, 1 reply; 19+ messages in thread
From: Ricardo Wurmus @ 2023-12-10 17:50 UTC (permalink / raw)
  To: Attila Lendvai; +Cc: Saku Laesvuori, Liliana Marie Prikler, guix-devel


Attila Lendvai <attila@lendvai.name> writes:

>> I guess "required" here means that in some cases Guix's policy is to
>> prefer small commits over buildable commits (with the previous
>> definition). I at least don't see any technical reasons why it would be
>> required. The question then becomes whether that policy applies in this
>> case.
>
>
> FWIW, this commit policy has always bothered me as a newcomer to
> Guix. pretty much everywhere else it's a major offence against your
> colleagues to commit something that breaks the build in any way.

In the last few months I’ve repeatedly seen assertions in a similar
style as this one.  They always genuinely surprise me, and it’s probably
not just because I’m oblivious and out of touch.

Also in Guix it is not okay to commit intermediate things that break
stuff.

Commits should, however, also tell a reviewable story and not be a big
blob of thousands of lines of non-trivial changes.  (The one exception
has always been fixes of repeated typos, but this is not what this
discussion is about.)

For what it’s worth: when I do bulk R upgrades I generally have one
commit per package upgrade, because it’s easy to review in a git log in
the future and easy to revert individual changes with fine grain access.
Of course I group upgrades that depend on one another (e.g. r-arrow
together with apache-arrow and the inheriting python-arrow).

-- 
Ricardo


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2023-12-10 17:50       ` Ricardo Wurmus
@ 2023-12-10 23:20         ` Attila Lendvai
  2023-12-10 23:56           ` Philip McGrath
  0 siblings, 1 reply; 19+ messages in thread
From: Attila Lendvai @ 2023-12-10 23:20 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: Saku Laesvuori, Liliana Marie Prikler, guix-devel

> > FWIW, this commit policy has always bothered me as a newcomer to
> > Guix. pretty much everywhere else it's a major offence against your
> > colleagues to commit something that breaks the build in any way.
> 
> 
> In the last few months I’ve repeatedly seen assertions in a similar
> style as this one. They always genuinely surprise me, and it’s probably
> not just because I’m oblivious and out of touch.


well, both point of views are reasonable. they just make different tradeoffs.

i think an abstraction is missing here, let's call it guix log for this mail. it's something like the git log, but one that lists the buildable and substitutable states of the guix repo.

it's probably the same thing that causes the discrepancy between git commits and substitutes: the build servers are not building every commit of the git repo. they pick an unpredictable (?) series of commits, skipping some inbetween. if i guix pull, or guix time-machine to the "wrong" commit, then i'll need to build some stuff locally. sometimes these can be heavy packages.

this hypothetical 'guix log' is probably also what's missing between a hypothetical staging branch and master, whose role would be to make sure that commits don't reach the users prior to having substitutes for them.

does this make sense?

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“Sometimes I wonder whether the world is being run by smart people who are putting us on or by imbeciles who really mean it.”
	— Mark Twain (1835-1910)



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2023-12-10 23:20         ` Attila Lendvai
@ 2023-12-10 23:56           ` Philip McGrath
  2023-12-11 10:51             ` Attila Lendvai
  0 siblings, 1 reply; 19+ messages in thread
From: Philip McGrath @ 2023-12-10 23:56 UTC (permalink / raw)
  To: Attila Lendvai, Ricardo Wurmus
  Cc: Saku Laesvuori, Liliana Marie Prikler, guix-devel

On 12/10/23 18:20, Attila Lendvai wrote:
>>> FWIW, this commit policy has always bothered me as a newcomer to
>>> Guix. pretty much everywhere else it's a major offence against your
>>> colleagues to commit something that breaks the build in any way.
>>
>>
>> In the last few months I’ve repeatedly seen assertions in a similar
>> style as this one. They always genuinely surprise me, and it’s probably
>> not just because I’m oblivious and out of touch.
> 
> 
> well, both point of views are reasonable. they just make different tradeoffs.
> 

I find it hard to see any benefit to anyone from making commits so small 
that they are known to break things that will be fixed later in the same 
series. Even aside from `guix time-machine`, substitute building, and 
the like, a human reading the diff won't be able to see what the true 
impact of the change is.

On the concrete issue:


On 12/10/23 10:28, Saku Laesvuori wrote:
 >> However, in each commit at least the package touched in that
 >> commit ought to build.
 > This should, of course, be theoretically possible with at least one
 > update order but I don't know how would I discover that order (more
 > efficiently than by trial and error. I don't want to try ~800² different
 > combinations).

Preparing a large set of updates like this is already a great deal of 
work. It does not seem to me like a good use of volunteers' time to ask 
them to break such an update into hundreds of tiny pieces, especially 
not if the result is hundreds of broken commits to Guix.

Philip


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2023-12-10 23:56           ` Philip McGrath
@ 2023-12-11 10:51             ` Attila Lendvai
  2023-12-11 11:51               ` Ricardo Wurmus
  0 siblings, 1 reply; 19+ messages in thread
From: Attila Lendvai @ 2023-12-11 10:51 UTC (permalink / raw)
  To: Philip McGrath
  Cc: Ricardo Wurmus, Saku Laesvuori, Liliana Marie Prikler, guix-devel

> Preparing a large set of updates like this is already a great deal of
> work. It does not seem to me like a good use of volunteers' time to ask
> them to break such an update into hundreds of tiny pieces, especially
> not if the result is hundreds of broken commits to Guix.


fair enough. in that paragraph i did not consider the consts, only the benefits of the two approaches.

i myself also had headaches multiple times when i fixed something that needed to touch several different packages, and they would only work when applied in one transaction:

how many debbugs issues? multiple issues and record the dependencies? little gain for much more effort on both sides... but if one issue, then what should be the name of the debbugs issue? etc...

the contribution process has quite some accidental complexity, and it most probably turns away valuable potential contributors... which is something that is both hard to notice, and has a strong impact. but this has already been discussed in a long thread recently.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“As long as habit and routine dictate the pattern of living, new dimensions of the soul will not emerge.”
	— Henry Van Dyke (1852–1933)



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2023-12-11 10:51             ` Attila Lendvai
@ 2023-12-11 11:51               ` Ricardo Wurmus
  2024-03-04 21:38                 ` John Kehayias
  0 siblings, 1 reply; 19+ messages in thread
From: Ricardo Wurmus @ 2023-12-11 11:51 UTC (permalink / raw)
  To: Attila Lendvai
  Cc: Philip McGrath, Saku Laesvuori, Liliana Marie Prikler, guix-devel


Attila Lendvai <attila@lendvai.name> writes:

> i myself also had headaches multiple times when i fixed something that
> needed to touch several different packages, and they would only work
> when applied in one transaction:
>
> how many debbugs issues? multiple issues and record the dependencies?
> little gain for much more effort on both sides... but if one issue,
> then what should be the name of the debbugs issue? etc...

We often have just one debbugs issue per transaction, i.e. multiple
patches that all belong to the same issue.  The free form cover letter
should describe what the relation between the patches is.

This can be done for dependencies (e.g. an R package and all the
unpackaged things it needs) or for a feature with several related but
atomically recorded changes.

A misconception I read earlier is that there’s a desire to have one
commit per touched file, but that’s certainly not true.  Atomicity
doesn’t care about file boundaries, and we don’t care about them either.
Commits should tell a story and often it is easier to tell it by making
consecutive independent changes.

-- 
Ricardo


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2023-12-11 11:51               ` Ricardo Wurmus
@ 2024-03-04 21:38                 ` John Kehayias
  2024-03-05  4:32                   ` dan
  2024-03-05  5:19                   ` Liliana Marie Prikler
  0 siblings, 2 replies; 19+ messages in thread
From: John Kehayias @ 2024-03-04 21:38 UTC (permalink / raw)
  To: Ricardo Wurmus
  Cc: Attila Lendvai, Philip McGrath, Saku Laesvuori,
	Liliana Marie Prikler, guix-devel, dan, 69461

Hi everyone,

And sorry for reviving an old thread, but I am faced with a similar issue for updating vulkan, with the patch series submitted by dan (cc'ed): <https://issues.guix.gnu.org/69461>. I thought I would get some opinions here, please see below:

On Mon, Dec 11, 2023 at 12:51 PM, Ricardo Wurmus wrote:

> Attila Lendvai <attila@lendvai.name> writes:
>
>> i myself also had headaches multiple times when i fixed something that
>> needed to touch several different packages, and they would only work
>> when applied in one transaction:
>>

In this case all the vulkan packages share a version through a variable name. I would assume packages wouldn't like mixed versions, but maybe some would work (I haven't tried). I'll be taking this series on mesa-updates with related changes, so the plan is that when it hits master there are no/few broken packages and full substitute coverage. So perhaps this makes this more of a style and convention question.

Some options:

1. Essentially squash to one commit where all of vulkan is updated in one commit. The main upside is that nothing should break (within vulkan, dependents to be fixed as needed) and it shows as "one" change; the main downside is that the proposed changes are not just trivial version bumps. Harder to then disentangle as needed.

2. Make each commit updating a package, but don't use the variable %vulkan-sdk-version, updating each package with a version as it is done. Then do a commit where all the versions are replaced by the variable. This seems like unnecessary work to me and while it stops the obvious breaking (source hashes don't match once variable is updated but package hasn't yet) versions are still mixed which is likely a problem.

3. Go with the series as proposed: this means after the first commit for sure all other vulkan packages and dependents don't build, as the source hashes won't match until the commit that updates that package. Along with version mixing, this perhaps doesn't give you a helpful git bisect either?

None are perfect. What do people think?

My instinct is to go with the series as proposed (after review) accepting that there will be for sure builds failing if time traveling to the middle of the series. I don't think we can really avoid that anyway, as sometimes we only see an issue after a commit and it is fixed some time later. We could have a note in the first commit that this requires the next n commits to update vulkan packages. That might help if someone is on an intermediate commit and can see quickly in git log this note.

Or perhaps we can note something is part of a dependent series when we make commits so this is easier for someone to tell in general?

Thanks!
John



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2024-03-04 21:38                 ` John Kehayias
@ 2024-03-05  4:32                   ` dan
  2024-03-05  5:19                   ` Liliana Marie Prikler
  1 sibling, 0 replies; 19+ messages in thread
From: dan @ 2024-03-05  4:32 UTC (permalink / raw)
  To: John Kehayias, Ricardo Wurmus
  Cc: Attila Lendvai, Philip McGrath, Saku Laesvuori,
	Liliana Marie Prikler, guix-devel, 69461

Hi John,

On 3/5/2024 5:38 AM, John Kehayias wrote:
> In this case all the vulkan packages share a version through a variable name. I would assume packages wouldn't like mixed versions, but maybe some would work (I haven't tried). I'll be taking this series on mesa-updates with related changes, so the plan is that when it hits master there are no/few broken packages and full substitute coverage. So perhaps this makes this more of a style and convention question.
> 
> Some options:
> 
> 1. Essentially squash to one commit where all of vulkan is updated in one commit. The main upside is that nothing should break (within vulkan, dependents to be fixed as needed) and it shows as "one" change; the main downside is that the proposed changes are not just trivial version bumps. Harder to then disentangle as needed.
> 
> 2. Make each commit updating a package, but don't use the variable %vulkan-sdk-version, updating each package with a version as it is done. Then do a commit where all the versions are replaced by the variable. This seems like unnecessary work to me and while it stops the obvious breaking (source hashes don't match once variable is updated but package hasn't yet) versions are still mixed which is likely a problem.
> 
> 3. Go with the series as proposed: this means after the first commit for sure all other vulkan packages and dependents don't build, as the source hashes won't match until the commit that updates that package. Along with version mixing, this perhaps doesn't give you a helpful git bisect either?
> 
> None are perfect. What do people think?
> 
> My instinct is to go with the series as proposed (after review) accepting that there will be for sure builds failing if time traveling to the middle of the series. I don't think we can really avoid that anyway, as sometimes we only see an issue after a commit and it is fixed some time later. We could have a note in the first commit that this requires the next n commits to update vulkan packages. That might help if someone is on an intermediate commit and can see quickly in git log this note.
> 
> Or perhaps we can note something is part of a dependent series when we make commits so this is easier for someone to tell in general?

I think to make each commit able to build, it's feasible to remove this 
%vulkan-sdk-version variable. However, this doesn't fundamentally solve 
the problem: when updating several packages in a patch series, some 
packages might be broken since their dependencies are updated.

Another question is how should we treat vulkan packages. Some distros 
package them on a per package basis (I see in Arch Linux, vulkan-headers 
and vulkan-icd-loaders have version 1.3.276 while other packages like 
spirv-headers has 1.3.275). I had to admit that I'm not that familiar 
with vulkan packages, but I feel it's safer to keep their version 
matched since each vulkan-sdk release makes sure every vulkan packages 
are compatible with others. Thus, I prefer updating them in batch.

I think maybe it's a good option that we mark these commits are a series.

-- 
dan



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2024-03-04 21:38                 ` John Kehayias
  2024-03-05  4:32                   ` dan
@ 2024-03-05  5:19                   ` Liliana Marie Prikler
  2024-03-25  1:15                     ` John Kehayias
  1 sibling, 1 reply; 19+ messages in thread
From: Liliana Marie Prikler @ 2024-03-05  5:19 UTC (permalink / raw)
  To: John Kehayias, Ricardo Wurmus
  Cc: Attila Lendvai, Philip McGrath, Saku Laesvuori, guix-devel, dan,
	69461

Hi,

Am Montag, dem 04.03.2024 um 21:38 +0000 schrieb John Kehayias:
> [...]
> 1. Essentially squash to one commit where all of vulkan is updated in
> one commit. The main upside is that nothing should break (within
> vulkan, dependents to be fixed as needed) and it shows as "one"
> change; the main downside is that the proposed changes are not just
> trivial version bumps. Harder to then disentangle as needed.
> 
> 2. Make each commit updating a package, but don't use the variable
> %vulkan-sdk-version, updating each package with a version as it is
> done. Then do a commit where all the versions are replaced by the
> variable. This seems like unnecessary work to me and while it stops
> the obvious breaking (source hashes don't match once variable is
> updated but package hasn't yet) versions are still mixed which is
> likely a problem.
> 
> 3. Go with the series as proposed: this means after the first commit
> for sure all other vulkan packages and dependents don't build, as the
> source hashes won't match until the commit that updates that package.
> Along with version mixing, this perhaps doesn't give you a helpful
> git bisect either?
> 
> None are perfect. What do people think?
I think 1 would be workable if the changes to the packages are minimal.
You should also check whether you can just do the version bumps and
then the other changes – or flip the order.

I don't really see the benefit with 2.  Normally, we'd have "-next"
variants to catch nontrivial updates (among other things), but those
don't seem a good approach here.

If nothing else works, 3 is indeed an option to fall back to, albeit
begrudgingly.  As noted for 1, you could check whether bumping all the
hashes and then only fixing whatever else for the builds is an option
here.

Alternative 4 would be to build those -next variants and then replace
the base vulkan all at once.  This has the advantage of not doing any
version mixing in-between IIUC.


Cheers


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2024-03-05  5:19                   ` Liliana Marie Prikler
@ 2024-03-25  1:15                     ` John Kehayias
  2024-03-25  3:23                       ` dan
  0 siblings, 1 reply; 19+ messages in thread
From: John Kehayias @ 2024-03-25  1:15 UTC (permalink / raw)
  To: Liliana Marie Prikler, dan
  Cc: Ricardo Wurmus, Attila Lendvai, Philip McGrath, Saku Laesvuori,
	guix-devel, 69461

Hi,

Apologies for the delay. I would like to get things rolling on
mesa-updates and building, including the vulkan updates, so a choice
will have to be made :)

Thanks for the input so far!

On Tue, Mar 05, 2024 at 06:19 AM, Liliana Marie Prikler wrote:

> Hi,
>
> Am Montag, dem 04.03.2024 um 21:38 +0000 schrieb John Kehayias:
>> [...]
>> 1. Essentially squash to one commit where all of vulkan is updated in
>> one commit. The main upside is that nothing should break (within
>> vulkan, dependents to be fixed as needed) and it shows as "one"
>> change; the main downside is that the proposed changes are not just
>> trivial version bumps. Harder to then disentangle as needed.
>>
>> 2. Make each commit updating a package, but don't use the variable
>> %vulkan-sdk-version, updating each package with a version as it is
>> done. Then do a commit where all the versions are replaced by the
>> variable. This seems like unnecessary work to me and while it stops
>> the obvious breaking (source hashes don't match once variable is
>> updated but package hasn't yet) versions are still mixed which is
>> likely a problem.
>>
>> 3. Go with the series as proposed: this means after the first commit
>> for sure all other vulkan packages and dependents don't build, as the
>> source hashes won't match until the commit that updates that package.
>> Along with version mixing, this perhaps doesn't give you a helpful
>> git bisect either?
>>
>> None are perfect. What do people think?
> I think 1 would be workable if the changes to the packages are minimal.
> You should also check whether you can just do the version bumps and
> then the other changes – or flip the order.
>

As currently proposed, the changes are not minimal.

dan: Do you know if just a version/hash bump is at least buildable? Or
are the changes necessary for the packages to build/function at all?
Or I guess if the non-version changes are applicable to the current
version first?

> I don't really see the benefit with 2.  Normally, we'd have "-next"
> variants to catch nontrivial updates (among other things), but those
> don't seem a good approach here.
>
> If nothing else works, 3 is indeed an option to fall back to, albeit
> begrudgingly.  As noted for 1, you could check whether bumping all the
> hashes and then only fixing whatever else for the builds is an option
> here.
>

That's what I'll have to do I think, unless indeed the versions
changes can be made separately and still build. I can mark each patch
in the commit log that it is part of a series updating all the vulkan
packages. That might be something worth doing in general for cases
like this, to help out future time travelers and when e.g. searching
the log and finding a commit.

> Alternative 4 would be to build those -next variants and then replace
> the base vulkan all at once.  This has the advantage of not doing any
> version mixing in-between IIUC.
>

That's also an idea. Add a %vulkan-version-next or something like
that, and -next variants of all the packages using that version
instead. A bit clumsy and perhaps convoluted with the extra work for
maybe minimal gain.

I'll wait to see if dan has any information of what changes can be
made independently, but I guess I'll just have to make a decision on
mesa-updates.

Thanks!

John



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Should commits rather be buildable or small
  2024-03-25  1:15                     ` John Kehayias
@ 2024-03-25  3:23                       ` dan
  2024-03-25  3:23                         ` [bug#69461] " dan
  0 siblings, 1 reply; 19+ messages in thread
From: dan @ 2024-03-25  3:23 UTC (permalink / raw)
  To: John Kehayias, Liliana Marie Prikler
  Cc: Ricardo Wurmus, Attila Lendvai, Philip McGrath, Saku Laesvuori,
	guix-devel, 69461

Hi John,

On 3/25/2024 9:15 AM, John Kehayias wrote:
> dan: Do you know if just a version/hash bump is at least buildable? Or
> are the changes necessary for the packages to build/function at all?
> Or I guess if the non-version changes are applicable to the current
> version first?

I'll give it a try tomorrow.  I think if packages are updated in certain 
sequence, it should be at least buildable.  I'll give update or send a 
new patch series if I have any progress on this.

-- 
dan



^ permalink raw reply	[flat|nested] 19+ messages in thread

* [bug#69461] Should commits rather be buildable or small
  2024-03-25  3:23                       ` dan
@ 2024-03-25  3:23                         ` dan
  0 siblings, 0 replies; 19+ messages in thread
From: dan @ 2024-03-25  3:23 UTC (permalink / raw)
  To: John Kehayias, Liliana Marie Prikler
  Cc: Philip McGrath, Saku Laesvuori, 69461, Ricardo Wurmus, guix-devel,
	Attila Lendvai

Hi John,

On 3/25/2024 9:15 AM, John Kehayias wrote:
> dan: Do you know if just a version/hash bump is at least buildable? Or
> are the changes necessary for the packages to build/function at all?
> Or I guess if the non-version changes are applicable to the current
> version first?

I'll give it a try tomorrow.  I think if packages are updated in certain 
sequence, it should be at least buildable.  I'll give update or send a 
new patch series if I have any progress on this.

-- 
dan





^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2024-03-25 13:45 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-12-08  8:42 Should commits rather be buildable or small Saku Laesvuori
2023-12-08 11:41 ` Tomas Volf
2023-12-08 12:05 ` Lars-Dominik Braun
2023-12-08 16:35   ` Felix Lechner via Development of GNU Guix and the GNU System distribution.
2023-12-08 15:44 ` Liliana Marie Prikler
2023-12-10 15:28   ` Saku Laesvuori
2023-12-10 15:50     ` Liliana Marie Prikler
2023-12-10 17:02     ` Attila Lendvai
2023-12-10 17:50       ` Ricardo Wurmus
2023-12-10 23:20         ` Attila Lendvai
2023-12-10 23:56           ` Philip McGrath
2023-12-11 10:51             ` Attila Lendvai
2023-12-11 11:51               ` Ricardo Wurmus
2024-03-04 21:38                 ` John Kehayias
2024-03-05  4:32                   ` dan
2024-03-05  5:19                   ` Liliana Marie Prikler
2024-03-25  1:15                     ` John Kehayias
2024-03-25  3:23                       ` dan
2024-03-25  3:23                         ` [bug#69461] " dan

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).