bug#41604: guix pull impossible after rebasing a local repository

bug#41604: guix pull impossible after rebasing a local repository

[John Soo]

Hi Guix,

I use a local git repo with branch that I specify in channels.scm.
My usual workflow is:

1. rebase on origin
2. guix pull

This stopped working with the following error:

Updating channel 'guix' from Git repository at 'file:///home/john/projects/guix/.git'...
guix pull: error: aborting update of channel 'guix' to commit 1444040933ac35b967720288dc30ed70e5481ed3, which is not a descendant of 57518fc7bf1efc899c0dabaa76685a319661f8e4
hint: This could indicate that the channel has been tampered with and is trying to
force a roll-back, preventing you from getting the latest updates.  If you think
this is not the case, explicitly allow non-forward updates.

After removing $HOME/.cache/guix I get the following error:
guix pull: error: Git error: object not found - no match for id (57518fc7bf1efc899c0dabaa76685a319661f8e4)

Many thanks,

John

bug#41604: Acknowledgement (guix pull impossible after rebasing a local repository)

[John Soo]

One other note I forgot to mention. Using guix pull --allow-downgrades
gives the same error.

bug#41604: guix pull impossible after rebasing a local repository

[zimoun]

Hi John,

On Fri, 29 May 2020 at 18:39, John Soo <jsoo1@asu.edu> wrote:

> Updating channel 'guix' from Git repository at 'file:///home/john/projects/guix/.git'...
> guix pull: error: aborting update of channel 'guix' to commit 1444040933ac35b967720288dc30ed70e5481ed3, which is not a descendant of 57518fc7bf1efc899c0dabaa76685a319661f8e4
> hint: This could indicate that the channel has been tampered with and is trying to
> force a roll-back, preventing you from getting the latest updates.  If you think
> this is not the case, explicitly allow non-forward updates.
>
> After removing $HOME/.cache/guix I get the following error:
> guix pull: error: Git error: object not found - no match for id (57518fc7bf1efc899c0dabaa76685a319661f8e4)

Have you used '--allow-downgrades' before or after the big remove?
And your current 'guix describe' is 57518fc7, right?


All the best,
simon

bug#41604: guix pull impossible after rebasing a local repository

[John Soo]

Hi there,

zimoun <zimon.toutoune@gmail.com> writes:

> Hi John,
>
> On Fri, 29 May 2020 at 18:39, John Soo <jsoo1@asu.edu> wrote:
>
> Have you used '--allow-downgrades' before or after the big remove?
> And your current 'guix describe' is 57518fc7, right?

I used it after the cache removal unfortunately.

Guix describe has 57518fc7bf1efc899c0dabaa76685a319661f8e4
as the current comment.

Kindly,

John

bug#41604: guix pull impossible after rebasing a local repository

[zimoun]

On Fri, 29 May 2020 at 19:10, John Soo <jsoo1@asu.edu> wrote:

> Guix describe has 57518fc7bf1efc899c0dabaa76685a319661f8e4
> as the current comment.

Maybe, you can try:

   guix pull --commit=57518fc7bf1efc899c0dabaa76685a319661f8e4
--allow-downgrades

if 57518fc7 is really in your channel at ~/john/projects/guix/.git  --
I mean, you have replaced 'https://' by a local clone where you are
committing stuff, right?


Otherwise, you could try:

   guix pull --rollback
   guix pull --commit=57518fc7
   guix pull --allow-downgrades


Well, I do not know if it helps...


Cheers,
simon

bug#41604: guix pull impossible after rebasing a local repository

[John Soo]

Hello,

zimoun <zimon.toutoune@gmail.com> writes:

> Maybe, you can try:
>
>    guix pull --commit=57518fc7bf1efc899c0dabaa76685a319661f8e4
> --allow-downgrades

> if 57518fc7 is really in your channel at ~/john/projects/guix/.git  --
> I mean, you have replaced 'https://' by a local clone where you are
> committing stuff, right?

Ah, I think that is the problem. I want to keep my same patches which
are rebased on top of origin/master. The patch contents are the same but
the commit hash has changed to something else.

$ git rev-parse HEAD
1444040933ac35b967720288dc30ed70e5481ed3

Commit 57518fc7bf1efc899c0dabaa76685a319661f8e4 no longer exists in my
branch's history:

$ git log --oneline --grep='57518'
(nothing)

I specify the branch in channels.scm:

(list
 (channel
  (name 'guix)
  (url "file:///home/john/projects/guix/.git")
  (branch "my-master"))
 (channel
  (name 'private)
  (url "file:///home/john/projects/guix-channel/.git")
  (branch "master")))

> Otherwise, you could try:
>
>    guix pull --rollback
>    guix pull --commit=57518fc7
>    guix pull --allow-downgrades

All of the following give the same error as reported:

guix pull --commit=57518fc7bf1efc899c0dabaa76685a319661f8e4
guix pull --commit=1444040933ac35b967720288dc30ed70e5481ed3
guix pull --allow-downgrades --commit=57518fc7bf1efc899c0dabaa76685a319661f8e4
guix pull --allow-downgrades --commit=1444040933ac35b967720288dc30ed70e5481ed3
guix pull --allow-downgrades

Using --rollback does not seem like an option

$ guix pull --rollback
guix pull: error: rollback: unrecognized option

> Well, I do not know if it helps...

Definitely! I am getting more familiar with the pull options. Thank you!

- John

bug#41604: guix pull impossible after rebasing a local repository

[zimoun]

On Fri, 29 May 2020 at 20:22, John Soo <jsoo1@asu.edu> wrote:

> Ah, I think that is the problem. I want to keep my same patches which
> are rebased on top of origin/master. The patch contents are the same but
> the commit hash has changed to something else.

Hehe! Dangerous zone. :-)

Personally, I keep clean ~/.config/guix/current by always pulling from
origin/master.
Then I have, as you, a local clone where I rebase, commit etc. But I
only pull to another profile than the default one, to avoid similar
situations as you currently are. ;-)

   /path/to/what-i-mean/bin/guix pull --url=/pah/local/clone
--branch=kikoo -p /path/to/next

and I have some channels files under ~/.config/guix/ to simply some
regular, e.g.,

   guix pull -C ~/.config/guix/extra.scm -p /path/to/extra
   /tmp/extra/bin/guix install foo -p /tmp/test

And so "guix pull" always works.
Anyway! :-)



> > Otherwise, you could try:
> >
> >    guix pull --rollback
> >    guix pull --commit=57518fc7
> >    guix pull --allow-downgrades
>
> All of the following give the same error as reported:
>
> guix pull --commit=57518fc7bf1efc899c0dabaa76685a319661f8e4
> guix pull --commit=1444040933ac35b967720288dc30ed70e5481ed3
> guix pull --allow-downgrades --commit=57518fc7bf1efc899c0dabaa76685a319661f8e4
> guix pull --allow-downgrades --commit=1444040933ac35b967720288dc30ed70e5481ed3
> guix pull --allow-downgrades
>
> Using --rollback does not seem like an option

What do you mean?  An option for you or an option of "guix pull"?
Ah my bad, it is "--roll-back".  The double "--roll-back" and
"--rollback" options is only allowed with "guix package".  Thank you
for the notification. :-)


Is "guix pull --commit=1444040933 --allow-downgrades" not working for you?


Cheers,
simon

bug#41604: guix pull impossible after rebasing a local repository

[John Soo]

Hello,

zimoun <zimon.toutoune@gmail.com> writes:

> Is "guix pull --commit=1444040933 --allow-downgrades" not working for you?

Yeah guix pull --commit=1444040933 --allow-downgrades failed.

> What do you mean?  An option for you or an option of "guix pull"?
> Ah my bad, it is "--roll-back".  The double "--roll-back" and
> "--rollback" options is only allowed with "guix package".  Thank you
> for the notification. :-)

Ah, nice! roll-back helped me work around the problem. Very helpful,
thanks!

> Hehe! Dangerous zone. :-)

I think I found that out, haha!


> Personally, I keep clean ~/.config/guix/current by always pulling from
> origin/master.
> Then I have, as you, a local clone where I rebase, commit etc. But I
> only pull to another profile than the default one, to avoid similar
> situations as you currently are. ;-)
>
>    /path/to/what-i-mean/bin/guix pull --url=/pah/local/clone
> --branch=kikoo -p /path/to/next
>
> and I have some channels files under ~/.config/guix/ to simply some
> regular, e.g.,
>
>    guix pull -C ~/.config/guix/extra.scm -p /path/to/extra
>    /tmp/extra/bin/guix install foo -p /tmp/test
>
> And so "guix pull" always works.
> Anyway! :-)

Nice!  I like the idea of having a "next" profile.

I guess that makes me wonder what the desired specification is.
There is a lot of problem space to explore.  As a user I would want to
be able to take my local patches as "the real truth".

Because guix has a linear git history, that means the user needs to
always rebase.  Often I have patches open for months that I am currently
testing and working on. It would be convenient for me to be able to guix
pull into my default user profile.

On the other hand, as you point out, using a hash that disappears from
the git history is dangerous. The git history no longer tracks the guix
pull history and then there may exist pulls in history that may never be
recoverable.

There seems to be some existing support for the rebasing into the
default profile since branches are allowed as references in the channels
configuration. Has the rebase use case been discussed before? Oh, also,
history items can be deleted in other places with --delete-generations
and friends.  I am not sure what to classify this problem, bug or
something else.

What do you think?

- John

bug#41604: guix pull impossible after rebasing a local repository

[zimoun]

Hi John,

On Sat, 30 May 2020 at 04:26, John Soo <jsoo1@asu.edu> wrote:

> Yeah guix pull --commit=1444040933 --allow-downgrades failed.

Hum?  with this message:

--8<---------------cut here---------------start------------->8---
guix pull: error: Git error: object not found - no match for id
(57518fc7bf1efc899c0dabaa76685a319661f8e4)
--8<---------------cut here---------------end--------------->8---

?


> > Hehe! Dangerous zone. :-)
>
> I think I found that out, haha!

To be honest, I have tried to use the checkout
~/.cache/guix/checkout/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq
 as local clone in order to rebase, worktree and apply patches.  Bad,
really bad idea!  The discipline to avoid breakage is too high. ;-)


> I guess that makes me wonder what the desired specification is.
> There is a lot of problem space to explore.  As a user I would want to
> be able to take my local patches as "the real truth".
>
> Because guix has a linear git history, that means the user needs to
> always rebase.  Often I have patches open for months that I am currently
> testing and working on. It would be convenient for me to be able to guix
> pull into my default user profile.
>
> On the other hand, as you point out, using a hash that disappears from
> the git history is dangerous. The git history no longer tracks the guix
> pull history and then there may exist pulls in history that may never be
> recoverable.
>
> There seems to be some existing support for the rebasing into the
> default profile since branches are allowed as references in the channels
> configuration. Has the rebase use case been discussed before? Oh, also,
> history items can be deleted in other places with --delete-generations
> and friends.  I am not sure what to classify this problem, bug or
> something else.

I am not sure to see what is the desired specification.  Well, I do
not know if my workflow is the good one but let me describe how it is
organised.

The local (master) checkout is at ~/src/guix/guix/ and then I only
checkout the branches using 'worktree' (thanks to Ricardo to show me
the tips during a "Skill Share" session on Dec. 2018 :-)).  Well,
Magit helps a lot here.
Basically all my branches live under ~/src/guix/wk/.  And when I test
patches, I always create a new branch.  Well, I do not often rebase
enough so I never remember the 'onto' direction so I always read the
doc for that. :-)

(The price to pay with worktree is that I "compile" all Guix really
often because it does not reuse .go files; even if I tried to
synchronize the .go files to avoid such annoyance but I have failed.)

Just 'guix' refers to the one living at ~/.config/guix/current because
I always want that "guix install foo" refers to upstream.  Well, I
never pull the latest commit from
https://git.savannah.gnu.org/git/guix.git because the substitutes are
almost never available so I pull 2 weeks behind.  Basically, I do:
"git -C ~/.cache/guix/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq
pull" then

   guix pull --commit=$(git -C ~/.cache/guix/<long-hash> \
                                   --before=$(date --date='2 weeks
ago' +%Y-%m-%d) \
                                   --format="%h" | head -n1)

And when I want to install something from one of the branches, it depends:

  a- inside "guix environment guix" and ~/src/guix/wk/<name>
       ./pre-inst-env guix install foo -p /tmp/profile
and then I use this /tmp/profile as any other.  Or I go with "guix
environment".  Well, if it is something I want to hold, then I write
down the manifest and channel files.

 b- inside "guix environment guix" and ~/src/guix/wk/<name>
       ./pre-inst-env guix pull --branch=<name> --url=$PWD -p ~/tmp/<name>
       /tmp/<name>/bin/guix blabla
and I almost never track "permanently" such other guix.

For custom packages, I am more often using '--load-path' than 'channel'.

Last, I use more and more "guix time-machine".  Well actually more
than b-, which I only use to test patches about "guix pull".  Roughly
speaking,
 - "guix environment" creates temporary profile
 - "guix time-machine" create temporary pull
And with this new "authentication" of "guix pull", I think now, it
would be better to do:
       guix time-machine --branch=<name> --url=~/src/wk/<name> -- blabla


Well, I do not know if all this is the correct way.  But it is more or
less what I am doing to avoid annoyance.


All the best,
simon

bug#41604: guix pull impossible after rebasing a local repository

[Arne Babenhauserheide]

John Soo <jsoo1@asu.edu> writes:

> I guess that makes me wonder what the desired specification is.
> There is a lot of problem space to explore.  As a user I would want to
> be able to take my local patches as "the real truth".
>
> Because guix has a linear git history, that means the user needs to
> always rebase.  Often I have patches open for months that I am currently
> testing and working on. It would be convenient for me to be able to guix
> pull into my default user profile.
>
> On the other hand, as you point out, using a hash that disappears from
> the git history is dangerous. The git history no longer tracks the guix
> pull history and then there may exist pulls in history that may never be
> recoverable.
>
> There seems to be some existing support for the rebasing into the
> default profile since branches are allowed as references in the channels
> configuration. Has the rebase use case been discussed before? Oh, also,
> history items can be deleted in other places with --delete-generations
> and friends.  I am not sure what to classify this problem, bug or
> something else.
>
> What do you think?

This sounds like a use-case for changeset evolution with
hidden-but-retrievable commits:
https://www.mercurial-scm.org/doc/evolution/index.html

With that you can rebase and have linear surface history, but the
rewritten commits still exist as hidden commits with references to the
commits that superseded them.

Problem: It not possible with Git. It would require switching to
Mercurial — which would also enable useful abilities like coordinated
rebasing in a group: https://blog.disy.net/hg-evolution/
(and which is GPLv3-compatible)

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

bug#41604: guix pull impossible after rebasing a local repository

[John Soo]

Hi Arne,

Interesting idea about mercurial.

My problem largely comes from the fact that I specified a branch in channels.scm for over a year with the same workflow. If a branch can be specified for a channel repo then the system probably should handle the current pull commit not being available in the history.

The commit history authentication is nice but seems to have regressed this particular use case. 

- John

bug#41604: guix pull impossible after rebasing a local repository

[zimoun]

Dear Arne,

On Sat, 30 May 2020 at 18:18, Arne Babenhauserheide <arne_bab@web.de> wrote:

> This sounds like a use-case for changeset evolution with
> hidden-but-retrievable commits:
> https://www.mercurial-scm.org/doc/evolution/index.html
>
> With that you can rebase and have linear surface history, but the
> rewritten commits still exist as hidden commits with references to the
> commits that superseded them.

Thank you for the pointer.  Yes it could be cool that "git rebase -i"
still stores the rewritten commits as hidden -- which would be removed
by "git gc".
However, IMHO, the question seems more about how to the use the weapon
than the weapon itself.  Other said, it seems more an issue about
practise than tool.
Well, rewrite the history is always dangerous, whatever the tools; and
one solution when one is not sure is to duplicate the branch with
e.g., "git branch --copy".

In this use-case, there is 2 incompatible requirements:
 - content-addressed for reproducibility etc. well the core principles of Guix
 - rewrite these addresses on the fly
Well, without care, for sure it would break one way or another,
whatever the tool is, IMHO.


> Problem: It not possible with Git. It would require switching to
> Mercurial — which would also enable useful abilities like coordinated
> rebasing in a group: https://blog.disy.net/hg-evolution/
> (and which is GPLv3-compatible)

I am sad too that 'hg' has lost against 'git'.  All my PhD was under
mercurial and I was really happy with it; tool easier to grab.  Now, I
am happier with Magit. ;-)
Speaking about DVSC, pijul and theory of patches looks very
interesting.  Whatever! :-)
https://pijul.org/


All the best,
simon

bug#41604: guix pull impossible after rebasing a local repository

[zimoun]

Dear John,

On Sun, 31 May 2020 at 07:04, John Soo <jsoo1@asu.edu> wrote:

> My problem largely comes from the fact that I specified a branch in channels.scm for over a year with the same workflow. If a branch can be specified for a channel repo then the system probably should handle the current pull commit not being available in the history.

What do you mean?


> The commit history authentication is nice but seems to have regressed this particular use case.

I am not sure that the new history authentication is the issue here.
IMHO, it is just the feature that shows up the flaw with your
workflow. ;-)
From my understanding, Guix is built around content-addressed
principles (channel, store, etc.) so try to change on the fly the
address of such content would lead to break one way or another, IMHO.


Well, is the issue fixed for you now?


All the best,
simon

bug#41604: guix pull impossible after rebasing a local repository

[John Soo]

Hello again,

zimoun <zimon.toutoune@gmail.com> writes:

> On Sun, 31 May 2020 at 07:04, John Soo <jsoo1@asu.edu> wrote:
>
>> My problem largely comes from the fact that I specified a branch in
>> channels.scm for over a year with the same workflow. If a branch can
>> be specified for a channel repo then the system probably should
>> handle the current pull commit not being available in the history.
>
> What do you mean?

Well if a channel specification has a branch then the channel
specification explicitly allows the commit history to not always have
the commit from "guix describe".  For example, say the channels.scm file
has the following:

(list
 (channel
  (name 'example-channel)
  (url "https://git.example.com/repo")
  (branch "example-branch"))
 ...)

the maintainer of example-channel may rebase example-branch on some
other branch.  Then guix-pull will indeed be impossible for anyone who
has example-channel specified with the branch field in channels.scm.

So what I am saying is that if unrelated histories are to be disallowed
during "guix pull" then branches should not be allowed in channel
specifications.

>> The commit history authentication is nice but seems to have
>> regressed this particular use case.
>
> I am not sure that the new history authentication is the issue here.
> IMHO, it is just the feature that shows up the flaw with your
> workflow. ;-)

You are right, I should not assume what introduced my problem. What I do
know is that this workflow worked for the last year or more until last
week. I should do a bisect to find the exact commit.

>>From my understanding, Guix is built around content-addressed
> principles (channel, store, etc.) so try to change on the fly the
> address of such content would lead to break one way or another, IMHO.
>
>
> Well, is the issue fixed for you now?

The issue is not fixed and I think I found another problem with the
workaround.  To reiterate, my workaround is to "guix pull --roll-back"
until a generation that has a the commit that is in my history then
"guix pull" again to get my new work. The problem is the next "guix
pull" shows all news from the old commit until the newest commit. In
other words I get the news I saw from the previous time I pulled plus
any new work. In this way the news continues to accumulate making the
news less and less useful and more and more noisy.

Btw, I am totally with you on pijul/darcs or some patch theory version
control. Pijul does indeed look pretty promising. I packaged it in my
channel if you want to try it :).

Kindly,

John

bug#41604: guix pull impossible after rebasing a local repository

[Ludovic Courtès]

Hi John,

John Soo <jsoo1@asu.edu> skribis:

> I use a local git repo with branch that I specify in channels.scm.
> My usual workflow is:
>
> 1. rebase on origin
> 2. guix pull
>
> This stopped working with the following error:
>
> Updating channel 'guix' from Git repository at 'file:///home/john/projects/guix/.git'...
> guix pull: error: aborting update of channel 'guix' to commit 1444040933ac35b967720288dc30ed70e5481ed3, which is not a descendant of 57518fc7bf1efc899c0dabaa76685a319661f8e4
> hint: This could indicate that the channel has been tampered with and is trying to
> force a roll-back, preventing you from getting the latest updates.  If you think
> this is not the case, explicitly allow non-forward updates.

What happens is that ‘guix pull’ ensures that it only ever makes
“fast-forward” updates by default, in Git parlance.  The goal is to
detect obvious “downgrade attacks”:

  https://issues.guix.gnu.org/41425

(This can be overridden this by passing ‘--allow-downgrades’.)

The rebase workflow you describe unavoidably triggers the error because
every time you pull, you do a non-fast-forward pull (because the commit
you were previously using, as shown in ‘guix describe’, has been
rewritten and no longer exists in the new commit graph.)  So at least,
it shows that the machinery works as advertised.  :-)

What I would recommend is for your channel to be a regular “fork”: you
create a branch containing your patches and regularly merge upstream
master back into your branch.  That way, you don’t need to rewrite
history and ‘guix pull’ is happy.

Alternately, if you like to have linear history (for example because you
intend to eventually submit your patches upstream), you could use
TopGit, which roughly allows you to version-control your rebases.

HTH!

From a pure Guix perspective, it’s “not-a-bug”.  If one of the above
suggestions works for you, I think we can close this issue.

Thanks for your feedback!

Ludo’.

bug#41604: guix pull impossible after rebasing a local repository

[John Soo]

Hi Ludo!

I hope you are well.

Ludovic Courtès <ludo@gnu.org> writes:

> What happens is that ‘guix pull’ ensures that it only ever makes
> “fast-forward” updates by default, in Git parlance.  The goal is to
> detect obvious “downgrade attacks”:
>
>   https://issues.guix.gnu.org/41425

Oh I see, I'm sorry I did not participate in that issue, now.  That
makes sense and I'm glad to know this is part of the design.

> (This can be overridden this by passing ‘--allow-downgrades’.)

Does '--allow-downgrades' support unrelated git histories?  I tried that
flag and it did not work.

> The rebase workflow you describe unavoidably triggers the error because
> every time you pull, you do a non-fast-forward pull (because the commit
> you were previously using, as shown in ‘guix describe’, has been
> rewritten and no longer exists in the new commit graph.)  So at least,
> it shows that the machinery works as advertised.  :-)

Totally. I think I understand the design goal now.

> What I would recommend is for your channel to be a regular “fork”: you
> create a branch containing your patches and regularly merge upstream
> master back into your branch.  That way, you don’t need to rewrite
> history and ‘guix pull’ is happy.

So far the need to avoid downgrade attacks makes sense. Here are the
things I think should be considered:

I have branches based on master in savannah that contain specific patch
sets associated to patch requests upstream. I think I have 3 or 4 right
now.  My patches are also in the branch I have in channels.scm.  I do
that for a few reasons:

1. To test the patches
2. To workaround or use bugs/features/packages I want but are not upstream yet.

That means I tend to want to use my patches whether or not they are
upstream yet.  In fact I stopped working on my channel because it was so
easy to just make patches on upstream to contribute back.

It can take many months for patches to be merged.  That is expected
since we are all volunteers.  Rebasing the patches is the easiest way to
keep them up to date so they can be applied cleanly.

There are two design and community goals I love about Guix: hackability
and inclusivity. I feel that disallowing linear history makes the
easiest way to contribute to, hack on, and participate in Guix much
harder without proper support. For instance: instead of making patches
on top of upstream it is now easier just to work on my channel.

Certainly some tradeoffs should be made for security and I think your
recent commit authentication work does that elegantly.  Perhaps we can
easily have hackability and security with a flag like --allow-downgrades
called --allow-unrelated that allows the rebase workflow.

WDYT?


> Alternately, if you like to have linear history (for example because you
> intend to eventually submit your patches upstream), you could use
> TopGit, which roughly allows you to version-control your rebases.

Hmm. I am unaware of TopGit but I find rebasing to be the simplest and
easiest way to do my work. I'll look into it but I would rather not have
to use another tool for simplicity's sake.


> From a pure Guix perspective, it’s “not-a-bug”.  If one of the above
> suggestions works for you, I think we can close this issue.

I think I understand the reasoning now. I do hope a flag like
--allow-unrelated might solve all the things.

> HTH!

Thanks as always. I really appreciate your communications.

- John

bug#41604: guix pull impossible after rebasing a local repository

[Ludovic Courtès]

Hello,

John Soo <jsoo1@asu.edu> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> (This can be overridden this by passing ‘--allow-downgrades’.)
>
> Does '--allow-downgrades' support unrelated git histories?  I tried that
> flag and it did not work.

It supports unrelated Git histories.  It could really be called
‘--allow-anything’ but I thought it’d be less descriptive.  :-)

If you hit a problem with that, please report it (perhaps I just
overlooked it in the other issue.)

> I have branches based on master in savannah that contain specific patch
> sets associated to patch requests upstream. I think I have 3 or 4 right
> now.  My patches are also in the branch I have in channels.scm.  I do
> that for a few reasons:
>
> 1. To test the patches
> 2. To workaround or use bugs/features/packages I want but are not upstream yet.
>
> That means I tend to want to use my patches whether or not they are
> upstream yet.  In fact I stopped working on my channel because it was so
> easy to just make patches on upstream to contribute back.
>
> It can take many months for patches to be merged.  That is expected
> since we are all volunteers.  Rebasing the patches is the easiest way to
> keep them up to date so they can be applied cleanly.

Yes.

> There are two design and community goals I love about Guix: hackability
> and inclusivity. I feel that disallowing linear history makes the
> easiest way to contribute to, hack on, and participate in Guix much
> harder without proper support. For instance: instead of making patches
> on top of upstream it is now easier just to work on my channel.
>
> Certainly some tradeoffs should be made for security and I think your
> recent commit authentication work does that elegantly.  Perhaps we can
> easily have hackability and security with a flag like --allow-downgrades
> called --allow-unrelated that allows the rebase workflow.

Interesting, I hadn’t thought about how this mechanism would give an
incentive to have a channel vs. contributing directly upstream.

Normally, ‘--allow-downgrades’ does exactly what you need, at least
that’s the intent.  I’d argue that it’s also reasonable to use it in
this case because obviously you know what you’re doing, and you’re
pulling from a local Git repository, so that’s fine.

>> Alternately, if you like to have linear history (for example because you
>> intend to eventually submit your patches upstream), you could use
>> TopGit, which roughly allows you to version-control your rebases.
>
> Hmm. I am unaware of TopGit but I find rebasing to be the simplest and
> easiest way to do my work. I'll look into it but I would rather not have
> to use another tool for simplicity's sake.

Yeah, that makes sense to me.

Thanks,
Ludo’.

bug#41604: guix pull impossible after rebasing a local repository

[John Soo]

Hi Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

> It supports unrelated Git histories.  It could really be called
> ‘--allow-anything’ but I thought it’d be less descriptive.  :-)
>
> If you hit a problem with that, please report it (perhaps I just
> overlooked it in the other issue.)

Oh nice, then this whole issue I think should be to report that
--allow-downgrades does not allow everything.  When I first reported the
issue I tried --allow-downgrades thanks to some help on IRC.  The first
attempt failed as I mentioned previously in the thread.  However I did
just try --allow-downgrades again after a rebase and it seems to have
succeeded.  With that, I think this issue can be closed.  Thanks for
thinking of this case!

> Interesting, I hadn’t thought about how this mechanism would give an
> incentive to have a channel vs. contributing directly upstream.

I think I will add some notes about a rebase workflow and
--allow-downgrades to the contributing documentation. Looking just now
I'm not sure using a local source tree as a channel is mentioned in the
documentation either. I will open a different patch set to deal with
those issues.

> Normally, ‘--allow-downgrades’ does exactly what you need, at least
> that’s the intent.  I’d argue that it’s also reasonable to use it in
> this case because obviously you know what you’re doing, and you’re
> pulling from a local Git repository, so that’s fine.

100% it does what I need.

Thanks again, feel free to close.

- John

bug#41604: guix pull impossible after rebasing a local repository

[zimoun]

Hi Ludo,

On Wed, 3 Jun 2020 at 17:14, Ludovic Courtès <ludo@gnu.org> wrote:

> >> (This can be overridden this by passing ‘--allow-downgrades’.)
> >
> > Does '--allow-downgrades' support unrelated git histories?  I tried that
> > flag and it did not work.
>
> It supports unrelated Git histories.  It could really be called
> ‘--allow-anything’ but I thought it’d be less descriptive.  :-)
>
> If you hit a problem with that, please report it (perhaps I just
> overlooked it in the other issue.)

I think it is not a bug and it is a feature* :-) but the behaviour has
changed for the commits which do not belong to the repo anymore.
That's why John has not seen the issue of his "rebase workflow"
before.

*feature: at least, it seems expected from what I understand of the code. :-)


Let remind the commit history.  Instead of create a channel file, I
directly use the raw repo, but it is the same for any channel (Git
repo).

--8<---------------cut here---------------start------------->8---
$ SRC=~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq
$ git -C $SRC --no-pager log --oneline c873980d18^..8bd0b533b3
8bd0b533b3 gnu: libexif: Update to 0.6.22 [security fixes].
e451612602 gnu: libgphoto2: Update to 2.5.25.
9744cc7b46 pull: Protect against downgrade attacks.
872898f768 channels: 'latest-channel-instances' guards against
non-forward updates.
8d1d56578a git: 'update-cached-checkout' returns the commit relation.
9b049de84e channels: 'latest-channel-instances' doesn't leak internal state.
c098c11be8 git: Add 'commit-relation'.
86ac14b2f3 (HEAD -> master) gnu: protonvpn-cli: Tweak description.
c873980d18 gnu: Add protonvpn-cli.
--8<---------------cut here---------------end--------------->8---


Here, a first session from a commit before the "downgrade attacks"
commit.  A commit is added, then pulled, then rebased where this
addition is totally deleted of the Git repo, then another pulled.  No
error at all.

--8<---------------cut here---------------start------------->8---
$ guix describe
Generation 34    Jun 05 2020 02:16:22    (current)
  guix 86ac14b
    repository URL: https://git.savannah.gnu.org/git/guix.git
    commit: 86ac14b2f37efbb6f4a3ed1c3e183fbc9496b7a5

$ echo hello >> $SRC/README && git -C $SRC commit -am hello
[master 20e984e931] hello
 1 file changed, 1 insertion(+)

$ guix pull --commit=$(git -C $SRC rev-parse HEAD)
Updating channel 'guix' from Git repository at
'https://git.savannah.gnu.org/git/guix.git'...
Building from this channel:
  guix      https://git.savannah.gnu.org/git/guix.git    20e984e
Computing Guix derivation for 'x86_64-linux'... /

$ guix describe
Generation 35    Jun 05 2020 02:32:43    (current)
  guix 20e984e
    repository URL: https://git.savannah.gnu.org/git/guix.git
    commit: 20e984e9311404295c9c82b54eac1c277709b0a0

$ git -C $SRC reset --hard HEAD^
HEAD is now at 86ac14b2f3 gnu: protonvpn-cli: Tweak description.

$ git -C $SRC reflog expire --expire-unreachable=now --all
$ git -C $SRC gc --prune=now --quiet

$ git -C $SRC show 20e984e9311404295c9c82b54eac1c277709b0a0
fatal: bad object 20e984e9311404295c9c82b54eac1c277709b0a0

$ guix pull --commit=c873980d18
Updating channel 'guix' from Git repository at
'https://git.savannah.gnu.org/git/guix.git'...
Building from this channel:
  guix      https://git.savannah.gnu.org/git/guix.git    c873980
Computing Guix derivation for 'x86_64-linux'... /
--8<---------------cut here---------------end--------------->8---


Now the same session after the introduction of '--allow-downgrades'.

--8<---------------cut here---------------start------------->8---
$ guix describe
Generation 37    Jun 05 2020 01:28:52    (current)
  guix 8bd0b53
    repository URL: https://git.savannah.gnu.org/git/guix.git
    commit: 8bd0b533b30d7ee5e03aee99a2eb96d5b0b1c836

$ echo hello >> $SRC/README && git -C $SRC commit -am hello
[master 09f6e9b34c] hello
 1 file changed, 1 insertion(+)

$ guix pull --commit=$(git -C $SRC rev-parse HEAD)
Updating channel 'guix' from Git repository at
'https://git.savannah.gnu.org/git/guix.git'...
Building from this channel:
  guix      https://git.savannah.gnu.org/git/guix.git    09f6e9b
Computing Guix derivation for 'x86_64-linux'... /

$ guix describe
Generation 38    Jun 05 2020 02:57:13    (current)
  guix 09f6e9b
    repository URL: https://git.savannah.gnu.org/git/guix.git
    commit: 09f6e9b34c6239bcdd8ca9e030d698b5244507a6

$ git -C $SRC reset --hard HEAD^
HEAD is now at 8bd0b533b3 gnu: libexif: Update to 0.6.22 [security fixes].

$ git -C $SRC reflog expire --expire-unreachable=now --all
$ git -C $SRC gc --prune=now --quiet
$ git -C $SRC show 09f6e9b34c6239bcdd8ca9e030d698b5244507a6
fatal: bad object 09f6e9b34c6239bcdd8ca9e030d698b5244507a6

$ guix pull --commit=e451612602 --allow-downgrades
Updating channel 'guix' from Git repository at
'https://git.savannah.gnu.org/git/guix.git'...
guix pull: error: Git error: object not found - no match for id
(09f6e9b34c6239bcdd8ca9e030d698b5244507a6)
--8<---------------cut here---------------end--------------->8---


Well, I admit it is an unexpected use-case.  The solution here is to
'--roll-back' but it can be tedious if we are talking about several
commits which had been removed in the channel by the user.  Therefore,
the best is to use "--list-generations" to find a generation with an
existing commit and switch to it with "guix pull
--switch-generation=NN".  However, some '--news' will be lost.  No
free lunch. ;-)

I do not think it will be worth to allow this kind of workflow but one
solution could be to add a flag, i.e., '--allow-downgrade=dangerous',
and bypass 'commit-relation' which is somehow the culprit here.

IMHO, it is better to document what to do when someone does a mistake
by removing the current commit where they is currently (describe).

WDYT?

All the best,
simon

bug#41604: guix pull impossible after rebasing a local repository

[zimoun]

Hi John,

On Thu, 4 Jun 2020 at 16:12, John Soo <jsoo1@asu.edu> wrote:

> Oh nice, then this whole issue I think should be to report that
> --allow-downgrades does not allow everything.  When I first reported the
> issue I tried --allow-downgrades thanks to some help on IRC.  The first
> attempt failed as I mentioned previously in the thread.  However I did
> just try --allow-downgrades again after a rebase and it seems to have
> succeeded.  With that, I think this issue can be closed.  Thanks for
> thinking of this case!

Well, there is 2 issues, I guess.  One is fixed by allow downgrade.

The other one is not about '--allow-downgrades' but simply about pull
from a removed commit (rebase).  I do not think it could be considered
as a bug but the behaviour has changed and '--allow-downgrades' is not
enough.  Well, if I understand correctly what you described before in
this thread. :-)


> I think I will add some notes about a rebase workflow and
> --allow-downgrades to the contributing documentation. Looking just now
> I'm not sure using a local source tree as a channel is mentioned in the
> documentation either. I will open a different patch set to deal with
> those issues.

IMHO, the trick to be able to pull from a removed commit is to switch
generation first.  The price to pay is that history will be lost and
so news will be screwed up.

Well, for sure describe your workflow could be an entry of the Cookbook. :-)


All the best,
simon

bug#41604: guix pull impossible after rebasing a local repository

[Ludovic Courtès]

Hi,

John Soo <jsoo1@asu.edu> skribis:

>> Normally, ‘--allow-downgrades’ does exactly what you need, at least
>> that’s the intent.  I’d argue that it’s also reasonable to use it in
>> this case because obviously you know what you’re doing, and you’re
>> pulling from a local Git repository, so that’s fine.
>
> 100% it does what I need.
>
> Thanks again, feel free to close.

OK, closing.  Let me know if you stumble upon issues in that area!

Ludo’.

bug#41604: guix pull impossible after rebasing a local repository

[Ludovic Courtès]

Hi Simon,

zimoun <zimon.toutoune@gmail.com> skribis:

> $ guix describe
> Generation 37    Jun 05 2020 01:28:52    (current)
>   guix 8bd0b53
>     repository URL: https://git.savannah.gnu.org/git/guix.git
>     commit: 8bd0b533b30d7ee5e03aee99a2eb96d5b0b1c836
>
> $ echo hello >> $SRC/README && git -C $SRC commit -am hello
> [master 09f6e9b34c] hello
>  1 file changed, 1 insertion(+)
>
> $ guix pull --commit=$(git -C $SRC rev-parse HEAD)
> Updating channel 'guix' from Git repository at
> 'https://git.savannah.gnu.org/git/guix.git'...
> Building from this channel:
>   guix      https://git.savannah.gnu.org/git/guix.git    09f6e9b
> Computing Guix derivation for 'x86_64-linux'... /
>
> $ guix describe
> Generation 38    Jun 05 2020 02:57:13    (current)
>   guix 09f6e9b
>     repository URL: https://git.savannah.gnu.org/git/guix.git
>     commit: 09f6e9b34c6239bcdd8ca9e030d698b5244507a6
>
> $ git -C $SRC reset --hard HEAD^
> HEAD is now at 8bd0b533b3 gnu: libexif: Update to 0.6.22 [security fixes].
>
> $ git -C $SRC reflog expire --expire-unreachable=now --all
> $ git -C $SRC gc --prune=now --quiet
> $ git -C $SRC show 09f6e9b34c6239bcdd8ca9e030d698b5244507a6
> fatal: bad object 09f6e9b34c6239bcdd8ca9e030d698b5244507a6
>
> $ guix pull --commit=e451612602 --allow-downgrades
> Updating channel 'guix' from Git repository at
> 'https://git.savannah.gnu.org/git/guix.git'...
> guix pull: error: Git error: object not found - no match for id
> (09f6e9b34c6239bcdd8ca9e030d698b5244507a6)

Ah I see, thanks for the reproducer.

Generally, rebasing does not necessarily implies ‘git gc’, so I think
one has to be unlucky to have the former commit disappear right away,
no?

But yeah, I don’t think there’s much we can do.  Or perhaps we could
have ‘commit-relation’ report 'unrelated when one of the commits does
not exist, that’d be clearer and more useful than this error.

WDYT?

Ludo’.

bug#41604: guix pull impossible after rebasing a local repository

[zimoun]

Hi Ludo,

On Fri, 5 Jun 2020 at 18:17, Ludovic Courtès <ludo@gnu.org> wrote:

> Generally, rebasing does not necessarily implies ‘git gc’, so I think

It depends on how "git gc" is configured.  Well, I am not a Git
specialist but from my understanding of the manual, by default, these
unreachable commits are kepts 30 days and then garbage collected.


> one has to be unlucky to have the former commit disappear right away,
> no?

Unluck happens! as Forest Gump said. ;-)


> But yeah, I don’t think there’s much we can do.  Or perhaps we could
> have ‘commit-relation’ report 'unrelated when one of the commits does
> not exist, that’d be clearer and more useful than this error.

And report a hint.
For example, guix pull --switch-generation


Cheers,
simon

bug#41604: guix pull impossible after rebasing a local repository

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

>> But yeah, I don’t think there’s much we can do.  Or perhaps we could
>> have ‘commit-relation’ report 'unrelated when one of the commits does
>> not exist, that’d be clearer and more useful than this error.
>
> And report a hint.

I did this:

  https://git.savannah.gnu.org/cgit/guix.git/commit/?id=1fd7de45f218ce572a3fe87764ad15927e3dbdc4

So now, in a situation like John described, ‘guix pull’ will print the
error and hint for unrelated commits.

I think we can close now!

Ludo’.

bug#41604: guix pull impossible after rebasing a local repository

[zimoun]

Hi Ludo,

On Sun, 7 Jun 2020 at 23:16, Ludovic Courtès <ludo@gnu.org> wrote:

> I did this:
>
>   https://git.savannah.gnu.org/cgit/guix.git/commit/?id=1fd7de45f218ce572a3fe87764ad15927e3dbdc4

Cool!  Thank you.

> So now, in a situation like John described, ‘guix pull’ will print the
> error and hint for unrelated commits.

Yes! Perfect! :-)

From a removed commit: "guix pull" says:

--8<---------------cut here---------------start------------->8---
guix pull: error: aborting update of channel 'guix' to commit
e78275608065ef073775fabb9f1a757da65851f2, which is not a descendant of
b50fb18a610e2a495624bf4570bec60265332615
hint: Use `--allow-downgrades' to force this downgrade.
--8<---------------cut here---------------end--------------->8---

then using the hint, it says:

--8<---------------cut here---------------start------------->8---
guix pull: warning: moving channel 'guix' from
b50fb18a610e2a495624bf4570bec60265332615 to unrelated commit
e78275608065ef073775fabb9f1a757da65851f2
Building from this channel:
  guix      https://git.savannah.gnu.org/git/guix.git   e782756
--8<---------------cut here---------------end--------------->8---


Just to note that the hint

--8<---------------cut here---------------start------------->8---
hint: This could indicate that the channel has been tampered with and is trying
to force a roll-back, preventing you from getting the latest updates.  If
you think this is not the case, explicitly allow non-forward updates.
--8<---------------cut here---------------end--------------->8---

does not report explicitly the option '--allow-downgrades'.


> I think we can close now!

Definitively! :-)

Cheers,
simon

bug#41604: guix pull impossible after rebasing a local repository

[John Soo]

Perfect! Thank you!

bug#41604: guix pull impossible after rebasing a local repository

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> On Sun, 7 Jun 2020 at 23:16, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> I did this:
>>
>>   https://git.savannah.gnu.org/cgit/guix.git/commit/?id=1fd7de45f218ce572a3fe87764ad15927e3dbdc4
>
> Cool!  Thank you.
>
>> So now, in a situation like John described, ‘guix pull’ will print the
>> error and hint for unrelated commits.
>
> Yes! Perfect! :-)
>
> From a removed commit: "guix pull" says:
>
> guix pull: error: aborting update of channel 'guix' to commit
> e78275608065ef073775fabb9f1a757da65851f2, which is not a descendant of
> b50fb18a610e2a495624bf4570bec60265332615
> hint: Use `--allow-downgrades' to force this downgrade.
>
>
> then using the hint, it says:
>
> guix pull: warning: moving channel 'guix' from
> b50fb18a610e2a495624bf4570bec60265332615 to unrelated commit
> e78275608065ef073775fabb9f1a757da65851f2
> Building from this channel:
>   guix      https://git.savannah.gnu.org/git/guix.git   e782756

Thanks for testing, closing!

> Just to note that the hint
>
> hint: This could indicate that the channel has been tampered with and is trying
> to force a roll-back, preventing you from getting the latest updates.  If
> you think this is not the case, explicitly allow non-forward updates.
>
> does not report explicitly the option '--allow-downgrades'.

Ah, right.

Ludo’.