bug#65787: time-machine is doing too much network requests

[Simon Tournier <zimon.toutoune@gmail.com>]

Hi Ludo,

>                             Maybe this is the purpose of your message:
> reducing Git remote accesses in those cases? 

Yes. :-)

On Sun, 10 Sep 2023 at 22:10, Ludovic Courtès <ludo@gnu.org> wrote:

>                      It would probably be easier if you could come with
> an example where there’s Git-related network traffic where there
> shouldn’t.

Do we agree that the both Guile-Git procedures ’remote-fetch’ and
’branch-lookup’ are doing network traffic?

If yes, here two examples:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix pull -q -p /tmp/new
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...

;;; (remote-fetch NETWORK)

;;; (branch-lookup NETWORK)
Authenticating channel 'guix', commits 9edb3f6 to 2eb6df5 (83 new commits)...
Building from this channel:
  guix      https://git.savannah.gnu.org/git/guix.git	2eb6df5
--8<---------------cut here---------------end--------------->8---

Well, that’s not perfect… it becomes much worse:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix time-machine -q -- describe

;;; (remote-fetch NETWORK)

;;; (branch-lookup NETWORK)

;;; (remote-fetch NETWORK)

;;; (branch-lookup NETWORK)
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...

;;; (remote-fetch NETWORK)

;;; (branch-lookup NETWORK)
building /gnu/store/z8jwdgr7z6i8c00msdm2kzfv0n3zp25v-module-import-compiled.drv...
--8<---------------cut here---------------end--------------->8---

Do we agree that is suboptimal?


> ‘cached-channel-instance’ has 3 cases:
>
>   1. Obvious cache hit: This is when CHANNELS specifies the commit of
>      each target channel (this happens for example when you type ‘guix
>      time-machine -q --commit=a4c35c607cfd7d6b0bad90cfcc46188d489e1754)
>      *and* the combination of channels is already in
>      ~/.cache/guix/inferiors.  This is the optimal case: the Git repo
>      doesn’t even need to be opened.

Yes.


>   2. Cache hit: CHANNELS are pinned, but refer to tags (like “v1.2.0”)
>      or short commit IDs (like “a4c35c6”).  In that case,
>      ‘channel-full-commit’ opens the Git repo to resolve the identifier.
>      After that, we go to case #1 or #4.

That’s suboptimal.  Currently, it reads:

--8<---------------cut here---------------start------------->8---
$ guix describe
Generation 28	sept. 06 2023 14:54:50	(current)
  guix 6113e05
    repository URL: https://git.savannah.gnu.org/git/guix.git
    commit: 6113e0529d61df7425f64e30a6bf77f7cfdfe5a5

$ ./pre-inst-env guix time-machine -q --commit=6113e05 -- describe

;;; (remote-fetch NETWORK)

;;; (remote-fetch NETWORK)
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...

;;; (remote-fetch NETWORK)
Computing Guix derivation for 'x86_64-linux'... |  C-c C-c
--8<---------------cut here---------------end--------------->8---

Using patch from #65352 [1], it removes these useless traffic:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix time-machine -q --commit=6113e05 -- describe
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Computing Guix derivation for 'x86_64-linux'... |  C-c C-c
--8<---------------cut here---------------end--------------->8---

Using the proposed patch, it becomes optimal, IMHO.  Well, let discuss
it in #65352 – comments are welcome. :-)

>   3. Cache hit: CHANNELS are not pinned—i.e., they refer to a branch,
>      not a commit.  In that case we first need to perform a ‘git fetch’
>      and then we go to #1 or #4.

I hope that I am convincing you that this case is suboptimal: it does 3
’git fetch’ and more 3 others lookup requiring network.  So it is about
6 network access when only one is necessary, IMHO.


>   4. Cache miss: ‘reference-available?’ returns #f for the channel
>      commits, we got through ‘remote-fetch’ followed by
>      ‘build-derivations’.

This case is part of #2 and discussed in #65352, IMHO.


Cheers,
simon


1: [bug#65352] [PATCH v2] DRAFT git: Avoid touching the network unless needed in 'reference-available?'.
Simon Tournier <zimon.toutoune@gmail.com>
Wed, 06 Sep 2023 16:17:08 +0200
id:32d3fb5066e0b20e200dabef0fba897634e21dda.1694009405.git.zimon.toutoune@gmail.com
https://yhetil.org/guix/32d3fb5066e0b20e200dabef0fba897634e21dda.1694009405.git.zimon.toutoune@gmail.com
https://issues.guix.gnu.org/msgid/32d3fb5066e0b20e200dabef0fba897634e21dda.1694009405.git.zimon.toutoune@gmail.com