Why "update substitutes?"

Why "update substitutes?"

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi,

When updating, my equipment spends nearly as much time "updating
substitutes" as it does downloading substitutes.

The activity is probably exacerbated because I cross-publish substitutes
in my local network.

Would it be more efficient to check whether a particular substitute is
available in an individual query (like DNS) and stop querying for that
substitute after a server was found?

Kind regards
Felix

Re: Why "update substitutes?"

[Attila Lendvai]

> When updating, my equipment spends nearly as much time "updating
> substitutes" as it does downloading substitutes.


while we are at it, these messages never made sense to me:

substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 50.0%

i still don't really know what's happening there, but maybe something like "fetching the registry" or "fetching metadata" would be more descriptive of it?

to me "the substitutes" means the binary artifacts.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“The only time my education was interrupted was when I was in school.”
	— George Bernard Shaw (1856–1950)

Re: Why "update substitutes?"

[Tobias Geerinckx-Rice]

Hi Attila,

On 25 October 2024 23:42:32 UTC, Attila Lendvai <attila@lendvai.name> wrote:
>i still don't really know what's happening there, but maybe something like "fetching the registry" or "fetching metadata" would be more descriptive of it?

Howsabout…  ‘looking for suitable substitutes on <server>’ if we must change the string?

It describes what's happening, implies why it's happening, and avoids clichéd jargon.  I wouldn't even know what ‘fetching the registry’ means.

>to me "the substitutes" means the binary artifacts.

Yes.


Kind regards,

T G-R

Sent on the go.  Excuse or enjoy my brevity.

Re: Why "update substitutes?"

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1509 bytes --]

On Sat, Oct 26, 2024 at 05:07:21PM +0000, Tobias Geerinckx-Rice wrote:
> Hi Attila,
> 
> On 25 October 2024 23:42:32 UTC, Attila Lendvai <attila@lendvai.name> wrote:
> >i still don't really know what's happening there, but maybe something like "fetching the registry" or "fetching metadata" would be more descriptive of it?
> 
> Howsabout…  ‘looking for suitable substitutes on <server>’ if we must change the string?
> 
> It describes what's happening, implies why it's happening, and avoids clichéd jargon.  I wouldn't even know what ‘fetching the registry’ means.
> 
> >to me "the substitutes" means the binary artifacts.
> 
> Yes.

I think it comes from the assumption that if a substitute isn't
available then it's dependencies are (all) available. Thus we get stuck
in a recursion of looking further and further to try to find the base of
what is available and will allow us to build the derivation we're
actually looking for.

One option I've been mulling over would be to look for a substitute, and
if that failed to automatically look ALL the way back immediately. I
don't like it because I'm sure it's overkill and would probably add
strain on the substitute servers, but it would decrease the number of
times 'searching for substitutes' is printed.

-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Why "update substitutes?"

[Ludovic Courtès]

Hi,

Felix Lechner via "Development of GNU Guix and the GNU System
distribution." <guix-devel@gnu.org> skribis:

> When updating, my equipment spends nearly as much time "updating
> substitutes" as it does downloading substitutes.
>
> The activity is probably exacerbated because I cross-publish substitutes
> in my local network.
>
> Would it be more efficient to check whether a particular substitute is
> available in an individual query (like DNS) and stop querying for that
> substitute after a server was found?

The “updating substitutes” bit corresponds to making pipelined GET
requests for narinfos¹ to figure out upfront what’s going to be
available as substitutes and what will have to be built.

There’s a cache at /var/guix/substitute/cache so Guix doesn’t send the
same request several times in a row.

I’m not sure there’s much to be optimized in this area, but perhaps we’d
find something by studying more closely pathological cases?

Thanks,
Ludo’.

¹ Like <https://ci.guix.gnu.org/fk39d3y3zyr6ajyzy8d6ghd0sj524cs5.narinfo>.

Re: Why "update substitutes?"

[Suhail Singh]

Ludovic Courtès <ludo@gnu.org> writes:

> The “updating substitutes” bit corresponds to making pipelined GET
> requests for narinfos¹ to figure out upfront what’s going to be
> available as substitutes and what will have to be built.

"Fetching list of available substitutes" would be clearer, IMHO.

-- 
Suhail

Re: Why "update substitutes?"

[Tobias Geerinckx-Rice]

On 26 October 2024 16:54:04 UTC, Suhail Singh <suhailsingh247@gmail.com> wrote:
>"Fetching list of available substitutes" would be clearer, IMHO.

Less accurate though.  We never care about or download a complete list of available substitutes.



Kind regards,

T G-R

Sent on the go.  Excuse or enjoy my brevity.

Re: Why "update substitutes?"

[Suhail Singh]

Tobias Geerinckx-Rice <me@tobias.gr> writes:

>>"Fetching list of available substitutes" would be clearer, IMHO.
>
> Less accurate though.  We never care about or download a complete list
> of available substitutes.

To be fair it didn't say "the list" nor "complete list".  Regardless,
would "updating list of available substitutes" be better?

Surely we can do better than the cryptic "updating substitutes" which
never made sense to me prior to this discussion =/.

-- 
Suhail

Re: Why "update substitutes?"

[Tobias Geerinckx-Rice]

On 26 October 2024 17:23:37 UTC, Suhail Singh <suhailsingh247@gmail.com> wrote:
>To be fair it didn't say "the list"

Very fair, sorry.  I wasn't even aware than my brain added the ‘the’ there.

>would "updating list of available substitutes" be better?

Well, I prefer my suggestion, but then I would, wouldn't I?  ;-)  You go looking for things.  Somebody saying that they are going looking for a list of things is how you know they are secretly a robot.

Still, I think both are better than ‘updating [implied words that do make sense but you need to know them] substitutes’.



Kind regards,

T G-R

Sent on the go.  Excuse or enjoy my brevity.

Re: Why "update substitutes?"

[Tobias Geerinckx-Rice]

Hi Felix,

On 25 October 2024 20:50:46 UTC, "Felix Lechner via Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> wrote:
>Would it be more efficient to check whether a particular substitute is
>available in an individual query (like DNS) and stop querying for that
>substitute after a server was found?

Doesn't this describe what Guix already does?  If not, how?

I don't understand the 'like DNS' part, though, so maybe that's where it differs.  Do you mean substitute a different protocol for HTTP?

We already use a single TCP connection per server for all narinfo queries, and nars found on one server aren't queried again.



Kind regards,

T G-R

Sent on the go.  Excuse or enjoy my brevity.