unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* GitLab to plans to delete dormant projects
@ 2022-08-06 13:08 Olivier Dion via Development of GNU Guix and the GNU System distribution.
  2022-08-06 14:34 ` Maxime Devos
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Olivier Dion via Development of GNU Guix and the GNU System distribution. @ 2022-08-06 13:08 UTC (permalink / raw)
  To: guix-devel

Hi,

Following this article <https://lwn.net/Articles/903858/>, GitLab is
planning to start deleting project that were idle for > 12 months.

Many packages origin in Guix use an url to a GitLab project.  What are
the consequence of such deletion on Guix reproducibility?  Will it
affects the time-machine?

-- 
Olivier Dion
oldiob.dev



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: GitLab to plans to delete dormant projects
  2022-08-06 13:08 GitLab to plans to delete dormant projects Olivier Dion via Development of GNU Guix and the GNU System distribution.
@ 2022-08-06 14:34 ` Maxime Devos
  2022-08-06 14:41 ` Julien Lepiller
  2022-08-06 21:57 ` John Kehayias
  2 siblings, 0 replies; 6+ messages in thread
From: Maxime Devos @ 2022-08-06 14:34 UTC (permalink / raw)
  To: Olivier Dion, guix-devel


[-- Attachment #1.1.1: Type: text/plain, Size: 1038 bytes --]


On 06-08-2022 15:08, Olivier Dion via Development of GNU Guix and the 
GNU System distribution. wrote:
> Hi,
>
> Following this article<https://lwn.net/Articles/903858/>, GitLab is
> planning to start deleting project that were idle for > 12 months.
>
> Many packages origin in Guix use an url to a GitLab project.  What are
> the consequence of such deletion on Guix reproducibility?  Will it
> affects the time-machine?

software heritage should avoid some problems, but from what I've heard 
it doesn't support all edge cases yet (something about recursive 
checkouts?).

I think it would be a good idea to make some Guile script to find all 
GitLab git checkouts in Guix and run the swh linter on them to make sure 
they are archived.

Additionally, it would be nice to support multiple URLs as fallbacks in 
the 'origin' record for git-fetch (like we have for url-fetch) to avoid 
the three points of failures (SWH and the copy at the substitute 
servers) in the fallback mechanism.

Greetings,
Maxime.


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: GitLab to plans to delete dormant projects
  2022-08-06 13:08 GitLab to plans to delete dormant projects Olivier Dion via Development of GNU Guix and the GNU System distribution.
  2022-08-06 14:34 ` Maxime Devos
@ 2022-08-06 14:41 ` Julien Lepiller
  2022-08-06 14:50   ` Olivier Dion via Development of GNU Guix and the GNU System distribution.
  2022-08-06 21:57 ` John Kehayias
  2 siblings, 1 reply; 6+ messages in thread
From: Julien Lepiller @ 2022-08-06 14:41 UTC (permalink / raw)
  To: Olivier Dion,
	Olivier Dion via Development of GNU Guix and the GNU System
	distribution.,
	guix-devel

[-- Attachment #1: Type: text/plain, Size: 1254 bytes --]

Our build farms need those sources, so they keep them in cache. If you need a source, you can always substitute from the build farms if the origin disappeared (that's actually the default and you don't even need to trust the build farm for that to work).

Another fallback option when substitution is not possible is to get the source from Software Heritage. They keep an archive of almost everything. To do that, they have listers that help tgem find sources fsom different sesvices. They have a lister for GitLab, and even one for Guix. Also, as part of guix lint, a request is sert to swh if the oriqin is not yet archived.

Hopefully that means our origins are saved by Software Heritage, so we can transparently fall back to them.

Le 6 août 2022 15:08:21 GMT+02:00, "Olivier Dion via Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> a écrit :
>Hi,
>
>Following this article <https://lwn.net/Articles/903858/>, GitLab is
>planning to start deleting project that were idle for > 12 months.
>
>Many packages origin in Guix use an url to a GitLab project.  What are
>the consequence of such deletion on Guix reproducibility?  Will it
>affects the time-machine?
>
>-- 
>Olivier Dion
>oldiob.dev
>
>

[-- Attachment #2: Type: text/html, Size: 1627 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: GitLab to plans to delete dormant projects
  2022-08-06 14:41 ` Julien Lepiller
@ 2022-08-06 14:50   ` Olivier Dion via Development of GNU Guix and the GNU System distribution.
  2022-08-07 21:09     ` Ludovic Courtès
  0 siblings, 1 reply; 6+ messages in thread
From: Olivier Dion via Development of GNU Guix and the GNU System distribution. @ 2022-08-06 14:50 UTC (permalink / raw)
  To: Julien Lepiller,
	Olivier Dion via Development of GNU Guix and the GNU System
	distribution.,
	guix-devel

On Sat, 06 Aug 2022, Julien Lepiller <julien@lepiller.eu> wrote:
> Our build farms need those sources, so they keep them in cache. If you
> need a source, you can always substitute from the build farms if the
> origin disappeared (that's actually the default and you don't even
> need to trust the build farm for that to work).

Does the cache as a time to live?  For example, would a source from 2020 in
20 years still be available on the build farms?  Or would the build
farms make a request to Software Heritage?

> Another fallback option when substitution is not possible is to get
> the source from Software Heritage. They keep an archive of almost
> everything. To do that, they have listers that help tgem find sources
> fsom different sesvices. They have a lister for GitLab, and even one
> for Guix. Also, as part of guix lint, a request is sert to swh if the
> oriqin is not yet archived.
>
> Hopefully that means our origins are saved by Software Heritage, so we
> can transparently fall back to them.

Okay great then.  Maybe it would be a good idea to test this by
simulating the deletion of a Gitlab repo.  Better find it out now than
when it's too late.

-- 
Olivier Dion
oldiob.dev


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: GitLab to plans to delete dormant projects
  2022-08-06 13:08 GitLab to plans to delete dormant projects Olivier Dion via Development of GNU Guix and the GNU System distribution.
  2022-08-06 14:34 ` Maxime Devos
  2022-08-06 14:41 ` Julien Lepiller
@ 2022-08-06 21:57 ` John Kehayias
  2 siblings, 0 replies; 6+ messages in thread
From: John Kehayias @ 2022-08-06 21:57 UTC (permalink / raw)
  To: Olivier Dion; +Cc: guix-devel, Julien Lepiller

Hi all,

------- Original Message -------
On Saturday, August 6th, 2022 at 9:08 AM, Olivier Dion via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> wrote:

>
>
> Hi,
>
> Following this article https://lwn.net/Articles/903858/, GitLab is
> planning to start deleting project that were idle for > 12 months.
>
> Many packages origin in Guix use an url to a GitLab project. What are
> the consequence of such deletion on Guix reproducibility? Will it
> affects the time-machine?
>

I think having good backup and archival plans are great, so not to dissuade anyone on this, but as an update here looks like GitLab has walked back on this:

https://www.theregister.com/2022/08/05/gitlab_reverses_deletion_policy/

John


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: GitLab to plans to delete dormant projects
  2022-08-06 14:50   ` Olivier Dion via Development of GNU Guix and the GNU System distribution.
@ 2022-08-07 21:09     ` Ludovic Courtès
  0 siblings, 0 replies; 6+ messages in thread
From: Ludovic Courtès @ 2022-08-07 21:09 UTC (permalink / raw)
  To: Olivier Dion via Development of GNU Guix and the GNU System
	distribution.
  Cc: Julien Lepiller, Olivier Dion

Hi,

Olivier Dion via "Development of GNU Guix and the GNU System
distribution." <guix-devel@gnu.org> skribis:

> On Sat, 06 Aug 2022, Julien Lepiller <julien@lepiller.eu> wrote:
>> Our build farms need those sources, so they keep them in cache. If you
>> need a source, you can always substitute from the build farms if the
>> origin disappeared (that's actually the default and you don't even
>> need to trust the build farm for that to work).
>
> Does the cache as a time to live?  For example, would a source from 2020 in
> 20 years still be available on the build farms?  Or would the build
> farms make a request to Software Heritage?

Our build farms have limited capacity, so they probably won’t keep
everything forever, but that’s the mission of Software Heritage (SWH).

Guix automatically falls back to SWH for some time now, and SWH
archives all of gitlab.com, so I’m not very concerned:

  https://guix.gnu.org/en/blog/2019/connecting-reproducible-deployment-to-a-long-term-source-code-archive/

A shortcoming of our code, as Maxime mentions, is that it doesn’t
correctly handle retrieval of recursive checkouts from SWH.  That’s a
bug to fix, and there’s another one in this area:

  https://issues.guix.gnu.org/48540

> Okay great then.  Maybe it would be a good idea to test this by
> simulating the deletion of a Gitlab repo.  Better find it out now than
> when it's too late.

Other than checking whether ‘guix lint -c archival PKG’ complains, one
can run:

  guix build -S PKG --check

HTH!

Ludo’.


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2022-08-07 21:09 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-06 13:08 GitLab to plans to delete dormant projects Olivier Dion via Development of GNU Guix and the GNU System distribution.
2022-08-06 14:34 ` Maxime Devos
2022-08-06 14:41 ` Julien Lepiller
2022-08-06 14:50   ` Olivier Dion via Development of GNU Guix and the GNU System distribution.
2022-08-07 21:09     ` Ludovic Courtès
2022-08-06 21:57 ` John Kehayias

Code repositories for project(s) associated with this inbox:

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).