unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
* [bug#50040] [PATCH 0/2] publish: Always render nar/narinfo during backing.
@ 2021-08-13 10:28 Mathieu Othacehe
  2021-08-13 10:30 ` [bug#50040] [PATCH 1/2] publish: Defer narinfo string creation to the http-write Mathieu Othacehe
  2021-08-22  7:33 ` Mathieu Othacehe
  0 siblings, 2 replies; 13+ messages in thread
From: Mathieu Othacehe @ 2021-08-13 10:28 UTC (permalink / raw)
  To: 50040; +Cc: Mathieu Othacehe

Hello,

This patchset does two things:

* It ensures that the narinfo creation is always done in a dedicated thread to
  prevent blocking the main publish thread and cause the timeout connection
  errors that some users experimented.

* It removes the cache bypass support and always renders temporary narinfo and
  nar files during baking. This way, the first user asking for a substitute
  won't get a 404 error. It also removes the need for Cuirass to pre-bake nar
  files. Finally it will make way easier to use a publish server with caching
  in a CI context.

The downside is that it makes the publish server code harder to understand and
that it consumes more server resources by duplicating the nar/narinfo file
creation.

In the long run, spawning new threads constantly won't scale. I think it would
be nice to rewrite the publish server using Fiber or an HTTP server framework.

Thanks,

Mathieu

Mathieu Othacehe (2):
  publish: Defer narinfo string creation to the http-write.
  publish: Remove cache bypass support.

 doc/guix.texi            |  15 -----
 gnu/services/base.scm    |  11 +---
 guix/scripts/publish.scm | 136 +++++++++++++++++++++++----------------
 tests/publish.scm        |  88 +++----------------------
 4 files changed, 93 insertions(+), 157 deletions(-)

-- 
2.32.0





^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2021-10-08  7:05 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-13 10:28 [bug#50040] [PATCH 0/2] publish: Always render nar/narinfo during backing Mathieu Othacehe
2021-08-13 10:30 ` [bug#50040] [PATCH 1/2] publish: Defer narinfo string creation to the http-write Mathieu Othacehe
2021-08-13 10:30   ` [bug#50040] [PATCH 2/2] publish: Remove cache bypass support Mathieu Othacehe
2021-08-30 22:31   ` [bug#50040] [PATCH 0/2] publish: Always render nar/narinfo during backing Ludovic Courtès
2021-08-31  9:08     ` Mathieu Othacehe
2021-09-01 20:48       ` Ludovic Courtès
2021-10-06  8:58   ` Mathieu Othacehe
2021-10-08  7:04     ` Mathieu Othacehe
2021-08-22  7:33 ` Mathieu Othacehe
2021-08-30 22:33   ` Ludovic Courtès
2021-09-06 13:54     ` Ludovic Courtès
2021-09-17 15:27       ` Mathieu Othacehe
2021-10-05 10:07         ` Mathieu Othacehe

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).