From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id CCH/BJ3mrWPYowAAbAwnHQ (envelope-from ) for ; Thu, 29 Dec 2022 20:12:29 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id GIr0BJ3mrWOBTQAA9RJhRA (envelope-from ) for ; Thu, 29 Dec 2022 20:12:29 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id BEA161B326 for ; Thu, 29 Dec 2022 20:12:28 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pAxQt-0000Ei-AT; Thu, 29 Dec 2022 13:15:11 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pAxQm-0000BQ-Vl for guix-patches@gnu.org; Thu, 29 Dec 2022 13:15:05 -0500 Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pAxQm-0004E9-KY for guix-patches@gnu.org; Thu, 29 Dec 2022 13:15:04 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pAxQm-0007Rt-GA for guix-patches@gnu.org; Thu, 29 Dec 2022 13:15:04 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#52555] [PATCH v3 6/8] WIP: eris: Use HTTP to get ERIS blocks. Resent-From: pukkamustard Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 29 Dec 2022 18:15:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555@debbugs.gnu.org Cc: pukkamustard Received: via spool by 52555-submit@debbugs.gnu.org id=B52555.167233767828524 (code B ref 52555); Thu, 29 Dec 2022 18:15:04 +0000 Received: (at 52555) by debbugs.gnu.org; 29 Dec 2022 18:14:38 +0000 Received: from localhost ([127.0.0.1]:32814 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pAxQM-0007Pu-2H for submit@debbugs.gnu.org; Thu, 29 Dec 2022 13:14:38 -0500 Received: from mout01.posteo.de ([185.67.36.65]:33369) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pAxQE-0007ON-Br for 52555@debbugs.gnu.org; Thu, 29 Dec 2022 13:14:31 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id D4569240149 for <52555@debbugs.gnu.org>; Thu, 29 Dec 2022 19:14:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1672337664; bh=Gx1I1s+FDI7Xn3adSjvAKoHNcF9VxPncAf+3vjxJjPw=; h=From:To:Cc:Subject:Date:From; b=VLIn6U614l8dcsWWcXvdX0HzN0kjAU+NByfyD8z54eH9i+OMYHj8jvJc4N27EG2HG JQCb6Zz/w5tgR/x2UaeJhpk8FM3RqDpj6bqGeoCOxSxx0tMq4MfxvEdtRwfjis3vTE Uvj9ZoEaipWw7Hni65Z5fV0Xxg2VnfbUOU4Xr7eijyD7Em6vLTREdnaA8ApcrQMSFJ LMenOMtKfrEetYENYVlP/ngfEIZPBahQTzbP6ulaQtoI3qk5fWsGh0AP346SSO4p74 VP5C2A++caXplw+F3/HbLvfmfFl8NyUO4Rn0YUfCpAsdk3PQI3biLsZrohBM3gJ0o1 ds1obVD5zWwBg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4Njc3g5VGmz6tnF; Thu, 29 Dec 2022 19:14:23 +0100 (CET) From: pukkamustard Date: Thu, 29 Dec 2022 18:13:25 +0000 Message-Id: <20221229181327.758-7-pukkamustard@posteo.net> In-Reply-To: <20221229181327.758-1-pukkamustard@posteo.net> References: <20221229181327.758-1-pukkamustard@posteo.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1672341148; a=rsa-sha256; cv=none; b=Hdsrca9c+LwdfSdHX64ERd4V6Swwfo3Kr5L/MG8WfjGcoH+8o8O8TVVZmpjA4xSAIPMUYw glRW5pozaqQTV+vIFPdGchFt7YMZnCLZ35yusMtL05/z22+KDm38x8DuxrIrC+M9rlHUco VDOfBPaGGV8SV111tF6eo9hsPqRGone9Xjmh0x+Ae5rZL7HerTjGJfxp4CcRgWnpfbKpaB rkQpfPa6VdKD4JleRYsHJ/h5wFg213IGh0Uvx39ZvaIZ6e+lxzXQXRnAtHL3jXHwOkCoHc JTvB1IX7zTwhF5/7uG4qGOqaszX6VQwE2tpnk0KRPFojoQYxguAGDVe+wTuCbA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=posteo.net header.s=2017 header.b=VLIn6U61; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (strict)" header.from=posteo.net (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1672341148; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=Bv+L6Q9Sk4QXwJ9uwjBcoorcFwee/DGgWrgYCTz3QVI=; b=OAQLwgraIp+Ufwa6AW/iQAG0f/VrAk5H1ZbToRh4uqqZ7qXvK3cuACZ+1nWXXSNeMqOi/T 2RPezjCAnDz9BQVMHXJU12PfEwxiPrF2+0uMNtvViklrhYUhygQiQQFxhvKCE7fmAztrzl jUixJPKfWG9Lb9BIuZ2IBkTL7XxBZs3XxfGVzsvqpMw9HPzygcerbZ4U1m69n+SOvEa9ls l1Ju7PBNlmzBwy/Tzt+aE533cRK1fzG9bQniEsu7rfTm5bpiiKkKhL7y6ABGR+PkggZFil GJ1b0LWTwD6ZcyGzg8RNhR0xW7PAMcmlZ1q3hMJFyvbiO1RkGoR7TZtZrKpXow== X-Spam-Score: 0.27 X-Migadu-Queue-Id: BEA161B326 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=posteo.net header.s=2017 header.b=VLIn6U61; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (strict)" header.from=posteo.net (policy=none) X-Migadu-Scanner: scn0.migadu.com X-Migadu-Spam-Score: 0.27 X-TUID: kgP6f1idLKqI - guix/eris.scm (guix-eris-block-ref): Refactor from call-with-eris-block-ref and use eris-http-block-ref. - guix/eris/fs-store.scm (eris-fs-store-ref): Refactor to take keyword arguments. - guix/scripts/substitute.scm (download-nar): Use guix-eris-block-ref. - guix/scripts/publish.scm (make-request-handler): Use refactored eris-fs-store-ref. TODO: - default value for %eris-peers parameter in (eris) --- guix/eris.scm | 37 +++++++++++++++++++++++++++++++------ guix/eris/fs-store.scm | 21 ++++++++++----------- guix/scripts/publish.scm | 9 ++++----- guix/scripts/substitute.scm | 14 ++++++++------ 4 files changed, 53 insertions(+), 28 deletions(-) diff --git a/guix/eris.scm b/guix/eris.scm index d9a0914b67..4af17c2807 100644 --- a/guix/eris.scm +++ b/guix/eris.scm @@ -21,9 +21,13 @@ (define-module (guix eris) #:use-module (guix config) #:use-module (guix eris fs-store) + #:use-module (guix eris http) + + #:use-module (web uri) + #:use-module (ice-9 match) #:export (guix-eris-block-reducer - call-with-eris-block-ref + guix-eris-block-ref %eris-block-store-directory)) @@ -36,8 +40,29 @@ (define (guix-eris-block-reducer) "Returns a block reducer that stores blocks of ERIS encoded content." (eris-fs-store-reducer (%eris-block-store-directory))) -(define (call-with-eris-block-ref f) - (let ((fs-store-block-ref - (eris-fs-store-ref - (%eris-block-store-directory)))) - (f fs-store-block-ref))) +(define %eris-peers + (make-parameter + ;; TODO + (list (string->uri "http://localhost:8081")))) + +(define* (try-in-order ref #:key block-refs) + (match block-refs + ((block-ref . rest) + (let ((block (block-ref ref))) + (if block + block + (try-in-order ref #:block-refs rest)))) + (() #f))) + +(define* (guix-eris-block-ref ref #:key open-connection) + (try-in-order + ref + #:block-refs + (list + (lambda (ref) + (eris-fs-store-ref ref + #:store-directory (%eris-block-store-directory))) + (lambda (ref) + (eris-http-block-ref ref + #:host (string->uri "http://localhost:8081") + #:open-connection open-connection))))) diff --git a/guix/eris/fs-store.scm b/guix/eris/fs-store.scm index 2ef7607988..38f5926280 100644 --- a/guix/eris/fs-store.scm +++ b/guix/eris/fs-store.scm @@ -54,14 +54,13 @@ (define (eris-fs-store-reducer store-directory) #t)))) -(define (eris-fs-store-ref store-directory) - (lambda (ref) - (let* ((b32 (base32-encode ref)) - (pre (substring b32 0 2)) - (suf (substring b32 2)) - (path (string-append store-directory "/" pre "/" suf))) - (if (file-exists? path) - (call-with-input-file path - (lambda (port) (get-bytevector-all port)) - #:binary #t) - #f)))) +(define* (eris-fs-store-ref ref #:key store-directory) + (let* ((b32 (base32-encode ref)) + (pre (substring b32 0 2)) + (suf (substring b32 2)) + (path (string-append store-directory "/" pre "/" suf))) + (if (file-exists? path) + (call-with-input-file path + (lambda (port) (get-bytevector-all port)) + #:binary #t) + #f))) diff --git a/guix/scripts/publish.scm b/guix/scripts/publish.scm index 15bdf02670..0ce50b2942 100644 --- a/guix/scripts/publish.scm +++ b/guix/scripts/publish.scm @@ -1085,13 +1085,12 @@ (define nar-path? (let ((expected (split-and-decode-uri-path nar-path))) (cut equal? expected <>))) - ;; Get ERIS blocks directly from the filesystem store. - (define eris-block-ref - (eris-fs-store-ref (%eris-block-store-directory))) - ;; Create a handler for resolving blake2b URN queries. (define blake2b-urn-query-ref - (make-blake2b-urn-query-ref eris-block-ref)) + (make-blake2b-urn-query-ref + (lambda (ref) (eris-fs-store-ref + ref + #:store-directory (%eris-block-store-directory))))) (define (handle request body) (format #t "~a ~a~%" diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index 8cf011d7e6..14a21f6c37 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -482,12 +482,14 @@ (define (fetch uri) ((urn) (let ((read-capability (->eris-read-capability uri))) (if (eris-read-capability? read-capability) - (call-with-eris-block-ref - (lambda (block-ref) - (values (open-eris-input-port - read-capability - #:block-ref block-ref) - #f))) + (values + (open-eris-input-port + read-capability + #:block-ref (lambda (ref) + (guix-eris-block-ref + ref + #:open-connection open-connection-for-uri/cached))) + #f) (leave (G_ "unsupported substitute URI scheme: ~a~%") (uri->string uri))))) (else -- 2.38.1