From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id qLFjEPmj6WF3YwEAgWs5BA (envelope-from ) for ; Thu, 20 Jan 2022 19:03:37 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id aLUBDvmj6WExPQEA9RJhRA (envelope-from ) for ; Thu, 20 Jan 2022 19:03:37 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id D0A843CC2F for ; Thu, 20 Jan 2022 19:03:36 +0100 (CET) Received: from localhost ([::1]:43734 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nAbma-0008C4-0s for larch@yhetil.org; Thu, 20 Jan 2022 13:03:36 -0500 Received: from eggs.gnu.org ([209.51.188.92]:50372) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nAXBb-00042U-51 for guix-patches@gnu.org; Thu, 20 Jan 2022 08:09:09 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:35556) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nAXBW-0004Kp-Bu for guix-patches@gnu.org; Thu, 20 Jan 2022 08:09:05 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nAXBW-0004qL-7Q for guix-patches@gnu.org; Thu, 20 Jan 2022 08:09:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#53389] [PATCH 3/9] tests/minetest: Run a HTTP server instead of mocking. Resent-From: Maxime Devos Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 20 Jan 2022 13:09:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 53389 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 53389@debbugs.gnu.org Cc: ludo@gnu.org, Maxime Devos Received: via spool by 53389-submit@debbugs.gnu.org id=B53389.164268413518597 (code B ref 53389); Thu, 20 Jan 2022 13:09:02 +0000 Received: (at 53389) by debbugs.gnu.org; 20 Jan 2022 13:08:55 +0000 Received: from localhost ([127.0.0.1]:56688 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nAXBO-0004pT-3y for submit@debbugs.gnu.org; Thu, 20 Jan 2022 08:08:55 -0500 Received: from baptiste.telenet-ops.be ([195.130.132.51]:42012) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nAXBM-0004os-2W for 53389@debbugs.gnu.org; Thu, 20 Jan 2022 08:08:52 -0500 Received: from localhost.localdomain ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by baptiste.telenet-ops.be with bizsmtp id l18q2600H4UW6Th0118rNH; Thu, 20 Jan 2022 14:08:51 +0100 From: Maxime Devos Date: Thu, 20 Jan 2022 13:08:43 +0000 Message-Id: <20220120130849.292178-3-maximedevos@telenet.be> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220120130849.292178-1-maximedevos@telenet.be> References: <20220120130849.292178-1-maximedevos@telenet.be> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1642684131; bh=QPzeQ1OR9EZOc2aVtRHjROXo3bd0aTaZ2ZBGRw7ZSYs=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=NTjbY3JQB4RQt/A1eKbhb2VQCScm6SA/XOl4J4v4SoSl+jK3qeh90V+cZ2oSsKd7q v+TPEVntNyNjsLIStnAOCOQ82Nkdv3fZpMFJntbaZJdIJYa4Ffx379sm7/aJOKaQvs tE5M0swg7T3dzO+oAexyw7Ph95724HtnOIDi4kc2L5sXqChwajUAoG1EMo11HER7mV 3Qyq85saw7PtDRa967qcDFmyj0ttuDdhFC+CjrGVVdYBLmpzlWSamCtH+C76jGaUe2 5MuM+QNR+I0mT0dLGUOixSVV+Edd20QUOfLj89Cr2OsyQdK5MBLhUFiyBHKc/sAVBS BB7ZnfAKibyoQ== X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1642701816; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=QPzeQ1OR9EZOc2aVtRHjROXo3bd0aTaZ2ZBGRw7ZSYs=; b=fl/c19yqVD3jaLg/czGsUfs+e7Eu29pmF1TaRSOeywNV96AqefsKLHYutL+F3/w1jB9qrV wJNAn7QMig+tiWPOeKfH2fXnxVObHddJusRsooA2lpFYfEEALrIij0V48Fg6688WoGBYDq /Qq4KGUHAhhyz4C7mcrflfKelprv/NVhcE1WOF8/Zlc+7qZSbYenOoEVJ8rWmgJZcXtGPJ khInJRGCsBVI50iNuWqOZG1zq6dR3carYdSGlmbhQb74JEFfOjZ0L6Cm85V0NExrS3zsyj ijlJWiQ57BsAnKPVmAP8sySd7jpWpwHDs45FgFqRX+UEUVlXZSyozk0QAQt7LA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1642701816; a=rsa-sha256; cv=none; b=YdCSkXHNHNzxJeKLk+SERLvkGwdooEJ5JlpKnW/gpM7DjTbm/WFVqsrFkTtVFiDdwOvDJx hylBlij13s983QL9MMwn/iixVRQfO4humyDRpdPFLO9ipkXs3oFy60sP45XTD24VLEfsab GnZMxILedq7tG3HmYZkoMnPc5odUVbe5Wvy05owp04hRXZDjA0A/DKuX52XaaVrkrza/bj MFSlNqEyZepl542mnaiU+CKnbveXt/Q5do+6hc1hUhHZTXc5KlMatnMycR2yg8oxPZw+Ih PAdYE04eV3ZsMHmx1mKuvYurirSB8I076ULFXy3shd9iOJSvOgC26iQgwFdc4g== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=telenet.be header.s=r22 header.b=NTjbY3JQ; dmarc=fail reason="SPF not aligned (relaxed)" header.from=telenet.be (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -1.12 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=telenet.be header.s=r22 header.b=NTjbY3JQ; dmarc=fail reason="SPF not aligned (relaxed)" header.from=telenet.be (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: D0A843CC2F X-Spam-Score: -1.12 X-Migadu-Scanner: scn1.migadu.com X-TUID: kZSJAKDZaEih Fixes: Unfortunately, for some unknown reason (a limitation of (guix tests http) perhaps?), parallelism causes ECONNREFUSED in tests but not in the wild, so 'par-map' has to be mocked for now. * tests/minetest.scm (call-with-packages): Avoid mocking by running an actual HTTP server. * guix/import/minetest.scm (par-map): Allow mocking the Minetest importer's use of par-map without impacting anything else. Suggested-by: Ludovic Courtès --- guix/import/minetest.scm | 5 ++- tests/minetest.scm | 82 ++++++++++++++++++++++++---------------- 2 files changed, 53 insertions(+), 34 deletions(-) diff --git a/guix/import/minetest.scm b/guix/import/minetest.scm index 3b2cdcdcac..3eab5f703f 100644 --- a/guix/import/minetest.scm +++ b/guix/import/minetest.scm @@ -19,7 +19,6 @@ (define-module (guix import minetest) #:use-module (ice-9 match) #:use-module (ice-9 receive) - #:use-module (ice-9 threads) #:use-module (ice-9 hash-table) #:use-module (srfi srfi-1) #:use-module (srfi srfi-2) @@ -69,6 +68,10 @@ (define (delete-cr text) (string-delete #\cr text)) +;; Mocked by tests. +(define par-map (@ (ice-9 threads) par-map)) +(set! par-map par-map) + ;;; diff --git a/tests/minetest.scm b/tests/minetest.scm index cbb9e83889..bdd8bd0645 100644 --- a/tests/minetest.scm +++ b/tests/minetest.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2021 Maxime Devos +;;; Copyright © 2021, 2022 Maxime Devos ;;; ;;; This file is part of GNU Guix. ;;; @@ -23,6 +23,7 @@ #:use-module (guix import minetest) #:use-module (guix import utils) #:use-module (guix tests) + #:use-module (guix tests http) #:use-module (guix packages) #:use-module (guix git-download) #:use-module ((gnu packages minetest) @@ -30,6 +31,9 @@ #:use-module ((gnu packages base) #:select (hello)) #:use-module (json) + #:use-module (web request) + #:use-module (web uri) + #:use-module (web client) #:use-module (ice-9 match) #:use-module (srfi srfi-1) #:use-module (srfi srfi-26) @@ -152,7 +156,7 @@ (invalidate-memoization! minetest->guix-package) (define (scm->json-port scm) (open-input-string (scm->json-string scm))) - (define (handle-package url requested-author requested-name . rest) + (define (handle-package subresource requested-author requested-name . rest) (define relevant-argument-list (any (lambda (argument-list) (apply (lambda* (#:key (author "Author") (name "foo") @@ -164,14 +168,15 @@ argument-lists)) (when (not relevant-argument-list) (error "the package ~a/~a should be irrelevant, but ~a is fetched" - requested-author requested-name url)) - (scm->json-port - (apply (match rest - (("") make-package-json) - (("dependencies" "") make-dependencies-json) - (("releases" "") make-releases-json) - (_ (error "TODO ~a" rest))) - relevant-argument-list))) + requested-author requested-name subresource)) + (define json (apply + (match rest + (("") make-package-json) + (("dependencies" "") make-dependencies-json) + (("releases" "") make-releases-json) + (_ (error "TODO ~a" rest))) + relevant-argument-list)) + (values '() (lambda (port) (scm->json json port)))) (define (handle-mod-search sort) ;; Produce search results, sorted by SORT in descending order. (define arguments->key @@ -191,29 +196,40 @@ ("name" . ,name) ("type" . ,type)))) (define argument-list->json (cut apply arguments->json <>)) - (scm->json-port - (list->vector (filter-map argument-list->json sorted-argument-lists)))) - (mock ((guix http-client) http-fetch - (lambda* (url #:key headers) - (unless (string-prefix? "mock://api/packages/" url) - (error "the URL ~a should not be used" url)) - (define resource - (substring url (string-length "mock://api/packages/"))) - (define components (string-split resource #\/)) - (match components - ((author name . rest) - (apply handle-package url author name rest)) - (((? (cut string-prefix? "?type=mod&q=" <>) query)) - (handle-mod-search - (cond ((string-contains query "sort=score") "score") - ((string-contains query "sort=downloads") "downloads") - (#t (error "search query ~a has unknown sort key" - query))))) - (_ - (error "the URL ~a should have an author and name component" - url))))) - (parameterize ((%contentdb-api "mock://api/")) - (thunk)))) + (define json + (list->vector (filter-map argument-list->json sorted-argument-lists))) + (values '() + (lambda (port) (scm->json json port)))) + (with-http-server* + (lambda (request _) + (unless (eq? 'GET (request-method request)) + (error "wrong HTTP method")) + (define resource (uri-path (request-uri request))) + (unless (string-prefix? "/api/packages/" resource) + (error "the resource ~a should not be used" resource)) + (define subresource + (substring resource (string-length "/api/packages/"))) + (define components (string-split subresource #\/)) + (match components + ((author name . rest) + (apply handle-package subresource author name rest)) + (("") + (let ((query (uri-query (request-uri request)))) + (handle-mod-search + (cond ((string-contains query "sort=score") "score") + ((string-contains query "sort=downloads") "downloads") + (#t (error "search query ~a has unknown sort key" + query)))))) + (_ + (error "the resource ~a should have an author and name component" + resource)))) + (parameterize ((%contentdb-api + (format #f "http://localhost:~a/api/" (%http-server-port))) + (current-http-proxy #f)) + ;; XXX: for some unknown reason, parallelism causes ECONNREFUSED in + ;; tests but not in the wild. + (mock ((guix import minetest) par-map map) + (thunk))))) (define* (minetest->guix-package* #:key (author "Author") (name "foo") (sort %default-sort-key) -- 2.30.2