From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id sK1mBNMT8GHGvAAAgWs5BA (envelope-from ) for ; Tue, 25 Jan 2022 16:14:27 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id eFf9ANMT8GG1hQAAauVa8A (envelope-from ) for ; Tue, 25 Jan 2022 16:14:27 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 384E43C31F for ; Tue, 25 Jan 2022 16:14:26 +0100 (CET) Received: from localhost ([::1]:47202 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nCNWa-0002z0-GC for larch@yhetil.org; Tue, 25 Jan 2022 10:14:24 -0500 Received: from eggs.gnu.org ([209.51.188.92]:47570) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nCM1L-00030E-AC for guix-patches@gnu.org; Tue, 25 Jan 2022 08:38:08 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:54535) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nCM1K-00085X-DC for guix-patches@gnu.org; Tue, 25 Jan 2022 08:38:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nCM1K-0003F8-C7 for guix-patches@gnu.org; Tue, 25 Jan 2022 08:38:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#53389] [PATCH 0/9] Replace some mocking with with-http-server*, avoid hardcoding ports, Resent-From: Maxime Devos Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 25 Jan 2022 13:38:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 53389 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 53389@debbugs.gnu.org Received: via spool by 53389-submit@debbugs.gnu.org id=B53389.164311787612455 (code B ref 53389); Tue, 25 Jan 2022 13:38:02 +0000 Received: (at 53389) by debbugs.gnu.org; 25 Jan 2022 13:37:56 +0000 Received: from localhost ([127.0.0.1]:47438 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCM1E-0003Eo-3c for submit@debbugs.gnu.org; Tue, 25 Jan 2022 08:37:56 -0500 Received: from albert.telenet-ops.be ([195.130.137.90]:49220) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCM1A-0003Ed-Qv for 53389@debbugs.gnu.org; Tue, 25 Jan 2022 08:37:54 -0500 Received: from ptr-bvsjgyhxw7psv60dyze.18120a2.ip6.access.telenet.be ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by albert.telenet-ops.be with bizsmtp id n1dq2600E4UW6Th061dqL8; Tue, 25 Jan 2022 14:37:51 +0100 Message-ID: <6b68d02fea18699d944b92e5b0351853201ff511.camel@telenet.be> From: Maxime Devos Date: Tue, 25 Jan 2022 14:37:39 +0100 In-Reply-To: <87wniop7wh.fsf@gnu.org> References: <6b1c1d98514b2547907a81a04c1241d9b865d6fa.camel@telenet.be> <20220120130849.292178-1-maximedevos@telenet.be> <87lez7zpgf.fsf_-_@gnu.org> <687d96c300852e684422de877cd87769daae7ccd.camel@telenet.be> <87wniop7wh.fsf@gnu.org> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-u1UFmVUxIIPZXK/uFFR9" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643117871; bh=BLGNv3t3EDK9ZskxJmTt/TuQBzD8HduV1rBhrvvcs0w=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=brpp17fcy0PQac4ksjZwj/cErflVkpnVU5shWqNAYhuhCWfsF1c4Z5hMXLZ67vw0U 95FC6eUNWhUlYE0ezx1EB7g7uhhmDLGtFNIsaAHgb6zWHcAGcLjyUSQwNhSbYu2abN +ES/12VPvuZTSjzAcNykXSyiUXKFcmauvOkCKGoJRPl76vvxeJTn+bt0o+XN2rlj/s D4YbCPg1rlU5c/kHW1gbavT/yM7CFqvndYnSFZA9tncKrUBbvHHKcl+OYMXBVmdz1o pTycAFb6m6ew+nRV35R0NjS+3COaPU4iKIPGfxIDK9fBL5YR2TPvi91o1ViPyomZha V+CFiLlNc7/mw== X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1643123666; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=BLGNv3t3EDK9ZskxJmTt/TuQBzD8HduV1rBhrvvcs0w=; b=eOD3mOgHUO04grjw1YMXbYYST3SGZbZytHxOzISCnlAium4UlN7nLQJDpLaaOzurIF3tI+ JkYKy43PhMllnPnCgLwDx8uO/BzfpZFLjLyEyKEeeb/XNrc5Oi4IdPTjwiJxv75rl7NOAi WblUqgSSXz0aU4dPN9hLfk+4OHUMhBWj2/xqRSExqh6JQaWPFecuyJpwhSBEQiOARfLgRr wnaiDcxAA4U8ddwQ6URYCJR+pVEfMPeIYV4AmNm8gBrhg6lSMj9YadYD+YjE0IMkrmA1xV vTBz4aZSJ1pJ2tMeO6OTPneF4cpYVDlBXmHte1y/KvtHCKDOcpJIQqh80N5HMg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1643123666; a=rsa-sha256; cv=none; b=TiVRuT47RerZtUuYqpCWOdm3n1fcsd+mfI6e8nAkUJ9f6GKLYS73sv4jBWjwMj7CiLFYjo Wa23//BmqtMvsvMvH4nk2FNR9seZFpCLe8oDvYoDSM0k8YbXV3uaD3PbHbKA48bXKxKlPw ZeTW7s7JgwQBCKy8l1JHJEw/vkgEduamqj1xi1UgdK7XFq3uZHkKx4vYgGeWKoqihD6m7r tn7xYpVOBJwKny1rXYqy+txERVmDL9D3iWYW+RxWgiM0llWMzhUdUAX+y5xVGWTZxnnjou yo58Sy84Y64DXxrByIMr6Vvdu8Y/VR3LvsO6kHd5Sj6R+P6q+ceyzhJbkHEK6w== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=telenet.be header.s=r22 header.b=brpp17fc; dmarc=fail reason="SPF not aligned (relaxed)" header.from=telenet.be (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -5.13 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=telenet.be header.s=r22 header.b=brpp17fc; dmarc=fail reason="SPF not aligned (relaxed)" header.from=telenet.be (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 384E43C31F X-Spam-Score: -5.13 X-Migadu-Scanner: scn0.migadu.com X-TUID: NdiUje+qwsga --=-u1UFmVUxIIPZXK/uFFR9 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s schreef op di 25-01-2022 om 08:54 [+0100]: > So tests/minetest.scm needs more than pre-programmed responses that are > returned in a specified order? >=20 > In ideal black-box testing, sure, you would make the test HTTP server > completely oblivious to what=E2=80=99s being tested, in particular oblivi= ous to > the order in which requests might arrive. >=20 > But in practice, you also want to keep the test infrastructure as simple > and concise as possible, or [...] I think the most concise test infrastructure here, with the least potential of breakage (and hence the least need to test the test infrastructure), would be to use mocking instead of creating a HTTP server listening to a port:=C2=A0 * mocking doesn't require threads, which eliminates the problem of deciding when to stop the thread (e.g., the current version doesn't stop the thread if the thunk throws an exception) and looking out for concurrency-related problems. * mocking doesn't use ports, which eliminates the problem of having to choose a _free_ port and eventually close it * somehow, when using mocking instead of threads, the ECONNREFUSED/par-map from [PATH 3/9] doesn't happen 'with-http-server' could then be renamed to 'with-http-mocking' and be based on mocking. 'with-http-server*' could be removed; tests/minetest.scm would keep mocking directly. As such, mocking seems a lot simpler to me and 'with-http-server' could be made simpler by implementing it on top of mocking. Guile's optimiser has been beginning to do some inlining for a while, so maybe not though. > [...] or you=E2=80=99ll need tests for that infrastructure > too. I guess that=E2=80=99s my main concern. I don't think guix/tests/http.scm has become significantly more complex and less concise, the changes seem more splitting a procedure doing multiple somewhat unrelated things (call-with-http-server, which did both construct a HTTP server and decide what to respond to a request) into two separate procedures (call-with-http-server*, which constructs a HTTP server and lets the handler procedure choose how to map requests to responses, and call-with-http-server's handler procedure). Additionally, it would seem to me that all tests using call-with-http-server and call-with-http-server* are also tests of guix/tests.scm Still, I could write some tests for (guix tests http)? > So I would opt for minimal changes. There are 6 files under tests/ > that mock =E2=80=98http-fetch=E2=80=99. Perhaps we can start converting = them one by > one to the (guix tests http) infrastructure as it exists, and only > change that infrastructure when needed? One of these files is tests/minetest.scm. The main purpose of this patch series was to convert tests/minetest.scm from mocking to (guix tests http). However, the tests in tests/minetest.scm did not fit the original (guix tests http). As such, some changes to the (guix tests http) infrastructure were needed, in [PATCH 1/9]. These changes seem rather minimal to me. That said, there might also be other minimal changes possible. E.g. call-with-packages could generate a map from URI -> response in advance. But that would require modifying both tests/minetest.scm quite a bit and (guix tests http) (to allow optionally ignoring ordering, adding a new flag and hence some complexity). That doesn't seem minimal to me? It would also make things more complicated later, e.g. I would like to someday teach the Minetest importer to use http-fetch/cached, If- Modified-Since and friends to reduce network traffic and some degree of resiliency (in case of flaky interruptions or even being offline) (*). To test that, a static URI->response map would not suffice. Another tweak to the tests would be to verify the content type (for the Minetest importer, ContentDB doesn't care currently, but for the GitHub updater, GitHub does IIUC). (*) Could be useful for supporting something like (packages->manifest (map specification->imported-package '("minetest-not-in-guix-yet@2.1.0" "minetest-mod-old-or-newer-version@9= .0.0"))) without incurring excessive network traffic, and having a chance of working when offline. Greetings, Maxime. p.s. I'll take some time off and write a v2 for the Minetest documentation patch later (before the v2 of this patch series). --=-u1UFmVUxIIPZXK/uFFR9 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYe/9IxccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7kY0AP4vgU/2gv6+eAvLFf/BqjcgdA+5 O6UuD4tSWLCf8xV5KwEAvQAX05x2qWs7Yn4Um56sxs+pY1g6482zvN8c8zZzTwQ= =Gwm/ -----END PGP SIGNATURE----- --=-u1UFmVUxIIPZXK/uFFR9--