From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms1.migadu.com with LMTPS id oBWNEkDCF2ZeBgEAe85BDQ:P1 (envelope-from ) for ; Thu, 11 Apr 2024 12:58:08 +0200 Received: from aspmx1.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id oBWNEkDCF2ZeBgEAe85BDQ (envelope-from ) for ; Thu, 11 Apr 2024 12:58:08 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gnu.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1712833088; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=k8eQa6meQ/C3swJhf82QftFpTpYIrqjaqSji+CaY4v8=; b=j2RQDAs9p3sbUMYIaeJ38O8DpR4tQZNfYsinU70RrcfLfTD2966IA/VJoMrq9Q7TFVCCB4 VnY6pkiUfvEwZld7v8vGuyz19+RM1KqPLjKhHwlOnvr58y5g0zfnDThRvm1we/p0GEDAeB sEif6MmrVZw06MvVQpn4qIiOSrdYRBP3qwcbhoNxR+XbRil4kuPDaGL9amn9J0f/rC1+0P FGPnZ+yhsjPAv8r9Kaz9ojyggXv1HOjaS66kwItoHXiN3Sf6UQ+dS0IGQlQvfkZCavc46w L5cmqPgFNyqZMm55GE8Pq95o5+dl86FVIvS1ql2klT2GrLHb+mby7zn0tZKF2w== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1712833088; a=rsa-sha256; cv=none; b=cdVBQIdJo4WNHUvJRMRC2ovJDnPmlX/K4w/+fdE9IiYnALWEvDfBJh+ZboKiXd97u7v26u f14FG3eegLNBm2x1DlzdRQ7K//9Pix4O07ZnFjmvA1PpYfEU8DXnR9fsrXETSx1D0x7lgb KRdhLVyPZubhoPiEuUElNMba2H6n0ERHZvTFj9RMkfaoMcvYvKyWc2KOhGWS0tNTRpoEDE TwBfq0gdT5ecWeYU4jo6N7CSCC5bc2kgDkKd5+JuYB9Gg/tO7jvUi+mF0pI2X6euCPXJBY 5lN+Odrb+ueR5/b0c8vo+/8b3joqDZJ/Acknr2o8ngJb7ejgA0OhWY9cvI8esQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gnu.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id CB3F960566 for ; Thu, 11 Apr 2024 12:58:07 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rus7x-000792-Dx; Thu, 11 Apr 2024 06:57:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rus7v-00075z-KM for guix-patches@gnu.org; Thu, 11 Apr 2024 06:57:55 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rus7v-00074D-BX for guix-patches@gnu.org; Thu, 11 Apr 2024 06:57:55 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rus84-0005ZR-Gs for guix-patches@gnu.org; Thu, 11 Apr 2024 06:58:04 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#62202] [PATCH v4 6/6] tests: juliahub: Add unit tests for (guix import juliahub). Resent-From: Nicolas Graves Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 11 Apr 2024 10:58:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62202 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch moreinfo To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: zimoun.toutoune@gmail.com, 62202@debbugs.gnu.org Received: via spool by 62202-submit@debbugs.gnu.org id=B62202.171283303620993 (code B ref 62202); Thu, 11 Apr 2024 10:58:04 +0000 Received: (at 62202) by debbugs.gnu.org; 11 Apr 2024 10:57:16 +0000 Received: from localhost ([127.0.0.1]:55049 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rus7I-0005SW-0g for submit@debbugs.gnu.org; Thu, 11 Apr 2024 06:57:16 -0400 Received: from 16.mo550.mail-out.ovh.net ([178.33.104.224]:34415) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rus7C-0005RT-JA for 62202@debbugs.gnu.org; Thu, 11 Apr 2024 06:57:14 -0400 Received: from director5.ghost.mail-out.ovh.net (unknown [10.108.17.147]) by mo550.mail-out.ovh.net (Postfix) with ESMTP id 4VFc8W6Yt3z1SWB for <62202@debbugs.gnu.org>; Thu, 11 Apr 2024 10:56:59 +0000 (UTC) Received: from ghost-submission-6684bf9d7b-cpszs (unknown [10.110.168.195]) by director5.ghost.mail-out.ovh.net (Postfix) with ESMTPS id 92CF51FE3E; Thu, 11 Apr 2024 10:56:58 +0000 (UTC) Received: from ngraves.fr ([37.59.142.105]) by ghost-submission-6684bf9d7b-cpszs with ESMTPSA id 2NLjMfrBF2YsegAAq7K0Lg (envelope-from ); Thu, 11 Apr 2024 10:56:58 +0000 X-OVh-ClientIp: 81.67.146.208 In-Reply-To: <87bk6jdmdk.fsf@ngraves.fr> References: <20240203230807.25751-1-ngraves@ngraves.fr> <20240203230807.25751-6-ngraves@ngraves.fr> <87plv8bye8.fsf@gnu.org> <87bk6jdmdk.fsf@ngraves.fr> Date: Thu, 11 Apr 2024 12:56:58 +0200 Message-ID: <874jc8np51.fsf@ngraves.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Ovh-Tracer-Id: 13977765870705107559 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedvledrudehkedgfeehucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecuhedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujfgurhephffvvefujghffffkgggtgfesthhqredttddtjeenucfhrhhomheppfhitgholhgrshcuifhrrghvvghsuceonhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrqeenucggtffrrghtthgvrhhnpeekgeffjeelhedvgeevteevffeggeeuvdekheekvddvhfefveejudetvdekfeegffenucffohhmrghinhephhhtthhpqdhfvghttghhuhhnvgigphgvtghtvgguuhhrlhhurhhluggvfhhinhgvmhhotghkqdhhthhtphdqghgvthhtvghsthgtrghsvghlrghmsggurghurhhlrdhrvghsthdpghhithdqshgtmhdrtghomhenucfkphepuddvjedrtddrtddruddpkedurdeijedrudegiedrvddtkedpfeejrdehledrudegvddruddtheenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpeduvdejrddtrddtrddupdhmrghilhhfrhhomhepnhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrpdhnsggprhgtphhtthhopedupdhrtghpthhtohepiedvvddtvdesuggvsggsuhhgshdrghhnuhdrohhrghdpoffvtefjohhsthepmhhoheehtddpmhhouggvpehsmhhtphhouhht X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Nicolas Graves X-ACL-Warn: , Nicolas Graves via Guix-patches From: Nicolas Graves via Guix-patches via Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -6.87 X-Spam-Score: -6.87 X-Migadu-Queue-Id: CB3F960566 X-Migadu-Scanner: mx13.migadu.com X-TUID: FsYy68P5UfBf On 2024-04-09 09:29, Nicolas Graves wrote: > On 2024-04-01 22:50, Ludovic Court=C3=A8s wrote: > >> Hi, >> >> As part of this v4, I would recommend merging patches 2, 3, and 6, such >> that there=E2=80=99s a single self-contained patch adding =E2=80=98guix = import >> juliahub=E2=80=99. (That=E2=80=99s what we usually do and I find it cle= arer because we >> immediately see what goes together.) >> >> Nicolas Graves skribis: >> >>> * tests/juliahub.scm : Add unit tests juliahub-redirect, >>> julia-general-registry-parsing, juliahub-fetch. >> >> Just =E2=80=9CNew file.=E2=80=9D >> >> Some of the other files lack a commit log; we can add it for you, but >> it=E2=80=99d be great if you could do it upfront. >> >>> --- >>> tests/juliahub.scm | 185 +++++++++++++++++++++++++++++++++++++++++++++ >> >> Please add it to =E2=80=98Makefile.am=E2=80=99. >> >> [...] >> >>> +(define (mock-http-fetch testcase) >>> + (lambda (url . rest) >>> + (let ((body (assoc-ref testcase url))) >>> + (if body >>> + (open-input-string body) >>> + (error "mocked http-fetch Unexpected URL: " url))))) >>> + >>> +(define (mock-http-get testcase) >>> + (lambda (url . rest) >>> + (let ((body (assoc-ref testcase url)) >>> + (response-header >>> + (build-response >>> + #:version '(1 . 1) >> >> I strongly encourage using =E2=80=98with-http-server=E2=80=99 using the = same strategy >> that=E2=80=99s used in =E2=80=98tests/pypi.scm=E2=80=99 and others inste= ad of mocking. (=E2=80=98mock=E2=80=99 >> is very sensitive to inlining, plus you sorta have to make assumptions >> about the code path to be able to mock the right things.) > > I can't however mock a git server, right? I still must mock at least the > git repo instead of getting it through a custom server, or is there a > better solution here? It's actually simpler than I thought, but there's an impediment in guile http server implementation that doesn't allow me to push this effort to the end. https://git-scm.com/book/en/v2/Git-Internals-Transfer-Protocols I'm currently writing it, it'll result in a handy helper for tests, such as : (with-git-forge ; spawns a dumb but functional git server '(("MyPackage" . ((add "a.txt" "A") (commit "First commit") (tag "v1.0.0" "Release 1.0")))) (with-julia-test-servers `(("/juliahub/MyPackage/" 200 ,juliahub-redirect.html) ("/juliahub/MyPackage/" 200 ,juliahub-redirect.html) ("/juliahub/MyPackage/MySlg/1.0.0/pkg.json" 200 ,(lambda (port) (display (fixture-pkg.json) port))) ("/general/M/MyPackage/Package.toml" 200 ,(lambda (port) (display (pk 'd (general-Package.toml)) port)))) (juliahub->guix-package "MyPackage"))) However, for that I'll need the http server to be able to respond with a (content-type . (application/x-git-upload-pack-advertisement)) header to git. But in guile's web server implementation, this is not possible because of sanitize-response's charset addition, which is not configurable.=20 That's outside my field, how can we progress further ? We do indeed need such a server to properly test juliahub since we go get the tag from the actual repo (this is justified in the patch series). ___________________________________________________________________________= __ ;;; Git Forge =3D Git HTTP Server with Dump transfer protocol and repositor= ies (define (call-with-temporary-git-repositories names+directives proc) "Call PROC with populated git temporary directories as per NAMES+DIRECTIV= ES; close the directories and delete them when leaving the dynamic extent of th= is call." (call-with-temporary-directory (lambda (directory) (for-each (match-lambda ((name . directives) (populate-git-repository (string-append directory "/" name ".git") directives))) names+directives) (proc directory)))) (define %git-forge-port ;; TCP port to use for the dumb git server. ;; If 0, the OS will automatically choose ;; a port. (make-parameter 0)) (define (binary-file-dump file) "Return a procedure that dumps binary FILE to the given port." (lambda (output) (call-with-input-file file (lambda (input) (put-bytevector output (get-bytevector-all input))) #:binary #t))) (define (serialize-git-ref ref oid) (format #f "~a ~a\n" oid ref)) (define (refs->alist repo refs) (let ((repository (repository-open repo))) (map (lambda (ref) (cons ref (oid->string (reference-name->oid repository ref)))) refs))) (define* (call-with-git-forge repositories+directives thunk) "Call THUNK with a running GIT test forge, i.e. an HTTP server implementi= ng the git dumb protocol (see https://git-scm.com/book/en/v2/Git-Internals-Transfer-Protocols) running. This server behaves like a GIT forge with the repositories constructed from REPOSITORIES+DIRECTIVES. Each element of REPOSITORIES+DIRECTIVES must be a tuple containing a repository name and a list of DIRECTIVES. %git-forge-port will be set to the port listened at The port listened at will be set for the dynamic extent of THUNK." (call-with-temporary-git-repositories repositories+directives (lambda (dir-with-repos) (define responses+data (let ((repos (scandir dir-with-repos (lambda (name) (not (member name '("." ".."))))))) (append-map (lambda (relative-repo) (let* ((name (string-drop-right relative-repo (string-length ".= git"))) (repo (string-append dir-with-repos "/" relative-repo))) `((,(string-append "/" name ".git/info/refs") 200 ((content-type . (application/x-git-upload-pack-advertisem= ent))) ,((@ (gnu services configuration) generic-serialize-alist) string-append serialize-git-ref (refs->alist repo (remote-refs repo)))) (,(string-append "/" name ".git/HEAD") 200 "ref: refs/heads/master") ,@(map (lambda (object) `(,(string-append "/" name ".git/objects/" (string-take-right object 41)) 200 ,(binary-file-dump (string-append repo "/.git/objects/" object)))) (find-files (string-append repo "/.git/objects"))) (,(string-append "/" name ".git/objects/info/http-alternate= s") 200 "") (,(string-append "/" name ".git/objects/info/packs") 200 "")))) repos))) (parameterize ((%http-server-port (%git-forge-port))) (call-with-http-server (pk 'responses+data responses+data) thunk))))) (define-syntax with-git-forge (syntax-rules () ((_ repositories+directives body ...) (call-with-git-forge repositories+directives (lambda () body ...))))) ___________________________________________________________________________= _______ >> >>> +(test-equal "juliahub-fetch" >>> + #t >>> + (mock ((web client) http-get >>> + (mock-http-get fixtures-juliahub-check-test)) >>> + (mock ((guix http-client) http-fetch >>> + (mock-http-fetch fixtures-juliahub-check-test)) >>> + (mock ((guix import utils) git->origin mock-git->origin) >>> + ((@@ (guix import juliahub) juliahub-package?) >>> + ((@@ (guix import juliahub) juliahub-fetch) "MyPa= ckage")))))) >> >> Checking for =E2=80=98juliahub-package?=E2=80=99 doesn=E2=80=99t tell us= much; what about >> checking the whole package, similar to what is done in other importer >> tests? >> >> Thanks, >> Ludo=E2=80=99. --=20 Best regards, Nicolas Graves