From: Nicolas Graves via Guix-patches via <guix-patches@gnu.org>
To: "Ludovic Courtès" <ludo@gnu.org>
Cc: zimoun.toutoune@gmail.com, 62202@debbugs.gnu.org
Subject: [bug#62202] [PATCH v4 6/6] tests: juliahub: Add unit tests for (guix import juliahub).
Date: Thu, 11 Apr 2024 12:56:58 +0200 [thread overview]
Message-ID: <874jc8np51.fsf@ngraves.fr> (raw)
In-Reply-To: <87bk6jdmdk.fsf@ngraves.fr>
On 2024-04-09 09:29, Nicolas Graves wrote:
> On 2024-04-01 22:50, Ludovic Courtès wrote:
>
>> Hi,
>>
>> As part of this v4, I would recommend merging patches 2, 3, and 6, such
>> that there’s a single self-contained patch adding ‘guix import
>> juliahub’. (That’s what we usually do and I find it clearer because we
>> immediately see what goes together.)
>>
>> Nicolas Graves <ngraves@ngraves.fr> skribis:
>>
>>> * tests/juliahub.scm : Add unit tests juliahub-redirect,
>>> julia-general-registry-parsing, juliahub-fetch.
>>
>> Just “New file.”
>>
>> Some of the other files lack a commit log; we can add it for you, but
>> it’d be great if you could do it upfront.
>>
>>> ---
>>> tests/juliahub.scm | 185 +++++++++++++++++++++++++++++++++++++++++++++
>>
>> Please add it to ‘Makefile.am’.
>>
>> [...]
>>
>>> +(define (mock-http-fetch testcase)
>>> + (lambda (url . rest)
>>> + (let ((body (assoc-ref testcase url)))
>>> + (if body
>>> + (open-input-string body)
>>> + (error "mocked http-fetch Unexpected URL: " url)))))
>>> +
>>> +(define (mock-http-get testcase)
>>> + (lambda (url . rest)
>>> + (let ((body (assoc-ref testcase url))
>>> + (response-header
>>> + (build-response
>>> + #:version '(1 . 1)
>>
>> I strongly encourage using ‘with-http-server’ using the same strategy
>> that’s used in ‘tests/pypi.scm’ and others instead of mocking. (‘mock’
>> is very sensitive to inlining, plus you sorta have to make assumptions
>> about the code path to be able to mock the right things.)
>
> I can't however mock a git server, right? I still must mock at least the
> git repo instead of getting it through a custom server, or is there a
> better solution here?
It's actually simpler than I thought, but there's an impediment in guile
http server implementation that doesn't allow me to push this effort to
the end.
https://git-scm.com/book/en/v2/Git-Internals-Transfer-Protocols
I'm currently writing it, it'll result in a handy helper for tests, such
as :
(with-git-forge ; spawns a dumb but functional git server
'(("MyPackage" . ((add "a.txt" "A")
(commit "First commit")
(tag "v1.0.0" "Release 1.0"))))
(with-julia-test-servers
`(("/juliahub/MyPackage/" 200 ,juliahub-redirect.html)
("/juliahub/MyPackage/" 200 ,juliahub-redirect.html)
("/juliahub/MyPackage/MySlg/1.0.0/pkg.json" 200
,(lambda (port) (display (fixture-pkg.json) port)))
("/general/M/MyPackage/Package.toml" 200
,(lambda (port) (display (pk 'd (general-Package.toml)) port))))
(juliahub->guix-package "MyPackage")))
However, for that I'll need the http server to be able to respond with a
(content-type . (application/x-git-upload-pack-advertisement))
header to git. But in guile's web server implementation, this is not
possible because of sanitize-response's charset addition, which is not
configurable.
That's outside my field, how can we progress further ? We do indeed need
such a server to properly test juliahub since we go get the tag from the
actual repo (this is justified in the patch series).
_____________________________________________________________________________
;;; Git Forge = Git HTTP Server with Dump transfer protocol and repositories
(define (call-with-temporary-git-repositories names+directives proc)
"Call PROC with populated git temporary directories as per NAMES+DIRECTIVES;
close the directories and delete them when leaving the dynamic extent of this
call."
(call-with-temporary-directory
(lambda (directory)
(for-each
(match-lambda
((name . directives)
(populate-git-repository
(string-append directory "/" name ".git") directives)))
names+directives)
(proc directory))))
(define %git-forge-port
;; TCP port to use for the dumb git server.
;; If 0, the OS will automatically choose
;; a port.
(make-parameter 0))
(define (binary-file-dump file)
"Return a procedure that dumps binary FILE to the given port."
(lambda (output)
(call-with-input-file file
(lambda (input)
(put-bytevector output (get-bytevector-all input)))
#:binary #t)))
(define (serialize-git-ref ref oid)
(format #f "~a ~a\n" oid ref))
(define (refs->alist repo refs)
(let ((repository (repository-open repo)))
(map
(lambda (ref)
(cons ref (oid->string (reference-name->oid repository ref))))
refs)))
(define* (call-with-git-forge repositories+directives thunk)
"Call THUNK with a running GIT test forge, i.e. an HTTP server implementing
the git dumb protocol (see
https://git-scm.com/book/en/v2/Git-Internals-Transfer-Protocols) running.
This server behaves like a GIT forge with the repositories constructed from
REPOSITORIES+DIRECTIVES. Each element of REPOSITORIES+DIRECTIVES must be a
tuple containing a repository name and a list of DIRECTIVES.
%git-forge-port will be set to the port listened at
The port listened at will be set for the dynamic extent of THUNK."
(call-with-temporary-git-repositories
repositories+directives
(lambda (dir-with-repos)
(define responses+data
(let ((repos (scandir dir-with-repos
(lambda (name)
(not (member name '("." "..")))))))
(append-map
(lambda (relative-repo)
(let* ((name (string-drop-right relative-repo (string-length ".git")))
(repo (string-append dir-with-repos "/" relative-repo)))
`((,(string-append "/" name ".git/info/refs")
200
((content-type . (application/x-git-upload-pack-advertisement)))
,((@ (gnu services configuration) generic-serialize-alist)
string-append
serialize-git-ref
(refs->alist repo (remote-refs repo))))
(,(string-append "/" name ".git/HEAD")
200
"ref: refs/heads/master")
,@(map
(lambda (object)
`(,(string-append "/" name ".git/objects/"
(string-take-right object 41))
200
,(binary-file-dump
(string-append repo "/.git/objects/" object))))
(find-files (string-append repo "/.git/objects")))
(,(string-append "/" name ".git/objects/info/http-alternates")
200
"")
(,(string-append "/" name ".git/objects/info/packs")
200 ""))))
repos)))
(parameterize ((%http-server-port (%git-forge-port)))
(call-with-http-server (pk 'responses+data responses+data) thunk)))))
(define-syntax with-git-forge
(syntax-rules ()
((_ repositories+directives body ...)
(call-with-git-forge repositories+directives (lambda () body ...)))))
__________________________________________________________________________________
>>
>>> +(test-equal "juliahub-fetch"
>>> + #t
>>> + (mock ((web client) http-get
>>> + (mock-http-get fixtures-juliahub-check-test))
>>> + (mock ((guix http-client) http-fetch
>>> + (mock-http-fetch fixtures-juliahub-check-test))
>>> + (mock ((guix import utils) git->origin mock-git->origin)
>>> + ((@@ (guix import juliahub) juliahub-package?)
>>> + ((@@ (guix import juliahub) juliahub-fetch) "MyPackage"))))))
>>
>> Checking for ‘juliahub-package?’ doesn’t tell us much; what about
>> checking the whole package, similar to what is done in other importer
>> tests?
>>
>> Thanks,
>> Ludo’.
--
Best regards,
Nicolas Graves
next prev parent reply other threads:[~2024-04-11 10:58 UTC|newest]
Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-03-15 12:47 [bug#62202] [PATCH 0/21] Juliahub import script Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 01/21] import: juliahub: first script draft Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 02/21] import: utils: Change git->origin function to git->origin+version Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 03/21] import: juliahub: Add support for native-inputs Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 04/21] import: juliahub: Correct source parsing Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 05/21] import: juliahub: Add indirect dependencies Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 06/21] import: juliahub: Add updater and recursive-importer Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 07/21] import: juliahub: Filter out julia stdlibs Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 08/21] import: juliahub: Simplify juliahub dependency management Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 09/21] import: juliahub: Improve " Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 10/21] import: juliahub: Add functions to parse the git repo for a git tag Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 11/21] import: juliahub: Improve test dependencies parsing Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 12/21] import: juliahub: Handle the case where we have a subdirectory Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 13/21] import: juliahub: Add support for versions for juliahub-fetch Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 14/21] import: juliahub: Filter out stdlibs from test-dependencies Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 15/21] import: juliahub: More robust toml regex parser Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 16/21] import: juliahub: Beautify description Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 17/21] import: juliahub: Fix license management Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 18/21] import: juliahub: Fix version management Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 19/21] import: juliahub: Fix undefined homepages Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 20/21] import: utils: Rule out texinfo common syntax from @ escape Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 21/21] import: juliahub: output package names as symbols Nicolas Graves via Guix-patches via
2023-04-07 16:14 ` [bug#62202] [PATCH 0/21] Juliahub import script Simon Tournier
2023-04-08 22:07 ` Ludovic Courtès
2023-08-08 15:24 ` Ludovic Courtès
2023-08-16 15:43 ` Simon Tournier
2023-09-15 9:45 ` Giovanni Biscuolo
2023-09-15 13:32 ` Nicolas Graves via Guix-patches via
2023-09-15 14:01 ` Simon Tournier
2023-09-18 9:31 ` Nicolas Graves via Guix-patches via
2023-09-18 18:06 ` Simon Tournier
2023-09-19 6:23 ` Giovanni Biscuolo
2023-09-19 6:37 ` Simon Tournier
2023-09-19 6:51 ` Nicolas Graves via Guix-patches via
2023-10-02 9:34 ` Simon Tournier
2023-12-19 18:28 ` Nicolas Graves via Guix-patches via
2023-09-18 18:03 ` [bug#62202] [PATCH v2 01/23] DRAFT guix: import: go: Add optional transform-version to vcs->origin Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 02/23] DRAFT TODO guix: import: utils: Add git->origin+dir Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 03/23] DRAFT TODO: guix: import: utils: Fix corner cases beautify-descritption Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 04/23] DRAFT import: Add julia Simon Tournier
2023-09-18 18:03 ` [bug#66088] [PATCH v2 05/23] DRAFT import: juliahub: Add support for native-inputs Simon Tournier
2023-09-18 18:03 ` [bug#66077] [PATCH v2 06/23] DRAFT import: juliahub: Correct source parsing Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 07/23] DRAFT import: juliahub: Add indirect dependencies Simon Tournier
2023-09-18 18:03 ` [bug#66090] [PATCH v2 08/23] DRAFT import: juliahub: Add updater and recursive-importer Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 09/23] DRAFT import: juliahub: Filter out julia stdlibs Simon Tournier
2023-09-18 18:03 ` [bug#66087] " Simon Tournier
2023-09-18 18:03 ` [bug#66076] [PATCH v2 10/23] DRAFT import: juliahub: Simplify juliahub dependency management Simon Tournier
2023-09-18 18:03 ` [bug#66092] [PATCH v2 11/23] DRAFT import: juliahub: Improve " Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 12/23] DRAFT import: juliahub: Add functions to parse the git repo for a git tag Simon Tournier
2023-09-18 18:03 ` [bug#66089] [PATCH v2 13/23] DRAFT import: juliahub: Improve test dependencies parsing Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 14/23] DRAFT import: juliahub: Handle the case where we have a subdirectory Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 15/23] DRAFT import: juliahub: Add support for versions for juliahub-fetch Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 16/23] DRAFT import: juliahub: Filter out stdlibs from test-dependencies Simon Tournier
2023-09-18 18:03 ` [bug#66086] [PATCH v2 17/23] DRAFT import: juliahub: More robust toml regex parser Simon Tournier
2023-09-18 18:03 ` [bug#66079] [PATCH v2 18/23] DRAFT import: juliahub: Beautify description Simon Tournier
2023-09-18 18:03 ` [bug#66080] [PATCH v2 19/23] DRAFT import: juliahub: Fix license management Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 20/23] DRAFT import: juliahub: Fix version management Simon Tournier
2023-09-18 18:03 ` [bug#66085] [PATCH v2 21/23] DRAFT import: juliahub: Fix undefined homepages Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 22/23] DRAFT import: juliahub: output package names as symbols Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 23/23] DRAFT guix: import: julia: Fix beautify Simon Tournier
2023-12-21 14:01 ` [bug#62202] [PATCH v3 1/4] import: utils: Add function git->origin Nicolas Graves via Guix-patches via
2023-12-21 14:01 ` [bug#62202] [PATCH v3 2/4] import: Add juliahub importer Nicolas Graves via Guix-patches via
2023-12-21 14:01 ` [bug#62202] [PATCH v3 3/4] import: juliahub: Beautify description Nicolas Graves via Guix-patches via
2023-12-21 14:01 ` [bug#62202] [PATCH v3 4/4] import: utils: Rule out texinfo common syntax from @ escape Nicolas Graves via Guix-patches via
2024-02-03 23:07 ` [bug#62202] [PATCH v4 1/6] import: utils: Add function git->origin Nicolas Graves via Guix-patches via
2024-02-03 23:07 ` [bug#62202] [PATCH v4 2/6] import: Add juliahub importer Nicolas Graves via Guix-patches via
2024-02-03 23:07 ` [bug#62202] [PATCH v4 3/6] import: juliahub: Beautify description Nicolas Graves via Guix-patches via
2024-02-03 23:07 ` [bug#62202] [PATCH v4 4/6] import: utils: Rule out texinfo common syntax from @ escape Nicolas Graves via Guix-patches via
2024-02-03 23:07 ` [bug#62202] [PATCH v4 5/6] tests: go: Add mock-git->origin function Nicolas Graves via Guix-patches via
2024-02-03 23:07 ` [bug#62202] [PATCH v4 6/6] tests: juliahub: Add unit tests for (guix import juliahub) Nicolas Graves via Guix-patches via
2024-04-01 20:50 ` Ludovic Courtès
2024-04-02 11:52 ` Nicolas Graves via Guix-patches via
2024-04-09 7:29 ` Nicolas Graves via Guix-patches via
2024-04-11 10:56 ` Nicolas Graves via Guix-patches via [this message]
2024-04-16 16:52 ` Ludovic Courtès
2024-04-16 19:11 ` Nicolas Graves via Guix-patches via
2024-04-17 8:51 ` Ludovic Courtès
2024-04-21 16:08 ` Nicolas Graves via Guix-patches via
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=874jc8np51.fsf@ngraves.fr \
--to=guix-patches@gnu.org \
--cc=62202@debbugs.gnu.org \
--cc=ludo@gnu.org \
--cc=ngraves@ngraves.fr \
--cc=zimoun.toutoune@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.