From: Simon Tournier <zimon.toutoune@gmail.com>
To: 62202@debbugs.gnu.org
Cc: zimon.toutoune@gmail.com
Subject: [bug#62202] [PATCH v2 12/23] DRAFT import: juliahub: Add functions to parse the git repo for a git tag.
Date: Mon, 18 Sep 2023 20:03:19 +0200 [thread overview]
Message-ID: <e42032f4db1c8b76ff5ba94805deba47992a7c39.1695060058.git.zimon.toutoune@gmail.com> (raw)
In-Reply-To: <c50c63bce62fc13aca5b51aaddf4158d2108c8ee.1695060058.git.zimon.toutoune@gmail.com>
From: Nicolas Graves via Guix-patches via <guix-patches@gnu.org>
Signed-off-by: Simon Tournier <zimon.toutoune@gmail.com>
---
guix/import/juliahub.scm | 62 ++++++++++++++++++++++++++++++++++++----
1 file changed, 56 insertions(+), 6 deletions(-)
diff --git a/guix/import/juliahub.scm b/guix/import/juliahub.scm
index fc25ba1d4220..5327e923251d 100644
--- a/guix/import/juliahub.scm
+++ b/guix/import/juliahub.scm
@@ -26,6 +26,7 @@ (define-module (guix import juliahub)
#:use-module (srfi srfi-9)
#:use-module (srfi srfi-11)
#:use-module (guix http-client)
+ #:use-module (guix git)
#:use-module (guix import utils)
#:use-module (guix import json)
#:use-module (guix base32)
@@ -38,6 +39,48 @@ (define-module (guix import juliahub)
%juliahub-updater
juliahub-recursive-import))
+;; Juliahub may be more up-to-date than the General registry or the actual git
+;; tag (it seems around 6 hours pass between the time a commit is supplied to
+;; JuliaRegistrator as a release, and the time Julia TagBot Github Action makes
+;; the git tag). We have no simple way to get the commit of the latest-version.
+;; Thus the best simple thing we can do is get the latest-git-version, and
+;; import this version instead. We do this by parsing Package.toml in the General
+;; registry, and then getting the refs of the git repo supplied by this
+;; file. Parsing this file is also necessary if the package is in a subdir of a
+;; git repository, because the information isn't present in Juliahub.
+;; There's a last case where some Julia packages are not based on a particular
+;; git tag, and where looking for the commit is tedious and artificial. In these
+;; cases, we introduce the tree-commit which is available in the Versions.toml
+;; file in the General repository. This is equivalent to a commit, since we have
+;; a unique hash of the listing of files and directories, thus it can be used to
+;; identify the state of a repository.
+
+(define %general-base-url
+ "https://raw.githubusercontent.com/JuliaRegistries/General/master/")
+
+(define (general-url package-name file)
+ (let ((folder (string-capitalize (string-take package-name 1))))
+ (string-append
+ %general-base-url folder "/" package-name "/" file)))
+
+(define (ini-list->alist lst)
+ (match lst
+ ((attribute '= value ooo ...)
+ `((,attribute . ,value) ,@(ini-list->alist ooo)))
+ ('()
+ '())))
+
+(define (ini-fetch url)
+ (let* ((port (http-fetch url #:text? #t))
+ (ini-list (stream->list (port->stream port read))))
+ (close-port port)
+ (ini-list->alist ini-list)))
+
+(define (latest-git-tag repo)
+ (let* ((last-ref (last (remote-refs repo #:tags? #t)))
+ (last-git-tag (last (string-split last-ref #\/))))
+ (string-drop last-git-tag 1)))
+
;; To update, see file sysimg.jl
(define %julia-stdlibs
(list "julia"
@@ -194,14 +237,21 @@ (define (make-julia-sexp name source home-page synopsis description
((license) (license->symbol license))
(_ `(list ,@(map license->symbol licenses)))))))
+;; TODO handle subdir case properly.
+
(define* (juliahub->guix-package package-name
#:key version #:allow-other-keys)
"Fetch the metadata for PACKAGE-NAME from juliahub.org, and return the
`package' s-expression corresponding to that package, or #f on failure.
Optionally include a VERSION string to fetch a specific version juliahub."
- (let ((package (if version
- (juliahub-fetch package-name version)
- (juliahub-fetch package-name))))
+ (let* ((package-toml (ini-fetch (general-url name "Package.toml")))
+ (subdir (assoc-ref package-toml 'subdir))
+ (tag (latest-git-tag (assoc-ref package-toml 'repo)))
+ (package (if version
+ (juliahub-fetch package-name version)
+ (if tag
+ (juliahub-fetch package-name tag)
+ (juliahub-fetch package-name)))))
(if package
(let-values (((source directory)
(git->origin+dir
@@ -233,13 +283,13 @@ (define* (juliahub->guix-package package-name
(define (guix-package->juliahub-name package)
(let* ((url (juliahub-package-url package))
- (git-name (car (last-pair (string-split url #\/))))
+ (git-name (last (string-split url #\/)))
(ungitted-name (if (string-suffix? ".git" git-name)
(string-drop-right git-name 4)
git-name))
(package-name (if (string-suffix? ".jl" ungitted-name)
- (string-drop-right ungitted-name 4)
- ungitted-name)))
+ (string-drop-right ungitted-name 4)
+ ungitted-name)))
package-name))
(define* (import-release package #:key (version #f))
--
2.38.1
next prev parent reply other threads:[~2023-09-18 18:08 UTC|newest]
Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-03-15 12:47 [bug#62202] [PATCH 0/21] Juliahub import script Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 01/21] import: juliahub: first script draft Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 02/21] import: utils: Change git->origin function to git->origin+version Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 03/21] import: juliahub: Add support for native-inputs Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 04/21] import: juliahub: Correct source parsing Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 05/21] import: juliahub: Add indirect dependencies Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 06/21] import: juliahub: Add updater and recursive-importer Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 07/21] import: juliahub: Filter out julia stdlibs Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 08/21] import: juliahub: Simplify juliahub dependency management Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 09/21] import: juliahub: Improve " Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 10/21] import: juliahub: Add functions to parse the git repo for a git tag Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 11/21] import: juliahub: Improve test dependencies parsing Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 12/21] import: juliahub: Handle the case where we have a subdirectory Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 13/21] import: juliahub: Add support for versions for juliahub-fetch Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 14/21] import: juliahub: Filter out stdlibs from test-dependencies Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 15/21] import: juliahub: More robust toml regex parser Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 16/21] import: juliahub: Beautify description Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 17/21] import: juliahub: Fix license management Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 18/21] import: juliahub: Fix version management Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 19/21] import: juliahub: Fix undefined homepages Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 20/21] import: utils: Rule out texinfo common syntax from @ escape Nicolas Graves via Guix-patches via
2023-03-15 12:51 ` [bug#62202] [PATCH 21/21] import: juliahub: output package names as symbols Nicolas Graves via Guix-patches via
2023-04-07 16:14 ` [bug#62202] [PATCH 0/21] Juliahub import script Simon Tournier
2023-04-08 22:07 ` Ludovic Courtès
2023-08-08 15:24 ` Ludovic Courtès
2023-08-16 15:43 ` Simon Tournier
2023-09-15 9:45 ` Giovanni Biscuolo
2023-09-15 13:32 ` Nicolas Graves via Guix-patches via
2023-09-15 14:01 ` Simon Tournier
2023-09-18 9:31 ` Nicolas Graves via Guix-patches via
2023-09-18 18:06 ` Simon Tournier
2023-09-19 6:23 ` Giovanni Biscuolo
2023-09-19 6:37 ` Simon Tournier
2023-09-19 6:51 ` Nicolas Graves via Guix-patches via
2023-10-02 9:34 ` Simon Tournier
2023-12-19 18:28 ` Nicolas Graves via Guix-patches via
2023-09-18 18:03 ` [bug#62202] [PATCH v2 01/23] DRAFT guix: import: go: Add optional transform-version to vcs->origin Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 02/23] DRAFT TODO guix: import: utils: Add git->origin+dir Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 03/23] DRAFT TODO: guix: import: utils: Fix corner cases beautify-descritption Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 04/23] DRAFT import: Add julia Simon Tournier
2023-09-18 18:03 ` [bug#66088] [PATCH v2 05/23] DRAFT import: juliahub: Add support for native-inputs Simon Tournier
2023-09-18 18:03 ` [bug#66077] [PATCH v2 06/23] DRAFT import: juliahub: Correct source parsing Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 07/23] DRAFT import: juliahub: Add indirect dependencies Simon Tournier
2023-09-18 18:03 ` [bug#66090] [PATCH v2 08/23] DRAFT import: juliahub: Add updater and recursive-importer Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 09/23] DRAFT import: juliahub: Filter out julia stdlibs Simon Tournier
2023-09-18 18:03 ` [bug#66087] " Simon Tournier
2023-09-18 18:03 ` [bug#66076] [PATCH v2 10/23] DRAFT import: juliahub: Simplify juliahub dependency management Simon Tournier
2023-09-18 18:03 ` [bug#66092] [PATCH v2 11/23] DRAFT import: juliahub: Improve " Simon Tournier
2023-09-18 18:03 ` Simon Tournier [this message]
2023-09-18 18:03 ` [bug#66089] [PATCH v2 13/23] DRAFT import: juliahub: Improve test dependencies parsing Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 14/23] DRAFT import: juliahub: Handle the case where we have a subdirectory Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 15/23] DRAFT import: juliahub: Add support for versions for juliahub-fetch Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 16/23] DRAFT import: juliahub: Filter out stdlibs from test-dependencies Simon Tournier
2023-09-18 18:03 ` [bug#66086] [PATCH v2 17/23] DRAFT import: juliahub: More robust toml regex parser Simon Tournier
2023-09-18 18:03 ` [bug#66079] [PATCH v2 18/23] DRAFT import: juliahub: Beautify description Simon Tournier
2023-09-18 18:03 ` [bug#66080] [PATCH v2 19/23] DRAFT import: juliahub: Fix license management Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 20/23] DRAFT import: juliahub: Fix version management Simon Tournier
2023-09-18 18:03 ` [bug#66085] [PATCH v2 21/23] DRAFT import: juliahub: Fix undefined homepages Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 22/23] DRAFT import: juliahub: output package names as symbols Simon Tournier
2023-09-18 18:03 ` [bug#62202] [PATCH v2 23/23] DRAFT guix: import: julia: Fix beautify Simon Tournier
2023-12-21 14:01 ` [bug#62202] [PATCH v3 1/4] import: utils: Add function git->origin Nicolas Graves via Guix-patches via
2023-12-21 14:01 ` [bug#62202] [PATCH v3 2/4] import: Add juliahub importer Nicolas Graves via Guix-patches via
2023-12-21 14:01 ` [bug#62202] [PATCH v3 3/4] import: juliahub: Beautify description Nicolas Graves via Guix-patches via
2023-12-21 14:01 ` [bug#62202] [PATCH v3 4/4] import: utils: Rule out texinfo common syntax from @ escape Nicolas Graves via Guix-patches via
2024-02-03 23:07 ` [bug#62202] [PATCH v4 1/6] import: utils: Add function git->origin Nicolas Graves via Guix-patches via
2024-02-03 23:07 ` [bug#62202] [PATCH v4 2/6] import: Add juliahub importer Nicolas Graves via Guix-patches via
2024-02-03 23:07 ` [bug#62202] [PATCH v4 3/6] import: juliahub: Beautify description Nicolas Graves via Guix-patches via
2024-02-03 23:07 ` [bug#62202] [PATCH v4 4/6] import: utils: Rule out texinfo common syntax from @ escape Nicolas Graves via Guix-patches via
2024-02-03 23:07 ` [bug#62202] [PATCH v4 5/6] tests: go: Add mock-git->origin function Nicolas Graves via Guix-patches via
2024-02-03 23:07 ` [bug#62202] [PATCH v4 6/6] tests: juliahub: Add unit tests for (guix import juliahub) Nicolas Graves via Guix-patches via
2024-04-01 20:50 ` Ludovic Courtès
2024-04-02 11:52 ` Nicolas Graves via Guix-patches via
2024-04-09 7:29 ` Nicolas Graves via Guix-patches via
2024-04-11 10:56 ` Nicolas Graves via Guix-patches via
2024-04-16 16:52 ` Ludovic Courtès
2024-04-16 19:11 ` Nicolas Graves via Guix-patches via
2024-04-17 8:51 ` Ludovic Courtès
2024-04-21 16:08 ` Nicolas Graves via Guix-patches via
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e42032f4db1c8b76ff5ba94805deba47992a7c39.1695060058.git.zimon.toutoune@gmail.com \
--to=zimon.toutoune@gmail.com \
--cc=62202@debbugs.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).