From: "JOULAUD François via Guix-patches via" <guix-patches@gnu.org>
To: "Ludovic Courtès" <ludo@gnu.org>
Cc: "44178@debbugs.gnu.org" <44178@debbugs.gnu.org>,
Katherine Cox-Buday <cox.katherine.e@gmail.com>,
Maxim Cournoyer <maxim.cournoyer@gmail.com>
Subject: [bug#44178] Add a Go Module Importer
Date: Mon, 8 Mar 2021 13:54:04 +0000 [thread overview]
Message-ID: <20210308135025.vn32lypnivpsilcg@fjo-extia-HPdeb.example.avalenn.eu> (raw)
In-Reply-To: <871rcxte52.fsf_-_@gnu.org>
[-- Attachment #1: Type: text/plain, Size: 4760 bytes --]
Hello,
Please find attached v5 version of the patch. Hopefully this is the last one.
I took quite all changes from Maxim's proposal.
Things I did not took are related to html parsing. I did not use of
"%strict-tokenizer" because it needs a yet-to-be-packaged version of
Guile-Libe and did not change the result on any of my tests. I did not
take either the short-form sxpath expression for go-import meta parsing
as it is buggy on my test cases. We can revisit those choices on future
patches but for now I think I have a working version.
Other changes are mainly responses to Ludovic review.
On Tue, Mar 02, 2021 at 10:54:49PM +0100, Ludovic Courtès wrote:
> Nitpick: please mention the sections (for documentation) or variables
> changed
I tried to do it. Don't hesitate to modify message if needed before
commiting.
> (see <https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html).
I finally understood that the document refers to GNU Guidelines for
Changelogs. Some examples specific to Guix would be nice for noobs
like me.
> Some comments below, mostly stylistic as I’m not familiar with the
> actual file formats etc. that the importer implements.
I am not yet completely familiar with it either. All languages now try
to live in their own ecosystem with their own set of incompatible build
and distribution tools. I am just beginning to grasp how Go fo it.
> s/crate/go/
> s/guile-lib/Guile-Lib/
done.
> You can use ‘string-append’ instead of (string-concatenate (list …)).
> Use [[:xdigit:]] instead of [0-9A-Fa-f] for clarity and
> locale-independence.
Thanks for the string-append tip.
>
> Also, you can arrange to use ‘make-regexp’ so that the regexp is
> compiled once for all, and then just ‘regexp-exec’:
I thought about it but was lazy. Thanks to your remark it is now done.
> Please use ‘match’ instead of car/cdr (throughout):
This one was more difficult than I thought. It lead me to create some
specific record type, probably for the better.
> s/acons/alist-cons/ for consistency with the rest of the code.
I still must look at the difference between different type of alists. I
trusted you and just applied the substitution.
> You could rename ‘make-vcs’ above to ‘%make-vcs’ and do:
>
> (define (make-vcs prefix regexp type)
> (%make-vcs prefix (make-regex regexp) type))
>
> so that again you can rely on pre-compiled regexps.
Thanks for the tip.
I wonder when we use "%" prefix versus "*" suffix. I was under the
impression that "%" prefix was more for global (possibly mutable)
variables but you don't use it that way here.
> Keep ‘known-vcs’ in a global variable so it doesn’t have to be
> recomputed every time.
known-vcs is now a top-level variable with precompiled regexs.
> ‘guix lint’ wouldn’t like it. :-) Maybe "Write synopsis here" instead?
>
> Is there no info about the license?
Maxim's patch parse pkg.go.dev for synopsis, license and description.
It is not without flaws (Human review badly needed as it uses README
for trying to extract synopsis) but still better than before.
> New dependency; it’s a bit of a commitment, but hopefully Guile-Lib is
> stable enough and works on all the supported architectures.
It is a bit of commitment but we really needed a library for parsing
HTML. It is only useful on "import go" as of now so nothing critical
for using Guix itself if we can keep it optionnal.
> Please add guix/scripts/import/go.scm to ‘po/guix/POTFILES.in’ so it can
> be translated.
Done.
> > +++ b/tests/import-go.scm
>
> Looks nice! It should be called ‘tests/go.scm’ for consistency, with:
I renamed it. I also put in it one test for guix/build-system/go.scm.
I still am not satisfied with the overall look of this file which is
really difficult to read, but at least we have some basic tests.
> Would it be an option to also have an end-to-end test (checking the
> resulting ‘package’ sexp)? That’d be nice, but perhaps we can add it
> afterwards if you prefer.
I added one end-to-end test loosely based on github.com/go-check/check
example.
For end-to-end tests I reused the "mock" syntax from guix/tests.scm by
doing copy-paste because use-module of "(guix tests)" was really too
slow for me. I don't know what's going on here (it seems to rebuild all
of "gnu" scheme modules) but feel free to delete the copy and import
"(guix tests)" if you prefer.
> Let’s see how much of the comments above you can address for a v4, and
> then we can get that in and improve it from there if needed!
I hope all needed to get that in the tree is done now ;-)
François
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Create-importer-for-Go-modules-good-message.patch --]
[-- Type: text/x-diff; name="0001-Create-importer-for-Go-modules-good-message.patch", Size: 50117 bytes --]
From dab6612fecd809279f6fc4710176c78a0fdf28a8 Mon Sep 17 00:00:00 2001
From: Katherine Cox-Buday <cox.katherine.e@gmail.com>
Date: Thu, 22 Oct 2020 19:40:17 -0500
Subject: [PATCHv5] Create importer for Go modules (good message)
This patch add a `guix import go` command.
* doc/guix.texi: doc about go importer and guile-lib dependency
* gnu/packages/package-management.scm: added guile-lib dependency
* guix/self.scm: add guile-lib dependency
* guix/build-system/go.scm (go-version->git-ref): new procedure
* guix/import/go.scm: Created Go importer
* guix/scripts/import/go.scm: Subcommand for Go importer
* guix/scripts/import.scm: Declare subcommand guix import go
* tests/import-go.scm: Tests for Go importer and build-system
Signed-off-by: Francois Joulaud <francois.joulaud@radiofrance.com>
Co-Authored-By: Helio Machado <0x2b3bfa0@gmail.com>
Co-Authored-By: Francois Joulaud <francois.joulaud@radiofrance.com>
Co-Authored-By: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Closes: https://issues.guix.gnu.org/issue/44178
---
doc/guix.texi | 26 ++
gnu/packages/package-management.scm | 2 +
guix/build-system/go.scm | 34 +-
guix/import/go.scm | 490 ++++++++++++++++++++++++++++
guix/scripts/import.scm | 2 +-
guix/scripts/import/go.scm | 118 +++++++
guix/self.scm | 5 +-
po/guix/POTFILES.in | 1 +
tests/go.scm | 289 ++++++++++++++++
9 files changed, 964 insertions(+), 3 deletions(-)
create mode 100644 guix/import/go.scm
create mode 100644 guix/scripts/import/go.scm
create mode 100644 tests/go.scm
diff --git a/doc/guix.texi b/doc/guix.texi
index 5d28fca837..bb5b64b2fc 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -861,6 +861,10 @@ substitutes (@pxref{Invoking guix publish}).
@uref{https://ngyro.com/software/guile-semver.html, Guile-Semver} for
the @code{crate} importer (@pxref{Invoking guix import}).
+@item
+@uref{https://www.nongnu.org/guile-lib/doc/ref/htmlprag/, Guile-Lib} for
+the @code{go} importer (@pxref{Invoking guix import}).
+
@item
When @url{http://www.bzip.org, libbz2} is available,
@command{guix-daemon} can use it to compress build logs.
@@ -11493,6 +11497,28 @@ Select the given repository (a repository name). Possible values include:
of coq packages.
@end itemize
@end table
+
+@item go
+@cindex go
+Import metadata for a Go module using
+@uref{https://proxy.golang.org, proxy.golang.org}.
+
+This importer is highly experimental. See the source code for more info
+about the current state.
+
+@example
+guix import go gopkg.in/yaml.v2
+@end example
+
+Additional options include:
+
+@table @code
+@item --recursive
+@itemx -r
+Traverse the dependency graph of the given upstream package recursively
+and generate package expressions for all those packages that are not yet
+in Guix.
+@end table
@end table
The structure of the @command{guix import} code is modular. It would be
diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index 9fb8c40a31..06bb5bd2df 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -304,6 +304,7 @@ $(prefix)/etc/init.d\n")))
'((assoc-ref inputs "guile"))))
(avahi (assoc-ref inputs "guile-avahi"))
(gcrypt (assoc-ref inputs "guile-gcrypt"))
+ (guile-lib (assoc-ref inputs "guile-lib"))
(json (assoc-ref inputs "guile-json"))
(sqlite (assoc-ref inputs "guile-sqlite3"))
(zlib (assoc-ref inputs "guile-zlib"))
@@ -367,6 +368,7 @@ $(prefix)/etc/init.d\n")))
`(("guile-avahi" ,guile-avahi)))
("guile-gcrypt" ,guile-gcrypt)
("guile-json" ,guile-json-4)
+ ("guile-lib" ,guile-lib)
("guile-sqlite3" ,guile-sqlite3)
("guile-zlib" ,guile-zlib)
("guile-lzlib" ,guile-lzlib)
diff --git a/guix/build-system/go.scm b/guix/build-system/go.scm
index f8ebaefb27..ad3df9cc4e 100644
--- a/guix/build-system/go.scm
+++ b/guix/build-system/go.scm
@@ -26,9 +26,41 @@
#:use-module (guix build-system gnu)
#:use-module (guix packages)
#:use-module (ice-9 match)
+ #:use-module (ice-9 regex)
#:export (%go-build-system-modules
go-build
- go-build-system))
+ go-build-system
+
+ go-version->git-ref))
+
+(define %go-version-rx
+ (make-regexp (string-append
+ "(v?[0-9]\\.[0-9]\\.[0-9])" ;"v" prefix can be omitted in version prefix
+ "(-|-pre\\.0\\.|-0\\.)" ;separator
+ "([0-9]{14})-" ;timestamp
+ "([0-9A-Fa-f]{12})"))) ;commit hash
+
+(define (go-version->git-ref version)
+ "GO-VERSION->GIT-REF parse pseudo-versions and extract the commit hash from
+it, defaulting to full VERSION if a pseudo-version pattern is not recognized."
+ ;; A module version like v1.2.3 is introduced by tagging a revision in the
+ ;; underlying source repository. Untagged revisions can be referred to
+ ;; using a "pseudo-version" like v0.0.0-yyyymmddhhmmss-abcdefabcdef, where
+ ;; the time is the commit time in UTC and the final suffix is the prefix of
+ ;; the commit hash (see: https://golang.org/ref/mod#pseudo-versions).
+ (let* ((version
+ ;; If a source code repository has a v2.0.0 or later tag for a file
+ ;; tree with no go.mod, the version is considered to be part of the
+ ;; v1 module's available versions and is given an +incompatible
+ ;; suffix
+ ;; (see:https://golang.org/cmd/go/#hdr-Module_compatibility_and_semantic_versioning).
+ (if (string-suffix? "+incompatible" version)
+ (string-drop-right version 13)
+ version))
+ (match (regexp-exec %go-version-rx version)))
+ (if match
+ (match:substring match 4)
+ version)))
;; Commentary:
;;
diff --git a/guix/import/go.scm b/guix/import/go.scm
new file mode 100644
index 0000000000..c452d81b4a
--- /dev/null
+++ b/guix/import/go.scm
@@ -0,0 +1,490 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2020 Katherine Cox-Buday <cox.katherine.e@gmail.com>
+;;; Copyright © 2020 Helio Machado <0x2b3bfa0+guix@googlemail.com>
+;;; Copyright © 2021 François Joulaud <francois.joulaud@radiofrance.com>
+;;; Copyright © 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+;;; (guix import golang) attempts to make it easier to create Guix package
+;;; declarations for Go modules.
+;;;
+;;; Modules in Go are a "collection of related Go packages" which are "the
+;;; unit of source code interchange and versioning". Modules are generally
+;;; hosted in a repository.
+;;;
+;;; At this point it should handle correctly modules which have only Go
+;;; dependencies and are accessible from proxy.golang.org (or configured via
+;;; GOPROXY).
+;;;
+;;; We want it to work more or less this way:
+;;; - get latest version for the module from GOPROXY
+;;; - infer VCS root repo from which we will check-out source by
+;;; + recognising known patterns (like github.com)
+;;; + or recognizing .vcs suffix
+;;; + or parsing meta tag in HTML served at the URL
+;;; + or (TODO) if nothing else works by using zip file served by GOPROXY
+;;; - get go.mod from GOPROXY (which is able to synthetize one if needed)
+;;; - extract list of dependencies from this go.mod
+;;;
+;;; The Go module paths are translated to a Guix package name under the
+;;; assumption that there will be no collision.
+
+;;; TODO list
+;;; - get correct hash in vcs->origin
+;;; - print partial result during recursive imports (need to catch
+;;; exceptions)
+
+(define-module (guix import go)
+ #:use-module (guix build-system go)
+ #:use-module (guix git)
+ #:use-module (guix i18n)
+ #:use-module (guix import utils)
+ #:use-module (guix import json)
+ #:use-module (guix packages)
+ #:use-module (guix upstream)
+ #:use-module (guix utils)
+ #:use-module (guix http-client)
+ #:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix base16)
+ #:use-module (guix base32)
+ #:use-module (guix memoization)
+ #:use-module (htmlprag)
+ #:use-module (ice-9 match)
+ #:use-module (ice-9 rdelim)
+ #:use-module (ice-9 receive)
+ #:use-module (ice-9 regex)
+ #:use-module (json)
+ #:use-module (rnrs io ports)
+ #:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-9)
+ #:use-module (srfi srfi-11)
+ #:use-module (srfi srfi-26)
+ #:use-module (sxml xpath)
+ #:use-module (web client)
+ #:use-module (web response)
+ #:use-module (web uri)
+
+ #:export (go-module->guix-package
+ go-module-recursive-import))
+
+
+(define (go-path-escape path)
+ "Escape a module path by replacing every uppercase letter with an
+exclamation mark followed with its lowercase equivalent, as per the module
+Escaped Paths specification (see:
+https://godoc.org/golang.org/x/mod/module#hdr-Escaped_Paths)."
+ (define (escape occurrence)
+ (string-append "!" (string-downcase (match:substring occurrence))))
+ (regexp-substitute/global #f "[A-Z]" path 'pre escape 'post))
+
+(define (go-module-latest-version goproxy-url module-path)
+ "Fetch the version number of the latest version for MODULE-PATH from the
+given GOPROXY-URL server."
+ (assoc-ref (json-fetch (format #f "~a/~a/@latest" goproxy-url
+ (go-path-escape module-path)))
+ "Version"))
+
+
+(define (go-package-licenses name)
+ "Retrieve the list of licenses that apply to NAME, a Go package or module
+name (e.g. \"github.com/golang/protobuf/proto\"). The data is scraped from
+the https://pkg.go.dev/ web site."
+ (let*-values (((url) (string-append "https://pkg.go.dev/" name
+ "?tab=licenses"))
+ ((response body) (http-get url))
+ ;; Extract the text contained in a h2 child node of any
+ ;; element marked with a "License" class attribute.
+ ((select) (sxpath `(// (* (@ (equal? (class "License"))))
+ h2 // *text*))))
+ (and (eq? (response-code response) 200)
+ (match (select (html->sxml body))
+ (() #f) ;nothing selected
+ (licenses licenses)))))
+
+(define (go.pkg.dev-info name)
+ (http-get (string-append "https://pkg.go.dev/" name)))
+(define go.pkg.dev-info*
+ (memoize go.pkg.dev-info))
+
+(define (go-package-description name)
+ "Retrieve a short description for NAME, a Go package name,
+e.g. \"google.golang.org/protobuf/proto\". The data is scraped from the
+https://pkg.go.dev/ web site."
+ (let*-values (((response body) (go.pkg.dev-info* name))
+ ;; Extract the text contained in a h2 child node of any
+ ;; element marked with a "License" class attribute.
+ ((select) (sxpath
+ `(// (section
+ (@ (equal? (class "Documentation-overview"))))
+ (p 1)))))
+ (and (eq? (response-code response) 200)
+ (match (select (html->sxml body))
+ (() #f) ;nothing selected
+ (((p . strings))
+ ;; The paragraph text is returned as a list of strings embedding
+ ;; newline characters. Join them and strip the newline
+ ;; characters.
+ (string-delete #\newline (string-join strings)))))))
+
+(define (go-package-synopsis module-name)
+ "Retrieve a short synopsis for a Go module named MODULE-NAME,
+e.g. \"google.golang.org/protobuf\". The data is scraped from
+the https://pkg.go.dev/ web site."
+ ;; Note: Only the *module* (rather than package) page has the README title
+ ;; used as a synopsis on the https://pkg.go.dev web site.
+ (let*-values (((response body) (go.pkg.dev-info* module-name))
+ ;; Extract the text contained in a h2 child node of any
+ ;; element marked with a "License" class attribute.
+ ((select) (sxpath
+ `(// (div (@ (equal? (class "UnitReadme-content"))))
+ // h3 *text*))))
+ (and (eq? (response-code response) 200)
+ (match (select (html->sxml body))
+ (() #f) ;nothing selected
+ ((title more ...) ;title is the first string of the list
+ (string-trim-both title))))))
+
+(define (list->licenses licenses)
+ "Given a list of LICENSES mostly following the SPDX conventions, return the
+corresponding Guix license or 'unknown-license!"
+ (filter-map (lambda (license)
+ (and (not (string-null? license))
+ (not (any (cut string=? <> license)
+ '("AND" "OR" "WITH")))
+ ;; Adjust the license names scraped from
+ ;; https://pkg.go.dev to an equivalent SPDX identifier,
+ ;; if they differ (see: https://github.com/golang/pkgsite
+ ;; /internal/licenses/licenses.go#L174).
+ (or (spdx-string->license
+ (match license
+ ("BlueOak-1.0" "BlueOak-1.0.0")
+ ("BSD-0-Clause" "0BSD")
+ ("BSD-2-Clause" "BSD-2-Clause-FreeBSD")
+ ("GPL2" "GPL-2.0")
+ ("GPL3" "GPL-3.0")
+ ("NIST" "NIST-PD")
+ (_ license)))
+ 'unknown-license!)))
+ licenses))
+
+(define (fetch-go.mod goproxy-url module-path version)
+ "Fetches go.mod from the given GOPROXY-URL server for the given MODULE-PATH
+and VERSION."
+ (let ((url (format #f "~a/~a/@v/~a.mod" goproxy-url
+ (go-path-escape module-path)
+ (go-path-escape version))))
+ (http-fetch url)))
+
+(define %go.mod-require-directive-rx
+ ;; A line in a require directive is composed of a module path and
+ ;; a version separated by whitespace and an optionnal '//' comment at
+ ;; the end.
+ (make-regexp
+ (string-append
+ "^[[:blank:]]*"
+ "([^[:blank:]]+)[[:blank:]]+([^[:blank:]]+)"
+ "([[:blank:]]+//.*)?")))
+
+(define %go.mod-replace-directive-rx
+ ;; ReplaceSpec = ModulePath [ Version ] "=>" FilePath newline
+ ;; | ModulePath [ Version ] "=>" ModulePath Version newline .
+ (make-regexp
+ (string-append
+ "([^[:blank:]]+)([[:blank:]]+([^[:blank:]]+))?"
+ "[[:blank:]]+" "=>" "[[:blank:]]+"
+ "([^[:blank:]]+)([[:blank:]]+([^[:blank:]]+))?")))
+
+(define (parse-go.mod port)
+ "Parse the go.mod file accessible via the input PORT, returning a list of
+requirements."
+ (define-record-type <results>
+ (make-results requirements replacements)
+ results?
+ (requirements results-requirements)
+ (replacements results-replacements))
+ ;; We parse only a subset of https://golang.org/ref/mod#go-mod-file-grammar
+ ;; which we think necessary for our use case.
+ (define (toplevel results)
+ "Main parser, RESULTS is a pair of alist serving as accumulator for
+ all encountered requirements and replacements."
+ (let ((line (read-line)))
+ (cond
+ ((eof-object? line)
+ ;; parsing ended, give back the result
+ results)
+ ((string=? line "require (")
+ ;; a require block begins, delegate parsing to IN-REQUIRE
+ (in-require results))
+ ((string=? line "replace (")
+ ;; a replace block begins, delegate parsing to IN-REPLACE
+ (in-replace results))
+ ((string-prefix? "require " line)
+ ;; a standalone require directive
+ (let* ((stripped-line (string-drop line 8))
+ (new-results (require-directive results stripped-line)))
+ (toplevel new-results)))
+ ((string-prefix? "replace " line)
+ ;; a standalone replace directive
+ (let* ((stripped-line (string-drop line 8))
+ (new-results (replace-directive results stripped-line)))
+ (toplevel new-results)))
+ (#t
+ ;; unrecognised line, ignore silently
+ (toplevel results)))))
+
+ (define (in-require results)
+ (let ((line (read-line)))
+ (cond
+ ((eof-object? line)
+ ;; this should never happen here but we ignore silently
+ results)
+ ((string=? line ")")
+ ;; end of block, coming back to toplevel
+ (toplevel results))
+ (#t
+ (in-require (require-directive results line))))))
+
+ (define (in-replace results)
+ (let ((line (read-line)))
+ (cond
+ ((eof-object? line)
+ ;; this should never happen here but we ignore silently
+ results)
+ ((string=? line ")")
+ ;; end of block, coming back to toplevel
+ (toplevel results))
+ (#t
+ (in-replace (replace-directive results line))))))
+
+ (define (replace-directive results line)
+ "Extract replaced modules and new requirements from replace directive
+ in LINE and add to RESULTS."
+ (match results
+ (($ <results> requirements replaced)
+ (let* ((rx-match (regexp-exec %go.mod-replace-directive-rx line))
+ (module-path (match:substring rx-match 1))
+ (version (match:substring rx-match 3))
+ (new-module-path (match:substring rx-match 4))
+ (new-version (match:substring rx-match 6))
+ (new-replaced (alist-cons module-path version replaced))
+ (new-requirements
+ (if (string-match "^\\.?\\./" new-module-path)
+ requirements
+ (alist-cons new-module-path new-version requirements))))
+ (make-results new-requirements new-replaced)))))
+ (define (require-directive results line)
+ "Extract requirement from LINE and add it to RESULTS."
+ (let* ((rx-match (regexp-exec %go.mod-require-directive-rx line))
+ (module-path (match:substring rx-match 1))
+ ;; we saw double-quoted string in the wild without escape
+ ;; sequences so we just trim the quotes
+ (module-path (string-trim-both module-path #\"))
+ (version (match:substring rx-match 2)))
+ (match results
+ (($ <results> requirements replaced)
+ (make-results (alist-cons module-path version requirements) replaced)))))
+
+ (with-input-from-port port
+ (lambda ()
+ (let ((results (toplevel (make-results '() '()))))
+ (match results
+ (($ <results> requirements replaced)
+ ;; At last we remove replaced modules from the requirements list
+ (fold
+ (lambda (replacedelem requirements)
+ (alist-delete! (car replacedelem) requirements))
+ requirements
+ replaced)))))))
+
+(define-record-type <vcs>
+ (%make-vcs url-prefix root-regex type)
+ vcs?
+ (url-prefix vcs-url-prefix)
+ (root-regex vcs-root-regex)
+ (type vcs-type))
+(define (make-vcs prefix regexp type)
+ (%make-vcs prefix (make-regexp regexp) type))
+(define known-vcs
+ ;; See the following URL for the official Go equivalent:
+ ;; https://github.com/golang/go/blob/846dce9d05f19a1f53465e62a304dea21b99f910/src/cmd/go/internal/vcs/vcs.go#L1026-L1087
+ (list
+ (make-vcs
+ "github.com"
+ "^(github\\.com/[A-Za-z0-9_.\\-]+/[A-Za-z0-9_.\\-]+)(/[A-Za-z0-9_.\\-]+)*$"
+ 'git)
+ (make-vcs
+ "bitbucket.org"
+ "^(bitbucket\\.org/([A-Za-z0-9_.\\-]+/[A-Za-z0-9_.\\-]+))(/[A-Za-z0-9_.\\-]+)*$"
+ 'unknown)
+ (make-vcs
+ "hub.jazz.net/git/"
+ "^(hub\\.jazz\\.net/git/[a-z0-9]+/[A-Za-z0-9_.\\-]+)(/[A-Za-z0-9_.\\-]+)*$"
+ 'git)
+ (make-vcs
+ "git.apache.org"
+ "^(git\\.apache\\.org/[a-z0-9_.\\-]+\\.git)(/[A-Za-z0-9_.\\-]+)*$"
+ 'git)
+ (make-vcs
+ "git.openstack.org"
+ "^(git\\.openstack\\.org/[A-Za-z0-9_.\\-]+/[A-Za-z0-9_.\\-]+)(\\.git)?\
+(/[A-Za-z0-9_.\\-]+)*$"
+ 'git)))
+
+(define (module-path->repository-root module-path)
+ "Infer the repository root from a module path. Go modules can be
+defined at any level of a repository tree, but querying for the meta tag
+usually can only be done from the web page at the root of the repository,
+hence the need to derive this information."
+
+ ;; For reference, see: https://golang.org/ref/mod#vcs-find.
+ (define vcs-qualifiers '(".bzr" ".fossil" ".git" ".hg" ".svn"))
+
+ (define (vcs-qualified-module-path->root-repo-url module-path)
+ (let* ((vcs-qualifiers-group (string-join vcs-qualifiers "|"))
+ (pattern (format #f "^(.*(~a))(/|$)" vcs-qualifiers-group))
+ (m (string-match pattern module-path)))
+ (and=> m (cut match:substring <> 1))))
+
+ (or (and=> (find (lambda (vcs)
+ (string-prefix? (vcs-url-prefix vcs) module-path))
+ known-vcs)
+ (lambda (vcs)
+ (match:substring (regexp-exec (vcs-root-regex vcs)
+ module-path) 1)))
+ (vcs-qualified-module-path->root-repo-url module-path)
+ module-path))
+
+(define (go-module->guix-package-name module-path)
+ "Converts a module's path to the canonical Guix format for Go packages."
+ (string-downcase (string-append "go-" (string-replace-substring
+ (string-replace-substring
+ module-path
+ "." "-")
+ "/" "-"))))
+
+(define-record-type <module-meta>
+ (make-module-meta import-prefix vcs repo-root)
+ module-meta?
+ (import-prefix module-meta-import-prefix)
+ (vcs module-meta-vcs) ;a symbol
+ (repo-root module-meta-repo-root))
+
+(define (fetch-module-meta-data module-path)
+ "Retrieve the module meta-data from its landing page. This is necessary
+because goproxy servers don't currently provide all the information needed to
+build a package."
+ ;; <meta name="go-import" content="import-prefix vcs repo-root">
+ (let* ((port (http-fetch (format #f "https://~a?go-get=1" module-path)))
+ (select (sxpath `(// head (meta (@ (equal? (name "go-import"))))
+ // content))))
+ (match (select (call-with-port port html->sxml))
+ (() #f) ;nothing selected
+ (((content content-text))
+ (match (string-split content-text #\space)
+ ((root-path vcs repo-url)
+ (make-module-meta root-path (string->symbol vcs) repo-url)))))))
+
+(define (module-meta-data-repo-url meta-data goproxy-url)
+ "Return the URL where the fetcher which will be used can download the
+source."
+ (if (member (module-meta-vcs meta-data) '(fossil mod))
+ goproxy-url
+ (module-meta-repo-root meta-data)))
+
+(define (vcs->origin vcs-type vcs-repo-url version)
+ "Generate the `origin' block of a package depending on what type of source
+control system is being used."
+ (case vcs-type
+ ((git)
+ `(origin
+ (method git-fetch)
+ (uri (git-reference
+ (url ,vcs-repo-url)
+ (commit (go-version->git-ref version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ ;; FIXME: populate hash for git repo checkout
+ "0000000000000000000000000000000000000000000000000000"))))
+ ((hg)
+ `(origin
+ (method hg-fetch)
+ (uri (hg-reference
+ (url ,vcs-repo-url)
+ (changeset ,version)))
+ (file-name (string-append name "-" version "-checkout"))
+ (sha256
+ (base32
+ ;; FIXME: populate hash for hg repo checkout
+ "0000000000000000000000000000000000000000000000000000"))))
+ ((svn)
+ `(origin
+ (method svn-fetch)
+ (uri (svn-reference
+ (url ,vcs-repo-url)
+ (revision (string->number version))))
+ (file-name (string-append name "-" version "-checkout"))
+ (sha256
+ (base32
+ ;; FIXME: populate hash for svn repo checkout
+ "0000000000000000000000000000000000000000000000000000"))))
+ (else
+ (raise (formatted-message (G_ "unsupported vcs type '~a' for package '~a'" vcs-type vcs-repo-url))))))
+
+(define* (go-module->guix-package module-path #:key
+ (goproxy-url "https://proxy.golang.org"))
+ (let* ((latest-version (go-module-latest-version goproxy-url module-path))
+ (port (fetch-go.mod goproxy-url module-path latest-version))
+ (dependencies (map car (call-with-port port parse-go.mod)))
+ (guix-name (go-module->guix-package-name module-path))
+ (root-module-path (module-path->repository-root module-path))
+ ;; The VCS type and URL are not included in goproxy information. For
+ ;; this we need to fetch it from the official module page.
+ (meta-data (fetch-module-meta-data root-module-path))
+ (vcs-type (module-meta-vcs meta-data))
+ (vcs-repo-url (module-meta-data-repo-url meta-data goproxy-url))
+ (synopsis (go-package-synopsis root-module-path))
+ (description (go-package-description module-path))
+ (licenses (go-package-licenses module-path)))
+ (values
+ `(package
+ (name ,guix-name)
+ ;; Elide the "v" prefix Go uses
+ (version ,(string-trim latest-version #\v))
+ (source
+ ,(vcs->origin vcs-type vcs-repo-url latest-version))
+ (build-system go-build-system)
+ (arguments
+ '(#:import-path ,root-module-path))
+ ,@(maybe-inputs (map go-module->guix-package-name dependencies))
+ (home-page ,(format #f "https://~a" root-module-path))
+ (synopsis ,synopsis)
+ (description ,description)
+ (license ,(and=> licenses list->licenses)))
+ dependencies)))
+
+(define go-module->guix-package* (memoize go-module->guix-package))
+
+(define* (go-module-recursive-import package-name
+ #:key (goproxy-url "https://proxy.golang.org"))
+ (recursive-import
+ package-name
+ #:repo->guix-package (lambda* (name . _)
+ (go-module->guix-package*
+ name
+ #:goproxy-url goproxy-url))
+ #:guix-name go-module->guix-package-name))
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 0a3863f965..1d2b45d942 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -77,7 +77,7 @@ rather than \\n."
;;;
(define importers '("gnu" "nix" "pypi" "cpan" "hackage" "stackage" "elpa" "gem"
- "cran" "crate" "texlive" "json" "opam"))
+ "go" "cran" "crate" "texlive" "json" "opam"))
(define (resolve-importer name)
(let ((module (resolve-interface
diff --git a/guix/scripts/import/go.scm b/guix/scripts/import/go.scm
new file mode 100644
index 0000000000..fde7555973
--- /dev/null
+++ b/guix/scripts/import/go.scm
@@ -0,0 +1,118 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2020 Katherine Cox-Buday <cox.katherine.e@gmail.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix scripts import go)
+ #:use-module (guix ui)
+ #:use-module (guix utils)
+ #:use-module (guix scripts)
+ #:use-module (guix import go)
+ #:use-module (guix scripts import)
+ #:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-11)
+ #:use-module (srfi srfi-37)
+ #:use-module (ice-9 match)
+ #:use-module (ice-9 format)
+ #:export (guix-import-go))
+
+
+;;;
+;;; Command-line options.
+;;;
+
+(define %default-options
+ '())
+
+(define (show-help)
+ (display (G_ "Usage: guix import go PACKAGE-PATH
+Import and convert the Go module for PACKAGE-PATH.\n"))
+ (display (G_ "
+ -h, --help display this help and exit"))
+ (display (G_ "
+ -V, --version display version information and exit"))
+ (display (G_ "
+ -r, --recursive generate package expressions for all Go modules\
+ that are not yet in Guix"))
+ (display (G_ "
+ -p, --goproxy=GOPROXY specify which goproxy server to use"))
+ (newline)
+ (show-bug-report-information))
+
+(define %options
+ ;; Specification of the command-line options.
+ (cons* (option '(#\h "help") #f #f
+ (lambda args
+ (show-help)
+ (exit 0)))
+ (option '(#\V "version") #f #f
+ (lambda args
+ (show-version-and-exit "guix import go")))
+ (option '(#\r "recursive") #f #f
+ (lambda (opt name arg result)
+ (alist-cons 'recursive #t result)))
+ (option '(#\p "goproxy") #t #f
+ (lambda (opt name arg result)
+ (alist-cons 'goproxy
+ (string->symbol arg)
+ (alist-delete 'goproxy result))))
+ %standard-import-options))
+
+
+;;;
+;;; Entry point.
+;;;
+
+(define (guix-import-go . args)
+ (define (parse-options)
+ ;; Return the alist of option values.
+ (args-fold* args %options
+ (lambda (opt name arg result)
+ (leave (G_ "~A: unrecognized option~%") name))
+ (lambda (arg result)
+ (alist-cons 'argument arg result))
+ %default-options))
+
+ (let* ((opts (parse-options))
+ (args (filter-map (match-lambda
+ (('argument . value)
+ value)
+ (_ #f))
+ (reverse opts))))
+ (match args
+ ((module-name)
+ (if (assoc-ref opts 'recursive)
+ (map (match-lambda
+ ((and ('package ('name name) . rest) pkg)
+ `(define-public ,(string->symbol name)
+ ,pkg))
+ (_ #f))
+ (go-module-recursive-import module-name
+ #:goproxy-url
+ (or (assoc-ref opts 'goproxy)
+ "https://proxy.golang.org")))
+ (let ((sexp (go-module->guix-package module-name
+ #:goproxy-url
+ (or (assoc-ref opts 'goproxy)
+ "https://proxy.golang.org"))))
+ (unless sexp
+ (leave (G_ "failed to download meta-data for module '~a'~%")
+ module-name))
+ sexp)))
+ (()
+ (leave (G_ "too few arguments~%")))
+ ((many ...)
+ (leave (G_ "too many arguments~%"))))))
diff --git a/guix/self.scm b/guix/self.scm
index 35fba1152d..ed5ee9ddea 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -814,6 +814,9 @@ itself."
(define guile-ssh
(specification->package "guile-ssh"))
+ (define guile-lib
+ (specification->package "guile-lib"))
+
(define guile-git
(specification->package "guile-git"))
@@ -842,7 +845,7 @@ itself."
(append-map transitive-package-dependencies
(list guile-gcrypt gnutls guile-git guile-avahi
guile-json guile-semver guile-ssh guile-sqlite3
- guile-zlib guile-lzlib guile-zstd)))
+ guile-lib guile-zlib guile-lzlib guile-zstd)))
(define *core-modules*
(scheme-node "guix-core"
diff --git a/po/guix/POTFILES.in b/po/guix/POTFILES.in
index 666e630adf..cbbfe1e76b 100644
--- a/po/guix/POTFILES.in
+++ b/po/guix/POTFILES.in
@@ -101,6 +101,7 @@ guix/scripts/import/cpan.scm
guix/scripts/import/crate.scm
guix/scripts/import/gem.scm
guix/scripts/import/gnu.scm
+guix/scripts/import/go.scm
guix/scripts/import/hackage.scm
guix/scripts/import/json.scm
guix/scripts/import/nix.scm
diff --git a/tests/go.scm b/tests/go.scm
new file mode 100644
index 0000000000..d9fe4dac81
--- /dev/null
+++ b/tests/go.scm
@@ -0,0 +1,289 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 François Joulaud <francois.joulaud@radiofrance.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+;;; Summary
+;; Tests for guix/import/go.scm
+
+(define-module (test-import-go)
+ #:use-module (guix base32)
+ #:use-module (guix build-system go)
+ #:use-module (guix import go)
+ #:use-module (ice-9 iconv)
+ #:use-module (ice-9 match)
+ #:use-module (srfi srfi-19)
+ #:use-module (srfi srfi-64)
+ #:use-module (web response))
+
+;; This is a copy of the syntax-rule in (guix tests) because I don't
+;; want to load all (guix tests) just for this macro.
+(define-syntax-rule (mock (module proc replacement) body ...)
+ "Within BODY, replace the definition of PROC from MODULE with the definition
+given by REPLACEMENT."
+ (let* ((m (resolve-module 'module))
+ (original (module-ref m 'proc)))
+ (dynamic-wind
+ (lambda () (module-set! m 'proc replacement))
+ (lambda () body ...)
+ (lambda () (module-set! m 'proc original)))))
+
+(define fixture-go-mod-simple
+ "module my/thing
+go 1.12
+require other/thing v1.0.2
+require new/thing/v2 v2.3.4
+exclude old/thing v1.2.3
+replace bad/thing v1.4.5 => good/thing v1.4.5
+")
+
+(define fixture-go-mod-with-block
+ "module M
+
+require (
+ A v1
+ B v1.0.0
+ C v1.0.0
+ D v1.2.3
+ E dev
+)
+
+exclude D v1.2.3
+")
+
+
+(define fixture-go-mod-complete
+ "module M
+
+go 1.13
+
+replace github.com/myname/myproject/myapi => ./api
+
+replace github.com/mymname/myproject/thissdk => ../sdk
+
+replace launchpad.net/gocheck => github.com/go-check/check v0.0.0-20140225173054-eb6ee6f84d0a
+
+require (
+ github.com/user/project v1.1.11
+ github.com/user/project/sub/directory v1.1.12
+ bitbucket.org/user/project v1.11.20
+ bitbucket.org/user/project/sub/directory v1.11.21
+ launchpad.net/project v1.1.13
+ launchpad.net/project/series v1.1.14
+ launchpad.net/project/series/sub/directory v1.1.15
+ launchpad.net/~user/project/branch v1.1.16
+ launchpad.net/~user/project/branch/sub/directory v1.1.17
+ hub.jazz.net/git/user/project v1.1.18
+ hub.jazz.net/git/user/project/sub/directory v1.1.19
+ k8s.io/kubernetes/subproject v1.1.101
+ one.example.com/abitrary/repo v1.1.111
+ two.example.com/abitrary/repo v0.0.2
+ \"quoted.example.com/abitrary/repo\" v0.0.2
+)
+
+replace two.example.com/abitrary/repo => github.com/corp/arbitrary-repo v0.0.2
+
+replace (
+ golang.org/x/sys => golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a // pinned to release-branch.go1.13
+ golang.org/x/tools => golang.org/x/tools v0.0.0-20190821162956-65e3620a7ae7 // pinned to release-branch.go1.13
+)
+
+")
+
+
+
+(define fixture-latest-for-go-check
+ "{\"Version\":\"v0.0.0-20201130134442-10cb98267c6c\",\"Time\":\"2020-11-30T13:44:42Z\"}")
+
+
+(define fixtures-go-check-test
+ (let ((version
+ "{\"Version\":\"v0.0.0-20201130134442-10cb98267c6c\",\"Time\":\"2020-11-30T13:44:42Z\"}")
+ (go.mod
+ "module gopkg.in/check.v1
+
+go 1.11
+
+require github.com/kr/pretty v0.2.1
+")
+ (go-get
+ "<!DOCTYPE html>
+<html lang=\"en\" >
+ <head>
+ <meta charset=\"utf-8\">
+ <link rel=\"dns-prefetch\" href=\"https://github.githubassets.com\">
+ <script crossorigin=\"anonymous\" defer=\"defer\" integrity=\"sha512-aw5tciVT0IsECUmMuwp9ez60QReE2/yFNL1diLgZnOom6RhU8+0lG3RlAKto4JwbCoEP15E41Pksd7rK5BKfCQ==\" type=\"application/javascript\" src=\"https://github.githubassets.com/assets/topic-suggestions-6b0e6d72.js\"></script>
+ <meta name=\"viewport\" content=\"width=device-width\">
+
+ <title>GitHub - go-check/check: Rich testing for the Go language</title>
+ <meta name=\"description\" content=\"Rich testing for the Go language. Contribute to go-check/check development by creating an account on GitHub.\">
+ <link rel=\"search\" type=\"application/opensearchdescription+xml\" href=\"/opensearch.xml\" title=\"GitHub\">
+ <link rel=\"fluid-icon\" href=\"https://github.com/fluidicon.png\" title=\"GitHub\">
+ <!-- To prevent page flashing, the optimizely JS needs to be loaded in the
+ <head> tag before the DOM renders -->
+ <meta name=\"hostname\" content=\"github.com\">
+ <meta name=\"user-login\" content=\"\">
+ <link href=\"https://github.com/go-check/check/commits/v1.atom\" rel=\"alternate\" title=\"Recent Commits to check:v1\" type=\"application/atom+xml\">
+ <meta name=\"go-import\" content=\"github.com/go-check/check git https://github.com/go-check/check.git\">
+ </head>
+ <body class=\"logged-out env-production page-responsive\" style=\"word-wrap: break-word;\">
+ </body>
+</html>
+")
+ (pkg.go.dev "<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n <meta charset=\"utf-8\">\n</head>\n<body class=\"Site Site--wide Site--redesign\">\n <div class=\"Site-content\">\n <div class=\"Container\">\n <div class=\"UnitDetails\" data-test-id=\"UnitDetails\">\n <div class=\"UnitDetails-content js-unitDetailsContent\" role=\"main\" data-test-id=\"UnitDetails-content\">\n <div class=\"UnitReadme js-readme\">\n <h2 class=\"UnitReadme-title\" id=\"section-readme\"><img height=\"25px\" width=\"20px\" src=\"/static/img/pkg-icon-readme_20x16.svg\" alt=\"\">README</h2>\n <div class=\"UnitReadme-content\" data-test-id=\"Unit-readmeContent\">\n <div class=\"Overview-readmeContent js-readmeContent\">\n <h3 class=\"h1\" id=\"readme-instructions\">Instructions</h3>\n <p>Install the package with:</p>\n <pre><code>go get gopkg.in/check.v1\n</code></pre>\n </div>\n <div class=\"UnitReadme-fadeOut\"></div>\n </div>\n </div>\n <div class=\"UnitDoc\">\n <h2 class=\"UnitDoc-title\" id=\"section-documentation\"><img height=\"25px\" width=\"20px\" src=\"/static/img/pkg-icon-doc_20x12.svg\" alt=\"\">Documentation</h2>\n <div class=\"Documentation js-documentation\">\n <div class=\"Documentation-content js-docContent\">\n <section class=\"Documentation-overview\">\n <h3 tabindex=\"-1\" id=\"pkg-overview\" class=\"Documentation-overviewHeader\">Overview <a href=\"#pkg-overview\">¶</a></h3>\n <div role=\"navigation\" aria-label=\"Table of Contents\">\n <ul class=\"Documentation-toc\"></ul>\n </div>\n <p>Package check is a rich testing extension for Go's testing package.</p>\n <p>For details about the project, see:</p>\n <pre><a href=\"http://labix.org/gocheck\">http://labix.org/gocheck</a>\n</pre>\n </section>\n <h3 tabindex=\"-1\" id=\"pkg-constants\" class=\"Documentation-constantsHeader\">Constants <a href=\"#pkg-constants\">¶</a></h3>\n </div>\n </div>\n </div>\n </div>\n </div>\n </div>\n </div>\n</body>\n</html>\n")
+ (pkg.go.dev-licence "<!DOCTYPE html>\n<html lang=\"en\">\n<meta charset=\"utf-8\">\n<meta http-equiv=\"X-UA-Compatible\" content=\"IE=edge\">\n<body class=\"Site Site--wide Site--redesign\">\n <div class=\"Unit-content\" role=\"main\">\n <section class=\"License\" id=\"lic-0\">\n <h2><div id=\"#lic-0\">BSD-2-Clause</div></h2>\n <p>This is not legal advice. <a href=\"/license-policy\">Read disclaimer.</a></p>\n <pre class=\"License-contents\">Gocheck - A rich testing framework for Go\n \nCopyright line\n\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without\nmodification, are permitted provided that the following conditions are met: \n\n1. Redistributions of source code must retain the above copyright notice, this\n list of conditions and the following disclaimer. \n2. Redistributions in binary form must reproduce the above copyright notice,\n this list of conditions and the following disclaimer in the documentation\n and/or other materials provided with the distribution. \n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND\nANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED\nWARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\nDISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR\nANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES\n(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\nLOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND\nON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS\nSOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n</pre>\n </section>\n <div class=\"License-source\">Source: github.com/go-check/check@v0.0.0-20201128035030-22ab2dfb190c/LICENSE</div>\n </div>\n </div>\n"))
+ `(("https://proxy.golang.org/github.com/go-check/check/@v/v0.0.0-20201130134442-10cb98267c6c.mod"
+ . ,go.mod)
+ ("https://proxy.golang.org/github.com/go-check/check/@latest"
+ . ,version)
+ ("https://github.com/go-check/check?go-get=1"
+ . ,go-get)
+ ("https://pkg.go.dev/github.com/go-check/check"
+ . ,pkg.go.dev)
+ ("https://pkg.go.dev/github.com/go-check/check?tab=licenses"
+ . ,pkg.go.dev-licence))))
+
+(test-begin "go")
+
+;;; Unit tests for go build-system
+
+(test-equal "go-version basic"
+ "v1.0.2"
+ (go-version->git-ref "v1.0.2"))
+
+(test-equal "go-version omited 'v' character"
+ "v1.0.2"
+ (go-version->git-ref "v1.0.2"))
+
+(test-equal "go-version with embeded git-ref"
+ "65e3620a7ae7"
+ (go-version->git-ref "v0.0.0-20190821162956-65e3620a7ae7"))
+
+(test-equal "go-version with complex embeded git-ref"
+ "daa7c04131f5"
+ (go-version->git-ref "v1.2.4-0.20191109021931-daa7c04131f5"))
+
+;;; Unit tests for (guix import go)
+
+(test-equal "go-path-escape"
+ "github.com/!azure/!avere"
+ ((@@ (guix import go) go-path-escape) "github.com/Azure/Avere"))
+
+
+;; We define a function for all similar tests with different go.mod files
+(define (testing-parse-mod name expected input)
+ (define (inf? p1 p2)
+ (string<? (car p1) (car p2)))
+ (let ((input-port (open-input-string input)))
+ (test-equal name
+ (sort expected inf?)
+ (sort
+ ( (@@ (guix import go) parse-go.mod)
+ input-port)
+ inf?))))
+
+(testing-parse-mod "parse-go.mod-simple"
+ '(("good/thing" . "v1.4.5")
+ ("new/thing/v2" . "v2.3.4")
+ ("other/thing" . "v1.0.2"))
+ fixture-go-mod-simple)
+
+(testing-parse-mod "parse-go.mod-with-block"
+ '(("A" . "v1")
+ ("B" . "v1.0.0")
+ ("C" . "v1.0.0")
+ ("D" . "v1.2.3")
+ ("E" . "dev"))
+ fixture-go-mod-with-block)
+
+(testing-parse-mod "parse-go.mod-complete"
+ '(("github.com/corp/arbitrary-repo" . "v0.0.2")
+ ("quoted.example.com/abitrary/repo" . "v0.0.2")
+ ("one.example.com/abitrary/repo" . "v1.1.111")
+ ("hub.jazz.net/git/user/project/sub/directory" . "v1.1.19")
+ ("hub.jazz.net/git/user/project" . "v1.1.18")
+ ("launchpad.net/~user/project/branch/sub/directory" . "v1.1.17")
+ ("launchpad.net/~user/project/branch" . "v1.1.16")
+ ("launchpad.net/project/series/sub/directory" . "v1.1.15")
+ ("launchpad.net/project/series" . "v1.1.14")
+ ("launchpad.net/project" . "v1.1.13")
+ ("bitbucket.org/user/project/sub/directory" . "v1.11.21")
+ ("bitbucket.org/user/project" . "v1.11.20")
+ ("k8s.io/kubernetes/subproject" . "v1.1.101")
+ ("github.com/user/project/sub/directory" . "v1.1.12")
+ ("github.com/user/project" . "v1.1.11")
+ ("github.com/go-check/check" . "v0.0.0-20140225173054-eb6ee6f84d0a"))
+ fixture-go-mod-complete)
+
+;;; End-to-end tests for (guix import go)
+(define (mock-http-fetch testcase)
+ (lambda (url . rest)
+ (let ((body (assoc-ref testcase url)))
+ (if body
+ (open-input-string body)
+ (error "mocked http-fetch Unexpected URL: " url)))))
+
+(define (mock-http-get testcase)
+ (lambda (url . rest)
+ (let ((body (assoc-ref testcase url))
+ (response-header
+ (build-response
+ #:version '(1 . 1)
+ #:code 200
+ #:reason-phrase "Ok"
+ #:headers `(
+ (content-type text/html (charset . "utf-8"))
+ (date . ,(make-date 0 10 58 12 6 3 2021 0))
+ (transfer-encoding (chunked)))
+ #:port #f
+ #:validate-headers? #t)))
+ (if body
+ (values response-header body)
+ (error "mocked http-get Unexpected URL: " url)))))
+
+(test-equal "go-module->guix-package"
+ '(package
+ (name "go-github-com-go-check-check")
+ (version "0.0.0-20201130134442-10cb98267c6c")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/go-check/check.git")
+ (commit (go-version->git-ref version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0000000000000000000000000000000000000000000000000000"))))
+ (build-system go-build-system)
+ (arguments
+ (quote (#:import-path "github.com/go-check/check")))
+ (inputs
+ (quasiquote (("go-github-com-kr-pretty"
+ (unquote go-github-com-kr-pretty)))))
+ (home-page "https://github.com/go-check/check")
+ (synopsis "Instructions")
+ (description #f)
+ (license (license:bsd-2)))
+ ;; Replace network resources with sample data.
+ (mock ((web client) http-get
+ (mock-http-get fixtures-go-check-test))
+ (mock ((guix http-client) http-fetch
+ (mock-http-fetch fixtures-go-check-test))
+ (go-module->guix-package "github.com/go-check/check"))))
+
+(test-end "go")
+
--
2.30.1
next prev parent reply other threads:[~2021-03-08 13:56 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-23 14:06 [bug#44178] Add a Go Module Importer Katherine Cox-Buday
2020-10-28 10:41 ` Ludovic Courtès
2020-10-28 10:42 ` Ludovic Courtès
2020-11-10 20:26 ` Marius Bakke
[not found] ` <CANe01w55ZO=_9v0HcDv248UsoLUXb_9WVAgM4LqiZ4E-r1XgXg@mail.gmail.com>
2020-11-11 1:23 ` Helio Machado
2021-01-23 21:35 ` [bug#44178] [PATCH] Create importer for Go modules guix-patches--- via
2021-01-23 22:41 ` Katherine Cox-Buday
2021-01-25 21:03 ` guix-patches--- via
2021-01-27 14:38 ` Katherine Cox-Buday
2021-01-28 13:27 ` Ludovic Courtès
2021-01-29 16:43 ` guix-patches--- via
2021-01-29 16:52 ` [bug#44178] [PATCHv2] " guix-patches--- via
2021-01-31 16:23 ` [bug#44178] [PATCH] " Ludovic Courtès
2021-02-19 15:51 ` JOULAUD François via Guix-patches via
2021-02-19 16:21 ` [bug#44178] [PATCHv3] " JOULAUD François via Guix-patches via
2021-03-02 21:54 ` [bug#44178] Add a Go Module Importer Ludovic Courtès
2021-03-04 5:40 ` [bug#44178] [PATCH v4] Re: bug#44178: " Maxim Cournoyer
2021-03-04 14:14 ` JOULAUD François via Guix-patches via
2021-03-04 15:47 ` Maxim Cournoyer
2021-03-08 13:54 ` JOULAUD François via Guix-patches via [this message]
2021-03-10 17:12 ` Ludovic Courtès
2021-01-28 5:01 ` [bug#44178] [PATCH] Create importer for Go modules Timmy Douglas
2020-11-11 20:48 ` [bug#44178] Add a Go Module Importer Katherine Cox-Buday
2020-12-09 14:22 ` [bug#44178] dftxbs3e
2020-12-10 2:42 ` [bug#44178] dftxbs3e
2020-12-10 3:14 ` [bug#44178] dftxbs3e
2021-01-28 7:29 ` [bug#44178] [PATCH] Create importer for Go modules guix-patches--- via
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210308135025.vn32lypnivpsilcg@fjo-extia-HPdeb.example.avalenn.eu \
--to=guix-patches@gnu.org \
--cc=44178@debbugs.gnu.org \
--cc=Francois.JOULAUD@radiofrance.com \
--cc=cox.katherine.e@gmail.com \
--cc=ludo@gnu.org \
--cc=maxim.cournoyer@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).