unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
From: "Ludovic Courtès" <ludo@gnu.org>
To: Katherine Cox-Buday <cox.katherine.e@gmail.com>
Cc: 44178@debbugs.gnu.org
Subject: [bug#44178] Add a Go Module Importer
Date: Wed, 28 Oct 2020 11:41:45 +0100	[thread overview]
Message-ID: <87a6w64opi.fsf@gnu.org> (raw)
In-Reply-To: <87sga5kpdp.fsf@gmail.com> (Katherine Cox-Buday's message of "Fri, 23 Oct 2020 09:06:58 -0500")

Hi Katherine,

Katherine Cox-Buday <cox.katherine.e@gmail.com> skribis:

>>From cc92cbcf5ae89891f478f319e955419800bdfcf9 Mon Sep 17 00:00:00 2001
> From: Katherine Cox-Buday <cox.katherine.e@gmail.com>
> Date: Thu, 22 Oct 2020 19:40:17 -0500
> Subject: [PATCH] * guix/import/go.scm: Created Go Importer *
>  guix/scripts/import.scm: Created Go Importer Subcommand * guix/import/go.scm
>  (importers): Added Go Importer Subcommand

Nice!  I think that can make a lot of people happy.  :-)

Here’s a quick review.  I won’t promise I can reply to followups in the
coming days because with the release preparation going on, I’d rather
focus on that.  So perhaps this patch will have to wait until after this
release, but certainly before the next one!

> +(define (escape-capital-letters s)
> +  "To avoid ambiguity when serving from case-insensitive file systems, the
> +$module and $version elements are case-encoded by replacing every uppercase
> +letter with an exclamation mark followed by the corresponding lower-case
> +letter."
> +  (let ((escaped-string (string)))
> +    (string-for-each-index
> +     (lambda (i)
> +       (let ((c (string-ref s i)))
> +         (set! escaped-string
> +           (string-concatenate
> +            (list escaped-string
> +                  (if (char-upper-case? c) "!" "")
> +                  (string (char-downcase c)))))))
> +     s)
> +    escaped-string))

As a general comment, the coding style in Guix is functional “by
default” (info "(guix) Coding Style").  That means we almost never use
‘set!’ and procedures that modify their arguments.

We also avoid idioms like car/cdr and ‘do’, which are more commonly used
in other Lisps, as you know very well.  ;-)

In the case above, I’d probably use ‘string-fold’.  The resulting code
should be easier to reason about and likely more efficient.

> +(define (fetch-latest-version goproxy-url module-path)
> +  "Fetches the version number of the latest version for MODULE-PATH from the
> +given GOPROXY-URL server."
> +  (assoc-ref
> +   (json-fetch (format #f "~a/~a/@latest" goproxy-url
> +                       (escape-capital-letters module-path)))
> +   "Version"))

I’d suggest using ‘define-json-mapping’ from (json) like in the other
importers.

> +(define (infer-module-root module-path)
> +  "Go modules can be defined at any level of a repository's tree, but querying
> +for the meta tag usually can only be done at the webpage at the root of the
> +repository. Therefore, it is sometimes necessary to try and derive a module's
> +root path from its path. For a set of well-known forges, the pattern of what
> +consists of a module's root page is known before hand."
> +  ;; See the following URL for the official Go equivalent:
> +  ;; https://github.com/golang/go/blob/846dce9d05f19a1f53465e62a304dea21b99f910/src/cmd/go/internal/vcs/vcs.go#L1026-L1087
> +  (define-record-type <scs>
> +    (make-scs url-prefix root-regex type)
> +    scs?
> +    (url-prefix	scs-url-prefix)
> +    (root-regex scs-root-regex)
> +    (type	scs-type))

Maybe VCS as “version control system”?  (It took me a while to guess
what “SCS” meant.)

> +(define (fetch-module-meta-data module-path)
> +  "Fetches module meta-data from a module's landing page. This is necessary
> +because goproxy servers don't currently provide all the information needed to
> +build a package."
> +  (let* ((port (http-fetch (string->uri (format #f "https://~a?go-get=1" module-path))))
> +         (module-metadata #f)
> +         (meta-tag-prefix "<meta name=\"go-import\" content=\"")
> +         (meta-tag-prefix-length (string-length meta-tag-prefix)))
> +    (do ((line (read-line port) (read-line port)))
> +        ((or (eof-object? line)
> +             module-metadata))
> +      (let ((meta-tag-index (string-contains line meta-tag-prefix)))
> +        (when meta-tag-index
> +          (let* ((start (+ meta-tag-index meta-tag-prefix-length))
> +                 (end (string-index line #\" start)))
> +            (set! module-metadata
> +              (string-split (substring/shared line start end) #\space))))))

I’d suggest a named ‘let’ or ‘fold’ here.

Likewise, instead of concatenating XML strings (which could lead to
malformed XML), I recommend using SXML: you would create an sexp like

  (meta (@ (name "go-import") (content …)))

and at the end pass it to ‘sxml->sxml’ (info "(guile) Reading and
Writing XML").

> +    (else
> +     (raise-exception (format #f "unsupported scs type: ~a" scs-type)))))

‘raise-exception’ takes an error condition.  In this case, we should use
(srfi srfi-34) for ‘raise’ write something like:

  (raise (condition (formatted-message (G_ "…" …))))

> +(define* (go-module->guix-package module-path #:key (goproxy-url "https://proxy.golang.org"))
> +  (call-with-temporary-output-file
> +   (lambda (temp port)
> +     (let* ((latest-version (fetch-latest-version goproxy-url module-path))
> +            (go.mod-path (fetch-go.mod goproxy-url module-path latest-version
> +                                       temp))

It seems that ‘go.mod-path’ isn’t used, and thus ‘fetch-go.mod’ &
co. aren’t used either, or am I overlooking something?

> +            (dependencies (map car (parse-go.mod temp)))

Please use ‘match’ instead, or perhaps define a record type for the
abstraction at hand.

> +            (guix-name (to-guix-package-name module-path))
> +            (root-module-path (infer-module-root module-path))
> +            ;; SCS type and URL are not included in goproxy information. For
> +            ;; this we need to fetch it from the official module page.
> +            (meta-data (fetch-module-meta-data root-module-path))
> +            (scs-type (module-meta-data-scs meta-data))
> +            (scs-repo-url (module-meta-data-repo-url meta-data goproxy-url)))
> +       (values
> +        `(package
> +           (name ,guix-name)
> +           ;; Elide the "v" prefix Go uses
> +           (version ,(string-trim latest-version #\v))
> +           (source
> +            ,(source-uri scs-type scs-repo-url temp))
> +           (build-system go-build-system)
> +           ,@(maybe-inputs (map to-guix-package-name dependencies))
> +           ;; TODO(katco): It would be nice to make an effort to fetch this
> +           ;; from known forges, e.g. GitHub
> +           (home-page ,(format #f "https://~a" root-module-path))
> +           (synopsis "A Go package")
> +           (description ,(format #f "~a is a Go package." guix-name))

Maybe something like “fill it out!” so we don’t get patch submissions
with the default synopsis/description.  :-)

> +           (license #f))

Likewise.

Two more things: could you (1) and an entry under “Invoking guix import”
in doc/guix.texi, and (2) add tests, taking inspiration from the
existing importer tests?

Thank you!

Ludo’.




  reply	other threads:[~2020-10-28 10:42 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-23 14:06 [bug#44178] Add a Go Module Importer Katherine Cox-Buday
2020-10-28 10:41 ` Ludovic Courtès [this message]
2020-10-28 10:42 ` Ludovic Courtès
2020-11-10 20:26 ` Marius Bakke
     [not found]   ` <CANe01w55ZO=_9v0HcDv248UsoLUXb_9WVAgM4LqiZ4E-r1XgXg@mail.gmail.com>
2020-11-11  1:23     ` Helio Machado
2021-01-23 21:35       ` [bug#44178] [PATCH] Create importer for Go modules guix-patches--- via
2021-01-23 22:41         ` Katherine Cox-Buday
2021-01-25 21:03           ` guix-patches--- via
2021-01-27 14:38             ` Katherine Cox-Buday
2021-01-28 13:27               ` Ludovic Courtès
2021-01-29 16:43                 ` guix-patches--- via
2021-01-29 16:52                   ` [bug#44178] [PATCHv2] " guix-patches--- via
2021-01-31 16:23                   ` [bug#44178] [PATCH] " Ludovic Courtès
2021-02-19 15:51                     ` JOULAUD François via Guix-patches via
2021-02-19 16:21                       ` [bug#44178] [PATCHv3] " JOULAUD François via Guix-patches via
2021-03-02 21:54                         ` [bug#44178] Add a Go Module Importer Ludovic Courtès
2021-03-04  5:40                           ` [bug#44178] [PATCH v4] Re: bug#44178: " Maxim Cournoyer
2021-03-04 14:14                             ` JOULAUD François via Guix-patches via
2021-03-04 15:47                               ` Maxim Cournoyer
2021-03-08 13:54                           ` [bug#44178] " JOULAUD François via Guix-patches via
2021-03-10 17:12                             ` bug#44178: " Ludovic Courtès
2021-01-28  5:01             ` [bug#44178] [PATCH] Create importer for Go modules Timmy Douglas
2020-11-11 20:48   ` [bug#44178] Add a Go Module Importer Katherine Cox-Buday
2020-12-09 14:22 ` [bug#44178] dftxbs3e
2020-12-10  2:42   ` [bug#44178] dftxbs3e
2020-12-10  3:14     ` [bug#44178] dftxbs3e
2021-01-28  7:29 ` [bug#44178] [PATCH] Create importer for Go modules guix-patches--- via

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87a6w64opi.fsf@gnu.org \
    --to=ludo@gnu.org \
    --cc=44178@debbugs.gnu.org \
    --cc=cox.katherine.e@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).