unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
* [bug#31736] [PATCH] Add an opam importer
@ 2018-06-06 17:23 Julien Lepiller
  2018-06-06 17:37 ` Julien Lepiller
  0 siblings, 1 reply; 6+ messages in thread
From: Julien Lepiller @ 2018-06-06 17:23 UTC (permalink / raw)
  To: 31736

[-- Attachment #1: Type: text/plain, Size: 77 bytes --]

Hi, this patch adds an importer for ocaml packages from the opam
repository.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-guix-Add-opam-importer.patch --]
[-- Type: text/x-patch, Size: 14277 bytes --]

From 04268bfc1a64c2c69f25977d76af8af34f7e0024 Mon Sep 17 00:00:00 2001
From: Julien Lepiller <julien@lepiller.eu>
Date: Wed, 6 Jun 2018 19:14:39 +0200
Subject: [PATCH] guix: Add opam importer.

* guix/scripts/import.scm (importers): Add opam.
* guix/scripts/import/opam.scm: New file.
* guix/import/opam.scm: New file.
* Makefile.am: Add them.
---
 Makefile.am                  |   2 +
 guix/import/opam.scm         | 210 +++++++++++++++++++++++++++++++++++
 guix/scripts/import.scm      |   2 +-
 guix/scripts/import/opam.scm |  92 +++++++++++++++
 4 files changed, 305 insertions(+), 1 deletion(-)
 create mode 100644 guix/import/opam.scm
 create mode 100644 guix/scripts/import/opam.scm

diff --git a/Makefile.am b/Makefile.am
index 7898a3648..6bf077d1b 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -230,11 +230,13 @@ MODULES +=					\
   guix/import/github.scm   			\
   guix/import/gnome.scm				\
   guix/import/json.scm				\
+  guix/import/opam.scm				\
   guix/import/pypi.scm				\
   guix/import/stackage.scm			\
   guix/scripts/import/crate.scm			\
   guix/scripts/import/gem.scm			\
   guix/scripts/import/json.scm  		\
+  guix/scripts/import/opam.scm			\
   guix/scripts/import/pypi.scm			\
   guix/scripts/import/stackage.scm		\
   guix/scripts/weather.scm
diff --git a/guix/import/opam.scm b/guix/import/opam.scm
new file mode 100644
index 000000000..92bcef292
--- /dev/null
+++ b/guix/import/opam.scm
@@ -0,0 +1,210 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2014 David Thompson <davet@gnu.org>
+;;; Copyright © 2015 Cyril Roelandt <tipecaml@gmail.com>
+;;; Copyright © 2015, 2016, 2017 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import opam)
+  #:use-module (ice-9 binary-ports)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 pretty-print)
+  #:use-module (ice-9 regex)
+  #:use-module ((ice-9 rdelim) #:select (read-line))
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
+  #:use-module (srfi srfi-34)
+  #:use-module (srfi srfi-35)
+  #:use-module (rnrs bytevectors)
+  #:use-module (web uri)
+  #:use-module (guix http-client)
+  #:use-module (guix ui)
+  #:use-module (guix utils)
+  #:use-module ((guix build utils)
+                #:select ((package-name->name+version
+                           . hyphen-package-name->name+version)))
+  #:use-module (guix import utils)
+  #:use-module ((guix download) #:prefix download:)
+  #:use-module (guix import json)
+  #:use-module (guix packages)
+  #:use-module (guix upstream)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix build-system python)
+  #:export (opam-urls
+            urls->htable
+            opam->guix-package
+            %opam-updater))
+
+(define (opam-urls)
+  "Fetch the urls.txt file from the opam repository and returns the list of
+URLs it contains."
+  (let ((port (http-fetch/cached (string->uri "https://opam.ocaml.org/urls.txt"))))
+    (let loop ((result '()))
+      (let ((line (read-line port)))
+        (if (eof-object? line)
+          (begin
+            (close port)
+            result)
+          (loop (cons line result)))))))
+
+(define (htable-update htable line)
+  "Parse @var{line} to get the name and version of the package and adds them
+to the hashtable."
+  (let* ((line (string-split line #\ ))
+         (url (car line)))
+    (unless (equal? url "repo")
+      (let ((sp (string-split url #\/)))
+        (when (equal? (car sp) "packages")
+          (let* ((versionstr (car (cdr (cdr sp))))
+                 (name1 (car (cdr sp)))
+                 (name2 (car (string-split versionstr #\.)))
+                 (version (string-join (cdr (string-split versionstr #\.)) ".")))
+            (when (equal? name1 name2)
+              (let ((curr (hash-ref htable name1 '())))
+                (hash-set! htable name1 (cons version curr))))))))))
+
+(define (urls->htable urls)
+  "Transform urls.txt in a hashtable whose keys are package names and values
+the list of available versions."
+  (let ((htable (make-hash-table)))
+    (let loop ((urls urls))
+      (if (eq? (length urls) 0)
+        htable
+        (begin
+          (htable-update htable (car urls))
+          (loop (cdr urls)))))))
+
+(define (latest-version versions)
+  "Find the most recent version from a list of versions."
+  (let loop ((versions (cdr versions)) (m (car versions)))
+    (if (eq? (length versions) 0)
+      m
+      (loop (cdr versions) (if (version>? m (car versions)) m (car versions))))))
+
+(define (fetch-url uri)
+  "Fetch and parse the url file.  Return the URL the package can be downloaded
+from."
+  (let ((port (http-fetch uri)))
+    (let loop ((result #f))
+      (let ((line (read-line port)))
+        (if (eof-object? line)
+          (begin
+            (close port)
+            result)
+          (let* ((line (string-split line #\ ))
+                 (key (car line)))
+            (if (equal? key "archive:")
+              (loop (string-trim-both (car (cdr line)) #\"))
+              (loop result))))))))
+
+(define (fetch-metadata uri)
+  "Fetch and parse the opam file.  Return an association list containing the
+homepage, the license and the list of inputs."
+  (let ((port (http-fetch uri)))
+    (let loop ((result '()) (deps? #f))
+      (let ((line (read-line port)))
+        (if (eof-object? line)
+          (begin
+            (close port)
+            result)
+          (let* ((line (string-split line #\ ))
+                 (key (car line))
+                 (deps? (if deps? (not (equal? key "]")) (equal? key "depends:")))
+                 (val (string-trim-both (string-join (cdr line) "") #\")))
+            (cond
+              ((equal? key "homepage:")
+               (loop (cons `("homepage" . ,val) result) deps?))
+              ((equal? key "license:")
+               (loop (cons `("license" . ,val) result) deps?))
+              ((and deps? (not (equal? val "[")))
+               (let ((curr (assoc-ref result "inputs"))
+                     (new (string-trim-both (car (string-split val #\{)) (list->char-set '(#\] #\[ #\")))))
+                 (loop (cons `("inputs" . ,(cons new (if curr curr '()))) result)
+                       (if (string-contains val "]") #f deps?))))
+              (else (loop result deps?)))))))))
+
+(define (string->license str)
+  (cond
+    ((equal? str "MIT") '(license:expat))
+    ((equal? str "GPL2") '(license:gpl2))
+    ((equal? str "LGPLv2") '(license:lgpl2))
+    (else `())))
+
+(define (deps->inputs deps)
+  "Transform the list of dependencies in a list of inputs.  Filter out anything
+that looks like a native-input."
+  (if (eq? deps #f)
+    '()
+    (let ((inputs
+            (map (lambda (input)
+                   (list input (list 'unquote (string->symbol input))))
+              (map (lambda (input)
+                     (cond
+                       ((equal? input "ocamlfind") "ocaml-findlib")
+                       ((string-prefix? "ocaml" input) input)
+                       (else (string-append "ocaml-" input))))
+                (filter (lambda (input) (not (string-prefix? "conf-" input))) deps)))))
+      (if (eq? (length inputs) 0) #f inputs))))
+
+(define (deps->native-inputs deps)
+  "Transform the list of dependencies in a list of native-inputs.  Filter out
+anything that doesn't look like a native-input."
+  (if (eq? deps #f)
+    '()
+    (let ((inputs
+            (map (lambda (input)
+                   (list input (list 'unquote (string->symbol input))))
+              (map (lambda (input) (substring input 5))
+                (filter (lambda (input) (string-prefix? "conf-" input)) deps)))))
+      (if (eq? (length inputs) 0) #f inputs))))
+
+(define (opam->guix-package name)
+  (let* ((htable (urls->htable (opam-urls)))
+         (versions (hash-ref htable name)))
+    (unless (eq? versions #f)
+      (let* ((version (latest-version versions))
+             (package-url (string-append "https://opam.ocaml.org/packages/" name
+                                         "/" name "." version "/"))
+             (url-url (string-append package-url "url"))
+             (opam-url (string-append package-url "opam"))
+             (source-url (fetch-url url-url))
+             (metadata (fetch-metadata opam-url))
+             (deps (assoc-ref metadata "inputs"))
+             (native-inputs (deps->native-inputs deps))
+             (inputs (deps->inputs deps)))
+        (call-with-temporary-output-file
+          (lambda (temp port)
+            (and (url-fetch source-url temp)
+                 `(package
+                    (name ,name)
+                    (version ,version)
+                    (source
+                      (origin
+                        (method url-fetch)
+                        (uri ,source-url)
+                        (sha256 (base32 ,(guix-hash-url temp)))))
+                    (build-system ocaml-build-system)
+                    ,@(if (eq? (length inputs) 0)
+                        '()
+                        `((inputs ,(list 'quasiquote inputs))))
+                    ,@(if (eq? (length native-inputs) 0)
+                        '()
+                        `((native-inputs ,(list 'quasiquote native-inputs))))
+                    (home-page ,(assoc-ref metadata "homepage"))
+                    (synopsis "")
+                    (description "")
+                    (license ,@(string->license (assoc-ref metadata "license")))))))))))
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 67bc7a755..bc03179e5 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -74,7 +74,7 @@ rather than \\n."
 ;;;
 
 (define importers '("gnu" "nix" "pypi" "cpan" "hackage" "stackage" "elpa" "gem"
-                    "cran" "crate" "texlive" "json"))
+                    "cran" "crate" "texlive" "json" "opam"))
 
 (define (resolve-importer name)
   (let ((module (resolve-interface
diff --git a/guix/scripts/import/opam.scm b/guix/scripts/import/opam.scm
new file mode 100644
index 000000000..ab8bbcb5b
--- /dev/null
+++ b/guix/scripts/import/opam.scm
@@ -0,0 +1,92 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2014 David Thompson <davet@gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix scripts import opam)
+  #:use-module (guix ui)
+  #:use-module (guix utils)
+  #:use-module (guix scripts)
+  #:use-module (guix import opam)
+  #:use-module (guix scripts import)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-11)
+  #:use-module (srfi srfi-37)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 format)
+  #:export (guix-import-opam))
+
+\f
+;;;
+;;; Command-line options.
+;;;
+
+(define %default-options
+  '())
+
+(define (show-help)
+  (display (G_ "Usage: guix import opam PACKAGE-NAME
+Import and convert the opam package for PACKAGE-NAME.\n"))
+  (display (G_ "
+  -h, --help             display this help and exit"))
+  (display (G_ "
+  -V, --version          display version information and exit"))
+  (newline)
+  (show-bug-report-information))
+
+(define %options
+  ;; Specification of the command-line options.
+  (cons* (option '(#\h "help") #f #f
+                 (lambda args
+                   (show-help)
+                   (exit 0)))
+         (option '(#\V "version") #f #f
+                 (lambda args
+                   (show-version-and-exit "guix import opam")))
+         %standard-import-options))
+
+\f
+;;;
+;;; Entry point.
+;;;
+
+(define (guix-import-opam . args)
+  (define (parse-options)
+    ;; Return the alist of option values.
+    (args-fold* args %options
+                (lambda (opt name arg result)
+                  (leave (G_ "~A: unrecognized option~%") name))
+                (lambda (arg result)
+                  (alist-cons 'argument arg result))
+                %default-options))
+
+  (let* ((opts (parse-options))
+         (args (filter-map (match-lambda
+                            (('argument . value)
+                             value)
+                            (_ #f))
+                           (reverse opts))))
+    (match args
+      ((package-name)
+       (let ((sexp (opam->guix-package package-name)))
+         (unless sexp
+           (leave (G_ "failed to download meta-data for package '~a'~%")
+                  package-name))
+         sexp))
+      (()
+       (leave (G_ "too few arguments~%")))
+      ((many ...)
+       (leave (G_ "too many arguments~%"))))))
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [bug#31736] [PATCH] Add an opam importer
  2018-06-06 17:23 [bug#31736] [PATCH] Add an opam importer Julien Lepiller
@ 2018-06-06 17:37 ` Julien Lepiller
  2018-06-10 21:28   ` Ludovic Courtès
  0 siblings, 1 reply; 6+ messages in thread
From: Julien Lepiller @ 2018-06-06 17:37 UTC (permalink / raw)
  To: 31736

[-- Attachment #1: Type: text/plain, Size: 267 bytes --]

Le Wed, 6 Jun 2018 19:23:29 +0200,
Julien Lepiller <julien@lepiller.eu> a écrit :

> Hi, this patch adds an importer for ocaml packages from the opam
> repository.

Whoops, the copyright lines and part of the code was wrong. This
version should be better :)

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-guix-Add-opam-importer.patch --]
[-- Type: text/x-patch, Size: 13395 bytes --]

From a5250186722305961f0a5d77cb8f7f36cdae0da0 Mon Sep 17 00:00:00 2001
From: Julien Lepiller <julien@lepiller.eu>
Date: Wed, 6 Jun 2018 19:14:39 +0200
Subject: [PATCH] guix: Add opam importer.

* guix/scripts/import.scm (importers): Add opam.
* guix/scripts/import/opam.scm: New file.
* guix/import/opam.scm: New file.
* Makefile.am: Add them.
---
 Makefile.am                  |   2 +
 guix/import/opam.scm         | 188 +++++++++++++++++++++++++++++++++++
 guix/scripts/import.scm      |   2 +-
 guix/scripts/import/opam.scm |  92 +++++++++++++++++
 4 files changed, 283 insertions(+), 1 deletion(-)
 create mode 100644 guix/import/opam.scm
 create mode 100644 guix/scripts/import/opam.scm

diff --git a/Makefile.am b/Makefile.am
index 7898a3648..6bf077d1b 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -230,11 +230,13 @@ MODULES +=					\
   guix/import/github.scm   			\
   guix/import/gnome.scm				\
   guix/import/json.scm				\
+  guix/import/opam.scm				\
   guix/import/pypi.scm				\
   guix/import/stackage.scm			\
   guix/scripts/import/crate.scm			\
   guix/scripts/import/gem.scm			\
   guix/scripts/import/json.scm  		\
+  guix/scripts/import/opam.scm			\
   guix/scripts/import/pypi.scm			\
   guix/scripts/import/stackage.scm		\
   guix/scripts/weather.scm
diff --git a/guix/import/opam.scm b/guix/import/opam.scm
new file mode 100644
index 000000000..608f8b449
--- /dev/null
+++ b/guix/import/opam.scm
@@ -0,0 +1,188 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Julien Lepiller <julien@lepiller.eu>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import opam)
+  #:use-module (ice-9 match)
+  #:use-module ((ice-9 rdelim) #:select (read-line))
+  #:use-module (srfi srfi-1)
+  #:use-module (web uri)
+  #:use-module (guix http-client)
+  #:use-module (guix utils)
+  #:use-module (guix import utils)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:export (opam->guix-package))
+
+(define (opam-urls)
+  "Fetch the urls.txt file from the opam repository and returns the list of
+URLs it contains."
+  (let ((port (http-fetch/cached (string->uri "https://opam.ocaml.org/urls.txt"))))
+    (let loop ((result '()))
+      (let ((line (read-line port)))
+        (if (eof-object? line)
+          (begin
+            (close port)
+            result)
+          (loop (cons line result)))))))
+
+(define (htable-update htable line)
+  "Parse @var{line} to get the name and version of the package and adds them
+to the hashtable."
+  (let* ((line (string-split line #\ ))
+         (url (car line)))
+    (unless (equal? url "repo")
+      (let ((sp (string-split url #\/)))
+        (when (equal? (car sp) "packages")
+          (let* ((versionstr (car (cdr (cdr sp))))
+                 (name1 (car (cdr sp)))
+                 (name2 (car (string-split versionstr #\.)))
+                 (version (string-join (cdr (string-split versionstr #\.)) ".")))
+            (when (equal? name1 name2)
+              (let ((curr (hash-ref htable name1 '())))
+                (hash-set! htable name1 (cons version curr))))))))))
+
+(define (urls->htable urls)
+  "Transform urls.txt in a hashtable whose keys are package names and values
+the list of available versions."
+  (let ((htable (make-hash-table)))
+    (let loop ((urls urls))
+      (if (eq? (length urls) 0)
+        htable
+        (begin
+          (htable-update htable (car urls))
+          (loop (cdr urls)))))))
+
+(define (latest-version versions)
+  "Find the most recent version from a list of versions."
+  (let loop ((versions (cdr versions)) (m (car versions)))
+    (if (eq? (length versions) 0)
+      m
+      (loop (cdr versions) (if (version>? m (car versions)) m (car versions))))))
+
+(define (fetch-url uri)
+  "Fetch and parse the url file.  Return the URL the package can be downloaded
+from."
+  (let ((port (http-fetch uri)))
+    (let loop ((result #f))
+      (let ((line (read-line port)))
+        (if (eof-object? line)
+          (begin
+            (close port)
+            result)
+          (let* ((line (string-split line #\ ))
+                 (key (car line)))
+            (if (equal? key "archive:")
+              (loop (string-trim-both (car (cdr line)) #\"))
+              (loop result))))))))
+
+(define (fetch-metadata uri)
+  "Fetch and parse the opam file.  Return an association list containing the
+homepage, the license and the list of inputs."
+  (let ((port (http-fetch uri)))
+    (let loop ((result '()) (deps? #f))
+      (let ((line (read-line port)))
+        (if (eof-object? line)
+          (begin
+            (close port)
+            result)
+          (let* ((line (string-split line #\ ))
+                 (key (car line))
+                 (deps? (if deps? (not (equal? key "]")) (equal? key "depends:")))
+                 (val (string-trim-both (string-join (cdr line) "") #\")))
+            (cond
+              ((equal? key "homepage:")
+               (loop (cons `("homepage" . ,val) result) deps?))
+              ((equal? key "license:")
+               (loop (cons `("license" . ,val) result) deps?))
+              ((and deps? (not (equal? val "[")))
+               (let ((curr (assoc-ref result "inputs"))
+                     (new (string-trim-both (car (string-split val #\{)) (list->char-set '(#\] #\[ #\")))))
+                 (loop (cons `("inputs" . ,(cons new (if curr curr '()))) result)
+                       (if (string-contains val "]") #f deps?))))
+              (else (loop result deps?)))))))))
+
+(define (string->license str)
+  (cond
+    ((equal? str "MIT") '(license:expat))
+    ((equal? str "GPL2") '(license:gpl2))
+    ((equal? str "LGPLv2") '(license:lgpl2))
+    (else `())))
+
+(define (deps->inputs deps)
+  "Transform the list of dependencies in a list of inputs.  Filter out anything
+that looks like a native-input."
+  (if (eq? deps #f)
+    '()
+    (let ((inputs
+            (map (lambda (input)
+                   (list input (list 'unquote (string->symbol input))))
+              (map (lambda (input)
+                     (cond
+                       ((equal? input "ocamlfind") "ocaml-findlib")
+                       ((string-prefix? "ocaml" input) input)
+                       (else (string-append "ocaml-" input))))
+                (filter (lambda (input) (not (string-prefix? "conf-" input))) deps)))))
+      (if (eq? (length inputs) 0) '() inputs))))
+
+(define (deps->native-inputs deps)
+  "Transform the list of dependencies in a list of native-inputs.  Filter out
+anything that doesn't look like a native-input."
+  (if (eq? deps #f)
+    '()
+    (let ((inputs
+            (map (lambda (input)
+                   (list input (list 'unquote (string->symbol input))))
+              (map (lambda (input) (substring input 5))
+                (filter (lambda (input) (string-prefix? "conf-" input)) deps)))))
+      (if (eq? (length inputs) 0) '() inputs))))
+
+(define (opam->guix-package name)
+  (let* ((htable (urls->htable (opam-urls)))
+         (versions (hash-ref htable name)))
+    (unless (eq? versions #f)
+      (let* ((version (latest-version versions))
+             (package-url (string-append "https://opam.ocaml.org/packages/" name
+                                         "/" name "." version "/"))
+             (url-url (string-append package-url "url"))
+             (opam-url (string-append package-url "opam"))
+             (source-url (fetch-url url-url))
+             (metadata (fetch-metadata opam-url))
+             (deps (assoc-ref metadata "inputs"))
+             (native-inputs (deps->native-inputs deps))
+             (inputs (deps->inputs deps)))
+        (call-with-temporary-output-file
+          (lambda (temp port)
+            (and (url-fetch source-url temp)
+                 `(package
+                    (name ,name)
+                    (version ,version)
+                    (source
+                      (origin
+                        (method url-fetch)
+                        (uri ,source-url)
+                        (sha256 (base32 ,(guix-hash-url temp)))))
+                    (build-system ocaml-build-system)
+                    ,@(if (eq? (length inputs) 0)
+                        '()
+                        `((inputs ,(list 'quasiquote inputs))))
+                    ,@(if (eq? (length native-inputs) 0)
+                        '()
+                        `((native-inputs ,(list 'quasiquote native-inputs))))
+                    (home-page ,(assoc-ref metadata "homepage"))
+                    (synopsis "")
+                    (description "")
+                    (license ,@(string->license (assoc-ref metadata "license")))))))))))
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 67bc7a755..bc03179e5 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -74,7 +74,7 @@ rather than \\n."
 ;;;
 
 (define importers '("gnu" "nix" "pypi" "cpan" "hackage" "stackage" "elpa" "gem"
-                    "cran" "crate" "texlive" "json"))
+                    "cran" "crate" "texlive" "json" "opam"))
 
 (define (resolve-importer name)
   (let ((module (resolve-interface
diff --git a/guix/scripts/import/opam.scm b/guix/scripts/import/opam.scm
new file mode 100644
index 000000000..b54987874
--- /dev/null
+++ b/guix/scripts/import/opam.scm
@@ -0,0 +1,92 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Julien Lepiller <julien@lepiller.eu>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix scripts import opam)
+  #:use-module (guix ui)
+  #:use-module (guix utils)
+  #:use-module (guix scripts)
+  #:use-module (guix import opam)
+  #:use-module (guix scripts import)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-11)
+  #:use-module (srfi srfi-37)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 format)
+  #:export (guix-import-opam))
+
+\f
+;;;
+;;; Command-line options.
+;;;
+
+(define %default-options
+  '())
+
+(define (show-help)
+  (display (G_ "Usage: guix import opam PACKAGE-NAME
+Import and convert the opam package for PACKAGE-NAME.\n"))
+  (display (G_ "
+  -h, --help             display this help and exit"))
+  (display (G_ "
+  -V, --version          display version information and exit"))
+  (newline)
+  (show-bug-report-information))
+
+(define %options
+  ;; Specification of the command-line options.
+  (cons* (option '(#\h "help") #f #f
+                 (lambda args
+                   (show-help)
+                   (exit 0)))
+         (option '(#\V "version") #f #f
+                 (lambda args
+                   (show-version-and-exit "guix import opam")))
+         %standard-import-options))
+
+\f
+;;;
+;;; Entry point.
+;;;
+
+(define (guix-import-opam . args)
+  (define (parse-options)
+    ;; Return the alist of option values.
+    (args-fold* args %options
+                (lambda (opt name arg result)
+                  (leave (G_ "~A: unrecognized option~%") name))
+                (lambda (arg result)
+                  (alist-cons 'argument arg result))
+                %default-options))
+
+  (let* ((opts (parse-options))
+         (args (filter-map (match-lambda
+                            (('argument . value)
+                             value)
+                            (_ #f))
+                           (reverse opts))))
+    (match args
+      ((package-name)
+       (let ((sexp (opam->guix-package package-name)))
+         (unless sexp
+           (leave (G_ "failed to download meta-data for package '~a'~%")
+                  package-name))
+         sexp))
+      (()
+       (leave (G_ "too few arguments~%")))
+      ((many ...)
+       (leave (G_ "too many arguments~%"))))))
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [bug#31736] [PATCH] Add an opam importer
  2018-06-06 17:37 ` Julien Lepiller
@ 2018-06-10 21:28   ` Ludovic Courtès
  2018-07-07 21:56     ` Julien Lepiller
  0 siblings, 1 reply; 6+ messages in thread
From: Ludovic Courtès @ 2018-06-10 21:28 UTC (permalink / raw)
  To: Julien Lepiller; +Cc: 31736

Salut Julien!

Julien Lepiller <julien@lepiller.eu> skribis:

> From a5250186722305961f0a5d77cb8f7f36cdae0da0 Mon Sep 17 00:00:00 2001
> From: Julien Lepiller <julien@lepiller.eu>
> Date: Wed, 6 Jun 2018 19:14:39 +0200
> Subject: [PATCH] guix: Add opam importer.
>
> * guix/scripts/import.scm (importers): Add opam.
> * guix/scripts/import/opam.scm: New file.
> * guix/import/opam.scm: New file.
> * Makefile.am: Add them.

Very nice!  I hope that’ll help create bridges with more fellow OCaml
hackers.  :-)

I have some comments, mostly about style:

> +(define (htable-update htable line)
> +  "Parse @var{line} to get the name and version of the package and adds them
> +to the hashtable."
> +  (let* ((line (string-split line #\ ))
> +         (url (car line)))
> +    (unless (equal? url "repo")
> +      (let ((sp (string-split url #\/)))
> +        (when (equal? (car sp) "packages")
> +          (let* ((versionstr (car (cdr (cdr sp))))
> +                 (name1 (car (cdr sp)))
> +                 (name2 (car (string-split versionstr #\.)))
> +                 (version (string-join (cdr (string-split versionstr #\.)) ".")))
> +            (when (equal? name1 name2)
> +              (let ((curr (hash-ref htable name1 '())))
> +                (hash-set! htable name1 (cons version curr))))))))))

There are a couple of things that don’t fully match the coding style
(see <https://www.gnu.org/software/guix/manual/html_node/Coding-Style.html>):
try to use full names for identifiers, favor a functional style (here
maybe you could use a vhash¹ instead of a hash table), and, last but not
least, use ‘match’ instead of ‘car’ and ‘cdr’.  :-)

¹ https://www.gnu.org/software/guile/manual/html_node/VHashes.html

> +(define (fetch-url uri)
> +  "Fetch and parse the url file.  Return the URL the package can be downloaded
> +from."

Maybe ‘fetch-url-list’ or ‘fetch-package-urls’?

> +(define (fetch-metadata uri)
> +  "Fetch and parse the opam file.  Return an association list containing the
> +homepage, the license and the list of inputs."

Maybe ‘fetch-package-metadata’ to clarify that it’s per-package?

> +(define (deps->inputs deps)
> +  "Transform the list of dependencies in a list of inputs.  Filter out anything
> +that looks like a native-input."

So that would be ‘dependencies->inputs’.  :-)

> +  (if (eq? deps #f)

Rather: (if (not dependencies) …)

> +    (let ((inputs
> +            (map (lambda (input)
> +                   (list input (list 'unquote (string->symbol input))))
> +              (map (lambda (input)
> +                     (cond
> +                       ((equal? input "ocamlfind") "ocaml-findlib")
> +                       ((string-prefix? "ocaml" input) input)
> +                       (else (string-append "ocaml-" input))))
> +                (filter (lambda (input) (not (string-prefix? "conf-" input))) deps)))))

The indentation is misleading: the 2nd argument to ‘map’ should be
aligned with the 1st.

Perhaps you can use ‘filter-map’ here?

> +      (if (eq? (length inputs) 0) '() inputs))))

(eq? (length inputs) 0) → (null? inputs)

Note that ‘null?’ is constant-time whereas ‘length’ is O(n).

Could you add:

  • A few lines in guix.texi, under “Invoking guix import”;

  • Tests in tests/opam.scm (you can take a look at tests/cpan.scm,
    tests/elpa.scm, etc. for inspiration.)

Thank you!

Ludo’.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [bug#31736] [PATCH] Add an opam importer
  2018-06-10 21:28   ` Ludovic Courtès
@ 2018-07-07 21:56     ` Julien Lepiller
  2018-07-09 12:44       ` Ludovic Courtès
  0 siblings, 1 reply; 6+ messages in thread
From: Julien Lepiller @ 2018-07-07 21:56 UTC (permalink / raw)
  To: 31736

[-- Attachment #1: Type: text/plain, Size: 3705 bytes --]

Le 2018-06-10 23:28, ludo@gnu.org a écrit :
> Salut Julien!
> 
> Julien Lepiller <julien@lepiller.eu> skribis:
> 
>> From a5250186722305961f0a5d77cb8f7f36cdae0da0 Mon Sep 17 00:00:00 2001
>> From: Julien Lepiller <julien@lepiller.eu>
>> Date: Wed, 6 Jun 2018 19:14:39 +0200
>> Subject: [PATCH] guix: Add opam importer.
>> 
>> * guix/scripts/import.scm (importers): Add opam.
>> * guix/scripts/import/opam.scm: New file.
>> * guix/import/opam.scm: New file.
>> * Makefile.am: Add them.
> 
> Very nice!  I hope that’ll help create bridges with more fellow OCaml
> hackers.  :-)
> 
> I have some comments, mostly about style:
> 
>> +(define (htable-update htable line)
>> +  "Parse @var{line} to get the name and version of the package and 
>> adds them
>> +to the hashtable."
>> +  (let* ((line (string-split line #\ ))
>> +         (url (car line)))
>> +    (unless (equal? url "repo")
>> +      (let ((sp (string-split url #\/)))
>> +        (when (equal? (car sp) "packages")
>> +          (let* ((versionstr (car (cdr (cdr sp))))
>> +                 (name1 (car (cdr sp)))
>> +                 (name2 (car (string-split versionstr #\.)))
>> +                 (version (string-join (cdr (string-split versionstr 
>> #\.)) ".")))
>> +            (when (equal? name1 name2)
>> +              (let ((curr (hash-ref htable name1 '())))
>> +                (hash-set! htable name1 (cons version curr))))))))))
> 
> There are a couple of things that don’t fully match the coding style
> (see 
> <https://www.gnu.org/software/guix/manual/html_node/Coding-Style.html>):
> try to use full names for identifiers, favor a functional style (here
> maybe you could use a vhash¹ instead of a hash table), and, last but 
> not
> least, use ‘match’ instead of ‘car’ and ‘cdr’.  :-)
> 
> ¹ https://www.gnu.org/software/guile/manual/html_node/VHashes.html
> 
>> +(define (fetch-url uri)
>> +  "Fetch and parse the url file.  Return the URL the package can be 
>> downloaded
>> +from."
> 
> Maybe ‘fetch-url-list’ or ‘fetch-package-urls’?
> 
>> +(define (fetch-metadata uri)
>> +  "Fetch and parse the opam file.  Return an association list 
>> containing the
>> +homepage, the license and the list of inputs."
> 
> Maybe ‘fetch-package-metadata’ to clarify that it’s per-package?
> 
>> +(define (deps->inputs deps)
>> +  "Transform the list of dependencies in a list of inputs.  Filter 
>> out anything
>> +that looks like a native-input."
> 
> So that would be ‘dependencies->inputs’.  :-)
> 
>> +  (if (eq? deps #f)
> 
> Rather: (if (not dependencies) …)
> 
>> +    (let ((inputs
>> +            (map (lambda (input)
>> +                   (list input (list 'unquote (string->symbol 
>> input))))
>> +              (map (lambda (input)
>> +                     (cond
>> +                       ((equal? input "ocamlfind") "ocaml-findlib")
>> +                       ((string-prefix? "ocaml" input) input)
>> +                       (else (string-append "ocaml-" input))))
>> +                (filter (lambda (input) (not (string-prefix? "conf-" 
>> input))) deps)))))
> 
> The indentation is misleading: the 2nd argument to ‘map’ should be
> aligned with the 1st.
> 
> Perhaps you can use ‘filter-map’ here?
> 
>> +      (if (eq? (length inputs) 0) '() inputs))))
> 
> (eq? (length inputs) 0) → (null? inputs)
> 
> Note that ‘null?’ is constant-time whereas ‘length’ is O(n).
> 
> Could you add:
> 
>   • A few lines in guix.texi, under “Invoking guix import”;
> 
>   • Tests in tests/opam.scm (you can take a look at tests/cpan.scm,
>     tests/elpa.scm, etc. for inspiration.)
> 
> Thank you!
> 
> Ludo’.


Here is a new version. What do you think?

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-guix-Add-opam-importer.patch --]
[-- Type: text/x-diff; name=0001-guix-Add-opam-importer.patch, Size: 19045 bytes --]

From b3fd75a804a4264cb083584591a3e589886d96c8 Mon Sep 17 00:00:00 2001
From: Julien Lepiller <julien@lepiller.eu>
Date: Wed, 6 Jun 2018 19:14:39 +0200
Subject: [PATCH] guix: Add opam importer.

* guix/scripts/import.scm (importers): Add opam.
* guix/scripts/import/opam.scm: New file.
* guix/import/opam.scm: New file.
* tests/opam.scm: New file.
* Makefile.am: Add them.
* doc/guix.texi (Invoking guix import): Document it.
---
 Makefile.am                  |   3 +
 doc/guix.texi                |   6 ++
 guix/import/opam.scm         | 193 +++++++++++++++++++++++++++++++++++
 guix/scripts/import.scm      |   2 +-
 guix/scripts/import/opam.scm |  92 +++++++++++++++++
 tests/opam.scm               | 118 +++++++++++++++++++++
 6 files changed, 413 insertions(+), 1 deletion(-)
 create mode 100644 guix/import/opam.scm
 create mode 100644 guix/scripts/import/opam.scm
 create mode 100644 tests/opam.scm

diff --git a/Makefile.am b/Makefile.am
index 5dc04de35..618d1653e 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -231,11 +231,13 @@ MODULES +=					\
   guix/import/github.scm   			\
   guix/import/gnome.scm				\
   guix/import/json.scm				\
+  guix/import/opam.scm				\
   guix/import/pypi.scm				\
   guix/import/stackage.scm			\
   guix/scripts/import/crate.scm			\
   guix/scripts/import/gem.scm			\
   guix/scripts/import/json.scm  		\
+  guix/scripts/import/opam.scm			\
   guix/scripts/import/pypi.scm			\
   guix/scripts/import/stackage.scm		\
   guix/scripts/weather.scm
@@ -382,6 +384,7 @@ if HAVE_GUILE_JSON
 
 SCM_TESTS += 					\
   tests/pypi.scm				\
+  tests/opam.scm				\
   tests/cpan.scm				\
   tests/gem.scm					\
   tests/crate.scm
diff --git a/doc/guix.texi b/doc/guix.texi
index aeac5aaa8..509aba539 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -6651,6 +6651,12 @@ in Guix.
 @cindex crate
 Import metadata from the crates.io Rust package repository
 @uref{https://crates.io, crates.io}.
+
+@item opam
+@cindex opam
+@cindex ocaml
+Import metadata from the @uref{https://opam.ocaml.org/, Opam} package
+repository used by the OCaml community.
 @end table
 
 The structure of the @command{guix import} code is modular.  It would be
diff --git a/guix/import/opam.scm b/guix/import/opam.scm
new file mode 100644
index 000000000..f252bdc31
--- /dev/null
+++ b/guix/import/opam.scm
@@ -0,0 +1,193 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Julien Lepiller <julien@lepiller.eu>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import opam)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 vlist)
+  #:use-module ((ice-9 rdelim) #:select (read-line))
+  #:use-module (srfi srfi-1)
+  #:use-module (web uri)
+  #:use-module (guix http-client)
+  #:use-module (guix utils)
+  #:use-module (guix import utils)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:export (opam->guix-package))
+
+(define (opam-urls)
+  "Fetch the urls.txt file from the opam repository and returns the list of
+URLs it contains."
+  (let ((port (http-fetch/cached (string->uri "https://opam.ocaml.org/urls.txt"))))
+    (let loop ((result '()))
+      (let ((line (read-line port)))
+        (if (eof-object? line)
+          (begin
+            (close port)
+            result)
+          (loop (cons line result)))))))
+
+(define (vhash-ref hashtable key default)
+  (match (vhash-assoc key hashtable)
+    (#f default)
+    ((_ . x) x)))
+
+(define (hashtable-update hashtable line)
+  "Parse @var{line} to get the name and version of the package and adds them
+to the hashtable."
+  (let* ((line (string-split line #\ )))
+    (match line
+      ((url foo ...)
+       (if (equal? url "repo")
+         hashtable
+         (match (string-split url #\/)
+           ((type name1 versionstr foo ...)
+            (if (equal? type "packages")
+              (match (string-split versionstr #\.)
+                ((name2 versions ...)
+                 (let ((version (string-join versions ".")))
+                   (if (equal? name1 name2)
+                     (let ((curr (vhash-ref hashtable name1 '())))
+                       (vhash-cons name1 (cons version curr) hashtable))
+                     hashtable)))
+                (_ hashtable))
+              hashtable))
+           (_ hashtable))))
+      (_ hashtable))))
+
+(define (urls->hashtable urls)
+  "Transform urls.txt in a hashtable whose keys are package names and values
+the list of available versions."
+  (let ((hashtable vlist-null))
+    (let loop ((urls urls) (hashtable hashtable))
+      (match urls
+        (() hashtable)
+        ((url rest ...) (loop rest (hashtable-update hashtable url)))))))
+
+(define (latest-version versions)
+  "Find the most recent version from a list of versions."
+  (match versions
+    ((first rest ...)
+     (let loop ((versions rest) (m first))
+       (match versions
+         (() m)
+         ((first rest ...)
+          (loop rest (if (version>? m first) m first))))))))
+
+(define (fetch-package-url uri)
+  "Fetch and parse the url file.  Return the URL the package can be downloaded
+from."
+  (let ((port (http-fetch uri)))
+    (let loop ((result #f))
+      (let ((line (read-line port)))
+        (if (eof-object? line)
+          (begin
+            (close port)
+            result)
+          (let* ((line (string-split line #\ )))
+            (match line
+              ((key value rest ...)
+               (if (member key '("archive:" "http:"))
+                 (loop (string-trim-both value #\"))
+                 (loop result))))))))))
+
+(define (fetch-package-metadata uri)
+  "Fetch and parse the opam file.  Return an association list containing the
+homepage, the license and the list of inputs."
+  (let ((port (http-fetch uri)))
+    (let loop ((result '()) (dependencies? #f))
+      (let ((line (read-line port)))
+        (if (eof-object? line)
+          (begin
+            (close port)
+            result)
+          (let* ((line (string-split line #\ )))
+            (match line
+               ((key value ...)
+                (let ((dependencies?
+                        (if dependencies?
+                          (not (equal? key "]"))
+                          (equal? key "depends:")))
+                      (val (string-trim-both (string-join value "") #\")))
+                  (cond
+                    ((equal? key "homepage:")
+                     (loop (cons `("homepage" . ,val) result) dependencies?))
+                    ((equal? key "license:")
+                     (loop (cons `("license" . ,val) result) dependencies?))
+                    ((and dependencies? (not (equal? val "[")))
+                     (match (string-split val #\{)
+                       ((val rest ...)
+                        (let ((curr (assoc-ref result "inputs"))
+                              (new (string-trim-both
+                                     val (list->char-set '(#\] #\[ #\")))))
+                          (loop (cons `("inputs" . ,(cons new (if curr curr '()))) result)
+                                (if (string-contains val "]") #f dependencies?))))))
+                    (else (loop result dependencies?))))))))))))
+
+(define (string->license str)
+  (cond
+    ((equal? str "MIT") '(license:expat))
+    ((equal? str "GPL2") '(license:gpl2))
+    ((equal? str "LGPLv2") '(license:lgpl2))
+    (else `())))
+
+(define (ocaml-name->guix-name name)
+  (cond
+    ((equal? name "ocamlfind") "ocaml-findlib")
+    ((string-prefix? "ocaml" name) name)
+    ((string-prefix? "conf-" name) (substring name 5))
+    (else (string-append "ocaml-" name))))
+
+(define (dependencies->inputs dependencies)
+  "Transform the list of dependencies in a list of inputs."
+  (if (not dependencies)
+    '()
+    (map (lambda (input)
+           (list input (list 'unquote (string->symbol input))))
+         (map ocaml-name->guix-name dependencies))))
+
+(define (opam->guix-package name)
+  (let* ((hashtable (urls->hashtable (opam-urls)))
+         (versions (vhash-ref hashtable name #f)))
+    (unless (eq? versions #f)
+      (let* ((version (latest-version versions))
+             (package-url (string-append "https://opam.ocaml.org/packages/" name
+                                         "/" name "." version "/"))
+             (url-url (string-append package-url "url"))
+             (opam-url (string-append package-url "opam"))
+             (source-url (fetch-package-url url-url))
+             (metadata (fetch-package-metadata opam-url))
+             (dependencies (assoc-ref metadata "inputs"))
+             (inputs (dependencies->inputs dependencies)))
+        (call-with-temporary-output-file
+          (lambda (temp port)
+            (and (url-fetch source-url temp)
+                 `(package
+                    (name ,(ocaml-name->guix-name name))
+                    (version ,version)
+                    (source
+                      (origin
+                        (method url-fetch)
+                        (uri ,source-url)
+                        (sha256 (base32 ,(guix-hash-url temp)))))
+                    (build-system ocaml-build-system)
+                    ,@(if (null? inputs)
+                        '()
+                        `((inputs ,(list 'quasiquote inputs))))
+                    (home-page ,(assoc-ref metadata "homepage"))
+                    (synopsis "")
+                    (description "")
+                    (license ,@(string->license (assoc-ref metadata "license")))))))))))
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 67bc7a755..bc03179e5 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -74,7 +74,7 @@ rather than \\n."
 ;;;
 
 (define importers '("gnu" "nix" "pypi" "cpan" "hackage" "stackage" "elpa" "gem"
-                    "cran" "crate" "texlive" "json"))
+                    "cran" "crate" "texlive" "json" "opam"))
 
 (define (resolve-importer name)
   (let ((module (resolve-interface
diff --git a/guix/scripts/import/opam.scm b/guix/scripts/import/opam.scm
new file mode 100644
index 000000000..b54987874
--- /dev/null
+++ b/guix/scripts/import/opam.scm
@@ -0,0 +1,92 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Julien Lepiller <julien@lepiller.eu>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix scripts import opam)
+  #:use-module (guix ui)
+  #:use-module (guix utils)
+  #:use-module (guix scripts)
+  #:use-module (guix import opam)
+  #:use-module (guix scripts import)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-11)
+  #:use-module (srfi srfi-37)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 format)
+  #:export (guix-import-opam))
+
+\f
+;;;
+;;; Command-line options.
+;;;
+
+(define %default-options
+  '())
+
+(define (show-help)
+  (display (G_ "Usage: guix import opam PACKAGE-NAME
+Import and convert the opam package for PACKAGE-NAME.\n"))
+  (display (G_ "
+  -h, --help             display this help and exit"))
+  (display (G_ "
+  -V, --version          display version information and exit"))
+  (newline)
+  (show-bug-report-information))
+
+(define %options
+  ;; Specification of the command-line options.
+  (cons* (option '(#\h "help") #f #f
+                 (lambda args
+                   (show-help)
+                   (exit 0)))
+         (option '(#\V "version") #f #f
+                 (lambda args
+                   (show-version-and-exit "guix import opam")))
+         %standard-import-options))
+
+\f
+;;;
+;;; Entry point.
+;;;
+
+(define (guix-import-opam . args)
+  (define (parse-options)
+    ;; Return the alist of option values.
+    (args-fold* args %options
+                (lambda (opt name arg result)
+                  (leave (G_ "~A: unrecognized option~%") name))
+                (lambda (arg result)
+                  (alist-cons 'argument arg result))
+                %default-options))
+
+  (let* ((opts (parse-options))
+         (args (filter-map (match-lambda
+                            (('argument . value)
+                             value)
+                            (_ #f))
+                           (reverse opts))))
+    (match args
+      ((package-name)
+       (let ((sexp (opam->guix-package package-name)))
+         (unless sexp
+           (leave (G_ "failed to download meta-data for package '~a'~%")
+                  package-name))
+         sexp))
+      (()
+       (leave (G_ "too few arguments~%")))
+      ((many ...)
+       (leave (G_ "too many arguments~%"))))))
diff --git a/tests/opam.scm b/tests/opam.scm
new file mode 100644
index 000000000..26832174a
--- /dev/null
+++ b/tests/opam.scm
@@ -0,0 +1,118 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Julien Lepiller <julien@lepiller.eu>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (test-opam)
+  #:use-module (guix import opam)
+  #:use-module (guix base32)
+  #:use-module (guix hash)
+  #:use-module (guix tests)
+  #:use-module ((guix build utils) #:select (delete-file-recursively mkdir-p which))
+  #:use-module (srfi srfi-64)
+  #:use-module (web uri)
+  #:use-module (ice-9 match))
+
+(define test-url-file
+  "http: \"https://example.org/foo-1.0.0.tar.gz\"
+checksum: \"ac8920f39a8100b94820659bc2c20817\"")
+
+(define test-source-hash
+  "")
+
+(define test-urls
+  "repo ac8920f39a8100b94820659bc2c20817 0o644
+packages/foo/foo.1.0.0/url ac8920f39a8100b94820659bc2c20817 0o644
+packages/foo/foo.1.0.0/opam ac8920f39a8100b94820659bc2c20817 0o644
+packages/foo/foo.1.0.0/descr ac8920f39a8100b94820659bc2c20817 0o644")
+
+(define test-opam-file
+"opam-version: 1.2
+maintainer: \"Alice Doe\"
+authors: \"Alice Doe, John Doe\"
+homepage: \"https://example.org/\"
+bug-reports: \"https://example.org/bugs\"
+license: \"MIT\"
+dev-repo: \"https://example.org/git\"
+build: [
+  \"ocaml\" \"pkg/pkg.ml\" \"build\" \"--pinned\" \"%{pinned}%\"
+]
+build-test: [
+  \"ocaml\" \"pkg/pkg.ml\" \"build\" \"--pinned\" \"%{pinned}%\" \"--tests\" \"true\"
+]
+depends: [
+  \"alcotest\" {test & >= \"0.7.2\"}
+  \"ocamlbuild\" {build & >= \"0.9.2\"}
+]")
+
+(test-begin "opam")
+
+(test-assert "opam->guix-package"
+  ;; Replace network resources with sample data.
+    (mock ((guix import utils) url-fetch
+           (lambda (url file-name)
+             (match url
+               ("https://example.org/foo-1.0.0.tar.gz"
+                (begin
+                  (mkdir-p "foo-1.0.0")
+                  (system* "tar" "czvf" file-name "foo-1.0.0/")
+                  (delete-file-recursively "foo-1.0.0")
+                  (set! test-source-hash
+                    (call-with-input-file file-name port-sha256))))
+               (_ (error "Unexpected URL: " url)))))
+          (mock ((guix http-client) http-fetch/cached
+                 (lambda (url . rest)
+                   (match (uri->string url)
+                     ("https://opam.ocaml.org/urls.txt"
+                      (values (open-input-string test-urls)
+                              (string-length test-urls)))
+                     (_ (error "Unexpected URL: " url)))))
+                (mock ((guix http-client) http-fetch
+                       (lambda (url . rest)
+                         (match url
+                           ("https://opam.ocaml.org/packages/foo/foo.1.0.0/url"
+                            (values (open-input-string test-url-file)
+                                    (string-length test-url-file)))
+                           ("https://opam.ocaml.org/packages/foo/foo.1.0.0/opam"
+                            (values (open-input-string test-opam-file)
+                                    (string-length test-opam-file)))
+                           (_ (error "Unexpected URL: " url)))))
+                      (match (opam->guix-package "foo")
+                        (('package
+                           ('name "ocaml-foo")
+                           ('version "1.0.0")
+                           ('source ('origin
+                                      ('method 'url-fetch)
+                                      ('uri "https://example.org/foo-1.0.0.tar.gz")
+                                      ('sha256
+                                       ('base32
+                                        (? string? hash)))))
+                           ('build-system 'ocaml-build-system)
+                           ('inputs
+                            ('quasiquote
+                             (("ocamlbuild" ('unquote 'ocamlbuild))
+                              ("ocaml-alcotest" ('unquote 'ocaml-alcotest)))))
+                           ('home-page "https://example.org/")
+                           ('synopsis "")
+                           ('description "")
+                           ('license 'license:expat))
+                         (string=? (bytevector->nix-base32-string
+                                    test-source-hash)
+                                   hash))
+                        (x
+                         (pk 'fail x #f)))))))
+
+(test-end "opam")
-- 
2.18.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [bug#31736] [PATCH] Add an opam importer
  2018-07-07 21:56     ` Julien Lepiller
@ 2018-07-09 12:44       ` Ludovic Courtès
  2018-07-10 11:47         ` bug#31736: " Julien Lepiller
  0 siblings, 1 reply; 6+ messages in thread
From: Ludovic Courtès @ 2018-07-09 12:44 UTC (permalink / raw)
  To: Julien Lepiller; +Cc: 31736

Hello Julien,

Julien Lepiller <julien@lepiller.eu> skribis:

> Here is a new version. What do you think?

I think it looks great.  :-)

> From b3fd75a804a4264cb083584591a3e589886d96c8 Mon Sep 17 00:00:00 2001
> From: Julien Lepiller <julien@lepiller.eu>
> Date: Wed, 6 Jun 2018 19:14:39 +0200
> Subject: [PATCH] guix: Add opam importer.
>
> * guix/scripts/import.scm (importers): Add opam.
> * guix/scripts/import/opam.scm: New file.
> * guix/import/opam.scm: New file.
> * tests/opam.scm: New file.
> * Makefile.am: Add them.
> * doc/guix.texi (Invoking guix import): Document it.

[...]

> +@item opam
> +@cindex opam
> +@cindex ocaml
> +Import metadata from the @uref{https://opam.ocaml.org/, Opam} package
> +repository used by the OCaml community.

Nitpick: the proper spelling is “OPAM” and “OCaml”, so I think you need
to fix these above (in the concept index as well.)

Otherwise LGTM!

As a next step, you can add an OPAM updater, which will hopefully take
you no more than a few lines of code (see the bottom of cpam.scm, for
example.)

:-)

Thank you!

Ludo’.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#31736: [PATCH] Add an opam importer
  2018-07-09 12:44       ` Ludovic Courtès
@ 2018-07-10 11:47         ` Julien Lepiller
  0 siblings, 0 replies; 6+ messages in thread
From: Julien Lepiller @ 2018-07-10 11:47 UTC (permalink / raw)
  To: 31736-done

Le Mon, 09 Jul 2018 14:44:20 +0200,
ludo@gnu.org (Ludovic Courtès) a écrit :

> Hello Julien,
> 
> Julien Lepiller <julien@lepiller.eu> skribis:
> 
> > Here is a new version. What do you think?  
> 
> I think it looks great.  :-)
> 
> > From b3fd75a804a4264cb083584591a3e589886d96c8 Mon Sep 17 00:00:00
> > 2001 From: Julien Lepiller <julien@lepiller.eu>
> > Date: Wed, 6 Jun 2018 19:14:39 +0200
> > Subject: [PATCH] guix: Add opam importer.
> >
> > * guix/scripts/import.scm (importers): Add opam.
> > * guix/scripts/import/opam.scm: New file.
> > * guix/import/opam.scm: New file.
> > * tests/opam.scm: New file.
> > * Makefile.am: Add them.
> > * doc/guix.texi (Invoking guix import): Document it.  
> 
> [...]
> 
> > +@item opam
> > +@cindex opam
> > +@cindex ocaml
> > +Import metadata from the @uref{https://opam.ocaml.org/, Opam}
> > package +repository used by the OCaml community.  
> 
> Nitpick: the proper spelling is “OPAM” and “OCaml”, so I think you
> need to fix these above (in the concept index as well.)
> 
> Otherwise LGTM!
> 
> As a next step, you can add an OPAM updater, which will hopefully take
> you no more than a few lines of code (see the bottom of cpam.scm, for
> example.)
> 
> :-)
> 
> Thank you!
> 
> Ludo’.

Pushed as b24443bff9f9f3f36353eea2ef35e6dc3745a417

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2018-07-10 11:49 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-06-06 17:23 [bug#31736] [PATCH] Add an opam importer Julien Lepiller
2018-06-06 17:37 ` Julien Lepiller
2018-06-10 21:28   ` Ludovic Courtès
2018-07-07 21:56     ` Julien Lepiller
2018-07-09 12:44       ` Ludovic Courtès
2018-07-10 11:47         ` bug#31736: " Julien Lepiller

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).