From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([209.51.188.92]:45607) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gxBhI-0004ZY-EE for guix-patches@gnu.org; Fri, 22 Feb 2019 09:21:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gxBhG-0008Oh-Pp for guix-patches@gnu.org; Fri, 22 Feb 2019 09:21:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:34156) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gxBhG-0008NV-G9 for guix-patches@gnu.org; Fri, 22 Feb 2019 09:21:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gxBhG-0005NY-7a for guix-patches@gnu.org; Fri, 22 Feb 2019 09:21:02 -0500 Subject: [bug#34548] [PATCH 06/24] gnu: Add r-leaflet. Resent-Message-ID: References: <87a7isv3u9.fsf@posteo.net> <87pnroudwk.fsf@elephly.net> <87va1faatv.fsf@posteo.net> <5AB7CFA8-FD34-48D3-8392-9102042F3596@riseup.net> <87d0nknbl1.fsf@posteo.net> From: swedebugia Message-ID: Date: Fri, 22 Feb 2019 10:03:23 +0100 MIME-Version: 1.0 In-Reply-To: <87d0nknbl1.fsf@posteo.net> Content-Type: multipart/alternative; boundary="------------FBAD7EC7CC62F930C0BE2730" Content-Language: en-US List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: Brett Gilio Cc: 34548@debbugs.gnu.org This is a multi-part message in MIME format. --------------FBAD7EC7CC62F930C0BE2730 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 02/21/2019 09:14 PM, Brett Gilio wrote: > swedebugia writes: > >> Brett Gilio skrev: (19 februari 2019 19:32:12 CET) snip >>> However, on that note, do we >>> have >>> any progress on an npm importer? I know that came up awhile back. >> Julien is sitting on the patchset right now since a few weeks. >> >> It still has some defects still but is mostly working. >> >> The problem of circular dependencies is still unsolved. I suggest we add to our importer the ability to parse and select versions recursively. I tried but failed to get it to work before I gave up. > Hey, where is the patchset located? I wouldn't mind taking a look. Here it is: https://framagit.org/tyreunom/guix/tree/npm Julien integrated my changes (which consisted of changing to use the npm-packages instead of github sources) on top of Jelles earlier work and made some changes after that (fixing the recursive importer :D) and rebased the whole thing. Most smaller packages include everything in the npm-packages but when reviewing we should check that we are actually building from source and not just installing some precompiles js-files (which you can imagine will take a long time). Fortunately it is "only" 1500-2000 packages i a couple of different versions that are used in most projects that I have explored with the very nice tool npm-explorer (see https://gitlab.com/swedebugia/guile-npm-explorer). We essentially need to write our own npm dependency resolver to avoid circular dependencies. What in my view is currently missing to decide which version of dependencies to import is a semver parser. I took a stab at that, see below[1], but it is messy and buggy and should probably be implemented with PEG[2]. Now that I recently succeded with PEG in my unpublished WIP quicklisp importer I could give it another try with PEG. Have fun with it! |;; FIXME consider even the patch versions. ;; See https://stackoverflow.com/questions/22343224/whats-the-difference-between-tilde-and-caret-in-package-json (define (parse-semver hashtable version) "return the newest version within the same major or minor version" (define (split list) (string-split list #\.)) (define (version-list hashtable) (map split (map first (hash-table->alist (hash-ref hashtable "versions"))))) (define (major list) (first list)) (define (minor list) (second list)) (define (minor->number list) (string->number (minor (split list)))) ;; Return latest minor with same major version. ;; e.g. ^1.1.0 -> 1.4.0 even though 2.0.0 is availiable (let* ((version (split (string-drop version 1))) (version-list (map first (hash-table->alist (hash-ref hashtable "versions")))) (same-major (if (equal? 3 (length version)) (fold ;; recurse through version-list (lambda (ver lst) (if (string-prefix? (major version) ver) (cons ver lst) lst)) '() version-list) ;; not a version triplet #f))) ;; From ;; https://www.gnu.org/software/guile/manual/html_node/SRFI_002d1-Fold-and-Map.html#SRFI_002d1-Fold-and-Map (fold-right (lambda (str prev) (if (> (minor->number str) (minor->number prev)) str prev)) ;;init with 0.0.0 work with minor->number "0.0.0" same-major))) [1] https://gitlab.com/swedebugia/guile-npm-explorer/blob/wip-versioning-parse/npm-explorer.scm#L148 [2] https://www.gnu.org/software/guile/manual/html_node/PEG-Tutorial.html#PEG-Tutorial | --------------FBAD7EC7CC62F930C0BE2730 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit
On 02/21/2019 09:14 PM, Brett Gilio wrote:
swedebugia writes:


Brett Gilio <brettg@posteo.net> skrev: (19 februari 2019 19:32:12 CET)
snip
 However, on that note, do we
have
any progress on an npm importer? I know that came up awhile back.

Julien is sitting on the patchset right now since a few weeks. 

It still has some defects still but is mostly working.

The problem of circular dependencies is still unsolved. I suggest we add to our importer the ability to parse and select versions recursively. I tried but failed to get it to work before I gave up.

Hey, where is the patchset located? I wouldn't mind taking a look.

Here it is: https://framagit.org/tyreunom/guix/tree/npm

Julien integrated my changes (which consisted of changing to use the npm-packages instead of github sources) on top of Jelles earlier work and made some changes after that (fixing the recursive importer :D) and rebased the whole thing.

Most smaller packages include everything in the npm-packages but when reviewing we should check that we are actually building from source and not just installing some precompiles js-files (which you can imagine will take a long time).

Fortunately it is "only" 1500-2000 packages i a couple of different versions that are used in most projects that I have explored with the very nice tool npm-explorer (see https://gitlab.com/swedebugia/guile-npm-explorer).

We essentially need to write our own npm dependency resolver to avoid circular dependencies.
What in my view is currently missing to decide which version of dependencies to import is a semver parser. I took a stab at that, see below[1], but it is messy and buggy and should probably be implemented with PEG[2].

Now that I recently succeded with PEG in my unpublished WIP quicklisp importer I could give it another try with PEG.

Have fun with it! 

;; FIXME consider even the patch versions.
;; See https://stackoverflow.com/questions/22343224/whats-the-difference-between-tilde-and-caret-in-package-json
(define (parse-semver hashtable version)
  "return the newest version within the same major or minor version"
  (define (split list)
    (string-split list #\.))
  (define (version-list hashtable)
    (map split
	 (map first
	      (hash-table->alist (hash-ref hashtable "versions")))))
  (define (major list)
    (first list))
  (define (minor list)
    (second list))
  (define (minor->number list)
    (string->number (minor (split list))))
  ;; Return latest minor with same major version.
  ;; e.g. ^1.1.0 -> 1.4.0 even though 2.0.0 is availiable
  (let* ((version (split (string-drop version 1)))
	 (version-list
	  (map first
	       (hash-table->alist (hash-ref hashtable "versions"))))
	 (same-major
	  (if (equal? 3 (length version))
	      (fold
	       ;; recurse through version-list
	       (lambda (ver lst)
		 (if (string-prefix? (major version) ver)
		     (cons ver lst)
		     lst))
	       '()
	       version-list)
	      ;; not a version triplet
	      #f)))
    ;; From
    ;; https://www.gnu.org/software/guile/manual/html_node/SRFI_002d1-Fold-and-Map.html#SRFI_002d1-Fold-and-Map
    (fold-right
     (lambda (str prev)
       (if (> (minor->number str) (minor->number prev))
      	   str
      	   prev))
     ;;init with 0.0.0 work with minor->number
     "0.0.0"
     same-major)))

[1] https://gitlab.com/swedebugia/guile-npm-explorer/blob/wip-versioning-parse/npm-explorer.scm#L148
[2] https://www.gnu.org/software/guile/manual/html_node/PEG-Tutorial.html#PEG-Tutorial
--------------FBAD7EC7CC62F930C0BE2730--