bug#27590: artanis has a bundled and customized copy of guile-json package

bug#27590: artanis has a bundled and customized copy of guile-json package

[Adonay Felipe Nogueira]

Commit 71b0418843ef0aa8de97734851879a0479c81f69 in the Git repository of
GNU Artanis introduces this issue.

The affected files are at "artanis/third-party/json/upstream". There is
another file at "artanis/third-party/json.scm", but I don't know if it
is from guile-json.

The file at "artanis/third-party/json/upstream/README" says that
upstream is a Git repository in GitHub (ugh), but I found out that our
guile-json package comes from a Git repository in GNU Savannah that is
the same as the one in GitHub (and this one actually has download links
pointing to GNU Savannah)... what a duplicated work. ;)

So far I have attempted:

- Comparison between GNU Artanis'
  "artanis/third-party/json/upstream/builder.scm" (from commit
  71b0418843ef0aa8de97734851879a0479c81f69) with guile-json's
  "json/builder.scm". After each comparison, the local copy of
  guile-json repository was checked-out for a commit prior to the
  current. This was repeated until an exact match was found, and until
  `git checkout` exited with an error.

- Comparison between GNU Artanis'
  "artanis/third-party/json/upstream/parser.scm" (id.) with guile-json's
  "json/parser.scm". After each comparison, the local copy of guile-json
  repository was checked-out for a commit prior to the current. This was
  repeated until an exact match was found, and until `git checkout`
  exited with an error.

However, none of these attempts resulted in exact matches (so that the
two bundled files could be compared). This seems to tell me that commit
71b0418843ef0aa8de97734851879a0479c81f69 already introduced customized
bundles.

For the sake of transparency, here is the GNU Bash script I used for the
comparison attempts:

# "${HOME}/Projetos/Software/guile-json" is my local copy of guile-json
# repository.
cd "${HOME}/Projetos/Software/guile-json"
for each_file in "builder.scm" "parser.scm"; do
  until diff -qwB -I '^[[:space:]]*;' -I 'artanis[/[:space:]]{1,}third-party' "json/${each_file}" "../artanis/artanis/third-party/json/upstream/${each_file}";  do
    git checkout HEAD~1
    if [ $? -ne 0 ]; then
      break
    fi
  done
  git fetch
  git update-ref "remotes/origin/master" "master"
  git checkout "master"
done

-- 
- [[https://libreplanet.org/wiki/User:Adfeno]]
- Palestrante e consultor sobre /software/ livre (não confundir com
  gratis).
- "WhatsApp"? Ele não é livre, por isso não uso. Iguais a ele prefiro
  GNU Ring, ou Tox. Quer outras formas de contato? Adicione o vCard
  que está no endereço acima aos teus contatos.
- Pretende me enviar arquivos .doc, .ppt, .cdr, ou .mp3? OK, eu
  aceito, mas não repasso. Entrego apenas em formatos favoráveis ao
  /software/ livre. Favor entrar em contato em caso de dúvida.

bug#27590: artanis has a bundled and customized copy of guile-json package

[Ludovic Courtès]

Hello,

Adonay Felipe Nogueira <adfeno@openmailbox.org> skribis:

> So far I have attempted:
>
> - Comparison between GNU Artanis'
>   "artanis/third-party/json/upstream/builder.scm" (from commit
>   71b0418843ef0aa8de97734851879a0479c81f69) with guile-json's
>   "json/builder.scm". After each comparison, the local copy of
>   guile-json repository was checked-out for a commit prior to the
>   current. This was repeated until an exact match was found, and until
>   `git checkout` exited with an error.
>
> - Comparison between GNU Artanis'
>   "artanis/third-party/json/upstream/parser.scm" (id.) with guile-json's
>   "json/parser.scm". After each comparison, the local copy of guile-json
>   repository was checked-out for a commit prior to the current. This was
>   repeated until an exact match was found, and until `git checkout`
>   exited with an error.
>
> However, none of these attempts resulted in exact matches (so that the
> two bundled files could be compared). This seems to tell me that commit
> 71b0418843ef0aa8de97734851879a0479c81f69 already introduced customized
> bundles.

Good catch.  Did you try removing the bundled Guile-JSON and adding
“ours” as an input?

Perhaps you could take it to upstream (Nala Ginrut), who I’m sure would
be happy to discuss this.

Thank you,
Ludo’.

bug#27590: artanis has a bundled and customized copy of guile-json package

[Adonay Felipe Nogueira]

I have asked for more information on the artanis mailing list, see
[[https://lists.gnu.org/archive/html/artanis/2017-07/msg00000.html]].

bug#27590: artanis has a bundled and customized copy of guile-json package

[Adonay Felipe Nogueira]

I have compared the two. Note however that I didn't test them, I only
read the diffs, so my notes might not accurately reflect nested/complex
cases.

Discarding the new features implemented in guile-json since the bundled
customization and the deprecation of (json ...) macro, it seems that
"json/builder.scm" is the file where most important changes are.

Considering upstream guile-json:

- (json-build-object ...) considers the given Scheme "native" directly as
  a list of pairs (that is: no conversion is done inside the procedure
  itself).

- (json-build ...) has conditionals for checking if a given Scheme
  "native" is a symbol, and also for checking if it's an association
  list.

- Inside (json-build ...), the conditional that checks if the givem
  Scheme object is a hash table also builds JSON documents for those
  cases, but instead of taking the Scheme "native" as it is, it will
  receive such converted to a list of pairs.

Considering the bundled and customized guile-json:

- (json-build-object ...) calls procedures to convert the given Scheme
  "native" to a list of pairs (that is: there is always a conversion made
  inside the procedure).

- (json-build ...) has no conditionals to check for symbols and
  association lists specially.

- Inside (json-build ...), the conditional that checks if the givem
  Scheme object is a hash table takes the Scheme "native" as it is.

This indicates that upstream guile-json seems to have more control on
what to do with each thing. However, I don't know if this really impacts
GNU Artanis such that there is an unexpected behavior. From the looks of
it, there might be no problem on removing the bundle. Of course, a test
is preferable.

bug#27590: artanis has a bundled and customized copy of guile-json package

[Adonay Felipe Nogueira]

[-- Attachment #1: Type: text/plain, Size: 368 bytes --]

The following patch is an attempt to fix this.

I noticed that the '->json-string procedure doesn't exist in the
official guile-json. I tried checking for it also in multiple revisions,
but couldn't find it. Due to my lack of understanding on how both GNU
Artanis and guile-json work internally, I assume however that
'->json-string was similar to 'scm->json-string.


[-- Attachment #2: gnu packages guile (artanis).patch --]
[-- Type: text/x-diff, Size: 8927 bytes --]

From 8ec0eaaa0e552e7d99be369a4251b2577ac927ac Mon Sep 17 00:00:00 2001
From: Adonay Felipe Nogueira <adfeno@openmailbox.org>
Date: Fri, 28 Jul 2017 11:18:03 -0300
Subject: [PATCH] gnu: artanis: Unbundled guile-json and version according to
 documentation.

* gnu/packages/guile.scm (artanis): Unbundled guile-json and version
according to documentation.
---
 gnu/packages/guile.scm | 129 +++++++++++++++++++++++++++++--------------------
 1 file changed, 77 insertions(+), 52 deletions(-)

diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm
index f1ffc0e..3bcfbc0 100644
--- a/gnu/packages/guile.scm
+++ b/gnu/packages/guile.scm
@@ -7,7 +7,7 @@
 ;;; Copyright © 2016 Erik Edrosa <erik.edrosa@gmail.com>
 ;;; Copyright © 2016 Eraim Flashner <efraim@flashner.co.il>
 ;;; Copyright © 2016, 2017 Alex Kost <alezost@gmail.com>
-;;; Copyright © 2016 Adonay "adfeno" Felipe Nogueira <https://libreplanet.org/wiki/User:Adfeno> <adfeno@openmailbox.org>
+;;; Copyright © 2016, 2017 Adonay "adfeno" Felipe Nogueira <https://libreplanet.org/wiki/User:Adfeno> <adfeno@openmailbox.org>
 ;;; Copyright © 2016 Amirouche <amirouche@hypermove.net>
 ;;; Copyright © 2016 Jan Nieuwenhuizen <janneke@gnu.org>
 ;;; Copyright © 2017 Andy Wingo <wingo@igalia.com>
@@ -67,6 +67,7 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build utils)
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system trivial)
   #:use-module (guix utils)
@@ -384,55 +385,79 @@ program can be installed in one go.")
 ;;;
 
 (define-public artanis
-  (package
-    (name "artanis")
-    (version "0.2.1")
-    (source (origin
-              (method url-fetch)
-              (uri (string-append "mirror://gnu/artanis/artanis-"
-                                  version ".tar.gz"))
-              (sha256
-               (base32
-                "041ajcg2pz918kd9iqcj4inpzddc3impvz3r2nhlpbv8zrz011hn"))))
-    (build-system gnu-build-system)
-    ;; TODO: Add guile-dbi and guile-dbd optional dependencies.
-    (inputs `(("guile" ,guile-2.2)))
-    (native-inputs `(("bash"       ,bash)         ;for the `source' builtin
-                     ("pkgconfig"  ,pkg-config)
-                     ("util-linux" ,util-linux))) ;for the `script' command
-    (arguments
-     '(#:make-flags
-       ;; TODO: The documentation must be built with the `docs' target.
-       (let* ((out (assoc-ref %outputs "out"))
-              (scm (string-append out "/share/guile/site/2.2"))
-              (go  (string-append out "/lib/guile/2.2/site-ccache")))
-         ;; Don't use (%site-dir) for site paths.
-         (list (string-append "MOD_PATH=" scm)
-               (string-append "MOD_COMPILED_PATH=" go)))
-       #:test-target "test"
-       #:phases
-       (modify-phases %standard-phases
-         (add-before 'install 'substitute-root-dir
-          (lambda* (#:key outputs #:allow-other-keys)
-            (let ((out  (assoc-ref outputs "out")))
-              (substitute* "Makefile"   ;ignore the execution of bash.bashrc
-                ((" /etc/bash.bashrc") " /dev/null"))
-              (substitute* "Makefile"   ;set the root of config files to OUT
-                ((" /etc") (string-append " " out "/etc")))
-              (mkdir-p (string-append out "/bin")) ;for the `art' executable
-              #t)))
-         (add-after 'install 'wrap-art
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((out (assoc-ref outputs "out"))
-                    (bin (string-append out "/bin"))
-                    (scm (string-append out "/share/guile/site/2.2"))
-                    (go  (string-append out "/lib/guile/2.2/site-ccache")))
-               (wrap-program (string-append bin "/art")
-                 `("GUILE_LOAD_PATH" ":" prefix (,scm))
-                 `("GUILE_LOAD_COMPILED_PATH" ":" prefix (,go)))
-               #t))))))
-    (synopsis "Web application framework written in Guile")
-    (description "GNU Artanis is a web application framework written in Guile
+  (let ((release "0.2.1")
+	(revision 3))
+    (package
+      (name "artanis")
+      (version (if (zero? revision)
+                   release
+                   (string-append release "-"
+                                  (number->string revision))))
+      (source (origin
+                (method url-fetch)
+                (uri (string-append "mirror://gnu/artanis/artanis-"
+                                    release ".tar.gz"))
+                (file-name (string-append name "-" version ".tar.gz"))
+                (sha256
+                 (base32
+                  "041ajcg2pz918kd9iqcj4inpzddc3impvz3r2nhlpbv8zrz011hn"))
+                (modules '((guix build utils)))
+                (snippet
+                 '(begin
+                    (delete-file-recursively "artanis/third-party/json.scm")
+                    (delete-file-recursively "artanis/third-party/json")
+                    (substitute* '("artanis/artanis.scm"
+                                   "artanis/oht.scm")
+                      (("(#:use-module \\()artanis third-party (json\\))" _ use-module json)
+                       (string-append use-module json)))
+                    (substitute* "artanis/oht.scm"
+                      (("([[:punct:][:space:]]+)(->json-string)([[:punct:][:space:]]+)" _ pre json-string post)
+                       (string-append pre
+                                      "scm" json-string
+                                      post)))
+                    (substitute* "artanis/artanis.scm"
+                      (("[[:punct:][:space:]]+->json-string[[:punct:][:space:]]+")
+                       ""))))))
+      (build-system gnu-build-system)
+      ;; TODO: Add guile-dbi and guile-dbd optional dependencies.
+      (inputs `(("guile" ,guile-2.2)
+                ("guile-json" ,guile-json)))
+      (native-inputs `(("bash"       ,bash)         ;for the `source' builtin
+                       ("pkgconfig"  ,pkg-config)
+                       ("util-linux" ,util-linux))) ;for the `script' command
+      (arguments
+       '(#:make-flags
+         ;; TODO: The documentation must be built with the `docs' target.
+         (let* ((out (assoc-ref %outputs "out"))
+                (scm (string-append out "/share/guile/site/2.2"))
+                (go  (string-append out "/lib/guile/2.2/site-ccache")))
+           ;; Don't use (%site-dir) for site paths.
+           (list (string-append "MOD_PATH=" scm)
+                 (string-append "MOD_COMPILED_PATH=" go)))
+         #:test-target "test"
+         #:phases
+         (modify-phases %standard-phases
+           (add-before 'install 'substitute-root-dir
+             (lambda* (#:key outputs #:allow-other-keys)
+               (let ((out  (assoc-ref outputs "out")))
+                 (substitute* "Makefile"   ;ignore the execution of bash.bashrc
+                   ((" /etc/bash.bashrc") " /dev/null"))
+                 (substitute* "Makefile"   ;set the root of config files to OUT
+                   ((" /etc") (string-append " " out "/etc")))
+                 (mkdir-p (string-append out "/bin")) ;for the `art' executable
+                 #t)))
+           (add-after 'install 'wrap-art
+             (lambda* (#:key outputs #:allow-other-keys)
+               (let* ((out (assoc-ref outputs "out"))
+                      (bin (string-append out "/bin"))
+                      (scm (string-append out "/share/guile/site/2.2"))
+                      (go  (string-append out "/lib/guile/2.2/site-ccache")))
+                 (wrap-program (string-append bin "/art")
+                   `("GUILE_LOAD_PATH" ":" prefix (,scm))
+                   `("GUILE_LOAD_COMPILED_PATH" ":" prefix (,go)))
+                 #t))))))
+      (synopsis "Web application framework written in Guile")
+      (description "GNU Artanis is a web application framework written in Guile
 Scheme.  A web application framework (WAF) is a software framework that is
 designed to support the development of dynamic websites, web applications, web
 services and web resources.  The framework aims to alleviate the overhead
@@ -440,8 +465,8 @@ associated with common activities performed in web development.  Artanis
 provides several tools for web development: database access, templating
 frameworks, session management, URL-remapping for RESTful, page caching, and
 more.")
-    (home-page "https://www.gnu.org/software/artanis/")
-    (license (list license:gpl3+ license:lgpl3+)))) ;dual license
+      (home-page "https://www.gnu.org/software/artanis/")
+      (license (list license:gpl3+ license:lgpl3+))))) ;dual license
 
 (define-public guile-reader
   (package
-- 
1.9.1

bug#27590: artanis has a bundled and customized copy of guile-json package

[Adonay Felipe Nogueira]

Also see the reply given by Mx. Ginrut:
[[https://lists.gnu.org/archive/html/artanis/2017-07/msg00001.html]].

bug#27590: artanis has a bundled and customized copy of guile-json package

[Ludovic Courtès]

Hi,

Adonay Felipe Nogueira <adfeno@openmailbox.org> skribis:

> The following patch is an attempt to fix this.
>
> I noticed that the '->json-string procedure doesn't exist in the
> official guile-json. I tried checking for it also in multiple revisions,
> but couldn't find it. Due to my lack of understanding on how both GNU
> Artanis and guile-json work internally, I assume however that
> '->json-string was similar to 'scm->json-string.

It seems to be the case.

> From 8ec0eaaa0e552e7d99be369a4251b2577ac927ac Mon Sep 17 00:00:00 2001
> From: Adonay Felipe Nogueira <adfeno@openmailbox.org>
> Date: Fri, 28 Jul 2017 11:18:03 -0300
> Subject: [PATCH] gnu: artanis: Unbundled guile-json and version according to
>  documentation.
>
> * gnu/packages/guile.scm (artanis): Unbundled guile-json and version
> according to documentation.

Awesome.  Applied, thank you!

Ludo’.