LZ4 + LZO python bindings

LZ4 + LZO python bindings

[contact.ng0]

This adds python-lz4 and python-lzo, received some QA/review back in september. I rebased and applied changes for the current (new) python build-system.

[PATCH 1/2] gnu: Add python-lz4.

[contact.ng0]

From: ng0 <ngillmann@runbox.com>

* gnu/packages/compression.scm (python-lz4): New variable.
---
 gnu/packages/compression.scm | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index 73a35030e..64518fb6b 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -14,6 +14,7 @@
 ;;; Copyright © 2016 David Craven <david@craven.ch>
 ;;; Copyright © 2016 Kei Kebreau <kei@openmailbox.org>
 ;;; Copyright © 2016 Marius Bakke <mbakke@fastmail.com>
+;;; Copyright © 2017 ng0 <contact.ng0@cryptolab.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -38,6 +39,7 @@
   #:use-module (guix git-download)
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system perl)
+  #:use-module (guix build-system python)
   #:use-module (gnu packages)
   #:use-module (gnu packages assembly)
   #:use-module (gnu packages autotools)
@@ -45,6 +47,7 @@
   #:use-module (gnu packages base)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages pkg-config)
+  #:use-module (gnu packages python)
   #:use-module (gnu packages valgrind)
   #:use-module (ice-9 match)
   #:use-module ((srfi srfi-1) #:select (last)))
@@ -633,6 +636,31 @@ time for compression ratio.")
     ;; line interface programs (lz4, fullbench, fuzzer, datagen) are GPL2+.
     (license (list license:bsd-2 license:gpl2+))))
 
+(define-public python-lz4
+  (package
+    (name "python-lz4")
+    (version "0.8.2")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "lz4" version))
+       (sha256
+        (base32
+         "1irad4sq4hdr30fr53smvv3zzk4rddcf9b4jx19w8s9xsxhr1x3b"))))
+    (build-system python-build-system)
+    (native-inputs
+     `(("python-nose" ,python-nose)))
+    (home-page "https://github.com/python-lz4/python-lz4")
+    (synopsis "LZ4 Bindings for Python")
+    (description
+     "This package provides python bindings for the lz4 compression library
+by Yann Collet.  The project contains bindings for the LZ4 block format and
+the LZ4 frame format.")
+    (license license:bsd-3)))
+
+(define-public python2-lz4
+  (package-with-python2 python-lz4))
+
 (define-public squashfs-tools
   (package
     (name "squashfs-tools")
-- 
2.11.0

[PATCH 2/2] gnu: Add python-lzo.

[contact.ng0]

From: ng0 <ngillmann@runbox.com>

* gnu/packages/compression.scm (python-lzo): New variable.

Co-authored-by: Danny Milosavljevic <dannym@scratchpost.org>
---
 gnu/packages/compression.scm | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index 64518fb6b..2545c4517 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -365,6 +365,42 @@ LZO is written in ANSI C.  Both the source code and the compressed data
 format are designed to be portable across platforms.")
     (license license:gpl2+)))
 
+(define-public python-lzo
+  (package
+    (name "python-lzo")
+    (version "1.11")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "python-lzo" version))
+       (sha256
+        (base32
+         "11p3ifg14p086byhhin6azx5svlkg8dzw2b5abixik97xd6fm81q"))))
+    (build-system python-build-system)
+    (arguments
+     `(#:test-target "check"
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'patch-setuppy
+           (lambda _
+             (substitute* "setup.py"
+               (("include_dirs.append.*")
+                (string-append "include_dirs.append(\""
+                               (assoc-ref %build-inputs "lzo") "/include/lzo" "\")
+"))))))))
+    (inputs
+     `(("lzo" ,lzo)))
+    (home-page "https://github.com/jd-boyd/python-lzo")
+    (synopsis "Python bindings for the LZO data compression library")
+    (description
+     "Python-LZO provides Python bindings for LZO, i.e. you can access
+the LZO library from your Python scripts thereby compressing ordinary
+Python strings.")
+    (license license:gpl2+)))
+
+(define-public python2-lzo
+  (package-with-python2 python-lzo))
+
 (define-public lzop
   (package
     (name "lzop")
-- 
2.11.0

Re: [PATCH 2/2] gnu: Add python-lzo.

[Hartmut Goebel]

[-- Attachment #1: Type: text/plain, Size: 1072 bytes --]

Am 02.02.2017 um 16:16 schrieb contact.ng0@cryptolab.net:
> +                (string-append "include_dirs.append(\""
> +                               (assoc-ref %build-inputs "lzo") "/include/lzo" "\")

You could use single quotes for teh python string to make it more readable:


string-append "include_dirs.append('" + (assoc-ref %build-inputs "lzo")
"/include/lzo" "')


> +"))))))))

Also this line-break is confusing. I suggest either using "\n" or to
change the substitute into something like:


+             (substitute* "setup.py"
+               (("include_dirs.append\(.*\)")  ;; match up to the trailing parent only, not the new-line
+                (string-append "include_dirs.append('"
+                               (assoc-ref %build-inputs "lzo") "/include/lzo"
+                               "')"))))))))   ;; put these last few characters on the next line

Otherwise LGTM.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |


[-- Attachment #2: 0xBF773B65.asc --]
[-- Type: application/pgp-keys, Size: 14855 bytes --]

Re: [PATCH 1/2] gnu: Add python-lz4.

[Hartmut Goebel]

[-- Attachment #1: Type: text/plain, Size: 301 bytes --]

Am 02.02.2017 um 16:16 schrieb contact.ng0@cryptolab.net:
> * gnu/packages/compression.scm (python-lz4): New variable.

LGTM

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |


[-- Attachment #2: 0xBF773B65.asc --]
[-- Type: application/pgp-keys, Size: 14855 bytes --]

Re: [PATCH 2/2] gnu: Add python-lzo.

[ng0]

Hartmut Goebel writes:

> Am 02.02.2017 um 16:16 schrieb contact.ng0@cryptolab.net:
>> +                (string-append "include_dirs.append(\""
>> +                               (assoc-ref %build-inputs "lzo") "/include/lzo" "\")
>
> You could use single quotes for teh python string to make it more readable:

I think there was a reason for the way it was written. Your
suggestion below makes the substitute fail with:

ERROR: In procedure primitive-load-path:
ERROR: In procedure scm_lreadr: gnu/packages/compression.scm:387:44: illegal character in escape sequence: #\)

>
> string-append "include_dirs.append('" + (assoc-ref %build-inputs "lzo")
> "/include/lzo" "')
>
>
>> +"))))))))
>
> Also this line-break is confusing. I suggest either using "\n" or to
> change the substitute into something like:
>
>
> +             (substitute* "setup.py"
> +               (("include_dirs.append\(.*\)")  ;; match up to the trailing parent only, not the new-line
> +                (string-append "include_dirs.append('"
> +                               (assoc-ref %build-inputs "lzo") "/include/lzo"
> +                               "')"))))))))   ;; put these last few characters on the next line
>
> Otherwise LGTM.


-- 
ng0 . https://www.inventati.org/patternsinthechaos/

Re: [PATCH 2/2] gnu: Add python-lzo.

[Hartmut Goebel]

Am 03.02.2017 um 14:13 schrieb ng0:
> ERROR: In procedure scm_lreadr: gnu/packages/compression.scm:387:44: illegal character in escape sequence: #\)

IC, yes, this needs to be written as \\) (two backslashes)

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: [PATCH 2/2] gnu: Add python-lzo.

[ng0]

Hartmut Goebel writes:

> Am 03.02.2017 um 14:13 schrieb ng0:
>> ERROR: In procedure scm_lreadr: gnu/packages/compression.scm:387:44: illegal character in escape sequence: #\)
>
> IC, yes, this needs to be written as \\) (two backslashes)

I see. Okay, do you know which guix or guile module I do have to
rtf to understand once and for all how the (substitute*) behaves?
I'm doing this for too long to continue to run into problems with
this.

Just the (substitute*)? Or is there some complimentary literature
I should consider (something in guile, grep, sed, …)?
-- 
ng0 . https://www.inventati.org/patternsinthechaos/

python-{lzo,lz4} ,v3

[contact.ng0]

Applied the fixes as discussed in the previous email.
I'm not sure which parts Danny contributed back then, so I leave the name in co-authored-by.

[PATCH 1/2] gnu: Add python-lz4.

[contact.ng0]

From: ng0 <ngillmann@runbox.com>

* gnu/packages/compression.scm (python-lz4): New variable.
---
 gnu/packages/compression.scm | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index 73a35030e..64518fb6b 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -14,6 +14,7 @@
 ;;; Copyright © 2016 David Craven <david@craven.ch>
 ;;; Copyright © 2016 Kei Kebreau <kei@openmailbox.org>
 ;;; Copyright © 2016 Marius Bakke <mbakke@fastmail.com>
+;;; Copyright © 2017 ng0 <contact.ng0@cryptolab.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -38,6 +39,7 @@
   #:use-module (guix git-download)
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system perl)
+  #:use-module (guix build-system python)
   #:use-module (gnu packages)
   #:use-module (gnu packages assembly)
   #:use-module (gnu packages autotools)
@@ -45,6 +47,7 @@
   #:use-module (gnu packages base)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages pkg-config)
+  #:use-module (gnu packages python)
   #:use-module (gnu packages valgrind)
   #:use-module (ice-9 match)
   #:use-module ((srfi srfi-1) #:select (last)))
@@ -633,6 +636,31 @@ time for compression ratio.")
     ;; line interface programs (lz4, fullbench, fuzzer, datagen) are GPL2+.
     (license (list license:bsd-2 license:gpl2+))))
 
+(define-public python-lz4
+  (package
+    (name "python-lz4")
+    (version "0.8.2")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "lz4" version))
+       (sha256
+        (base32
+         "1irad4sq4hdr30fr53smvv3zzk4rddcf9b4jx19w8s9xsxhr1x3b"))))
+    (build-system python-build-system)
+    (native-inputs
+     `(("python-nose" ,python-nose)))
+    (home-page "https://github.com/python-lz4/python-lz4")
+    (synopsis "LZ4 Bindings for Python")
+    (description
+     "This package provides python bindings for the lz4 compression library
+by Yann Collet.  The project contains bindings for the LZ4 block format and
+the LZ4 frame format.")
+    (license license:bsd-3)))
+
+(define-public python2-lz4
+  (package-with-python2 python-lz4))
+
 (define-public squashfs-tools
   (package
     (name "squashfs-tools")
-- 
2.11.0

[PATCH 2/2] gnu: Add python-lzo.

[contact.ng0]

From: ng0 <ngillmann@runbox.com>

* gnu/packages/compression.scm (python-lzo): New variable.

Co-authored-by: Danny Milosavljevic <dannym@scratchpost.org>
---
 gnu/packages/compression.scm | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index 64518fb6b..3c38782df 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -365,6 +365,44 @@ LZO is written in ANSI C.  Both the source code and the compressed data
 format are designed to be portable across platforms.")
     (license license:gpl2+)))
 
+(define-public python-lzo
+  (package
+    (name "python-lzo")
+    (version "1.11")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "python-lzo" version))
+       (sha256
+        (base32
+         "11p3ifg14p086byhhin6azx5svlkg8dzw2b5abixik97xd6fm81q"))))
+    (build-system python-build-system)
+    (arguments
+     `(#:test-target "check"
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'patch-setuppy
+           (lambda _
+             (substitute* "setup.py"
+               (("include_dirs.append\\(.*\\)")
+                (string-append "include_dirs.append('"
+                               (assoc-ref %build-inputs "lzo")
+                               "/include/lzo"
+                               "')")))
+             #t)))))
+    (inputs
+     `(("lzo" ,lzo)))
+    (home-page "https://github.com/jd-boyd/python-lzo")
+    (synopsis "Python bindings for the LZO data compression library")
+    (description
+     "Python-LZO provides Python bindings for LZO, i.e. you can access
+the LZO library from your Python scripts thereby compressing ordinary
+Python strings.")
+    (license license:gpl2+)))
+
+(define-public python2-lzo
+  (package-with-python2 python-lzo))
+
 (define-public lzop
   (package
     (name "lzop")
-- 
2.11.0

substitute/regex (was [PATCH 2/2] gnu: Add python-lzo.)

[Hartmut Goebel]

Am 03.02.2017 um 17:36 schrieb ng0:
> I see. Okay, do you know which guix or guile module I do have to
> rtf to understand once and for all how the (substitute*) behaves?
> I'm doing this for too long to continue to run into problems with
> this.
>
> Just the (substitute*)? Or is there some complimentary literature
> I should consider (something in guile, grep, sed, …)?

I can't remember,

But it's two different things:

1) How to properly write strings in guile. For this I assume [1] is a
good guide (I did not read it yet). For regexp it comes down that you
need to escapw any backslash you need in the regex with another
backslash for passing guilse parser.

2) How to use substitute* properly. I'm not an expert on this, but
substitute* is defined in guix/build/utils.scm (grep is your friend :-)
Here I found a nice, but rarely used feature: Matching groups may be
assigned to variables like this:

  (substitute* file
     ((\"foo([a-z]+)bar(.*)$\" all letters end)
      (string-append \"baz\" letter end)))

Here ALL is bound to
the complete match, LETTERS is bound to the first sub-expression, and END is
bound to the last one.


[1]
https://www.gnu.org/software/guile/manual/html_node/Backslash-Escapes.html

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: python-{lzo,lz4} ,v3

[Hartmut Goebel]

[-- Attachment #1: Type: text/plain, Size: 447 bytes --]

Am 03.02.2017 um 17:56 schrieb contact.ng0@cryptolab.net:
> Applied the fixes as discussed in the previous email.
> I'm not sure which parts Danny contributed back then, so I leave the name in co-authored-by.
>
>
Applied, 1a0a5509dd169cc87c611c6c5fe38c787d1afe89
Thanks.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |


[-- Attachment #2: 0xBF773B65.asc --]
[-- Type: application/pgp-keys, Size: 14855 bytes --]

Re: [PATCH 2/2] gnu: Add python-lzo.

[Maxim Cournoyer]

[-- Attachment #1: Type: text/plain, Size: 2381 bytes --]

Hi ng0,

contact.ng0@cryptolab.net writes:

> From: ng0 <ngillmann@runbox.com>
>
> * gnu/packages/compression.scm (python-lzo): New variable.
>
> Co-authored-by: Danny Milosavljevic <dannym@scratchpost.org>
> ---
>  gnu/packages/compression.scm | 36 ++++++++++++++++++++++++++++++++++++
>  1 file changed, 36 insertions(+)
>
> diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
> index 64518fb6b..2545c4517 100644
> --- a/gnu/packages/compression.scm
> +++ b/gnu/packages/compression.scm
> @@ -365,6 +365,42 @@ LZO is written in ANSI C.  Both the source code and the compressed data
>  format are designed to be portable across platforms.")
>      (license license:gpl2+)))
>  
> +(define-public python-lzo
> +  (package
> +    (name "python-lzo")
> +    (version "1.11")
> +    (source
> +     (origin
> +       (method url-fetch)
> +       (uri (pypi-uri "python-lzo" version))
> +       (sha256
> +        (base32
> +         "11p3ifg14p086byhhin6azx5svlkg8dzw2b5abixik97xd6fm81q"))))
> +    (build-system python-build-system)
> +    (arguments
> +     `(#:test-target "check"
> +       #:phases
> +       (modify-phases %standard-phases
> +         (add-after 'unpack 'patch-setuppy
> +           (lambda _
> +             (substitute* "setup.py"
> +               (("include_dirs.append.*")
> +                (string-append "include_dirs.append(\""
> +                               (assoc-ref %build-inputs "lzo") "/include/lzo" "\")
> +"))))))))

A bit hard to read/long line here. Maybe the various string components
being appended could be aligned below the string-append call? Also, you
could use single quotes for the Python code which might help to
differentiate what is what.

Is the guix linter happy?

> +    (inputs
> +     `(("lzo" ,lzo)))
> +    (home-page "https://github.com/jd-boyd/python-lzo")
> +    (synopsis "Python bindings for the LZO data compression library")
> +    (description
> +     "Python-LZO provides Python bindings for LZO, i.e. you can access
> +the LZO library from your Python scripts thereby compressing ordinary
> +Python strings.")
> +    (license license:gpl2+)))
> +
> +(define-public python2-lzo
> +  (package-with-python2 python-lzo))
> +
>  (define-public lzop
>    (package
>      (name "lzop")

Otherwise these 2 patches LGTM!

Maxim

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: [PATCH 2/2] gnu: Add python-lzo.

[ng0]

On 17-02-05 01:31:18, Maxim Cournoyer wrote:
> Hi ng0,
> 
> contact.ng0@cryptolab.net writes:
> 
> > From: ng0 <ngillmann@runbox.com>
> >
> > * gnu/packages/compression.scm (python-lzo): New variable.
> >
> > Co-authored-by: Danny Milosavljevic <dannym@scratchpost.org>
> > ---
> >  gnu/packages/compression.scm | 36 ++++++++++++++++++++++++++++++++++++
> >  1 file changed, 36 insertions(+)
> >
> > diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
> > index 64518fb6b..2545c4517 100644
> > --- a/gnu/packages/compression.scm
> > +++ b/gnu/packages/compression.scm
> > @@ -365,6 +365,42 @@ LZO is written in ANSI C.  Both the source code and the compressed data
> >  format are designed to be portable across platforms.")
> >      (license license:gpl2+)))
> >  
> > +(define-public python-lzo
> > +  (package
> > +    (name "python-lzo")
> > +    (version "1.11")
> > +    (source
> > +     (origin
> > +       (method url-fetch)
> > +       (uri (pypi-uri "python-lzo" version))
> > +       (sha256
> > +        (base32
> > +         "11p3ifg14p086byhhin6azx5svlkg8dzw2b5abixik97xd6fm81q"))))
> > +    (build-system python-build-system)
> > +    (arguments
> > +     `(#:test-target "check"
> > +       #:phases
> > +       (modify-phases %standard-phases
> > +         (add-after 'unpack 'patch-setuppy
> > +           (lambda _
> > +             (substitute* "setup.py"
> > +               (("include_dirs.append.*")
> > +                (string-append "include_dirs.append(\""
> > +                               (assoc-ref %build-inputs "lzo") "/include/lzo" "\")
> > +"))))))))
> 
> A bit hard to read/long line here. Maybe the various string components
> being appended could be aligned below the string-append call? Also, you
> could use single quotes for the Python code which might help to
> differentiate what is what.

It is already applied. I think "it's hard to read" is more than just a
cosmetic fix, so if you have an idea how to make this more readable,
feel free to send a patch.

> Is the guix linter happy?
> 
> > +    (inputs
> > +     `(("lzo" ,lzo)))
> > +    (home-page "https://github.com/jd-boyd/python-lzo")
> > +    (synopsis "Python bindings for the LZO data compression library")
> > +    (description
> > +     "Python-LZO provides Python bindings for LZO, i.e. you can access
> > +the LZO library from your Python scripts thereby compressing ordinary
> > +Python strings.")
> > +    (license license:gpl2+)))
> > +
> > +(define-public python2-lzo
> > +  (package-with-python2 python-lzo))
> > +
> >  (define-public lzop
> >    (package
> >      (name "lzop")
> 
> Otherwise these 2 patches LGTM!
> 
> Maxim



-- 
ng0 -- https://www.inventati.org/patternsinthechaos/