unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Patches to add s3cmd and python-magic
@ 2016-03-28  3:29 Chris Marusich
  2016-03-28 20:35 ` Danny Milosavljevic
  2016-03-30  8:20 ` Ludovic Courtès
  0 siblings, 2 replies; 12+ messages in thread
From: Chris Marusich @ 2016-03-28  3:29 UTC (permalink / raw)
  To: guix-devel


[-- Attachment #1.1: Type: text/plain, Size: 1679 bytes --]

Hi,

I've packaged python-magic and s3cmd.  The former provides Python
bindings for libmagic.  It's an alternative to python-file, which is an
existing package that also provides Python bindings for libmagic.  The
latter is a command-line tool for Amazon S3 and CloudFront, which
expects python-magic (not python-file) to be available.

Please refer to the attached patches for details.  Guix lint says the
following about python-magic: "the source file name should contain the
package name".  However, I think maybe that's a bug in guix lint, since
the URI clearly contains the name "python-magic".

Just to be safe, I've tested python2-magic and python-magic, and
confirmed that module "magic" was importable from Python 2 (for the
former case) and Python 3 (for the latter case).  I've also confirmed
that python2-s3cmd installs fine and that I can use it for rudimentary
tasks, like uploading and downloading to/from S3.

I wasn't sure about a few things, so if you have feedback, I would love
to hear it:

* Do I need to provide setuptools as a native input, or will it be
  pulled in automatically?

* Does setuptools really need to be a native input, or can it be a
  regular input?  I understand that native inputs are important for
  cross-compiling, but does this apply to a language like Python which
  compiles to bytecode for a virtual machine?

* In the package definition for python2-s3cmd, should python2-magic be a
  propagated input instead of a regular input?  It seems to work as a
  normal input, so I think the answer is "no", but I understand that
  sometimes this matters for python packages.

Thank you,

-- 
Chris

[-- Attachment #1.2: 0001-gnu-add-python-magic.patch --]
[-- Type: text/x-patch, Size: 4625 bytes --]

From d0c858b6b717b837eb2eac95db2fb2f19ca4112a Mon Sep 17 00:00:00 2001
From: Chris Marusich <cmmarusich@gmail.com>
Date: Sun, 27 Mar 2016 19:10:48 -0700
Subject: [PATCH 1/2] gnu: add python-magic

* gnu/packages/python.scm (python-magic, python2-magic): New variables.
---
 gnu/packages/python.scm | 67 +++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 65 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 88aef9d..f2ef901 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -16,7 +16,7 @@
 ;;; Copyright © 2015, 2016 Erik Edrosa <erik.edrosa@gmail.com>
 ;;; Copyright © 2015, 2016 Efraim Flashner <efraim@flashner.co.il>
 ;;; Copyright © 2015 Kyle Meyer <kyle@kyleam.com>
-;;; Copyright © 2015 Chris Marusich <cmmarusich@gmail.com>
+;;; Copyright © 2015, 2016 Chris Marusich <cmmarusich@gmail.com>
 ;;; Copyright © 2016 Danny Milosavljevic <dannym+a@scratchpost.org>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -6056,7 +6056,10 @@ Python's @code{ctypes} foreign function interface (FFI).")
                         #t))))))
     (inputs `(("file" ,file)))
     (self-native-input? #f)
-    (synopsis "Python bindings to the libmagic file type guesser")))
+    (synopsis "Python bindings to the libmagic file type guesser.  Note that
+this module and the python-magic module both provide a \"magic.py\" file;
+these two modules, which are different and were developed separately, both
+serve the same purpose: provide Python bindings for libmagic.")))
 
 (define-public python2-file
   (package-with-python2 python-file))
@@ -8490,3 +8493,63 @@ is made as zipfile like as possible.")
 
 (define-public python2-rarfile
   (package-with-python2 python-rarfile))
+
+(define-public python-magic
+  (package
+    (name "python-magic")
+    (version "0.4.3")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "https://github.com/ahupp/python-magic/archive/"
+                           version ".tar.gz"))
+       (sha256
+        (base32
+         "17bgy92i7sb021f2s4mw1dcvpm6p1mi9jihridwy1pyn8mzvpjgk"))))
+    (build-system python-build-system)
+    (arguments
+     ;; The tests are unreliable, so don't run them.  The tests fail
+     ;; under Python3 because they were written for Python2 and
+     ;; contain import statements that do not work in Python3.  One of
+     ;; the tests fails under Python2 because its assertions are
+     ;; overly stringent; it relies on comparing output strings which
+     ;; are brittle and can change depending on the version of
+     ;; libmagic being used and the system on which the test is
+     ;; running.  In my case, under GuixSD 0.10.0, only one test
+     ;; failed, and it seems to have failed only because the version
+     ;; of libmagic that is packaged in Guix outputs a slightly
+     ;; different (but not wrong) string than the one that the test
+     ;; expected.
+     '(#:tests? #f
+       #:phases (modify-phases %standard-phases
+         ;; Replace a specific method call with a hard-coded
+         ;; path to the necessary libmagic.so file in the
+         ;; store.  If we don't do this, then the method call
+         ;; will fail to find the libmagic.so file, which in
+         ;; turn will cause any application using
+         ;; python-magic to fail.
+         (add-before 'build 'hard-code-path-to-libmagic
+           (lambda* (#:key inputs #:allow-other-keys)
+             (let ((file (assoc-ref inputs "file")))
+               (substitute* "magic.py"
+                 (("ctypes.util.find_library\\('magic'\\)")
+                  (string-append "'" file "/lib/libmagic.so'")))
+           #t))))))
+    (native-inputs
+     `(("python-setuptools" ,python-setuptools)))
+    (inputs
+     ;; python-magic needs to be able to find libmagic.so.
+     `(("file" ,file)))
+    (home-page "https://github.com/ahupp/python-magic")
+    (synopsis "File type identification using libmagic")
+    (description
+     "This module uses ctypes to access the libmagic file type
+identification library.  It makes use of the local magic database and
+supports both textual and MIME-type output.  Note that this module and
+the python-file module both provide a \"magic.py\" file; these two
+modules, which are different and were developed separately, both serve
+the same purpose: to provide Python bindings for libmagic.")
+    (license license:expat)))
+
+(define-public python2-magic
+  (package-with-python2 python-magic))
-- 
2.7.3


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.3: 0002-gnu-add-python2-s3cmd.patch --]
[-- Type: text/x-patch, Size: 2293 bytes --]

From 07c6fdfaab24d95fb6b318f5bd820cbe342ed271 Mon Sep 17 00:00:00 2001
From: Chris Marusich <cmmarusich@gmail.com>
Date: Sun, 27 Mar 2016 19:28:37 -0700
Subject: [PATCH 2/2] gnu: add python2-s3cmd

* gnu/packages/python.scm (python2-s3cmd): New variable.
---
 gnu/packages/python.scm | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index f2ef901..73b674e 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -8553,3 +8553,41 @@ the same purpose: to provide Python bindings for libmagic.")
 
 (define-public python2-magic
   (package-with-python2 python-magic))
+
+(define-public python2-s3cmd
+  (package
+    (name "python2-s3cmd")
+    (version "1.6.1")
+    (source
+      (origin
+        (method url-fetch)
+        (uri (string-append "mirror://sourceforge/s3tools/"
+                            "s3cmd-" version ".tar.gz"))
+        (sha256
+          (base32
+            "0ki1rzhm5icvi9ry5jswi4b22yqwyj0d2wsqsgilwx6qhi7pjxa6"))))
+    (build-system python-build-system)
+    (arguments
+     ;; s3cmd is written for python2 only and contains no tests.
+     `(#:python ,python-2
+       #:tests? #f))
+    (native-inputs
+     `(("python2-setuptools" ,python2-setuptools)))
+    (inputs
+     `(("python2-dateutil" ,python2-dateutil)
+       ;; The python-file package also provides a magic.py module.
+       ;; This is an unfortunate state of affairs; however, s3cmd
+       ;; fails to install if it cannot find specifically the
+       ;; python-magic package.  Thus we include it, instead of using
+       ;; python-file.  Ironically, s3cmd sometimes works better
+       ;; without libmagic bindings at all:
+       ;; https://github.com/s3tools/s3cmd/issues/198
+       ("python2-magic" ,python2-magic)))
+    (home-page "http://s3tools.org/s3cmd")
+    (synopsis "Command line S3 Client and backup")
+    (description
+     "S3cmd lets you copy files from/to Amazon S3 (Simple Storage
+Service) using a simple to use command line client.  Supports
+rsync-like backup, GPG encryption, and more.  Also supports management
+of Amazon's CloudFront content delivery network.")
+    (license gpl2+)))
-- 
2.7.3


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

^ permalink raw reply related	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2016-03-30 20:58 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-03-28  3:29 Patches to add s3cmd and python-magic Chris Marusich
2016-03-28 20:35 ` Danny Milosavljevic
2016-03-29  5:37   ` Chris Marusich
2016-03-29  6:25     ` Chris Marusich
2016-03-29  7:17       ` Efraim Flashner
2016-03-29  7:35         ` Chris Marusich
2016-03-29 17:57       ` Leo Famulari
2016-03-30  8:17       ` Ludovic Courtès
2016-03-30 10:03         ` Chris Marusich
2016-03-30 20:58           ` Ludovic Courtès
2016-03-29  7:10     ` Efraim Flashner
2016-03-30  8:20 ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).