From mboxrd@z Thu Jan 1 00:00:00 1970 From: Chris Marusich Subject: Patches to add s3cmd and python-magic Date: Sun, 27 Mar 2016 20:29:51 -0700 Message-ID: <87io071dqo.fsf@gmail.com> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:44097) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1akNsQ-0000tB-DR for guix-devel@gnu.org; Sun, 27 Mar 2016 23:30:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1akNsN-0004Py-4V for guix-devel@gnu.org; Sun, 27 Mar 2016 23:30:02 -0400 Received: from mail-pa0-x22c.google.com ([2607:f8b0:400e:c03::22c]:33083) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1akNsM-0004Pu-Nf for guix-devel@gnu.org; Sun, 27 Mar 2016 23:29:59 -0400 Received: by mail-pa0-x22c.google.com with SMTP id zm5so2619870pac.0 for ; Sun, 27 Mar 2016 20:29:58 -0700 (PDT) Received: from garuda (c-73-169-163-30.hsd1.wa.comcast.net. [73.169.163.30]) by smtp.gmail.com with ESMTPSA id zu10sm31415112pab.31.2016.03.27.20.29.56 for (version=TLSv1/SSLv3 cipher=OTHER); Sun, 27 Mar 2016 20:29:57 -0700 (PDT) List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: guix-devel@gnu.org --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hi, I've packaged python-magic and s3cmd. The former provides Python bindings for libmagic. It's an alternative to python-file, which is an existing package that also provides Python bindings for libmagic. The latter is a command-line tool for Amazon S3 and CloudFront, which expects python-magic (not python-file) to be available. Please refer to the attached patches for details. Guix lint says the following about python-magic: "the source file name should contain the package name". However, I think maybe that's a bug in guix lint, since the URI clearly contains the name "python-magic". Just to be safe, I've tested python2-magic and python-magic, and confirmed that module "magic" was importable from Python 2 (for the former case) and Python 3 (for the latter case). I've also confirmed that python2-s3cmd installs fine and that I can use it for rudimentary tasks, like uploading and downloading to/from S3. I wasn't sure about a few things, so if you have feedback, I would love to hear it: * Do I need to provide setuptools as a native input, or will it be pulled in automatically? * Does setuptools really need to be a native input, or can it be a regular input? I understand that native inputs are important for cross-compiling, but does this apply to a language like Python which compiles to bytecode for a virtual machine? * In the package definition for python2-s3cmd, should python2-magic be a propagated input instead of a regular input? It seems to work as a normal input, so I think the answer is "no", but I understand that sometimes this matters for python packages. Thank you, =2D-=20 Chris --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename=0001-gnu-add-python-magic.patch Content-Transfer-Encoding: quoted-printable From=20d0c858b6b717b837eb2eac95db2fb2f19ca4112a Mon Sep 17 00:00:00 2001 From: Chris Marusich Date: Sun, 27 Mar 2016 19:10:48 -0700 Subject: [PATCH 1/2] gnu: add python-magic * gnu/packages/python.scm (python-magic, python2-magic): New variables. =2D-- gnu/packages/python.scm | 67 +++++++++++++++++++++++++++++++++++++++++++++= ++-- 1 file changed, 65 insertions(+), 2 deletions(-) diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index 88aef9d..f2ef901 100644 =2D-- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -16,7 +16,7 @@ ;;; Copyright =C2=A9 2015, 2016 Erik Edrosa ;;; Copyright =C2=A9 2015, 2016 Efraim Flashner ;;; Copyright =C2=A9 2015 Kyle Meyer =2D;;; Copyright =C2=A9 2015 Chris Marusich +;;; Copyright =C2=A9 2015, 2016 Chris Marusich ;;; Copyright =C2=A9 2016 Danny Milosavljevic ;;; ;;; This file is part of GNU Guix. @@ -6056,7 +6056,10 @@ Python's @code{ctypes} foreign function interface (F= FI).") #t)))))) (inputs `(("file" ,file))) (self-native-input? #f) =2D (synopsis "Python bindings to the libmagic file type guesser"))) + (synopsis "Python bindings to the libmagic file type guesser. Note th= at +this module and the python-magic module both provide a \"magic.py\" file; +these two modules, which are different and were developed separately, both +serve the same purpose: provide Python bindings for libmagic."))) =20 (define-public python2-file (package-with-python2 python-file)) @@ -8490,3 +8493,63 @@ is made as zipfile like as possible.") =20 (define-public python2-rarfile (package-with-python2 python-rarfile)) + +(define-public python-magic + (package + (name "python-magic") + (version "0.4.3") + (source + (origin + (method url-fetch) + (uri (string-append "https://github.com/ahupp/python-magic/archive/" + version ".tar.gz")) + (sha256 + (base32 + "17bgy92i7sb021f2s4mw1dcvpm6p1mi9jihridwy1pyn8mzvpjgk")))) + (build-system python-build-system) + (arguments + ;; The tests are unreliable, so don't run them. The tests fail + ;; under Python3 because they were written for Python2 and + ;; contain import statements that do not work in Python3. One of + ;; the tests fails under Python2 because its assertions are + ;; overly stringent; it relies on comparing output strings which + ;; are brittle and can change depending on the version of + ;; libmagic being used and the system on which the test is + ;; running. In my case, under GuixSD 0.10.0, only one test + ;; failed, and it seems to have failed only because the version + ;; of libmagic that is packaged in Guix outputs a slightly + ;; different (but not wrong) string than the one that the test + ;; expected. + '(#:tests? #f + #:phases (modify-phases %standard-phases + ;; Replace a specific method call with a hard-coded + ;; path to the necessary libmagic.so file in the + ;; store. If we don't do this, then the method call + ;; will fail to find the libmagic.so file, which in + ;; turn will cause any application using + ;; python-magic to fail. + (add-before 'build 'hard-code-path-to-libmagic + (lambda* (#:key inputs #:allow-other-keys) + (let ((file (assoc-ref inputs "file"))) + (substitute* "magic.py" + (("ctypes.util.find_library\\('magic'\\)") + (string-append "'" file "/lib/libmagic.so'"))) + #t)))))) + (native-inputs + `(("python-setuptools" ,python-setuptools))) + (inputs + ;; python-magic needs to be able to find libmagic.so. + `(("file" ,file))) + (home-page "https://github.com/ahupp/python-magic") + (synopsis "File type identification using libmagic") + (description + "This module uses ctypes to access the libmagic file type +identification library. It makes use of the local magic database and +supports both textual and MIME-type output. Note that this module and +the python-file module both provide a \"magic.py\" file; these two +modules, which are different and were developed separately, both serve +the same purpose: to provide Python bindings for libmagic.") + (license license:expat))) + +(define-public python2-magic + (package-with-python2 python-magic)) =2D-=20 2.7.3 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0002-gnu-add-python2-s3cmd.patch Content-Transfer-Encoding: quoted-printable From=2007c6fdfaab24d95fb6b318f5bd820cbe342ed271 Mon Sep 17 00:00:00 2001 From: Chris Marusich Date: Sun, 27 Mar 2016 19:28:37 -0700 Subject: [PATCH 2/2] gnu: add python2-s3cmd * gnu/packages/python.scm (python2-s3cmd): New variable. =2D-- gnu/packages/python.scm | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index f2ef901..73b674e 100644 =2D-- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -8553,3 +8553,41 @@ the same purpose: to provide Python bindings for lib= magic.") =20 (define-public python2-magic (package-with-python2 python-magic)) + +(define-public python2-s3cmd + (package + (name "python2-s3cmd") + (version "1.6.1") + (source + (origin + (method url-fetch) + (uri (string-append "mirror://sourceforge/s3tools/" + "s3cmd-" version ".tar.gz")) + (sha256 + (base32 + "0ki1rzhm5icvi9ry5jswi4b22yqwyj0d2wsqsgilwx6qhi7pjxa6")))) + (build-system python-build-system) + (arguments + ;; s3cmd is written for python2 only and contains no tests. + `(#:python ,python-2 + #:tests? #f)) + (native-inputs + `(("python2-setuptools" ,python2-setuptools))) + (inputs + `(("python2-dateutil" ,python2-dateutil) + ;; The python-file package also provides a magic.py module. + ;; This is an unfortunate state of affairs; however, s3cmd + ;; fails to install if it cannot find specifically the + ;; python-magic package. Thus we include it, instead of using + ;; python-file. Ironically, s3cmd sometimes works better + ;; without libmagic bindings at all: + ;; https://github.com/s3tools/s3cmd/issues/198 + ("python2-magic" ,python2-magic))) + (home-page "http://s3tools.org/s3cmd") + (synopsis "Command line S3 Client and backup") + (description + "S3cmd lets you copy files from/to Amazon S3 (Simple Storage +Service) using a simple to use command line client. Supports +rsync-like backup, GPG encryption, and more. Also supports management +of Amazon's CloudFront content delivery network.") + (license gpl2+))) =2D-=20 2.7.3 --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJW+KUxAAoJEN1AmhXYIkadT8UP/2vFvtO2CHAJybWaw3BzKkqV O3PSswLULpQxvIVSTX+/SPt4ElN97hQAS0Ex/f892aRBPaWnrEHNxTOjUHViwbzx R3uvBM7JEnwISCUSj3hCSV2Na6PnFgmKMHhbgqlFLHUi2qXOPHLINAYRk8nH9dXX yCEFYLPikf6m/hHfNPPWY/Sv4bDUTnetc2NOP7DXxJxs1CpcUKJUGtjuOUrtF2WN dYqSiSCI3N9p011R8/aUnykDpYM1Z0RTuFiYRsR6X63vINQR9E7y5ki+Dqf3iPg/ douwM5MDERxq6PAFuJQDB7F5NbefW1RdFmZhpb2x7tpujZkx6C4BUd62qgkmnHmz v3HWijwQZBqplz5jKQZrQ8p2dI2Z3AEyP8Rvf1FS2vIaupK9GfgfR1rdCrM50rs7 66gJIxW36BAI/CIvqh1/J/QhfAZ4egpYMXSYW0j5CvFClY+psLX5WXoB4VioDuOz EQkJYKorsWN+5lP+QN1TcFudjFKxr16SEWAeMlw32SVzTp7agEBowbPr+FpvgRpk NTrL9K4xq0aILPsCcpW9Js2hGUWTRaR9gM8JlXaSfPb+1xisrr8uvYURqY7KvVAp duYCEvru7yUvavPt2+Gl4pl8IGbTUG9H+mD2azVpa7iytk0f41FEE+gw2hiym//p xrJoOgxQS/D/j4SNKFDC =uzNg -----END PGP SIGNATURE----- --==-=-=--