pypi->guix script

pypi->guix script

[David Thompson]

Hello all,

Over the past several days I've begun the long process towards packaging
GNU MediaGoblin, a Python web application with a lot of dependencies.
To make the process easier, I threw together a little script that can
automatically generate Guix package snippets for Python libraries using
metadata available on pypi.python.org.  Packages generated by this
script will still need some tweaking like adding the necessary build
inputs and verifying the license, but a lot of the tedious work has been
removed such as finding the latest release tarball and computing its
hash.

For more details and source code, check out the git repository:
https://gitorious.org/davexunit/pypi2guix

See the "Future Improvements" section of the README if you're interested
in automating this process even more.

Happy hacking!

-- 
David Thompson
Web Developer - Free Software Foundation - http://fsf.org
GPG Key: 0FF1D807
Support the FSF: https://fsf.org/donate

Re: pypi->guix script

[Nikita Karetnikov]

[-- Attachment #1: Type: text/plain, Size: 135 bytes --]

> For more details and source code, check out the git repository:
> https://gitorious.org/davexunit/pypi2guix

Why not add it to Guix?

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: pypi->guix script

[Ludovic Courtès]

David Thompson <dthompson2@worcester.edu> skribis:

> For more details and source code, check out the git repository:
> https://gitorious.org/davexunit/pypi2guix

This is very cool!

It would indeed be nice to turn it into a ‘guix pipy’ command or
something like that.  If need be, I can help with the details of
integrating it.

> * Remove dependency on guile-curl (use gnutls instead?)

It could use (guix build download), which uses (web client) + (gnutls).

Also, the thing could have two back-ends: one that generates code (what
it already does), and one that produces a <package> object.  That way,
we could have commands like:

  guix pipy build https://pipy...

Thanks,
Ludo’.

Re: pypi->guix script

[Thompson, David]

On Mon, Sep 8, 2014 at 3:06 AM, Ludovic Courtès <ludo@gnu.org> wrote:
> David Thompson <dthompson2@worcester.edu> skribis:
>
>> For more details and source code, check out the git repository:
>> https://gitorious.org/davexunit/pypi2guix
>
> This is very cool!
>

Thanks!

> It would indeed be nice to turn it into a ‘guix pipy’ command or
> something like that.  If need be, I can help with the details of
> integrating it.
>

Yes, that would be nice.  What are your thoughts on this?

>> * Remove dependency on guile-curl (use gnutls instead?)
>
> It could use (guix build download), which uses (web client) + (gnutls).
>

Perfect.  I will use that.

> Also, the thing could have two back-ends: one that generates code (what
> it already does), and one that produces a <package> object.  That way,
> we could have commands like:
>
>   guix pipy build https://pipy...
>

That would be neat.  That way one doesn't have to insert the code
anywhere to check if the package builds.

- Dave

Re: pypi->guix script

[Ludovic Courtès]

"Thompson, David" <dthompson2@worcester.edu> skribis:

> On Mon, Sep 8, 2014 at 3:06 AM, Ludovic Courtès <ludo@gnu.org> wrote:

[...]

>> It would indeed be nice to turn it into a ‘guix pipy’ command or
>> something like that.  If need be, I can help with the details of
>> integrating it.
>>
>
> Yes, that would be nice.  What are your thoughts on this?

The way forward would be to drop the file as guix/scripts/pipy.scm, and
then adjust it to follow the calling convention that ‘run-guix-command’
expects.  Similarly, it should use (guix build download).

Since it uses Guile-JSON, maybe ./configure would have to check for its
availability and disable compilation/installation when Guile-JSON is not
found (in a way similar to BUILD_DAEMON_OFFLOAD, in config-daemon.ac.)

Then a test file and an entry in the manual would be perfect.

Ludo’.

Re: pypi->guix script

[Thompson, David]

On Mon, Sep 8, 2014 at 10:42 AM, Ludovic Courtès <ludo@gnu.org> wrote:
> "Thompson, David" <dthompson2@worcester.edu> skribis:
>
>> On Mon, Sep 8, 2014 at 3:06 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>
> [...]
>
>>> It would indeed be nice to turn it into a ‘guix pipy’ command or
>>> something like that.  If need be, I can help with the details of
>>> integrating it.
>>>
>>
>> Yes, that would be nice.  What are your thoughts on this?
>
> The way forward would be to drop the file as guix/scripts/pipy.scm, and
> then adjust it to follow the calling convention that ‘run-guix-command’
> expects.  Similarly, it should use (guix build download).
>
> Since it uses Guile-JSON, maybe ./configure would have to check for its
> availability and disable compilation/installation when Guile-JSON is not
> found (in a way similar to BUILD_DAEMON_OFFLOAD, in config-daemon.ac.)
>
> Then a test file and an entry in the manual would be perfect.
>

Thanks for the explanation.  I will add this to my list of projects. :)

- Dave

Re: pypi->guix script

[Ludovic Courtès]

On second though, all the importers should have shared infrastructure.
There’s already the Nix importer, and the ‘guix import’ command.
Perhaps importers could use ‘guix import’ as a shared front-end.

Ludo’.

Re: pypi->guix script

[Thompson, David]

On Tue, Sep 9, 2014 at 3:39 AM, Ludovic Court�s <ludo@gnu.org> wrote:
> On second though, all the importers should have shared infrastructure.
> There\x19s already the Nix importer, and the \x18guix import\x19 command.
> Perhaps importers could use \x18guix import\x19 as a shared front-end.
>

That sounds reasonable.  I will try to do that when I get around to
integrating it with Guix.

Last night, I removed the guile-curl dependency and added another
backend that generates a package object.  I'm debating if I should
tackle automatically generating the proper inputs before integrating
it with the rest of Guix.

Re: pypi->guix script

[Ludovic Courtès]

"Thompson, David" <dthompson2@worcester.edu> skribis:

> That sounds reasonable.  I will try to do that when I get around to
> integrating it with Guix.

Cool!

> Last night, I removed the guile-curl dependency and added another
> backend that generates a package object.

Great.  I guess importers could simply generate a package object, and
then there would be a ‘package->sexp’ converter.

> I'm debating if I should tackle automatically generating the proper
> inputs before integrating it with the rest of Guix.

Either way is fine with me.

Thanks,
Ludo’.

Re: pypi->guix script

[Cyril Roelandt]

On 09/08/2014 09:06 AM, Ludovic Courtès wrote:
> David Thompson <dthompson2@worcester.edu> skribis:
> 
>> For more details and source code, check out the git repository:
>> https://gitorious.org/davexunit/pypi2guix
> 
> This is very cool!
> 
> It would indeed be nice to turn it into a ‘guix pipy’ command or
> something like that.  If need be, I can help with the details of
> integrating it.


I'm not too found of "guix pypi". I would rather use a "guix
autopackage" command that would make it easy to package software from
GNU, PyPI, and other sources.

We already have a "guix refresh" command that could probably be merged
with "guix autopackage". This way, we could have most of the
packaging/upgrading work done for us, with a single unified command.

WDYT ?


Cyril.

Re: pypi->guix script

[Cyril Roelandt]

On 09/09/2014 03:05 PM, Thompson, David wrote:
> On Tue, Sep 9, 2014 at 3:39 AM, Ludovic Court�s <ludo@gnu.org> wrote:
>> On second though, all the importers should have shared infrastructure.
>> There\x19s already the Nix importer, and the \x18guix import\x19 command.
>> Perhaps importers could use \x18guix import\x19 as a shared front-end.
>>
> 
> That sounds reasonable.  I will try to do that when I get around to
> integrating it with Guix.
> 
> Last night, I removed the guile-curl dependency and added another
> backend that generates a package object.  I'm debating if I should
> tackle automatically generating the proper inputs before integrating
> it with the rest of Guix.
> 

First of all, congrats! This looks really good and will be really useful.

I think you should release early :) It might be quite a pain to list all
inputs: you may have to read (test-)requirements.txt, setup.py (and
maybe actually evaluate some Python code) and deal with less standards
way of listing dependencies. Also, you might need some binaries or C
libraries installed, and it's usually not something that is well
specified in Python packages :/

This looks already good enough to be included; I'd just love to see
something more generic, with an architecture that would make it easy to
package software from other sources, as said in my previous email.

My 2 cents,
Cyril.

Re: pypi->guix script

[Ludovic Courtès]

Cyril Roelandt <tipecaml@gmail.com> skribis:

> I'm not too found of "guix pypi". I would rather use a "guix
> autopackage" command that would make it easy to package software from
> GNU, PyPI, and other sources.

Agreed.  As I wrote before, I think ‘guix import’ would be the natural
place for this.  ‘guix refresh’ can remain a separate thing IMO.

Ludo’.