From: Vladilen Kozin <vladilen.kozin@gmail.com>
To: guix-devel@gnu.org
Subject: Help adding a python package: chia-blockchain
Date: Fri, 16 Apr 2021 09:57:11 +0100 [thread overview]
Message-ID: <CACw=CXPhQFVsH7yoQxXg5WMdqXod0dZhP7-qJ_9eNzT5ro+c8w@mail.gmail.com> (raw)
Hi all.
Could a kind soul handhold me while I try to port a python package to
guix please. I am not a python programmer, so not on first name basis
with python ecosystem and new to guix - double trouble.
I've made some progress, but could use a bit of your help. Current
state of affairs is tracked in this repo:
https://github.com/vkz/ze-guix
Repo and package I'm trying to import:
https://github.com/Chia-Network/chia-blockchain
Both it and all of its dependencies are open source, most on pypi but
sadly some are only available there as binary wheels. Building from
source entirely maybe complicated since it very quickly escalates to
multiple build systems e.g. chiapos (proof of space lib) and chiavdf
are C++ and rely on cmake, clvm_rs (their homegrown lisp) is written
in Rust etc.
Conceptually, installation is very simple and basically relies on
binary wheels and amounts to:
- checkout repo,
- init and update submodule(s) - really only mozilla-ca,
- create virtual env and activate,
- upgrade to the latest pip to use binary wheels,
- install deps and the package with pip
You can see the steps I extracted from their install.sh here:
https://github.com/vkz/ze-guix/blob/master/packages/manual-install.log
Now, guix obviously heads and shoulders above virtualenv or at least
it should be, however trying to build from source escalates quickly as
I've mentioned. So, first question is if I could define a package by
"replicating" the above essentially binary-only steps but replace
virtualenv with e.g. guix profile or smth. This should be possible to
do in code - as a proper package with some rudimentary build system
that amounts to building everything inside a separate profile or maybe
just pulling binary wheels into a directory in store and then adding
that to whatever Python or pip expect and use as their PATH
equivalent. IMO doing something trivial like this should be easy. To
be sure this approach has security and "freedom" implications, but
this is orthogonal to having something that works installed quickly.
Building from source is very often involved borderline crazy. Could
someone kindly teach me how to do this "binary" only packaging.
Now, to building from source.
First step I tried was to `guix import pypi -r`. This fails because
not all deps have source pushed to pypi and only offer binary wheels.
IMO `guix import` has no business failing like this or even at all for
whatever reason. It is best effort tool to begin with that should
result in a template that I can fill in. If there are errors - skip
the dependency, report it, but please continue.
Since, recursive `guix import` failed me I had to manually track deps
and essentially call `guix import` for each one. Result you can see
here:
https://github.com/vkz/ze-guix/blob/master/packages/chia.scm
This won't build for several reasons. Not all deps are there, some
probably shouldn't be like win32 something or other. Another reason,
I'd have to figure out how to build the inputs that require cmake and
rust and python (some all at once). And this is where I could really
use help. Examples as mentioned earlier are these dependencies:
https://github.com/Chia-Network/chiapos
https://github.com/Chia-Network/clvm_rs
maybe others - it is a fairly big project and I've not even touched
NPM/node that they rely on for their GUI client. That's another scary
can of worms I'm not prepared to peek inside at this time.
Ok, so really two questions:
- how can I build a "binary" only python package that simply relies on
binary wheels e.g. from pypi,
- how to deal with multi-build systems e.g. python + cmake or python +
rust, maybe someone could think of a good educational (so, not too
convoluted) example of such package already in source tree.
Latter would probably go much faster and teach better if done as
pair-programming exercise, but that maybe a lot to ask. If you're up
for it, I'm game though.
Thank you
--
Best regards
Vlad Kozin
next reply other threads:[~2021-04-16 16:26 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-16 8:57 Vladilen Kozin [this message]
2021-04-16 13:57 ` Help adding a python package: chia-blockchain Vladilen Kozin
2021-04-17 1:20 ` Leo Prikler
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CACw=CXPhQFVsH7yoQxXg5WMdqXod0dZhP7-qJ_9eNzT5ro+c8w@mail.gmail.com' \
--to=vladilen.kozin@gmail.com \
--cc=guix-devel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).