From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:8:6d80::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id CJxkELG6eWCaFwEAgWs5BA (envelope-from ) for ; Fri, 16 Apr 2021 18:26:25 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id CNELDLG6eWAdQwAA1q6Kng (envelope-from ) for ; Fri, 16 Apr 2021 16:26:25 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id D80F125C8F for ; Fri, 16 Apr 2021 18:26:24 +0200 (CEST) Received: from localhost ([::1]:60970 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXRIW-00086O-2C for larch@yhetil.org; Fri, 16 Apr 2021 12:26:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37846) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXKI5-0001RO-UP for guix-devel@gnu.org; Fri, 16 Apr 2021 04:57:29 -0400 Received: from mail-io1-xd35.google.com ([2607:f8b0:4864:20::d35]:34656) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXKI0-00028X-Uj for guix-devel@gnu.org; Fri, 16 Apr 2021 04:57:29 -0400 Received: by mail-io1-xd35.google.com with SMTP id x16so27016712iob.1 for ; Fri, 16 Apr 2021 01:57:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=f4eUl+IQtr9O6cbdQ4VJfYlxNSZDU8dRW/J8Ej9iwyQ=; b=lLOFdygK4gceSV2KD4zfQ+cwo3itaifcWvNnhHohqD5D0aflO/0ZCB8FydhAH+Lx6y gsaN6YQU91bg9gMpKuhjvNzG4e8pFIL0AIe+Et7V1AUw454KbNr0gwKxit2iDCdecbR4 HEf1UcXxyz/kKlLFPoFLzbqBr6rv7aey/hZKhIxB5BJMGs1jqpNooucpVp6sxWk8LEfW ZopVaStRKdSC+cr4GKF6/g2VuFKAJ4jowg3jYtr6XbSy4Ea+RIMZpMMi7hzf/y8O88mP rS34Hy7u0yRaqrmaJrgkllLISoldM/LmPnebxcqweAM2xKQC/TPJ/hJyn2QPX0NezJtL WDjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=f4eUl+IQtr9O6cbdQ4VJfYlxNSZDU8dRW/J8Ej9iwyQ=; b=XtLEIbE+hGonUvkJ86Bbi72gbxFiHiXAZi6CfVSf7H09Gd9DwDo4y6q9Jvju6Aiiv3 stOlbLhBUCbSNmuntYezP02lMt/3jnjE9la/8iQyaywQLzP6xOcI+6K661Kan8fvlRU6 vEE4Mc4RlICcP/Gwd0r8xcW2b52OjM5r3xa4C4YmmHhbLil1B6L44njQoDI8KywJGUNK CSdwwdFCu351lxFDiQ4u7bdUaf8fg+P87aBAtsuiCCVXtFIR5vkcjn2QTLLkAc4RKAFM O03s6uRzLTR7snZaautjOW9ZH+u+0scbkdhhqrFc8/oZ+voVCDrk9C81QL2ZcT5O63Gz xveA== X-Gm-Message-State: AOAM531V0018flBMS14FhNrPVTApvy3ReZfQy0IAmL5gO+Q9jylVH3OD sCRUI1NQy/jI4xdOJ2BWnkczK+zHlhY1xi+KaNIHKNxZaVU= X-Google-Smtp-Source: ABdhPJw0ZcKV7aGY1RyyhKhiScX/k+dpPulhZWWx8s6SHZErKBTDIKfHmfMZwEvkjX2eD5K3H8BUAFKZZyjFsQoGHMg= X-Received: by 2002:a5d:9598:: with SMTP id a24mr2649065ioo.24.1618563442981; Fri, 16 Apr 2021 01:57:22 -0700 (PDT) MIME-Version: 1.0 From: Vladilen Kozin Date: Fri, 16 Apr 2021 09:57:11 +0100 Message-ID: Subject: Help adding a python package: chia-blockchain To: guix-devel@gnu.org Content-Type: text/plain; charset="UTF-8" Received-SPF: pass client-ip=2607:f8b0:4864:20::d35; envelope-from=vladilen.kozin@gmail.com; helo=mail-io1-xd35.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 16 Apr 2021 12:26:11 -0400 X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1618590385; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=f4eUl+IQtr9O6cbdQ4VJfYlxNSZDU8dRW/J8Ej9iwyQ=; b=ugqET7o0AvR7idraPMxnm1GxMMWBV6bX3ufcX5j2Y1Y07rgGEPUghMOmsTBJStp7PZMQoo Rtkn1YNSpcToNtHpR1RPGW5tlBDaUyOfYx+lrBkVF4yjLzUgAD86U33muAWO8O+L80Hl6Q D/Uo8G5uxSBkV1269iFClS9XK+KAI3ebLBSUnv9raAtmVMmo/Mhu4kgB68rxgNe/2kfdVR 5pZcVFmkLyFZwnPS5iJ8pIY4KGXlH3kKRTmMsElQn+hF/qhNvn1doA37UlBd0TfHo5ZLsK hF3llJZkZHY4kYEgGbLP7wTpC0/kGHfs55pCE5WeW/pM9UJUcSLo+Vnpfp1AkA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1618590385; a=rsa-sha256; cv=none; b=dAkmcz+hPT7/hBl9kivOaydrQt20kkpBRvMVpMNd0704ZYVu5sViwvJHexJbHMBZV5OaNy TCebBXV0JbiAMzBRwBgOnYQ6Oi4B6cD74vhFawD028mDF9IgaogacR4pFxgky0dgeGStME aAe6mvI6qIDDZ5o9TYH4saELJ/m5YBGzJCtcCzmQbMJ+n3heE3TvTBUjjX5ofFHZZGB43s sjZf0A/mlCb2xr0dZ/SxnyaIkau9owRDPSovS6uUffpdtqNUWHY5G6Ao8pPu0PjxNnrl5H lgs7bnWVaNYMocqnEBod8DCrZtZfLIjl5sXvIr5iAjnUZwdD+aA08g1U2h5gAA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=lLOFdygK; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -3.14 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=lLOFdygK; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: D80F125C8F X-Spam-Score: -3.14 X-Migadu-Scanner: scn0.migadu.com X-TUID: eBRExSXnu8pI Hi all. Could a kind soul handhold me while I try to port a python package to guix please. I am not a python programmer, so not on first name basis with python ecosystem and new to guix - double trouble. I've made some progress, but could use a bit of your help. Current state of affairs is tracked in this repo: https://github.com/vkz/ze-guix Repo and package I'm trying to import: https://github.com/Chia-Network/chia-blockchain Both it and all of its dependencies are open source, most on pypi but sadly some are only available there as binary wheels. Building from source entirely maybe complicated since it very quickly escalates to multiple build systems e.g. chiapos (proof of space lib) and chiavdf are C++ and rely on cmake, clvm_rs (their homegrown lisp) is written in Rust etc. Conceptually, installation is very simple and basically relies on binary wheels and amounts to: - checkout repo, - init and update submodule(s) - really only mozilla-ca, - create virtual env and activate, - upgrade to the latest pip to use binary wheels, - install deps and the package with pip You can see the steps I extracted from their install.sh here: https://github.com/vkz/ze-guix/blob/master/packages/manual-install.log Now, guix obviously heads and shoulders above virtualenv or at least it should be, however trying to build from source escalates quickly as I've mentioned. So, first question is if I could define a package by "replicating" the above essentially binary-only steps but replace virtualenv with e.g. guix profile or smth. This should be possible to do in code - as a proper package with some rudimentary build system that amounts to building everything inside a separate profile or maybe just pulling binary wheels into a directory in store and then adding that to whatever Python or pip expect and use as their PATH equivalent. IMO doing something trivial like this should be easy. To be sure this approach has security and "freedom" implications, but this is orthogonal to having something that works installed quickly. Building from source is very often involved borderline crazy. Could someone kindly teach me how to do this "binary" only packaging. Now, to building from source. First step I tried was to `guix import pypi -r`. This fails because not all deps have source pushed to pypi and only offer binary wheels. IMO `guix import` has no business failing like this or even at all for whatever reason. It is best effort tool to begin with that should result in a template that I can fill in. If there are errors - skip the dependency, report it, but please continue. Since, recursive `guix import` failed me I had to manually track deps and essentially call `guix import` for each one. Result you can see here: https://github.com/vkz/ze-guix/blob/master/packages/chia.scm This won't build for several reasons. Not all deps are there, some probably shouldn't be like win32 something or other. Another reason, I'd have to figure out how to build the inputs that require cmake and rust and python (some all at once). And this is where I could really use help. Examples as mentioned earlier are these dependencies: https://github.com/Chia-Network/chiapos https://github.com/Chia-Network/clvm_rs maybe others - it is a fairly big project and I've not even touched NPM/node that they rely on for their GUI client. That's another scary can of worms I'm not prepared to peek inside at this time. Ok, so really two questions: - how can I build a "binary" only python package that simply relies on binary wheels e.g. from pypi, - how to deal with multi-build systems e.g. python + cmake or python + rust, maybe someone could think of a good educational (so, not too convoluted) example of such package already in source tree. Latter would probably go much faster and teach better if done as pair-programming exercise, but that maybe a lot to ask. If you're up for it, I'm game though. Thank you -- Best regards Vlad Kozin