From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id IHxoHr7OSmCbUAAA0tVLHw (envelope-from ) for ; Fri, 12 Mar 2021 02:15:26 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id 2JErGr7OSmDccAAAbx9fmQ (envelope-from ) for ; Fri, 12 Mar 2021 02:15:26 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C6D012795D for ; Fri, 12 Mar 2021 03:15:25 +0100 (CET) Received: from localhost ([::1]:34802 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lKXKl-0007bk-T1 for larch@yhetil.org; Thu, 11 Mar 2021 21:15:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41562) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lKXKa-0007Zc-Nk for help-guix@gnu.org; Thu, 11 Mar 2021 21:15:12 -0500 Received: from elegua.eauchat.org ([91.224.149.118]:58506 helo=eauchat.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lKXKY-000058-5a for help-guix@gnu.org; Thu, 11 Mar 2021 21:15:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eauchat.org; s=mail; t=1615515304; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nWz88NYhj1FBmBjMtMHhn0IwQXlElPnZUsrBJjgDcD8=; b=jz/8e3f8/UgqtvV/KLxdWbCR0w+zVR5eZTd5uTZeDiYYqcySd7BQXq0FoVGtKQwVig1J6t JVHxy4kkogfOa34hHH6WOsuLR7DG1Y4ThKn101QQ2d8zs1hT+JTPr8M2D4+FTuYMOigAdx XD3S1IlR/d7N1Bqz1iZmBdySUrYLIck= References: <87pn0731c1.fsf@eauchat.org> <85a6rbmin8.fsf@beadling.co.uk> From: =?utf-8?Q?S=C3=A9bastien?= Lerique To: Phil Subject: Re: Python, poetry, and pandas In-reply-to: <85a6rbmin8.fsf@beadling.co.uk> Date: Fri, 12 Mar 2021 11:14:58 +0900 Message-ID: <877dmdnmn1.fsf@eauchat.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=91.224.149.118; envelope-from=sl@eauchat.org; helo=eauchat.org X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: help-guix@gnu.org Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1615515326; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=nWz88NYhj1FBmBjMtMHhn0IwQXlElPnZUsrBJjgDcD8=; b=iwA/g4i91PRtxi9DqHXQhXlBEeEQxlvW74UYvpDvQIzhEacwRZbUBlLWrShsAHB4Hf4cVo +RUNj4+qWQ1gMJ1RIBMTuyc6Wrp5QpTUGbINUHd+UB/xwwavNMSi5oD6N3ASV03oG1H7cX UcGw5VXHKPorQv2u73Sjix/S+czT0uNHZ+SaLz6uZwkdd5yf8J/kCD3l5M/scwiQd0lA9f uavySoIm2uvb/yTgRKXwhcbVxpo1RD6XagDUM4JlWKPkMIyO5L8tq47NihiraQjjaH2Mvg rE5uZwTa9tYXONeo+bSPgABe7DzK8e45cXauJVuQBqziDWyhXyG9lBclEYSkNw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1615515326; a=rsa-sha256; cv=none; b=AU5VecZpBWf+aObtxMNt8kQaO2VvD+HRnKaA4LVwWlv2+aH7Ut43O2s+rRTI4fsTlfhkJ7 Tr6y1DTy/jD+HZ3652N1RNNDf/oMT8+JMMM63LBW6uuPrH9epDhpMe8OJHNm3gbs9h/GJp /l/ktcS5dTztdXSRLB/6BqhcahYBbuqE+pMJfjcbQlUirli+7e1w91g7Top9ok2e1KE1fy IPU0fHvEbCR5DvnAFYXEym2TBUFtY2SF8nsIaj8Leo61AaezD8x1UvbVfb33fkeCer4Ryd pqRbP//VeaqmrHhy5tYp1Rlzu8880CLwcpNQyhd+jrp6gD1tQ/ukhaby/wJaRQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=eauchat.org header.s=mail header.b="jz/8e3f8"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=eauchat.org (policy=none); spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Migadu-Spam-Score: -1.29 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=eauchat.org header.s=mail header.b="jz/8e3f8"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=eauchat.org (policy=none); spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Migadu-Queue-Id: C6D012795D X-Spam-Score: -1.29 X-Migadu-Scanner: scn0.migadu.com X-TUID: KZG19xgdyWvJ Hi Phil and Simon, Thanks both for the input and advice (not citing it all here). First the details. Simon wrote: > How did you installed poetry? Via Guix or something else? > Which python packages do you have in your profile? > Is Guix System or Guix on foreign distro? I'm using Guix System, and installed poetry via Guix. I have the=20 following python packages installed: python, poetry,=20 python-jupyterlab (from guix-science), and python-black. And=20 guix-jupyter. >> (I am aware that python-pandas is packaged for guix, but some >> packages I will need down the road might not be; this project=20 >> is >> also in collaboration with non guix users, so I am looking for=20 >> a >> workflow which will work for them too.) > > What are these packages? I don't know yet, but I suspect the situation might crop up. Phil=20 pointed to the PyPi importer, and I haven't used it yet, but it=20 does seem to make things simpler if I need to add a package to=20 Guix. Phil wrote: >> ImportError: libstdc++.so.6: cannot open shared object file: No=20 >> such >> file or directory > > If it's only missing a reference to the underlying C++ library=20 > you could > install this library using Guix, or if you're using a foreign=20 > distro point to > the non-Guix version under /usr/lib which, at a guess, Poetry=20 > may have > picked up - but this is going to cause you more trouble down the=20 > line > than repackaging in Guix in my experience. Yes, my experience with trying this for some Julia packages=20 (GLMakie) convinced me not to try LD_LIBRARY_PATH hacks any more. Now the broader view. Phil: > Guix offers pretty much everything that virtualenvs offers, but=20 > it does > things a little differently - you'll need to look at=20 > environments, > profiles, manifests, and use of a guix.scm to recreate (IMHO) a=20 > better > experience in the end. I just set up a simple guix.scm for my current project, which=20 indeed does the job, and of course provides all the great=20 guarantees of Guix! Given the number of python packages currently,=20 and the existence of the PyPi importer, I'm pretty convinced it=20 covers my personal needs. Now my wish is to find ways to collaborate with others on this=20 kind of project, who unfortunately often use macOS. I was=20 exploring the possibility of Guix on Darwin, but the best way to=20 do that looks like running Guix System inside Docker: http://logs.guix.gnu.org/guix/2021-03-11.log#044004 https://lists.gnu.org/archive/html/guix-devel/2017-10/msg00111.html So it seems that to collaborate with non-guix and non-GNU/Linux=20 users, the options are 1) maintain a second environment management=20 configuration (e.g. using poetry) to provide a stop-gap, or 2)=20 convince colleagues to run Guix in Docker. The situation for=20 Windows might be a bit better using the Linux Subsystem. Thanks again all, S=C3=A9bastien On 10 Mar 2021 at 19:02, Phil wrote: > Hi S=C3=A9bastien, > > S=C3=A9bastien Lerique writes: > >> Dear all, >> >> (I am aware that python-pandas is packaged for guix, but some=20 >> packages >> I will need down the road might not be; this project is also in >> collaboration with non guix users, so I am looking for a=20 >> workflow >> which will work for them too.) >> > > Under the covers Poetry is just using regular Python virtualenvs=20 > I think. > These don't work well with Guix for a number of reasons - but in=20 > my > previous attempts to use them I found that Guix's use of=20 > PYTHONPATH > meant that virtualenvs where never isolated from packages=20 > installed by > Guix. > > Guix offers pretty much everything that virtualenvs offers, but=20 > it does > things a little differently - you'll need to look at=20 > environments, > profiles, manifests, and use of a guix.scm to recreate (IMHO) a=20 > better > experience in the end. > > When I started using Guix I went to some lengths to harmonize > virtualenvs and Guix - and others have made more substantial=20 > efforts > than I did - but you end up creating as many problems as you=20 > fix: > https://lists.gnu.org/archive/html/help-guix/2020-11/msg00237.html > > If you search the mail archive a couple of other people have=20 > asked > similar questions to do with poetry, pyenv, pip, etc - they're=20 > all > interesting reading and easy to grep with those keywords: > https://lists.gnu.org/archive/html/help-guix/ > > The advice I was given at the time was don't mix package=20 > managers, and > in hindsight I'm glad repackaged what I needed under Guix. > > I've packaged quite a few python libraries now, and most are=20 > pretty > trivial to do - if they are in PyPi you can just use this: > > guix import pypi > > If you need a later version than Guix provides (eg pandas), my=20 > advice is > to use the ability to "inherit" the current definition in Guix=20 > and tweak > the version. For Pandas I did this for a more recent version I=20 > needed > and all I had to do was disable 2 unit tests (which failed for a > non-Guix reason). > > Apologies - not exactly the answer you want (I have a stab at=20 > that > below), but it's worth considering moving to Guix packaging=20 > exclusively > to keep your life simple. > > >> File >> "/home/sl/.cache/pypoetry/virtualenvs/poetry-test-vOWX4_Vr-py3.8/lib/pyt= hon3.8/site-packages/pandas/core/window/ewm.py", >> line 9, in >> import pandas._libs.window.aggregations as=20 >> window_aggregations >> ImportError: libstdc++.so.6: cannot open shared object file: No=20 >> such >> file or directory > > If it's only missing a reference to the underlying C++ library=20 > you could > install this library using Guix, or if you're using a foreign=20 > distro point to > the non-Guix version under /usr/lib which, at a guess, Poetry=20 > may have > picked up - but this is going to cause you more trouble down the=20 > line > than repackaging in Guix in my experience.