From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Prototype for using Guix with python packages Date: Sun, 11 Sep 2016 15:43:13 +0200 Message-ID: <87inu2lfgu.fsf@gnu.org> References: <4174c012-8e2b-d6f2-8387-031e309b7daf@cbaines.net> <87fupbn713.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:54194) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bj52Y-00055q-Hr for guix-devel@gnu.org; Sun, 11 Sep 2016 09:43:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bj52W-0004Bz-7u for guix-devel@gnu.org; Sun, 11 Sep 2016 09:43:21 -0400 In-Reply-To: (Christopher Baines's message of "Sat, 10 Sep 2016 14:09:58 +0100") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Christopher Baines Cc: guix-devel@gnu.org Hi, Christopher Baines skribis: > On 07/09/16 15:01, Ludovic Court=C3=A8s wrote: >>> For a few months now, I've been assembling a prototype for how packages >>> could be produced for software released as Python source distributions >>> (sdists) [1]. >>=20 >> Woow, quite an achievement! Do you know how many of the automatically >> generated packages built from source flawlessly? > > Not exactly, I disabled quite a few test suites, and made some > requirements stricter in some cases among other things. OK, not too bad. >> IIRC, =E2=80=98guix import pypi=E2=80=99 currently produces templates th= at can require >> extra tweaks, although that was improved by reading metadata from >> Wheels. >>=20 >> How does sdist metadata differ from PyPI or Wheels metadata? Is it >> generally more complete, or of better quality? > > sdists normally contain a .egg-info directory (e.g. sentry.egg-info), > which contains a requires.txt file, which as far as I can tell contains > the same information as the metadata.json in a wheel. > > I use that, as it means I can get the data without having to rely on the > existence of a wheel. =E2=80=98guix import=E2=80=99 uses Wheels data when available, and otherwis= e falls back to requirements.txt. AIUI, the rationale was that Wheels dependency information that is more detailed/accurate than requirements.txt: https://lists.gnu.org/archive/html/guix-devel/2016-01/msg00822.html I=E2=80=99m Ccing Cyril who knows this better. :-) > I think the next steps towards this would be: > - Switch from downloading wheels to just using the requires.txt in the > .egg-info directory > - Add support for https://www.python.org/dev/peps/pep-0518/ > - Identify when PBR (Python Build Reasonableness) is in use, as it may > mean that the test requirements are available (in test-requirements.txt) > > In my attempt to do this in a very automated way, I've only done the > first point, but I build the package and parse the build log for signs > of missing dependencies if the build fails, and then repeat the build. > This is done first with the tests disabled, and then if it eventually > builds, again with the tests enabled. I'm not sure if you would want > guix import to do this? Anything that improves the quality of what =E2=80=98guix import pypi=E2=80= =99 produces would be welcome. > The main reason why I didn't just improve the importer is that I was > looking for a way to collaborate around getting multiple versions of > python packages building and working, and as far as I am aware, Guix > only contains multiple versions of the same piece of software in some > special cases? That=E2=80=99s right: we keep multiple versions only when people or packages expect to be able to choose among them (e.g., the various GnuPG series, GCC, Python 2.x vs. 3.x). That=E2=80=99s usually a case-by-case decision, because every additional version that is kept entails additional maintenance work. The same would apply here: we could have multiple versions of some packages, and only one version of the majority of them. What do you think of this approach in the Python context? Thanks, Ludo=E2=80=99.