From mboxrd@z Thu Jan 1 00:00:00 1970 From: Pjotr Prins Subject: A registry for distributed sources and binaries Date: Sun, 24 Jul 2016 05:30:27 +0200 Message-ID: <20160724033027.GA20236@thebird.nl> References: <579027b7.VHXjhpPxQC3AAmeY%pjotr.public12@email> <8760rznoh1.fsf@gnu.org> <20160722004130.GA10340@thebird.nl> <874m7hk6dz.fsf_-_@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:58271) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bRAB2-0003s2-VY for guix-devel@gnu.org; Sat, 23 Jul 2016 23:34:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bRAAy-0000a3-PV for guix-devel@gnu.org; Sat, 23 Jul 2016 23:34:03 -0400 Content-Disposition: inline In-Reply-To: <874m7hk6dz.fsf_-_@gnu.org> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ludovic =?iso-8859-1?Q?Court=E8s?= Cc: guix-devel@gnu.org After some thought I am coming to the following: my primary goals are to lower the barrier to entry, scale out development of Guix packages and have people collaborate on each others packages without some centralized 'opinion'. The main problems with the current GUIX_PACKAGE_PATH approach are 1. People are not aware about the work of others 2. Slightly complicated (you need a Guix source tree etc.) 3. No binary distribution My immediate idea was to have a separate project, i.e. a fast and lossy tree next to the current 'strict' tree. With a distribution server 'guix publish' that could work for those who are inclined to use a lossy server (call it experimental or agile package definitions). After some thought I decided there is still a major downside - it will depend on central people to manage this second tree - even if it is only for merging packages and git trees. That competes with the main effort too which is inefficient. Now I think we may better solve this with something akin to a plugin system that we have with Rubygems, Python pip etc. A plugin system that is truly distributed from source (you just need to provide a registry). One example I worked on before is http://biogems.info/ for Ruby packages in bioinformatics. How about the following: 1. Separate from the GNU project, we create a number of registries of online git repos without opinion (i.e., anything goes, it is up to the authors). A registry can contain the work of multiple packages and multiple authors. 2. Each repo in the registry can create package definitions online 3. When someone wants to use a registry, run guix with guix package --registry http://biogems.info/bioinformatics.scm -A ruby i.e. to list all packages that have ruby, including the ones in that registry. Likewise for an incubator registry: guix package --registry http://incubator-guix.org/unstable.scm -A ruby generates a different list. 4. At install of a registry package, the entry can (optionally) point to a Guix publish server with its key, so we can do 'add key' and 'install binary or source package' for the registry. This way we get trusted binary distribution for distributed packages in registries. This would also compete nicely with the likes of Docker, especially when using containers, in a more secure way. Registries solve the mentioned problems of GUIX_PACKAGE_PATH: 1. People are not aware about the work of others 2. Slightly complicated (you need a Guix source tree etc.) 3. No binary distribution It also takes away the centralized approach of GNU Guix for those who don't want that - though the main tree will still be developed against for building a solid distribution etc. I.e., the core stuff continues, but people are free to try other routes. All a registry needs to provide is a guile file containing information of the package(s) whith a pointer to online pointers of documentation, overview, guix publish server and key. Personally I think this will be very exciting. We can have a metaregistry that lists all these packages so everyone can track them. Scalability. Less opinion, less duplication of work. True incubators. Pj.