From mboxrd@z Thu Jan 1 00:00:00 1970 From: ng0 Subject: Re: GeoIP database redistribution? Date: Tue, 24 Jan 2017 11:01:19 +0000 Message-ID: <8737g8en7k.fsf@wasp.i-did-not-set--mail-host-address--so-tickle-me> References: <8760l7t5ax.fsf@wasp.i-did-not-set--mail-host-address--so-tickle-me> <87k29l63pz.fsf@kirby.i-did-not-set--mail-host-address--so-tickle-me> <20170124060621.GA30899@mail.thebird.nl> <20170124082002.GF11257@macbook42.flashner.co.il> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:60246) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cVyq5-0001UT-86 for guix-devel@gnu.org; Tue, 24 Jan 2017 06:00:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cVypy-00081U-8u for guix-devel@gnu.org; Tue, 24 Jan 2017 06:00:37 -0500 Received: from fragranza.investici.org ([2a00:1dc0:2479::19]:55908) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cVypx-00080f-UF for guix-devel@gnu.org; Tue, 24 Jan 2017 06:00:30 -0500 In-Reply-To: <20170124082002.GF11257@macbook42.flashner.co.il> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Efraim Flashner Cc: guix-devel@gnu.org Efraim Flashner writes: > On Tue, Jan 24, 2017 at 06:06:21AM +0000, Pjotr Prins wrote: >> On Mon, Jan 23, 2017 at 07:17:12PM +0100, Marius Bakke wrote: >> > ng0 writes: >> >=20 >> > > I want to slowly package OONI (https://ooni.torproject.org/). >> > > One of its dependencies, txtorcon, requires python-geoip which >> > > depends on geoip-c-api. I've got both covered, but both of them >> > > want (either to download or to be present) for tests (a/the) >> > > legacy database file of maxmind. >> > > >> > > Question 1: Can we distribute the database in a source? I can't >> > > access the homepage of maxmind for cloudflare reasons. >> >=20 >> > The database is distributed freely under cc-by-sa4.0: >> >=20 >> > https://dev.maxmind.com/geoip/legacy/geolite/#License >> >=20 >> > So packaging it should be fine. :) >>=20 >> This actually raises the issue of packaging large data files (we are >> getting into TB's). Could there be a way Guix fetches external >> datasets as part of the distribution? I think that if it is not >> executable code and SHA values/pfff values match it would be safe to >> do. >>=20 > > The other thing is that guix downloads the files into ram and then > writes them into the store, which can be a problem if the source is > larger than the available memory. > > --=20 > Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7= =9D =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 > GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 > Confidentiality cannot be guaranteed on emails sent or received unencry= pted TB? Oh. Due to cloudflare I had no clue how big this is. In this case, for the current state of the way we distribute packages is not itself using a distributed network (if just keeping the space on hydra is the problem?), I would suggest that I package the updater I mentioned and mention it in the description for other MaxMind software. This database is also an optional dependency for tor and not just ooni if I understand it correctly, so even if it's not 100% accurate (their commercial solution claims to be more accurate) it would be nice to find a way to include it. But I think size is really an issue, for example Gentoo dropped the database from their source distribution network and points users to the updater I mentioned (that's how I learned about the updater). --=20 =E2=99=A5=E2=92=B6 ng0 -- https://www.inventati.org/patternsinthechaos/