From mboxrd@z Thu Jan 1 00:00:00 1970 From: ng0 Subject: Re: GeoIP database redistribution? Date: Tue, 28 Nov 2017 16:19:37 +0000 Message-ID: <20171128161937.gx6l6v7z22g4klnj@abyayala> References: <8760l7t5ax.fsf@wasp.i-did-not-set--mail-host-address--so-tickle-me> <87k29l63pz.fsf@kirby.i-did-not-set--mail-host-address--so-tickle-me> <20170124060621.GA30899@mail.thebird.nl> <20170124082002.GF11257@macbook42.flashner.co.il> <8737g8en7k.fsf@wasp.i-did-not-set--mail-host-address--so-tickle-me> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="ar2stuopkkqphc3x" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:46461) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eJibv-00009m-Uq for guix-devel@gnu.org; Tue, 28 Nov 2017 11:19:55 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eJibn-0005fy-Ny for guix-devel@gnu.org; Tue, 28 Nov 2017 11:19:49 -0500 Received: from aibo.runbox.com ([91.220.196.211]:51514) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eJibn-0005fc-Gv for guix-devel@gnu.org; Tue, 28 Nov 2017 11:19:43 -0500 Received: from [10.9.9.210] (helo=mailfront10.runbox.com) by mailtransmit03.runbox with esmtp (Exim 4.86_2) (envelope-from ) id 1eJibl-0007M2-Pa for guix-devel@gnu.org; Tue, 28 Nov 2017 17:19:41 +0100 Received: from dslb-092-073-184-243.092.073.pools.vodafone-ip.de ([92.73.184.243] helo=localhost) by mailfront10.runbox.com with esmtpsa (uid:892961 ) (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) id 1eJibi-0005tI-0G for guix-devel@gnu.org; Tue, 28 Nov 2017 17:19:38 +0100 Content-Disposition: inline In-Reply-To: <8737g8en7k.fsf@wasp.i-did-not-set--mail-host-address--so-tickle-me> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: guix-devel@gnu.org --ar2stuopkkqphc3x Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi! It's better to reply late than never. My latency on some tasks is high. For anyone reading the old thread and wondering WTF happened, I've just re-read the thread and giving it shot soon as I want to analyze some logs and we already have a Perl module that wants this DB. I prefer Guix supplied solutions when I can make use of it, so long text short nonsense: I'll package it. Thanks for the provided feedback everyone. And Pjotr, the mentioning of TB's without any reference to an external project really threw me off. Just specify the "We" next time ;) ng0 transcribed 2.4K bytes: > Efraim Flashner writes: >=20 > > On Tue, Jan 24, 2017 at 06:06:21AM +0000, Pjotr Prins wrote: > >> On Mon, Jan 23, 2017 at 07:17:12PM +0100, Marius Bakke wrote: > >> > ng0 writes: > >> >=20 > >> > > I want to slowly package OONI (https://ooni.torproject.org/). > >> > > One of its dependencies, txtorcon, requires python-geoip which > >> > > depends on geoip-c-api. I've got both covered, but both of them > >> > > want (either to download or to be present) for tests (a/the) > >> > > legacy database file of maxmind. > >> > > > >> > > Question 1: Can we distribute the database in a source? I can't > >> > > access the homepage of maxmind for cloudflare reasons. > >> >=20 > >> > The database is distributed freely under cc-by-sa4.0: > >> >=20 > >> > https://dev.maxmind.com/geoip/legacy/geolite/#License > >> >=20 > >> > So packaging it should be fine. :) > >>=20 > >> This actually raises the issue of packaging large data files (we are > >> getting into TB's). Could there be a way Guix fetches external > >> datasets as part of the distribution? I think that if it is not > >> executable code and SHA values/pfff values match it would be safe to > >> do. > >>=20 > > > > The other thing is that guix downloads the files into ram and then > > writes them into the store, which can be a problem if the source is > > larger than the available memory. > > > > --=20 > > Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7= =9D =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 > > GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 > > Confidentiality cannot be guaranteed on emails sent or received unencry= pted >=20 > TB? Oh. Due to cloudflare I had no clue how big this is. In this > case, for the current state of the way we distribute packages is > not itself using a distributed network (if just keeping the space > on hydra is the problem?), I would suggest that I package the > updater I mentioned and mention it in the description for other > MaxMind software. >=20 > This database is also an optional dependency for tor and not just > ooni if I understand it correctly, so even if it's not 100% > accurate (their commercial solution claims to be more accurate) > it would be nice to find a way to include it. > But I think size is really an issue, for example Gentoo dropped > the database from their source distribution network and points > users to the updater I mentioned (that's how I learned about the > updater). > --=20 > =E2=99=A5=E2=92=B6 ng0 -- https://www.inventati.org/patternsinthechaos/ >=20 --=20 GnuPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588 GnuPG: https://c.n0.is/ng0_pubkeys/tree/keys WWW: https://n0.is --ar2stuopkkqphc3x Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEqIyK3RKYKNfqwC5S4i+bv+40hYgFAlodjJgACgkQ4i+bv+40 hYg/yRAAsTWOMwHTLWj0JrtwJUBpLbZN2d0iLo7Ma9DLeFB9qpDSnJVcn8P+GsJV 0uCBYbnM8yIDEj35NGcI9Q1bkWIFqaU02pKujwogyRu8KZAztxFcejrE+JLJqKm1 NUZXzrVDXjqpwRUBRsa3BvLEx7eDlpSoIMAoAOoNOXxvyqE6btZE9BJWMUtPuCLS ONYMd4yA03sptOhh1SLg+fIYccO06kfAAS70bdwukCetneXHCc8GFBR4lyp++2I6 ZHNItfmQJskoTlrFBYyWeRcRMdbv4StjyU19O58EYGk9xAmCjv/f1DVY7qXPIsi1 bLFbT4wxuX8JMhoVDEohLv0CcT408JJfhpSsolD+VT4Uu2O4z33Ht4w6uPIPxs0z s2QKKcMoEg50yUYYcFBzbxa+f/jVsWe/gv4acYxu30R68P7miLLB4GfSTTYxS6DS R8T+cPS8I7GAH7w8fRrk/FGx8o4R1g93PPSAg6n+bANBd3xBkSSXRWlRVNL2EgOg 9TyPFiWxMByGUgdUs3xP5tNB+xHHT3L0R09wjkJopBkPBeq9d9HSktrv2k4ttRxq S2bxejGwe6e1yRiN5QhYp+XywdkG5wH+TOoI7vb35Z5oQgpCUqec222W+NNWNAZw wYIP1bd0I9eEM/1nnv9J25MEJLt8eC9QFbGkMHfBeQTD22nv9AA= =JG7N -----END PGP SIGNATURE----- --ar2stuopkkqphc3x--