From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:470:142:3::10]:35005) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jJ0mO-0004X4-7x for guix-patches@gnu.org; Mon, 30 Mar 2020 16:13:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jJ0mM-0004H5-SF for guix-patches@gnu.org; Mon, 30 Mar 2020 16:13:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:51836) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1jJ0mM-0004Gr-Md for guix-patches@gnu.org; Mon, 30 Mar 2020 16:13:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jJ0mM-0002Yi-IZ for guix-patches@gnu.org; Mon, 30 Mar 2020 16:13:02 -0400 Subject: [bug#40115] [PATCH] download: Use correct system and guile in 'url-fetch/tarbomb' and 'url-fetch/zipbomb'. Resent-Message-ID: From: Diego Nicola Barbato References: <87d09927hw.fsf@GlaDOS.home> Date: Mon, 30 Mar 2020 22:11:57 +0200 In-Reply-To: <87d09927hw.fsf@GlaDOS.home> (Diego Nicola Barbato's message of "Wed, 18 Mar 2020 13:05:31 +0100") Message-ID: <871rp9d2lu.fsf@GlaDOS.home> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: 40115@debbugs.gnu.org --=-=-= Content-Type: text/plain Hey Guix, Here's some additional information. Diego Nicola Barbato writes: > The attached patch fixes a bug where e.g. > > guix build -s i686-linux ffmpeg > > builds a different derivation on i686-linux than on x86_64-linux. This > doesn't just affect ffmpeg but a whole class of packages which use or > depend on a package that uses 'url-fetch/tarbomb' or 'url-fetch/zipbomb' > as the origin method of its source. That's around 334 packages, among > them diffoscope, enlightenment, gnome, ungoogled-chromium, and wine. The number (348 for commit 151f3d4) and full list of affected packages can be computed by loading the attached script [0] into `guix repl' and running `(show-affected-packages)'. > The problem is fixed by explicitly passing the correct #:system and > #:guile-for-build to 'gexp->derivation' (as is done in other origin > methods such as 'git-fetch' or 'hg-fetch'). > > This shouldn't trigger any rebuils as it only affects the behaviour of > `guix build -s $system $package' if $system differs from the system type > of Guix itself. A closer look at some derivations and outputs suggests that this patch will actually trigger rebuilds for all affected packages on all systems except x86_64 because the build farm currently builds the wrong derivations as can be seen for e.g. QEMU by comparing the build on Cuirass https://ci.guix.gnu.org/build/2442001/details with the derivations computed by guix build -s i686-linux --no-grafts -d qemu on i686-linux and x86_64-linux (commit 151f3d4) respectively: Cuirass: /gnu/store/wc2k8h4iahbnfvl35220hvdx6mc70v7l-qemu-4.2.0.drv /gnu/store/fjg87f21qdzi7h5pqsxpd6rlf9mcy58h-qemu-4.2.0 <~ i686-linux: /gnu/store/019ccjdh1nxfkpjyzwmirvif1ra9v3lh-qemu-4.2.0.drv /gnu/store/8a0cg5ip9967y54gkwskfxmiwwk9mf1b-qemu-4.2.0 x86_64-linux: /gnu/store/iajzrw7lahcyhgyr7anmcjxa33607nqh-qemu-4.2.0.drv /gnu/store/fjg87f21qdzi7h5pqsxpd6rlf9mcy58h-qemu-4.2.0 <~ Consequently no substitutes are available for the affected packages on systems other than x86_64-linux as witnessed by the different number of available substitutes reported by guix weather -s i686-linux -m tarbomb-zipbomb-manifest-small.scm on i686-linux --8<---------------cut here---------------start------------->8--- computing 37 package derivations for i686-linux... looking for 37 store items on https://ci.guix.gnu.org... https://ci.guix.gnu.org 18.9% substitutes available (7 out of 37) at least 2.3 MiB of nars (compressed) 5.1 MiB on disk (uncompressed) 0.001 seconds per request (0.0 seconds in total) 1028.5 requests per second 'https://ci.guix.gnu.org/api/queue?nr=1000' returned 504 ("Gateway Time-out") --8<---------------cut here---------------end--------------->8--- and on x86_64-linux --8<---------------cut here---------------start------------->8--- computing 37 package derivations for i686-linux... looking for 37 store items on https://ci.guix.gnu.org... https://ci.guix.gnu.org 81.1% substitutes available (30 out of 37) at least 165.9 MiB of nars (compressed) 423.3 MiB on disk (uncompressed) 0.001 seconds per request (0.1 seconds in total) 703.3 requests per second 'https://ci.guix.gnu.org/api/queue?nr=1000' returned 504 ("Gateway Time-out") --8<---------------cut here---------------end--------------->8--- I have attached manifest files for the packages directly using `url-fetch/tarbomb' or `url-fetch/zipbomb' [1] and for all affected packages [2] (they use the aforementioned script). I think this patch can go on master even though it triggers more than 300 rebuilds, since there are currently no substitutes available for the affected packages anyway. Regards, Diego PS I hope I got all the terminology (e.g. computing vs. building a derivation) right. [0]: --=-=-= Content-Type: application/octet-stream Content-Disposition: attachment; filename=uses-tarbomb-zipbomb.scm Content-Transfer-Encoding: base64 KHVzZS1tb2R1bGVzIChnbnUgcGFja2FnZXMpCiAgICAgICAgICAgICAoZ3VpeCBwYWNrYWdlcykK ICAgICAgICAgICAgIChndWl4IGRvd25sb2FkKQogICAgICAgICAgICAgKHNyZmkgc3JmaS0xKSkK CihkZWZpbmUgKHVzZXMtb3JpZ2luLW1ldGhvZHM/IG1ldGhvZHMgcGFja2FnZSkKICAoYW5kIChw YWNrYWdlLXNvdXJjZSBwYWNrYWdlKQogICAgICAgKGlmIChsaXN0PyBtZXRob2RzKQogICAgICAg ICAgIChhbmQKICAgICAgICAgICAgKG1lbWJlciAob3JpZ2luLW1ldGhvZCAocGFja2FnZS1zb3Vy Y2UgcGFja2FnZSkpIG1ldGhvZHMpCiAgICAgICAgICAgICN0KQogICAgICAgICAgIChlcT8gKG9y aWdpbi1tZXRob2QgKHBhY2thZ2Utc291cmNlIHBhY2thZ2UpKSBtZXRob2RzKSkpKQoKKGRlZmlu ZSAoc29tZWhvdy1kZXBlbmRzLW9uLW9yaWdpbi1tZXRob2RzPyBtZXRob2RzIHBhY2thZ2UpCiAg KG5vdAogICAobnVsbD8KICAgIChmb2xkCiAgICAgKGxhbWJkYSAocCBsc3QpCiAgICAgICAoaWYg KHVzZXMtb3JpZ2luLW1ldGhvZHM/IG1ldGhvZHMgcCkKICAgICAgICAgICAoY29ucyBwIGxzdCkK ICAgICAgICAgICBsc3QpKQogICAgICcoKQogICAgIChwYWNrYWdlLWNsb3N1cmUgKGxpc3QgcGFj a2FnZSkpKSkpKQoKKGRlZmluZSAoYWxsLWRpcmVjdGx5LWFmZmVjdGVkLXBhY2thZ2VzIG1ldGhv ZHMpCiAgKGZvbGQtcGFja2FnZXMKICAgKGxhbWJkYSAocCByKQogICAgIChpZiAodXNlcy1vcmln aW4tbWV0aG9kcz8gbWV0aG9kcyBwKQogICAgICAgICAoY29ucyBwIHIpIHIpKQogICAnKCkpKQoK KGRlZmluZSAoYWxsLWFmZmVjdGVkLXBhY2thZ2VzIG1ldGhvZHMpCiAgKGZvbGQtcGFja2FnZXMK ICAgKGxhbWJkYSAocCByKQogICAgIChpZiAoc29tZWhvdy1kZXBlbmRzLW9uLW9yaWdpbi1tZXRo b2RzPyBtZXRob2RzIHApCiAgICAgICAgIChjb25zIHAgcikgcikpCiAgICcoKSkpCgooZGVmaW5l IChwYWNrYWdlLW5hbWUrdmVyc2lvbiBwYWNrYWdlKQogIChzdHJpbmctYXBwZW5kCiAgIChwYWNr YWdlLW5hbWUgcGFja2FnZSkKICAgIkAiCiAgIChwYWNrYWdlLXZlcnNpb24gcGFja2FnZSkpKQoK KGRlZmluZSAoZGlzcGxheS1hZmZlY3RlZC1wYWNrYWdlcykKICAobGV0ICgodGFyYm9tYit6aXBi b21iLWRpcmVjdAogICAgICAgICAoYWxsLWRpcmVjdGx5LWFmZmVjdGVkLXBhY2thZ2VzIGAoLHVy bC1mZXRjaC90YXJib21iICx1cmwtZmV0Y2gvemlwYm9tYikpKQogICAgICAgICh0YXJib21iK3pp cGJvbWItYWxsCiAgICAgICAgIChhbGwtYWZmZWN0ZWQtcGFja2FnZXMgYCgsdXJsLWZldGNoL3Rh cmJvbWIgLHVybC1mZXRjaC96aXBib21iKSkpKQogICAgKGZvcm1hdCAjdCAifiZ+ZCBwYWNrYWdl cyB1c2UgdXJsLWZldGNoL3RhcmJvbWIgb3IgdXJsLWZldGNoL3ppcGJvbWI6fgogICAgICAgICAg ICAgICAgfiZ+e35hfkB7IH5hfn1+fX4lfgogICAgICAgICAgICAgICAgfiZ+ZCBwYWNrYWdlcyBk ZXBlbmQgaW5kaXJlY3RseSBvbiB1cmwtZmV0Y2gvdGFyYm9tYiBvciB1cmwtZmV0Y2gvemlwYm9t Yjp+CiAgICAgICAgICAgICAgICB+Jn57fmF+QHsgfmF+fX59fiV+IgogICAgICAgICAgICAobGVu Z3RoIHRhcmJvbWIremlwYm9tYi1kaXJlY3QpCiAgICAgICAgICAgIChzb3J0IChtYXAgcGFja2Fn ZS1uYW1lK3ZlcnNpb24gdGFyYm9tYit6aXBib21iLWRpcmVjdCkgc3RyaW5nPCkKICAgICAgICAg ICAgKGxlbmd0aCB0YXJib21iK3ppcGJvbWItYWxsKQogICAgICAgICAgICAoc29ydCAobWFwIHBh Y2thZ2UtbmFtZSt2ZXJzaW9uIHRhcmJvbWIremlwYm9tYi1hbGwpIHN0cmluZzwpKSkpCg== --=-=-= Content-Type: text/plain [1]: --=-=-= Content-Type: application/octet-stream Content-Disposition: attachment; filename=tarbomb-zipbomb-manifest-small.scm Content-Transfer-Encoding: base64 OzsgRXZhbHVhdGUgdG8gYSBtYW5pZmVzdCBjb250YWluaW5nIGFsbCBwYWNrYWdlcyB0aGF0IHVz ZQo7OyBgdXJsLWZldGNoL3RhcmJvbWInIG9yIGB1cmwtZmV0Y2gvemlwYm9tYicgYXMgdGhlIG9y aWdpbi1tZXRob2Qgb2YKOzsgdGhlaXIgc291cmNlLgoobG9hZCAidXNlcy10YXJib21iLXppcGJv bWIuc2NtIikKKHBhY2thZ2VzLT5tYW5pZmVzdAogKGFsbC1kaXJlY3RseS1hZmZlY3RlZC1wYWNr YWdlcyBgKCx1cmwtZmV0Y2gvdGFyYm9tYiAsdXJsLWZldGNoL3ppcGJvbWIpKSkK --=-=-= Content-Type: text/plain [2]: --=-=-= Content-Type: application/octet-stream Content-Disposition: attachment; filename=tarbomb-zipbomb-manifest-full.scm Content-Transfer-Encoding: base64 OzsgRXZhbHVhdGUgdG8gYSBtYW5pZmVzdCBjb250YWluaW5nIGFsbCBwYWNrYWdlcyB0aGF0IHNv bWVob3cgZGVwZW5kCjs7IG9uIGB1cmwtZmV0Y2gvdGFyYm9tYicgb3IgYHVybC1mZXRjaC96aXBi b21iJy4KKGxvYWQgInVzZXMtdGFyYm9tYi16aXBib21iLnNjbSIpCihwYWNrYWdlcy0+bWFuaWZl c3QKIChhbGwtYWZmZWN0ZWQtcGFja2FnZXMgYCgsdXJsLWZldGNoL3RhcmJvbWIgLHVybC1mZXRj aC96aXBib21iKSkpCg== --=-=-=--