From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id WPtpC3hIMmE2WgEAgWs5BA (envelope-from ) for ; Fri, 03 Sep 2021 18:08:24 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id 6HYrB3hIMmEaBQAAB5/wlQ (envelope-from ) for ; Fri, 03 Sep 2021 16:08:24 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 7E04343DA for ; Fri, 3 Sep 2021 18:08:23 +0200 (CEST) Received: from localhost ([::1]:58088 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mMBjp-0006DE-8P for larch@yhetil.org; Fri, 03 Sep 2021 12:08:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59252) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mMBdU-0004mU-Gz for guix-devel@gnu.org; Fri, 03 Sep 2021 12:01:48 -0400 Received: from imta-37.everyone.net ([216.200.145.37]:52048 helo=imta-38.everyone.net) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mMBdR-0005Kq-Dg for guix-devel@gnu.org; Fri, 03 Sep 2021 12:01:48 -0400 Received: from pps.filterd (omta004.sj2.proofpoint.com [127.0.0.1]) by imta-38.everyone.net (8.16.0.43/8.16.0.43) with SMTP id 183FrVEk017104; Fri, 3 Sep 2021 09:01:42 -0700 X-Eon-Originating-Account: H-F5mHgUQWKn2dU3cQ_i1JjgKWSuXok3zmuEEw9WvGU X-Eon-Dm: m0116787.ppops.net Received: by m0116787.mta.everyone.net (EON-AUTHRELAY2 - 53b92e8b) id m0116787.611f98ad.fb48a; Fri, 3 Sep 2021 09:01:40 -0700 X-Eon-Sig: AQMHrIJhMkbkNV/UBgIAAAAD,4fec92e082f3c1573b233d24070bf874 X-Eip: IxjjcxmPOq0ZgBZa4t6R_BVrXA2P1Qmcu1hgjdDv7RE Date: Fri, 3 Sep 2021 18:01:31 +0200 From: Bengt Richter To: Attila Lendvai Subject: Re: packaging go-ethereum, and ultimately bee (of ethswarm.org) Message-ID: <20210903160131.GA8979@LionPure> References: <3a88b1abb853f8217e8dcc05810f1fb3f8468004.camel@telenet.be> <0Tgtoa1VWCatLMuCtmTliGT1yG1LDISdAJEQcbQM1jmeNeYg3106tuLHgYxgnR_2f31L9jbREnxViN4jr19NQl1Nwcc2lzc7T64gu4xTKIU=@lendvai.name> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <0Tgtoa1VWCatLMuCtmTliGT1yG1LDISdAJEQcbQM1jmeNeYg3106tuLHgYxgnR_2f31L9jbREnxViN4jr19NQl1Nwcc2lzc7T64gu4xTKIU=@lendvai.name> User-Agent: Mutt/1.10.1 (2018-07-13) X-Proofpoint-GUID: cy4LCyxLe32GgZAtwC35qnsTs0Y11xbG X-Proofpoint-ORIG-GUID: cy4LCyxLe32GgZAtwC35qnsTs0Y11xbG X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-09-03_07:2021-09-03, 2021-09-03 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 priorityscore=1501 mlxscore=0 lowpriorityscore=0 adultscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxlogscore=999 spamscore=0 clxscore=1034 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2108310000 definitions=main-2109030096 Received-SPF: pass client-ip=216.200.145.37; envelope-from=bokr@oz.net; helo=imta-38.everyone.net X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Bengt Richter Cc: "guix-devel@gnu.org" Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1630685303; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=ozL+olA3pcHMTzvcjOPE1ARFfBug4z+HiVy07f80LbE=; b=RMpxpO+5bKHHSqfIjNs2HjJy5X4+yJxpPcxUvtE8AZspVyEB0fI0hAPZG0V90h5zrcYgsw Yq9Ymfhx3VI5wtzg68dhCuD6+IjWGmgEFkLuQ0t0SkOnRXPqvgYkHentu3Rf4mJPfPrXma +J7JkTWtKM6P6wmHcbTHfOuP/xW4Gke/Cr7Oyz//yqbX5Rf4PQmh/TYFv9IJHXzjzJ+CMK +vef26K3Dn1u8Bv3rFn084FiDD7yUa+WoRUceeU+7rpti49XZlKW3n0UApWQtt+jbslsBO 8WBHm7ZnMr//WBt9mXVSxgmLvqe+UqnK/MjI6kiBgO9TbeSNtSuMBdnItfOkVQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1630685303; a=rsa-sha256; cv=none; b=mVG9q2gaK9e5ADnROoxJ4W6NcIxz6U8WDV1qYl6rsCFDTdQDQMEZZkXuAovg8DHHfX3HsY zd4NDxMplkF/zC3pkbo5xL27lykavEVQ+rMDIIo5/bDIt38Z6JmdX0R7OhQEnEu/QgMWNC WL+B6Y77NcBCadp89QTVyfiTich12+v/sC/Vu2Yldph2KrgNGso7UBjPi64jlGgsg8wKhh oCX6/x91G5QKzCulHqBl2s5wj1KMxNeWmslFiw+nfAgSaa28oRLJuSktFvPJQF0ifj+3MO /EuG9sUOI2AaMPxfznpN0c/5Ikzal42NROnJXBhLuU7+4pqcxriWwX3Kr9wRaw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -1.92 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 7E04343DA X-Spam-Score: -1.92 X-Migadu-Scanner: scn1.migadu.com X-TUID: /V8SUUnybY0u On +2021-09-03 08:16:00 +0000, Attila Lendvai wrote: >=20 > [with some reorderings] >=20 > > I hope my reply has been valueable. It seems like this discussion > > is going to end with =E2=80=98agreeing to disagree=E2=80=99 and I think= I said all > > I've got to say about the philosophical part of the matter, so I > > don't think I'll sending further replies on the philosophical parts. > > But if you have any technical questions, feel free to ask me. >=20 >=20 > ok, noted. your feedback is much appreciated, and i don't mean to > argue against the guix philosopy. i'm converted after all! :) >=20 > so, i'll try to focus on only pointing out perspectives that you may > not be aware of, and proposing solutions. >=20 >=20 > > > this mechanism is part of how go builds are reproducible. > > > building go-ethereum locally should yield the same binary that they > > > have released. > > > > FWIW, =E2=80=98reproducibility=E2=80=99 in guix means that, if the buil= d is repeated on another > > machine, with the same architecture and guix version, the binaries will= be > > exactly the same. Bit-for-bit reproducibility isn't guaranteed when the= builds > > are performed on different distro's. >=20 >=20 > thank you for clarifying this! i'll use clearer language when talking > about reproducibility. >=20 >=20 > > the loader to be at /lib/lib(64)/ld-linux.[...].so.[...], but that does= n't even > > exist on Guix, so binaries created on another distro cannot be run on g= uix, >=20 >=20 > FTR, the official binary seems to start up fine on my guix, and begins > syncing with some peers: >=20 > $ ~/.guix-profile/lib/ld-linux-x86-64.so.2 ./geth-linux-amd64-1.10.8-2667= 5454/geth --syncmode light >=20 > it's not statically linked: >=20 > $ ldd ./geth-linux-amd64-1.10.8-26675454/geth > linux-vdso.so.1 (0x00007fff26da6000) > libpthread.so.0 =3D> /gnu/store/ksy2b6fwfmz40gjajvspl87ia4vsfzj7-glibc-2= =2E31/lib/libpthread.so.0 (0x00007f86794af000) > libm.so.6 =3D> /gnu/store/ksy2b6fwfmz40gjajvspl87ia4vsfzj7-glibc-2.31/li= b/libm.so.6 (0x00007f867936e000) > librt.so.1 =3D> /gnu/store/ksy2b6fwfmz40gjajvspl87ia4vsfzj7-glibc-2.31/l= ib/librt.so.1 (0x00007f8679364000) > libc.so.6 =3D> /gnu/store/ksy2b6fwfmz40gjajvspl87ia4vsfzj7-glibc-2.31/li= b/libc.so.6 (0x00007f86791a7000) > /lib64/ld-linux-x86-64.so.2 =3D> /gnu/store/ksy2b6fwfmz40gjajvspl87ia4vs= fzj7-glibc-2.31/lib/ld-linux-x86-64.so.2 (0x00007f86794d2000) >=20 > will this have any pitfalls? is there anything i'm not aware of? For the paranoid, from man ldd, red emphasis mine (if it worked) --8<---------------cut here---------------start------------->8--- Security Be aware that in some circumstances (e.g., where the program specif= ies an ELF interpreter other than ld-linux.so), some versions of ldd may at= tempt to obtain the dependency information by attempting to directly execu= te the program, which may lead to the execution of whatever code is define= d in the program's ELF interpreter, and perhaps to execution of the prog= ram itself. (In glibc versions before 2.27, the upstream ldd implementatio= n did this for example, although most distributions provided a modified ve= rsion that did not.) =1B[1m=1B[31m Thus, you should never employ ldd on an untrusted executable, since = this may result in the execution of arbitrary code. A safer alternative = when dealing with untrusted executables is: $ objdump -p /path/to/program | grep NEEDED Note, however, that this alternative shows only the direct depend= encies of the executable, while ldd shows the entire dependency tree of the= exe=E2=80=90 cutable.=1B[m=0F --8<---------------cut here---------------end--------------->8--- >=20 > if not, then i propose adding a package that downloads the official > release, authenticates it by checking its signature, and sets up a > wrapper script for it to be able to start it by its proper name. >=20 > so, maybe we should add two packages? one that fetches the official > release binary, and one that builds it from source (and doesn't > strictly follow the versions in the project's go.mod file)? >=20 > i, as a user, would be more relaxed running the authenticated official > binary release. maybe this is misguided, and in that case i'm happy to > be corrected. >=20 >=20 > > > building go-ethereum locally should yield the same binary that they > > > have released. > > > > ... why should it? >=20 >=20 > so that, ideally, we are able to use the trusted guix intrastructure > to rebuild the same binary locally that the project has > released... and with that double-check also the release process of the > upstream. >=20 >=20 > > > > Another problem is: if a go package has many (transitive) dependenc= ies, how do > > > > we check that it doesn't contain any malware or non-free components= ? That needs > > > > > > go-ethereum is a software that handles ethereum wallets with several > > > zeroes. they are probably equally worried about this if not more. > > > > I'm sure the go-ethereum peope wouldn't mind if we double-check that no= malware > > slipt through their review process. >=20 > [...] >=20 > > > for projects like go-ethereum, it's not an option for the packager to > > > make decisions about the version of any of its dependencies. > > > > Why isn't this an option? Choosing different versions from upstream is = already > > done in guix, see e.g. https://issues.guix.gnu.org/50217 (ok that's not= merged > > yet, not the best example --- note, editing the version requirement was > > on my advice, see https://logs.guix.gnu.org/guix/2021-08-26.log). >=20 >=20 > extra auditing on top of the official release is certainly welcome by > each player. but, realisticly, i doubt that the hundreds of guix go > packages will get comparable amount of security-auditing-attention any > time soon. let alone -- and this is of key importance here -- auditing > any possible cross-interactions of altered versions of the > dependencies! who else is qualified to do that better than the > developers themselves? >=20 > for a program like the ethereum client, any such semantic surprises > may be disasterous. (losing money in various ways. e.g. their client > forking off of the consensus of the ethereum network, and if e.g. they > are paricipating in staking, then they will suffer penalties for not > adhering to the rules of the network, etc.) >=20 > this is why i would prefer to have a solution that somehow reproduces, > compares, and runs the same binary as the officially released one. >=20 > in the follwig days i'll play with reproducing the geth release binary > on my guix, and also planning to mock up a binary downloader package, > and see how it goes. i'll report back with anything worth mentioning. >=20 >=20 > > I see a third viable option (3): treat the "go.sum" as a mere > > =E2=80=98friendly suggestion=E2=80=99, and just use the latest version = when feasible. > > Again, I don't see much difference with, say, haskell, python, ruby, > > guile, java ... packages. >=20 >=20 > my point here is not that all go projects should be packaged like this > (i.e. with exacly pinned dependencies), but that applications like > geth should be, regardless of the language they are written in. >=20 > in case of a random image viewer written in go, i'm all in for a > relaxed handling of dependencies. >=20 >=20 > > Have there been any problems in practice with just using the latest > > version (updating the version currently in guix where applicable)? >=20 >=20 > not that i'm aware of, but the first such identified issue could turn > out to be very expensive. >=20 > - attila >=20 >=20 --=20 Regards, Bengt Richter