From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id uA2rGzFsrmGUSgAAgWs5BA (envelope-from ) for ; Mon, 06 Dec 2021 21:01:53 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id WJUyFzFsrmGfYQAAbx9fmQ (envelope-from ) for ; Mon, 06 Dec 2021 20:01:53 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 01323829F for ; Mon, 6 Dec 2021 21:01:53 +0100 (CET) Received: from localhost ([::1]:39052 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1muKBM-0002As-42 for larch@yhetil.org; Mon, 06 Dec 2021 15:01:52 -0500 Received: from eggs.gnu.org ([209.51.188.92]:57090) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1muK91-0007ki-D9 for guix-devel@gnu.org; Mon, 06 Dec 2021 14:59:28 -0500 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:53251) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1muK8y-0006yP-Eb for guix-devel@gnu.org; Mon, 06 Dec 2021 14:59:27 -0500 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 516565C00CC for ; Mon, 6 Dec 2021 14:59:22 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 06 Dec 2021 14:59:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:message-id :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; bh=Rw3ILDGkVcxEN0U3uaquKzuMo0Q4u /wBKIhXhamtBoI=; b=a++7B2BO0zmF5cC29USnTHKD+jb5J9LHRaUjbA/aLoE5C TLLoIFhtm85ye1PoGY2A54UPP9zYndFaU8rg+n8JDBG6tqivYzGFUzsLb3J/RJHz jmhkj1a+l9c1Jda5gDJ2vysTfbqEUxP0Yr+YCYUM4kc/iCVXH/uNUaLCtBuHu8++ vr9fGXzDdQKSvMFShZxgIQ5gUFO5LNorpTWSqPhXmaqhHO1WO4C0J1tRhPumjJ5i GttYWjA2apZoW/91tBhdix02xESwQbRXowr3xNb6LTiQaVSAEg2f4AuvfmibyMXS a6Dy0C2RaZofxnjpD7gLO3PFFB6tl47cwi9kviJUg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvuddrjeefgddufedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffkfgggtgesmhdtreertd erjeenucfhrhhomhepvfhimhhothhhhicuufgrmhhplhgvuceoshgrmhhplhgvthesnhhg hihrohdrtghomheqnecuggftrfgrthhtvghrnheptdehieetgfejudffvefhgfekveffgf ejleevudeuleektdethfefueduleffjeegnecuffhomhgrihhnpehnghihrhhordgtohhm necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepshgrmh hplhgvthesnhhghihrohdrtghomh X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Mon, 6 Dec 2021 14:59:21 -0500 (EST) From: Timothy Sample To: guix-devel@gnu.org Subject: Preservation of Guix Report 2021-12-06 Date: Mon, 06 Dec 2021 14:59:20 -0500 Message-ID: <87o85to67b.fsf@ngyro.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=66.111.4.28; envelope-from=samplet@ngyro.com; helo=out4-smtp.messagingengine.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1638820913; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=Rw3ILDGkVcxEN0U3uaquKzuMo0Q4u/wBKIhXhamtBoI=; b=Up8DnXHXnNWq5h67yHumquMNHCN8NEwXFEXfoNfEZ/JwUk2Tr9k1/0+d1vXZpQNw/My+zI ImRxuNLbOnaMt0KpFm+ILV77SnTYkgLX19bT69bZMEqrztyPfm+Uns7RstIW3b3rqBNrh5 ed+TSiVCZqCCj2Mpfw7CD3m9C/uchU0A3zrlXlg2J1rjQ00umhyceCvTPSUfgvThge6TA8 eSdvYiUdfejex3gDSJ7o7cIVtHU6Dhm21jX2Xv/N04jniVfLLcRmOGbmr94T4f2Bn5OV/y Ds45sp6n5OCWqY/hOK0tNF8w974KCfoXOjFUwVQbKygBlHYeu40t3EOSE/GE2w== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1638820913; a=rsa-sha256; cv=none; b=R2QfFKRqQ6Yon2A+L+objBzjaEj1ZKonnojERLiTwM8P7z3xirlrbflndC6Z6x7qjVkKlt mt1zyX2Uh0l0gYNI4ugc7pZQOxWyOozC83ciiiE1vTrSpC6O3iq6DG7XiW7UObPxYufi6p f+OslcrqP3Wqzyp+NvD+JY9WVxUBdT2BSX/QB1/CU4Ryg/Pdto3xxxYGZGzOEH6sTwaYbM +EEMbo4xUezF7rIqoa/x3UIN3N4OI9I2tLq00jgfBru6T7MgmssbFZJ6ZBYLGFdilv6hzk 7kaf4NguZKYq2u+7KdvR64trRCXWxm6KYb035S6jxW5JSCCqCAP+MuKoKaiWcg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=a++7B2BO; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.74 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=a++7B2BO; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 01323829F X-Spam-Score: -4.74 X-Migadu-Scanner: scn0.migadu.com X-TUID: 4xfsBEke97Zf --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Guix, This is an update to the preservation of Guix report. There are no new commits or fixed-output derivations in this report, but I spent some time cleaning up the results, and I think the improvements are worth sharing. The last report generated a lot of questions. This one doesn=E2=80=99t answer all of them, but it=E2=80=99s a big improvement: Since the last report, I added many more reference categories and moved them to the database. The new categories are 'hg', 'svn', 'cvs', 'bzr', 'tar-bz2', 'tar', 'zip', and 'text'. Of these, only 'tar' and 'text' are being processed. The rest are currently unsupported by my scripts. Moving the categories to the database allows me to make manual corrections when needed. It also encouraged me to look through the references a bit more carefully to track down some of the weirder 'text' sources (like Bash patches) and fix up some other ones (in the style of =E2=80=9C/tar_gz?download=3Dyes=E2=80=9D). I also made the fetching code more tenacious. Now it uses the content-addressed mirrors from Guix and Nix to find regular files, and will recover =E2=80=9Ceasy=E2=80=9D Git references from SWH (=E2=80=9Ceasy= =E2=80=9D means the commit is specified). Between improving the fetching code and adding 'tar' and 'text' processing, I=E2=80=99ve computed another 2.5K SWHIDs. We now have SWHIDs = for 86% of our fixed-output derivations. There are only 51 =E2=80=9Cunknown=E2= =80=9D non-recursive Git sources now (the list is attached). But that=E2=80=99s not all! The scripts now categorize failures, so we have a better idea of what=E2=80= =99s going on with the remaining 14% =E2=80=9Cunknown=E2=80=9D sources: no-ref: 13 disarchive: 863 fetch: 1262 bail: 3324 ----------------- total: 5462 The =E2=80=9Cbail=E2=80=9D category is all the stuff my scripts don=E2=80= =99t yet process, like Mercurial repositories and bzip2 tarballs. The =E2=80=9Cfetch=E2=80=9D category is everything the scripts couldn=E2=80= =99t track down. The =E2=80=9Cdisarchive=E2=80=9D category is all the tarballs Disarchive fa= iled to process. An interesting thing here is that most of them are from Cargo. Long story short: older versions of Cargo used the =E2=80=9Cminiz=E2=80=9D implementation of DEFLATE (rewritten in Rust) to compress tarballs. Disarchive doesn=E2=80=99t support this (yet...?). There are 686 old-Cargo-produced tarballs in the =E2=80=9Cdisarchive=E2=80=9D category. The =E2=80=9Cno-ref=E2=80=9D category covers a few fixed-output derivations= used in bootstrapping that do not come from an origin record. I will probably just load them by hand eventually. (In the future I hope to put some of this in the report itself.) One last thing to add is that the SWH folks were very quick to fix the loading error, so the increase in missing sources for recent commits is now gone. -- Tim --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=git-missing.txt 00000nij3ray7nssvq0lzb352wmnab8ffzk7dgff2c68mvjbh1l6 (git-reference (url "https://github.com/fdik/libetpan") (commit "210ba2b3b310b8b7a6ee4a4e35e50f7fa379643f")) 03ym14g9qhjqmryr5z065kynqm8yhmvnbs2djl6vp3i9cmqln8cl (git-reference (url "https://git.savannah.gnu.org/git/emacsy.git") (commit "v0.4.1-37-g5f91ee6")) 04c2vqxg31mk15cfrhzrivykis8fmf0m1d8h1qdjdmlfxd4qwaqf (git-reference (url "https://github.com/hboetes/mg") (commit "20210609")) 06plnhi1489wqsag5wgm16hb1xd1a8nbnb9gw7635d3fidxyb0wp (git-reference (url "https://github.com/erlang/otp") (commit "OTP-24.0.2")) 06w86xk7sjl2x2h3z6msn8kpmwj05qdimcym77wzhz5s94dzh1bl (git-reference (url "https://github.com/hboetes/mg") (commit "20180408")) 07xansmhn4l0b9ghzf56vyx8cqg0q01aq3pz5ikx2i19v5f0rc66 (git-reference (url "https://github.com/tgvaughan/elpher") (commit "v1.4.6")) 08yca0a0prrnrc7ir7ajd56yxvxpcs4m1k8f5kf273f5whgr7wzw (git-reference (url "https://github.com/ProtonVPN/linux-cli") (commit "v2.2.4")) 0a066f56hnb9znbwnv1blm31j0ysv05n4wzlkli0zgw087c9047x (git-reference (url "https://source.atlas.engineer/public/next") (commit "1.2.0")) 0anmprm63a88kii251rl296v1g4iq62r6n4nssx5jbc0hzkknanz (git-reference (url "https://git.savannah.gnu.org/git/nomad.git/") (commit "0.2.0-alpha-100-g6a565d3")) 0d269474kk1933c55hx4azw3sak5ycfrxkw6ida0sb2cm00kfich (git-reference (url "https://gitlab.savoirfairelinux.com/sflphone/libiax2.git") (commit "0e5980f1d78ce462e2d1ed6bc39ff35c8341f201")) 0f8zr2jxr0v4zcd98zqx99zxdn768vjpzwxsbsd6ss3if405sq2a (git-reference (url "https://github.com/erlang/otp") (commit "OTP-24.0.5")) 0fy4qsss3i3pkq1rpgjds4aipbwlh1dr9hbbf7jn2a1c63kfks0r (git-reference (url "https://git.zx2c4.com/wireguard-go/") (commit "v0.0.20200320")) 0fzhwbpyndwrmxip9zlcwkrr675l5pzwcygi45hv7w1hn39w0hxp (git-reference (url "https://github.com/emacsattic/relative-buffers") (commit "9762fe268e9ff150dcec2e2e45d862d82d5c4008")) 0gv83i5ybj1z3ykbbldjzf7dbfjszp84c0yzrpshj611b9wp0176 (git-reference (url "https://github.com/erlang/otp.git") (commit "OTP-21.0.5")) 0ibq30xrf871pkpasi8p9krn0pmd86rsdzb3jqvz3wnp4wa3hl9d (git-reference (url "https://source.atlas.engineer/public/next") (commit "1.3.0")) 0ifnaclsz7w08mc485i3j1kkcpd1m8q5qamckrfwc375ac13xf4g (git-reference (url "git://anongit.kde.org/kdenlive.git") (commit "v18.08.1")) 0p0ha6prp7pyadp61clbhc6b55023vxzfwy14j2qygb2mkq7fhic (git-reference (url "https://git.savannah.gnu.org/git/nomad.git/") (commit "0.2.0-alpha-199-g3e7a475")) 0rac70p6rpvdx9v0bdd8nphgr7imdxb7nz0x77n3p7h3180zz9x0 (git-reference (url "https://github.com/OpenVisualCloud/SVT-HEVC") (commit "v1.5.1")) 0rgiypb9ig8x4rl3hfzpy7kwnx1q3064nvlrv4fk0dnp84girn0v (git-reference (url "https://github.com/scour-project/scour") (commit "v038.1")) 0w0dff3s7wv2d9m78a4jhckiik58q38wx6wpbba5hzbs4yxz35ck (git-reference (url "https://github.com/parallaxinc/propgcc") (commit "4c46ecbe79ffbecd2ce918497ace5b956736b5a3")) (git-reference (url "https://github.com/parallaxinc/propgcc.git") (commit "4c46ecbe79ffbecd2ce918497ace5b956736b5a3")) 0xxd9nhqiclpkdd9crymvba37fl0xs5mikwhya68nfzcgar7w480 (git-reference (url "https://github.com/libretro/RetroArch.git") (commit "v1.7.8")) 0y7v9ikrmy5dbjlpbpacp08gy838i8z54m8m4ps7ldk1j6kyia3n (git-reference (url "https://github.com/ProtonVPN/linux-cli") (commit "v2.2.6")) 0zw5wra9hc717srmcar1wm4i34kyj8c49ny4bb7y3nrvkjp2pdb5 (git-reference (url "https://github.com/aureliojargas/clitest") (commit "v0.3.0")) 106v177y3nrjv2l1yskch4phpqd8h97b67zj0jiq9pc3c69jr1ay (git-reference (url "https://github.com/powertab/rtmidi.git") (commit "2.1.0")) 12bs2rfmmy021087i10vxibdbbvd5vld0vk3h5hymhpz7rgszcmg (git-reference (url "https://github.com/sekrit-twc/zimg.git") (commit "release-2.9.3")) 13i7dczbqwhws08zzrdraki1zkqv0qkbgx9c1r8vmg5qr9f7hfzg (git-reference (url "https://github.com/KittyKatt/screenFetch") (commit "v3.9.0")) 14knnvfaskfz97vs3lfqrdpcbcx22s6qp16213wdnvnsf4c1lx1b (git-reference (url "https://chromium.googlesource.com/webm/libvpx") (commit "v1.9.0-88-g12059d956")) 14vrm8lvwksf697sqks7xfd1xaqjlqjc9afjk33sksq5p27wr203 (git-reference (url "https://github.com/hboetes/mg") (commit "20180927")) 15i5ixpryfrbf3vrrb5rici8fb585f25k0v1ljds16bp1f1msr4q (git-reference (url "https://git.code.sf.net/p/itk-snap/src") (commit "v3.8.0")) 1827jljs8mps489fm7xw63cakdqwc5grilrr5n9spr2rlk76jpx3 (git-reference (url "https://github.com/dom4j/dom4j.git") (commit "version-2.1.0")) 189jsj87hycs57a54x0b9lifwvhr63nypb9vfxdrq7rwrpcvi5f8 (git-reference (url "https://inqlab.net/git/guile-sodium.git") (commit "v0.1.0")) 19s6dbn47xy30dwfdd2p8fxz6z63rp5h7sm0barb69r7mvgnqvc1 (git-reference (url "https://source.atlas.engineer/public/next") (commit "1.2.1")) 1a87mka2sfzhbch2jip6wlvvs0glxq9lqwmyrp359d1rmwwmqiw9 (git-reference (url "https://github.com/markfasheh/duperemove") (commit "v0.11.2")) 1bif1k738knhifxhkn0d2x1m521zkx40pri44vyjqncp9r95hkbk (git-reference (url "https://source.atlas.engineer/public/next") (commit "1.2.2")) 1ccy7qz1wcmggqlf3hwigbqq4wrx1amds4x9bxz9py6bypglyjc5 (git-reference (url "https://framagit.org/contrapunctus/chronometrist.git") (commit "v0.4.2")) 1cljkkyi9dxqpqhx8y6l2ja4zjmlya26m26kqxml8gx08vyvddhx (git-reference (url "git://git.tuxfamily.org/gitroot/non/non.git") (commit "5ae43bb27c42387052a73e5ffc5d33efb9d946a9")) 1cs1i1hxwrv0a512j54yrvfh743nci1chx6qjgp4jyzq98ncvxgg (git-reference (url "https://git.savannah.gnu.org/git/emacsy.git") (commit "v0.4.1-31-g415d96f")) 1dj37vk712dx76y25g13na24wbpn7a5ddmlpf4n51gm10sib54wj (git-reference (url "https://github.com/erlang/otp") (commit "OTP-21.3.8.13")) (git-reference (url "https://github.com/erlang/otp.git") (commit "OTP-21.3.8.13")) 1f1gapvs9j89qr474103dqgsiyb96phlnsmq5hiv4ba242blg9lb (git-reference (url "https://git.umaneti.net/flycheck-grammalecte/") (commit "v1.3")) 1gw27lqc3f525n8qdcmr2nyn16y9g10z9f6dnmckyyxcdzvhq35n (git-reference (url "https://github.com/JohnCremona/eclib/") (commit "v20190909")) 1hnh2mnmw179gr094r561w6cw1haid0lpvpqvkc24wpj82vphzpa (git-reference (url "http://dr-qubit.org/git/undo-tree.git") (commit "release/0.6.6")) 1ijglmwkdy1l87gj429qfjis0v8b1zlxhbyfhx5za8664h68nqka (git-reference (url "https://inqlab.net/git/eris.git") (commit "v0.2.0")) 1ks1fa0027s3xp0z6qp0dxmayvrb4dwwscfhbx7da0khp153f2cp (git-reference (url "https://github.com/trezor/trezord-go") (commit "v2.0.29")) (git-reference (url "https://github.com/trezor/trezord-go.git") (commit "v2.0.29")) 1ljjqzghcap4admv0hvw6asm148b80mfgjgxjjcw6qc95fkjjjlr (git-reference (url "https://framagit.org/contrapunctus/chronometrist.git") (commit "v0.4.3")) 1lv1nckvzyhpn8cs6m40f2np15b3a8071kh7sy1216q2345s2ckc (git-reference (url "https://notabug.org/cage/tinmop") (commit "v0.8.1")) 1nr0149y2nvrxj56gc12jqnfl01g6z9ypfsgl6pfg85cw73hnggk (git-reference (url "http://dr-qubit.org/git/undo-tree.git") (commit "release/0.7.1")) 1p3lw4bcm2dph3pf1h4i0d9pzrcfr83r0iadqanxkwbmm1bl11pm (git-reference (url "https://github.com/erlang/otp") (commit "OTP-23.2.1")) 1pm4pwg2abd0j9cn5v3k2ksk9ig4vlwxmlw9rrglanziv9l967qp (git-reference (url "https://github.com/kr/pretty.git") (commit "v0.2.0")) 1s29hz44rb5dwzq8d4i4bfg77dr0v3ywpvidpa6xzg7hnnv3mhi5 (git-reference (url "https://github.com/jurplel/qView.git") (commit "2.0")) 1sx31zp6q2qc6fz3r78rx34zp2x4blrqzxwbpww71vb6lp1clmdm (git-reference (url "https://github.com/MaskRay/ccls") (commit "0.20190823.3")) 1xv3h6svw9aay5ixpql231md3pf00qxvhg62z88daraf18hlkfja (git-reference (url "https://github.com/OpenCPN/OpenCPN") (commit "v5.0.0")) (git-reference (url "https://github.com/OpenCPN/OpenCPN.git") (commit "v5.0.0")) --=-=-=--