From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id AbcqBbBJHmFCqAAAgWs5BA (envelope-from ) for ; Thu, 19 Aug 2021 14:08:16 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id MLJeALBJHmE5QgAAB5/wlQ (envelope-from ) for ; Thu, 19 Aug 2021 12:08:16 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 336DA1BF76 for ; Thu, 19 Aug 2021 14:08:15 +0200 (CEST) Received: from localhost ([::1]:50236 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mGgqD-0001FC-Uc for larch@yhetil.org; Thu, 19 Aug 2021 08:08:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55760) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mGgq2-0001F4-C7 for bug-guix@gnu.org; Thu, 19 Aug 2021 08:08:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:47201) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mGgq2-0006PC-54 for bug-guix@gnu.org; Thu, 19 Aug 2021 08:08:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mGgq1-0002Lh-TY for bug-guix@gnu.org; Thu, 19 Aug 2021 08:08:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#50121: Deduplication breaks store item repair Resent-From: Lars-Dominik Braun Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 19 Aug 2021 12:08:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 50121 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 50121@debbugs.gnu.org X-Debbugs-Original-To: bug-guix@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.16293748418980 (code B ref -1); Thu, 19 Aug 2021 12:08:01 +0000 Received: (at submit) by debbugs.gnu.org; 19 Aug 2021 12:07:21 +0000 Received: from localhost ([127.0.0.1]:58747 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGgpM-0002Km-Sb for submit@debbugs.gnu.org; Thu, 19 Aug 2021 08:07:21 -0400 Received: from lists.gnu.org ([209.51.188.17]:53628) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGgpM-0002Kf-0Z for submit@debbugs.gnu.org; Thu, 19 Aug 2021 08:07:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55554) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mGgpL-0001Cq-SU for bug-guix@gnu.org; Thu, 19 Aug 2021 08:07:19 -0400 Received: from mout-p-102.mailbox.org ([2001:67c:2050::465:102]:43034) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_CHACHA20_POLY1305:256) (Exim 4.90_1) (envelope-from ) id 1mGgpJ-00060S-8q for bug-guix@gnu.org; Thu, 19 Aug 2021 08:07:19 -0400 Received: from smtp1.mailbox.org (smtp1.mailbox.org [80.241.60.240]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-102.mailbox.org (Postfix) with ESMTPS id 4Gr3RK1818zQjTy for ; Thu, 19 Aug 2021 14:07:09 +0200 (CEST) X-Virus-Scanned: amavisd-new at heinlein-support.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6xq.net; s=MBO0001; t=1629374827; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=hip4WWL0yj+LCidJ0ONV74EaBMrKYibwWR+yJaUJvEU=; b=fvJMZUHj71vWEA5d0aRAfwqf8uZyoTKqC/b2BeV26nOcGsqrQczLVggKZfTjppoZTWRNUL A/dlLM059GQXVRFDlkY3DwWcjjkzPqywiyxcUGI31FKIBzOs17wFUoXkiWpO5JG/GjZpTY wMklhAdF/Bm4+urq9/G8gN6HbssdJ+k8IU/maivgkQeES6logk463qc3m3NvdqVhD4euak XKEOEmGq94h/O/AyUT7r6sdnArb8xm0bbwE8jwaIAugOyFRmIxahOToWNZkqVDiYBPVv4y J/45y9ZxUvJIB+lKz9Epw9uNm0EPIifwXSWav3sivmH1dY1CrtMZKIXuzvLWhQ== Received: from smtp1.mailbox.org ([80.241.60.240]) by spamfilter06.heinlein-hosting.de (spamfilter06.heinlein-hosting.de [80.241.56.125]) (amavisd-new, port 10030) with ESMTP id z7SfcixetRIL for ; Thu, 19 Aug 2021 14:07:02 +0200 (CEST) Date: Thu, 19 Aug 2021 14:07:01 +0200 From: Lars-Dominik Braun Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="vxz+bEefRHM5YCrX" Content-Disposition: inline Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: E69C618C2 X-Rspamd-UID: 4c6aa9 Received-SPF: pass client-ip=2001:67c:2050::465:102; envelope-from=lars@6xq.net; helo=mout-p-102.mailbox.org X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1629374895; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=hip4WWL0yj+LCidJ0ONV74EaBMrKYibwWR+yJaUJvEU=; b=hHm2chXy/oygf1N+Qo36yX6OUyd3jKxnlxnZcaffhIgE0QvodTc9CQ0pbsiGGph5ICRsFk H3P3NuMpmkWw16oQgEUF/sBKjAh3oZmDm0IyoXuoE//BSXejSociDndQoaU0nbGPWZoBYX i8JiEir589rftRFG0STlx9VUiDt5ktGTcvFq6b7oVxfCGFQFTmd8PUS/nIBeZY9H4gkT7A BPBywtcMluAA9pXRT0ALfsQS0AXOiq6wdaU96wPQDkk6P8HdYn4nQ9MyUH8i2EdbZVnZ1f FIZhRpJE1CyO7ICf5CyVleZCFfhPnr6t2CBjKwzX+eCoPajEvvj5mJ6of9cwSQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1629374895; a=rsa-sha256; cv=none; b=UmiZ1iy0o1uzqKYPNeqD2v/fjQsNcA1O+X8GON5DQXRUBAitQ2JqNlSEYKz9JtNrV6LDKS k9pa08xmJs0i26REFibgW2VPp/Q++NyTgykmWG10wzK6/chNMuw7M8+vIpa6UcRnw7Jgop LWaopt6VKtHE2KUo43RzNgAMQl2m4ZQWvP9nL86Qhj+SRoPOLYEwIH8bjNuLL7i6iT4QgL bJMyuRgJvIP9fS0eJcCFFEOZ1shmoe4d664ME7D5c22VA6+cDwz1EWIAIUrfTiXxcRJoCC /OIX8YLRATKtSZ7MMEMOYgqPLefdRej99fIhe7cOvz47vQB1mFwDpJyGJvWnRg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=6xq.net header.s=MBO0001 header.b=fvJMZUHj; dmarc=fail reason="SPF not aligned (relaxed)" header.from=6xq.net (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Spam-Score: -1.22 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=6xq.net header.s=MBO0001 header.b=fvJMZUHj; dmarc=fail reason="SPF not aligned (relaxed)" header.from=6xq.net (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Queue-Id: 336DA1BF76 X-Spam-Score: -1.22 X-Migadu-Scanner: scn0.migadu.com X-TUID: 25QEQ4kDq5YM --vxz+bEefRHM5YCrX Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Hi, I’ve been having issues with the filesystem that holds /gnu/store recently, causing corrupted/broken files. When trying to repair these broken files with `guix gc --verify=repair,contents` it properly detects that store items’ hashes do not match the ones recorded in the database and redownloads/rebuilds them. However, the corrupted store items are never actually repaired – not by `guix gc` and not by `guix build --repair`. Attached is a testcase showing that deduplication is the problem, because repairing will just hardlink the (broken) deduplicated file instead of replacing it with the downloaded/built file. I tried the daemon’s `--disable-deduplication` too with same results. Cheers, Lars --vxz+bEefRHM5YCrX Content-Type: application/x-sh Content-Disposition: attachment; filename="testcase.sh" Content-Transfer-Encoding: quoted-printable # use =E2=80=9Chello=E2=80=9D as a test subject=0Apath=3D`guix build hello`= =0Ahello=3D$path/bin/hello=0Ahellohash=3D`guix hash -r $hello`=0Als -lh $he= llo=0Aecho "$hello $hellohash"=0A# =E2=80=9Ccorrupt=E2=80=9D the file=0Atru= ncate --size=3D0 $hello=0Als -lh $hello=0A# Now try to repair it=0Aguix gc = --verify=3Drepair,contents=0A# But nothing happens=0Als -lh $hello=0A# Try = again=0Aguix build --repair hello=0A# Again nothing happens=0Als -lh $hello= =0A# Fix the problem by removing the broken file from the deduplication=0A#= store.=0Arm /gnu/store/.links/$hellohash=0Aguix build --repair hello=0A# A= nd it=E2=80=99s fine again.=0Als -lh $hello=0A --vxz+bEefRHM5YCrX Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename=log Content-Transfer-Encoding: 8bit /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10/bin/hello 0jxnp4f4rac2scvq9lhcvpr4n4w0zrx5wdhcqml4w7zfybbszswp -r-xr-xr-x 2 root root 0 19. Aug 13:54 /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10/bin/hello reading the store... checking path existence... checking hashes... path `/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10' was modified! expected hash `9c61184c4b1af09639cee8148bc0c3d7aced4a671615a6e0a3e7ccb927848ffa', got `3330b928ba2d3cf6acfdb0ef3a359fb686eac7ee6e9d49a6515ef7b1701537cf' fetching path `/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10'... Downloading https://ci.guix.gnu.org/nar/lzip/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10... hello-2.10 51KiB 528KiB/s 00:00 [##################] 100.0% -r-xr-xr-x 2 root root 0 1. Jan 1970 /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10/bin/hello path `/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10' is corrupted or missing! Substituiere /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10 … Lade von https://ci.guix.gnu.org/nar/lzip/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10 herunter … hello-2.10 51KiB 482KiB/s 00:00 [##################] 100.0% /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10 -r-xr-xr-x 2 root root 0 1. Jan 1970 /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10/bin/hello path `/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10' is corrupted or missing! Substituiere /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10 … Lade von https://ci.guix.gnu.org/nar/lzip/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10 herunter … hello-2.10 51KiB 455KiB/s 00:00 [##################] 100.0% /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10 -r-xr-xr-x 2 root root 37K 1. Jan 1970 /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10/bin/hello --vxz+bEefRHM5YCrX--