From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id INPaE0buP2CzTQAA0tVLHw (envelope-from ) for ; Wed, 03 Mar 2021 20:15:02 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id ELCeD0buP2BpRwAAbx9fmQ (envelope-from ) for ; Wed, 03 Mar 2021 20:15:02 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 5138824115 for ; Wed, 3 Mar 2021 21:15:01 +0100 (CET) Received: from localhost ([::1]:33146 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lHXtc-0006am-EB for larch@yhetil.org; Wed, 03 Mar 2021 15:15:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:54812) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lHXtT-0006Zw-EO for help-guix@gnu.org; Wed, 03 Mar 2021 15:14:51 -0500 Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]:34823) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lHXtR-0000Kh-N1 for help-guix@gnu.org; Wed, 03 Mar 2021 15:14:51 -0500 Received: by mail-wr1-x42c.google.com with SMTP id l12so25107276wry.2 for ; Wed, 03 Mar 2021 12:14:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=beadling-co-uk.20150623.gappssmtp.com; s=20150623; h=references:user-agent:from:to:cc:subject:message-id:in-reply-to :date:mime-version; bh=qEFTpx9QqJWC1tqysG+lsI1B7XUWPte88tWj2hjKjj0=; b=NXvhfaR+dBzB01wUcm884jwgFfks5iWD+gGQPCCxJhe5LoltIzpvJ0yVNm8OinjMqk OLTA9JgZ1AbTYxuxAPN/uBKxyPnjDkhB9l2IDduAv37xD36rAw2N1i0VOVvCL6st8P0J 6eXpJDs2ZbpkuD6uvqBUxOytk/7FDHZMLdTJUax9A9LK/NSY7AVom7QsmD1TmmD+um37 eVS66NynCp7PNhBiTFSGutOUrrIU6a6sktV3E40iSHLucgwlF8Riw4qcI1faaVGN+SCX lphSSww+0RJkWUQLW292G+h3wSDCMEpHEikjTEVFso8BwE2AobJw56Q4EtUtNCMtzQyq 8ZLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:references:user-agent:from:to:cc:subject :message-id:in-reply-to:date:mime-version; bh=qEFTpx9QqJWC1tqysG+lsI1B7XUWPte88tWj2hjKjj0=; b=ZOI9da1YQTjL/ROzNhKDGpiqRmRmPcySbnA2Rso5//8CPf9gv1jYJ/l1yLxiCyYSk3 riVQGAUpbTmAggCGs/hopN6yFDJK3oCyVoOj6FMDct1/jpL5HirCd0Oq67zWNdk3Bv5k f0jweqgbEMbTxLEiPNhtgiTnL1zC16TwFF0C7PH64Pt/ILHo8hLyFQBtpOCPgzHifJ0o Lqb5CFCCrRSpJtowUd65EwbJjDW1ZPDgsr2yjlztAyYNUnn2t+dNvIcI/EfKKI/CzBWF iKNdg4jsCJhMic5OBXuoAA27lMwyWV2LMzBUOHZ/aFAmePzEaLB+pkMjAqTMANKBYAt8 9LqQ== X-Gm-Message-State: AOAM533HQfg7JepNzVDYnPZq/zQ7N306/3hIsxQl8/fU8Bpsc4DWLVB4 SO774bMqQZG27G6QJeczXfvbMf9QP+hSVG4uA84= X-Google-Smtp-Source: ABdhPJy8lw6MAtRk9ZiD6CmqsH+cqXxa8DUL63bHP57675CsrO/Vzya1YlX3OA5B/jJei6vr29aOOQ== X-Received: by 2002:adf:ce0a:: with SMTP id p10mr347629wrn.255.1614802487704; Wed, 03 Mar 2021 12:14:47 -0800 (PST) Received: from phil-XPS-13-9360 (host-78-147-225-8.as13285.net. [78.147.225.8]) by smtp.gmail.com with ESMTPSA id t14sm34829937wru.64.2021.03.03.12.14.46 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 03 Mar 2021 12:14:46 -0800 (PST) References: <85mtvkmjvr.fsf@beadling.co.uk> <878s74yyyy.fsf@nckx> User-agent: mu4e 1.2.0; emacs 26.3 From: Phil To: Tobias Geerinckx-Rice Subject: Re: Moving /gnu/store safely? Message-ID: <85im68m1uf.fsf@beadling.co.uk> In-reply-to: <878s74yyyy.fsf@nckx> Date: Wed, 03 Mar 2021 20:14:46 +0000 MIME-Version: 1.0 Content-Type: text/plain Received-SPF: none client-ip=2a00:1450:4864:20::42c; envelope-from=phil@beadling.co.uk; helo=mail-wr1-x42c.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: help-guix@gnu.org Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1614802501; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=qEFTpx9QqJWC1tqysG+lsI1B7XUWPte88tWj2hjKjj0=; b=qAJLFBCL3FIsq4gN+EEIbrgp2YowOEnNijF/6wCBiWQPWolurj2sRc4DF/W+SmmJv5DA6A nzGtr0U1UyezO5MG2vgNHte1+W5TJMIiWYW8mDvOG6RqZLdC7M9wvvYrNVwTtgyb9xMh0l qjutg9zv836BQrlY+oa8NRJOwkifhOlt3HLsckkctXsVMTx8nmYkofP9PCXHNUcdOXsxNw igZK9WlKqZcA6ZhZ8I9aszOGKacTVo9yMX/73Pv5omDZFOdHeOzLN3COrPjsVyWAQfjSm7 4IIqHeMNWW3G2/+DTHJ8e3YaTgtUupl2Axv/JQr7rQz874zZycmgw+SD49z+gw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1614802501; a=rsa-sha256; cv=none; b=CsUFYexntaAuYsEyZ1zGm4hz2pAuGOlyhF379muUM4tLC2SGmJ8imdRsWxcn2u+OeLVm5Z YeTHXSsz7TACvAC4uf+k9Etjr3FpVzDcpLRIOGoqXSFkFnR3fO2t9cAJWk/OxLfdgwTxOq VzadVxer9Ob47dypMva/hVPFKlIzeRaqDY7WyKkhQELhWbKobOTAlkwtmVrs0Y138EiwB0 gBwaR2tZ00oAEEchO7nPxXX/xvd0K0ulAOPX9JLm2lvyDLVk5pj36GQdwldlSfme5otNFb dn4kNR5x8onSSlKHIrcTh6hGETbQG6Y7xXe4sopVLsGzP2ClYdUrTL29YooW5Q== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=beadling-co-uk.20150623.gappssmtp.com header.s=20150623 header.b=NXvhfaR+; spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Migadu-Spam-Score: -1.56 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=beadling-co-uk.20150623.gappssmtp.com header.s=20150623 header.b=NXvhfaR+; dmarc=none; spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Migadu-Queue-Id: 5138824115 X-Spam-Score: -1.56 X-Migadu-Scanner: scn1.migadu.com X-TUID: Ekt3C1Cbv9ec Thanks v much for the explanations. Tobias Geerinckx-Rice writes: > As gremlin's go this one is pretty tame, even cuddly: the Guix daemon > deduplicates identical files, and creates hard links instead, saving a > good amount of space. That's all. > > By default, rsync does not preserve hard links (because of potential Heh heh - yes I was surprised (and relieved) how quickly this gremlin was dealt with just by scanning the manual and throwing some commands together - this makes perfect sense so each hardlink just becomes a new file on the target. I'm assuming the doubling in size is more fluke than by design? i.e. there is no reason to assume each file is duplicated exactly once in Guix? It just turns out that hard linking was saving approx 50% by chance in my specific store - it could have been 30% or 70% or whatever and depends on your store contents? > No, the links were *lost* and replaced by separate files during > rsyncing. > Got it. At the risk of speculating, would you expect the subsequent execution of the garbage collector to deduplicate the identical rsync'd files again? I'm 100% clear on why the problem happened, just want to check I understand why it went away which I ran 'guix gc'. If I look at the link count on the files in my new /gnu/store I can see some files have a count >1, which certainly suggests that Guix has performed some sort of deduplication after the rsync?