From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id EOS9OaqwbmDsHgAAgWs5BA (envelope-from ) for ; Thu, 08 Apr 2021 09:28:42 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id qFKGM6qwbmAjSwAAbx9fmQ (envelope-from ) for ; Thu, 08 Apr 2021 07:28:42 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 97055238A8 for ; Thu, 8 Apr 2021 09:28:40 +0200 (CEST) Received: from localhost ([::1]:54534 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lUP5i-00064e-4e for larch@yhetil.org; Thu, 08 Apr 2021 03:28:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60352) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lUP5f-00062u-FQ for gwl-devel@gnu.org; Thu, 08 Apr 2021 03:28:35 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:57233) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lUP5d-0004JU-7x for gwl-devel@gnu.org; Thu, 08 Apr 2021 03:28:35 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 3FB345C00A8; Thu, 8 Apr 2021 03:28:31 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 08 Apr 2021 03:28:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.net; h= from:to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-type; s=fm3; bh=XlwJVXh90s1nmy6h5nAWFkwAiB QEejgQ6x9XkcGJmWw=; b=JQ7996hPsXcZUCWe9UCVoz8HsYxd1ktDWWc8pkaw1f 34XEgYLA87ngmQGwvQcWFBFaAjBoT8wKDUqJZZMWU1XRUN3Zu5iFFieAzs5lafG1 FhgKf2hP1Xx5xcmUYWJL34dv4eaVPRE0ld1YTbguSNnymL9Jj5IQzW3oEB+p4KLD FD6DwiSc8CFnnp2DrB9hl00mvbZtxfuGnpJ61sd/8gKxnggK77wddw+cv+kyrdWJ pmf8bZwAijjzUYe9Emz09ykplhWWMwQu6NhsQClmUu5OIew2homCpGRQGvuuaXsN rJEg0bVAonGMjfPoldHaEaO5xai00KySzBPrW6k3jQHA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=XlwJVX h90s1nmy6h5nAWFkwAiBQEejgQ6x9XkcGJmWw=; b=QYyWVVx3CwC/buVDZ4YOLB 8g/u3CL7gTB1UtCsXpN4uf74gDFcde7dQvU2dyMXzlgSJlVw3VB5aCd8LlP7SPRg JUl9ufmoVjZT3XiUVsmitPUsVlxQUPrPsKjBEz4fIIvR0PtCO52Rk0oBLNH2obuP +gkd3mV7weyMU8rGxsCpMrHJ5foR01bTgUR2y6Ut8HlnFSk5siALbLKmN3i8dNZW aWFqo0N5bfEgMEAhgjyBxLSNI3QoXPpFFi4XDx5ZV9o4EtxbslICGLpBZINa50M/ evOvB+/o/A+05P2G3iFjrymxKEAnbQ6m8XIOVLN0mnH5dlbKhokLd2/0W/qwpByA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudejkedguddvtdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhephffvufgjfhffkfggtgesthdtre dttddttdenucfhrhhomhepmfhonhhrrgguucfjihhnshgvnhcuoehkohhnrhgrugdrhhhi nhhsvghnsehfrghsthhmrghilhdrnhgvtheqnecuggftrfgrthhtvghrnhepieekjeejve euieefjeehtdegtefhudetgfevfffhueehjeevieeigedvhfdtieefnecukfhppeekvddr uddvfedrfeegrdduudejnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepkhhonhhrrggurdhhihhnshgvnhesfhgrshhtmhgrihhlrdhnvght X-ME-Proxy: Received: from ordinateur-de-catherine--konrad.home (lfbn-idf2-1-313-117.w82-123.abo.wanadoo.fr [82.123.34.117]) by mail.messagingengine.com (Postfix) with ESMTPA id 4A5A9240057; Thu, 8 Apr 2021 03:28:30 -0400 (EDT) From: Konrad Hinsen To: Ricardo Wurmus Subject: Re: Managing data files in workflows In-Reply-To: <87blaqz5mq.fsf@elephly.net> References: <87r1k2ti7k.fsf@elephly.net> <87czvmt5w3.fsf@elephly.net> <87h7kq2kzy.fsf@elephly.net> <87blaqz5mq.fsf@elephly.net> Date: Thu, 08 Apr 2021 09:28:29 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=66.111.4.28; envelope-from=konrad.hinsen@fastmail.net; helo=out4-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: gwl-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gwl-devel@gnu.org Errors-To: gwl-devel-bounces+larch=yhetil.org@gnu.org Sender: "gwl-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1617866922; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=XlwJVXh90s1nmy6h5nAWFkwAiBQEejgQ6x9XkcGJmWw=; b=qY0Nl225qmKZK83qysaF9ffxCUJGblUVyCzWgfKm2iyFM0S/aU+f3wgCWGiX/VPEeYa5s4 to705o6uo2QHxssRKK8asK7BZWRvaqSOugsber2FCuH0RcD45tu4p21bc49twhvdbzjQoF N5xv8XB+7iuDBaxfhnqfO6NmhECSj0lymkeVwfDaPvKU3/lzHsWLB9NTM/Dqf3fYkSFEev pApoaqB2cs6/sublZ/GTm6KULVnITWBqtpkOwJfwkzqVxHPLXf2gDNfOvpx86UeTdpU3Fr eJfOwha5PMJq0CLO3ZIfbuAqFuAO7Va+AGvohqwL8iEda2/uff9QS3DlZiwbwA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1617866922; a=rsa-sha256; cv=none; b=DHcnmofF30lwXeY0Sa3JQ0Gr0wnzqkf+1z/AnjhWRghDA6wzCP2OCVyKiZDCDVhfpNkqI8 6GWUaVz/99czg3/S4pgjrQ1JzUaCCtn8Hh0kuDHzcdyduOZfk8jpiJWVpv4pR1oYLdY2ax A6+1ZmH6JsSQZNiH2eSFuJUKZCM3hXd1g4BEHceWSYXgMPomkeasFAnsPiTWZ664kcPoL4 cueXo01BNCzK7WwdKD6G4Tk9Wc/7nDcsBkuqABjWsgjzCUyipBSPmaURn4kqpsueXbVrdA jMAaOFphxDikHvNqkvtBNdoCNRNjmRaQHtLxVDeHhOc3qazygEYWhc9ynYdWLQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=fastmail.net header.s=fm3 header.b=JQ7996hP; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=QYyWVVx3; spf=pass (aspmx1.migadu.com: domain of gwl-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=gwl-devel-bounces@gnu.org X-Migadu-Spam-Score: -3.14 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=fastmail.net header.s=fm3 header.b=JQ7996hP; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=QYyWVVx3; dmarc=pass (policy=none) header.from=fastmail.net; spf=pass (aspmx1.migadu.com: domain of gwl-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=gwl-devel-bounces@gnu.org X-Migadu-Queue-Id: 97055238A8 X-Spam-Score: -3.14 X-Migadu-Scanner: scn0.migadu.com X-TUID: R1PiRuuRvHmn Hi Ricardo, > We can fix the problem with symlinks by restoring the target of the link > instead of the link itself, but I feel that we need to take a step back > and consider what this cache is really to be used for. Indeed, and I have to admit that this isn't clear to me yet. What is it supposed to protect against? Modification of files by other processes of the workflow? Modification of files outside of the workflow? Both? For the second situation (modification outside of the workflow), I think it would be sufficient to store a checksum, and terminate the workflow with an error if it detects such tampering. The first situation is more difficult. There are actually two cases: 1. The workflow intentionally updates files as it proceeds. 2. The workflow modifies a file by mistake. Only the workflow author can make the distinction, so this needs some specific input syntax. Case 2 could then again be handled by a simple checksum test for signalling an error. This leaves case 1, for which the only good solution is to make a copy of the file at the end of each process, and restore it in later runs. Cheers, Konrad.