From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Newsgroups: gmane.lisp.guile.bugs Subject: bug#68504: [PATCH v3] Add copy-on-write support to scm_copy_file. Date: Tue, 12 Mar 2024 14:06:34 +0100 Message-ID: <878r2nmwf9.fsf@gnu.org> References: <20240116124817.14680-1-~@wolfsden.cz> <20240124191607.3571-1-~@wolfsden.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="4221"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: 68504-done@debbugs.gnu.org To: Tomas Volf <~@wolfsden.cz> Original-X-From: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Tue Mar 12 14:07:47 2024 Return-path: Envelope-to: guile-bugs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rk1r9-0000ua-1j for guile-bugs@m.gmane-mx.org; Tue, 12 Mar 2024 14:07:47 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rk1r0-0002oj-9j; Tue, 12 Mar 2024 09:07:38 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rk1qq-0002mW-6X for bug-guile@gnu.org; Tue, 12 Mar 2024 09:07:28 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rk1qp-0004rJ-TZ for bug-guile@gnu.org; Tue, 12 Mar 2024 09:07:27 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rk1rO-0005MU-3z for bug-guile@gnu.org; Tue, 12 Mar 2024 09:08:02 -0400 Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-To: bug-guile@gnu.org Resent-Date: Tue, 12 Mar 2024 13:08:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: cc-closed 68504 X-GNU-PR-Package: guile X-GNU-PR-Keywords: patch Mail-Followup-To: 68504@debbugs.gnu.org, ludo@gnu.org, ~@wolfsden.cz Original-Received: via spool by 68504-done@debbugs.gnu.org id=D68504.171024884320542 (code D ref 68504); Tue, 12 Mar 2024 13:08:01 +0000 Original-Received: (at 68504-done) by debbugs.gnu.org; 12 Mar 2024 13:07:23 +0000 Original-Received: from localhost ([127.0.0.1]:42019 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rk1ql-0005LG-26 for submit@debbugs.gnu.org; Tue, 12 Mar 2024 09:07:23 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:60186) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rk1qg-0005Kr-1S for 68504-done@debbugs.gnu.org; Tue, 12 Mar 2024 09:07:21 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rk1q1-0004kF-8G; Tue, 12 Mar 2024 09:06:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=5zszqepfYMDk91jZ37NC4r89H2kiZOxEpovRv31jfZQ=; b=aR5DoMA7I4VOYhb7RD3i Zui5KNclAWEM+azVtOoKGPrnFRux+InV5yrshpPeYF+F4kf362GJNDhFd4vUlpBQ4Bnnmgxb1t8c+ vd7jDGDuKRSi7sWnOfu7ANvNPFAOIT6gNLL4gbdmeMk7FQJ3/wIVej5d1Ogt6QknMe/JRqAdkyKWv /XRbyYF9lhhMgUOdmhZqZUutdBHWHqS3ol90En4k4ALFJkq96iE15RFkxtlQctulmap1oxI2Eg0RZ HqRej2PbmsA9OEGkUewjhq4YDDJ9qStCTLL1LYEphh0bO0Dqcx/b9SxUGb9oJ3D11asDtrj4PjyTL L04L3fn5mXe3ww==; In-Reply-To: <20240124191607.3571-1-~@wolfsden.cz> (Tomas Volf's message of "Wed, 24 Jan 2024 20:14:32 +0100") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Original-Sender: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.lisp.guile.bugs:10789 Archived-At: Hi Tomas, Tomas Volf <~@wolfsden.cz> skribis: > On modern file-systems (BTRFS, ZFS) it is possible to copy a file using > copy-on-write method. For large files it has the advantage of being > much faster and saving disk space (since identical extents are not > duplicated). This feature is stable and for example coreutils' `cp' > does use it automatically (see --reflink). > > This commit adds support for this feature into our copy-file procedure. > Same as `cp', it defaults to 'auto, meaning the copy-on-write is > attempted, and in case of failure the regular copy is performed. > > No tests are provided, because the behavior depends on the system, > underlying file-system and its configuration. That makes it challenging > to write a test for it. Manual testing was performed instead: > > $ btrfs filesystem du /tmp/cow* > Total Exclusive Set shared Filename > 36.00KiB 36.00KiB 0.00B /tmp/cow > > $ cat cow-test.scm > (copy-file "/tmp/cow" "/tmp/cow-unspecified") > (copy-file "/tmp/cow" "/tmp/cow-always" #:copy-on-write 'always) > (copy-file "/tmp/cow" "/tmp/cow-auto" #:copy-on-write 'auto) > (copy-file "/tmp/cow" "/tmp/cow-never" #:copy-on-write 'never) > (copy-file "/tmp/cow" "/dev/shm/cow-unspecified") > (copy-file "/tmp/cow" "/dev/shm/cow-auto" #:copy-on-write 'auto) > (copy-file "/tmp/cow" "/dev/shm/cow-never" #:copy-on-write 'never) > $ ./meta/guile -s cow-test.scm > > $ btrfs filesystem du /tmp/cow* > Total Exclusive Set shared Filename > 36.00KiB 0.00B 36.00KiB /tmp/cow > 36.00KiB 0.00B 36.00KiB /tmp/cow-always > 36.00KiB 0.00B 36.00KiB /tmp/cow-auto > 36.00KiB 36.00KiB 0.00B /tmp/cow-never > 36.00KiB 0.00B 36.00KiB /tmp/cow-unspecified > > $ sha1sum /tmp/cow* /dev/shm/cow* > 4c665f87b5dc2e7d26279c4b48968d085e1ace32 /tmp/cow > 4c665f87b5dc2e7d26279c4b48968d085e1ace32 /tmp/cow-always > 4c665f87b5dc2e7d26279c4b48968d085e1ace32 /tmp/cow-auto > 4c665f87b5dc2e7d26279c4b48968d085e1ace32 /tmp/cow-never > 4c665f87b5dc2e7d26279c4b48968d085e1ace32 /tmp/cow-unspecified > 4c665f87b5dc2e7d26279c4b48968d085e1ace32 /dev/shm/cow-auto > 4c665f87b5dc2e7d26279c4b48968d085e1ace32 /dev/shm/cow-never > 4c665f87b5dc2e7d26279c4b48968d085e1ace32 /dev/shm/cow-unspecified > > This commit also adds to new failure modes for (copy-file). > > Failure to copy-on-write when 'always was passed in: > > scheme@(guile-user)> (copy-file "/tmp/cow" "/dev/shm/cow" #:copy-on-w= rite 'always) > ice-9/boot-9.scm:1676:22: In procedure raise-exception: > In procedure copy-file: copy-on-write failed: Invalid cross-device li= nk > > Passing in invalid value for the #:copy-on-write keyword argument: > > scheme@(guile-user)> (copy-file "/tmp/cow" "/dev/shm/cow" #:copy-on-w= rite 'nevr) > ice-9/boot-9.scm:1676:22: In procedure raise-exception: > In procedure copy-file: invalid value for #:copy-on-write: nevr > > * NEWS: Add note for copy-file supporting copy-on-write. > * configure.ac: Check for linux/fs.h. > * doc/ref/posix.texi (File System)[copy-file]: Document the new > signature. > * libguile/filesys.c (clone_file): New function cloning a file using > FICLONE, if supported. > (k_copy_on_write): New keyword. > (sym_always, sym_auto, sym_never): New symbols. > (scm_copy_file2): Renamed from scm_copy_file. New #:copy-on-write > keyword argument. Attempt copy-on-write copy by default. > (scm_copy_file): Call scm_copy_file2. > * libguile/filesys.h: Add scm_copy_file2 as SCM_INTERNAL. > --- > v2: Introduce scm_copy_file2 in order to preserve backwards compatibility. > > v3: Remove mention of scm_copy_file from the commit message. Finally pushed as e1690f3fd251d69b3687ec12c6f4b41034047f0f. Note that I added copyright lines for you, let me know if I got it wrong. As a followup, we should add support for =E2=80=98copy_file_range=E2=80=99 = when FICLONE cannot be used; glibc supports it on all platforms but it returns ENOSYS on GNU/Hurd currently. WDYT? Thank you! Ludo=E2=80=99.