From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Paul Eggert Newsgroups: gmane.emacs.bugs Subject: bug#23904: Btrfs clone support in copy operations Date: Wed, 6 Jul 2016 13:48:26 +0200 Message-ID: <577CF00A.80707@cs.ucla.edu> References: <1467745602.28158.17.camel@kcolford.com> <577C5588.3060608@cs.ucla.edu> <7d9c224b-e525-de33-426e-15bc328e7108@yandex.ru> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------030504080808070801020508" X-Trace: ger.gmane.org 1467805855 17793 80.91.229.3 (6 Jul 2016 11:50:55 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 6 Jul 2016 11:50:55 +0000 (UTC) Cc: sbaugh@catern.com, Kieran Colford To: Dmitry Antipov , 23904@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed Jul 06 13:50:41 2016 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bKlLj-0007GT-Po for geb-bug-gnu-emacs@m.gmane.org; Wed, 06 Jul 2016 13:50:40 +0200 Original-Received: from localhost ([::1]:32898 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bKlLj-00045p-0R for geb-bug-gnu-emacs@m.gmane.org; Wed, 06 Jul 2016 07:50:39 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:54980) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bKlKE-0002ei-8c for bug-gnu-emacs@gnu.org; Wed, 06 Jul 2016 07:49:07 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bKlKA-0005RZ-3h for bug-gnu-emacs@gnu.org; Wed, 06 Jul 2016 07:49:05 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:54780) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bKlK9-0005RM-Ve for bug-gnu-emacs@gnu.org; Wed, 06 Jul 2016 07:49:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1bKlK9-000588-Qe for bug-gnu-emacs@gnu.org; Wed, 06 Jul 2016 07:49:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 06 Jul 2016 11:49:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23904 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 23904-submit@debbugs.gnu.org id=B23904.146780572619695 (code B ref 23904); Wed, 06 Jul 2016 11:49:01 +0000 Original-Received: (at 23904) by debbugs.gnu.org; 6 Jul 2016 11:48:46 +0000 Original-Received: from localhost ([127.0.0.1]:38884 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bKlJt-00057b-JW for submit@debbugs.gnu.org; Wed, 06 Jul 2016 07:48:45 -0400 Original-Received: from zimbra.cs.ucla.edu ([131.179.128.68]:58644) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bKlJr-00057N-9w for 23904@debbugs.gnu.org; Wed, 06 Jul 2016 07:48:43 -0400 Original-Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 487A51614BC; Wed, 6 Jul 2016 04:48:37 -0700 (PDT) Original-Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id RfVVPXUhf2Ip; Wed, 6 Jul 2016 04:48:33 -0700 (PDT) Original-Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 732241614C8; Wed, 6 Jul 2016 04:48:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Original-Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 5caQhV2k36Ig; Wed, 6 Jul 2016 04:48:33 -0700 (PDT) Original-Received: from [192.168.0.35] (89-159-79-138.rev.numericable.fr [89.159.79.138]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 5F9531614BC; Wed, 6 Jul 2016 04:48:32 -0700 (PDT) User-Agent: Mozilla/5.0 (X11; Linux i686; rv:38.0) Gecko/20100101 Thunderbird/38.8.0 In-Reply-To: <7d9c224b-e525-de33-426e-15bc328e7108@yandex.ru> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:120483 Archived-At: This is a multi-part message in MIME format. --------------030504080808070801020508 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 07/06/2016 04:45 AM, Dmitry Antipov wrote: > IMHO this should use FICLONERANGE in a loop where you can > handle errors and use process_pending_signals as usual. Thanks, that suggestion should resolve the FIXME about how FICLONE behaves when the output file is already larger than the input. However, what should the clone chunk size be,each time through the loop? I don't use btrfs so I don't know what a good size would be. A downside of the suggestion is that the file copy won't be atomic. However, the existing code isn't atomic, so this is nothing new. Revised patch attached. Basically untested, since I don't use any file systems that support FIOCLONERANGE. The patch still has FIXMEs for what happens if the FIOCLONERANGE ioctl is interrupted by a signal, and for a good clone chunk size (the patch guesses 1 GiB). --------------030504080808070801020508 Content-Type: text/x-patch; name="0001-copy-file-now-uses-GNU-Linux-file-cloning.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-copy-file-now-uses-GNU-Linux-file-cloning.patch" >From 9c60f6f603563c335737102fc54609b837f5d02a Mon Sep 17 00:00:00 2001 From: Paul Eggert Date: Wed, 6 Jul 2016 12:44:46 +0200 Subject: [PATCH] copy-file now uses GNU/Linux file cloning >From a suggestion by Kieran Colford (see Bug#23904). * configure.ac: Check for linux/fs.h. * src/fileio.c [HAVE_LINUX_FS_H]: Include sys/ioctl.h and linux/fs.h. (clone_file_range): New function. (Fcopy_file): Use it. --- configure.ac | 1 + src/fileio.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++------------- 2 files changed, 50 insertions(+), 13 deletions(-) diff --git a/configure.ac b/configure.ac index aaddfcd..1798fa3 100644 --- a/configure.ac +++ b/configure.ac @@ -1636,6 +1636,7 @@ AC_DEFUN dnl checks for header files AC_CHECK_HEADERS_ONCE( + linux/fs.h malloc.h sys/systeminfo.h sys/sysinfo.h diff --git a/src/fileio.c b/src/fileio.c index b1f9d3c..f980247 100644 --- a/src/fileio.c +++ b/src/fileio.c @@ -52,6 +52,11 @@ along with GNU Emacs. If not, see . */ #include "region-cache.h" #include "frame.h" +#ifdef HAVE_LINUX_FS_H +# include +# include +#endif + #ifdef WINDOWSNT #define NOMINMAX 1 #include @@ -1828,6 +1833,23 @@ barf_or_query_if_file_exists (Lisp_Object absname, bool known_to_exist, } } +/* Copy data to DEST from SOURCE if possible, starting at byte offset + OFF and continuing for LENGTH bytes. Return true if OK. */ +static bool +clone_file_range (int dest, int source, off_t off, off_t length) +{ +#ifdef FICLONERANGE + /* FIXME: Is the ioctl interruptible? If so, what happens if the + ioctl is interrupted by a signal? Might it partially copy the + range? */ + struct file_clone_range range = { src_fd: source, src_offset: off, + src_length: length, dest_offset: off }; + if (ioctl (dest, FICLONERANGE, &range) == 0) + return true; +#endif + return false; +} + DEFUN ("copy-file", Fcopy_file, Scopy_file, 2, 6, "fCopy file: \nGCopy %s to file: \np\nP", doc: /* Copy FILE to NEWNAME. Both args must be strings. @@ -1974,7 +1996,7 @@ permissions. */) record_unwind_protect_int (close_file_unwind, ofd); - off_t oldsize = 0, newsize = 0; + off_t oldsize = 0, newsize; if (already_exists) { @@ -1990,18 +2012,32 @@ permissions. */) immediate_quit = 1; QUIT; - while (true) - { - char buf[MAX_ALLOCA]; - ptrdiff_t n = emacs_read (ifd, buf, sizeof buf); - if (n < 0) - report_file_error ("Read error", file); - if (n == 0) - break; - if (emacs_write_sig (ofd, buf, n) != n) - report_file_error ("Write error", newname); - newsize += n; - } + + off_t insize = st.st_size; + /* FIXME: is 1 GiB a good number? */ + off_t max_clone_chunk_size = 1 << 30; + off_t chunk_size = min (insize, max_clone_chunk_size); + + if (clone_file_range (ofd, ifd, 0, chunk_size)) + for (newsize = chunk_size; newsize < insize; newsize += chunk_size) + { + chunk_size = min (insize - newsize, max_clone_chunk_size); + if (! clone_file_range (ofd, ifd, newsize, chunk_size)) + report_file_error ("Clone error", file); + } + else + for (newsize = 0; ; newsize += chunk_size) + { + char buf[MAX_ALLOCA]; + ptrdiff_t n = emacs_read (ifd, buf, sizeof buf); + if (n < 0) + report_file_error ("Read error", file); + if (n == 0) + break; + if (emacs_write_sig (ofd, buf, n) != n) + report_file_error ("Write error", newname); + chunk_size = n; + } /* Truncate any existing output file after writing the data. This is more likely to work than truncation before writing, if the -- 2.5.5 --------------030504080808070801020508--