From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: Paul Eggert Newsgroups: gmane.emacs.devel Subject: Re: Dataloss copying file using copy-file on RHEL 8. Date: Wed, 12 Feb 2020 17:32:55 -0800 Organization: UCLA Computer Science Department Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------6901160BFFB496FA0EBDA546" Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="54117"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 Cc: emacs-devel To: David Koppelman Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Feb 13 02:33:45 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1j23Nx-000DzD-4q for ged-emacs-devel@m.gmane-mx.org; Thu, 13 Feb 2020 02:33:45 +0100 Original-Received: from localhost ([::1]:45926 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j23Nw-0001ge-5k for ged-emacs-devel@m.gmane-mx.org; Wed, 12 Feb 2020 20:33:44 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:47720) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j23NF-0001DG-01 for emacs-devel@gnu.org; Wed, 12 Feb 2020 20:33:02 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j23ND-0006Kq-E7 for emacs-devel@gnu.org; Wed, 12 Feb 2020 20:33:00 -0500 Original-Received: from zimbra.cs.ucla.edu ([131.179.128.68]:54176) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j23ND-0006HS-54 for emacs-devel@gnu.org; Wed, 12 Feb 2020 20:32:59 -0500 Original-Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id E19651600B7; Wed, 12 Feb 2020 17:32:56 -0800 (PST) Original-Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id pjzuSlbh1oI0; Wed, 12 Feb 2020 17:32:56 -0800 (PST) Original-Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 0CD271600BA; Wed, 12 Feb 2020 17:32:56 -0800 (PST) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Original-Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id rRl8A8mXVAH8; Wed, 12 Feb 2020 17:32:55 -0800 (PST) Original-Received: from Penguin.CS.UCLA.EDU (Penguin.CS.UCLA.EDU [131.179.64.200]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id E1E661600B7; Wed, 12 Feb 2020 17:32:55 -0800 (PST) In-Reply-To: Content-Language: en-US X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 131.179.128.68 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:244917 Archived-At: This is a multi-part message in MIME format. --------------6901160BFFB496FA0EBDA546 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 2/12/20 2:37 PM, David Koppelman wrote: > Except I don't get the > efficient kernel-space-to-kernel-space transfer that copy_file_range > uses.) It's more than just kernel-space-to-kernel-space copying. When copying a file within an NFS server, you don't need to ship its contents over the network; the server can do the copy. Also, many modern filesystems can copy files by fiddling with pointers rather than data and thus can copy much faster than read+write would do, even on local filesystems. So avoiding copy_file_range entirely would mean a big performance loss on big files. > I do not experience the problem on the version of Emacs packaged with > rhel 8, "GNU Emacs 26.1 (build 1, x86_64-redhat-linux-gnu, GTK+ > Version 3.22.30) of 2018-09-10". Emacs 26.1 doesn't use copy_file_range, which explains why it doesn't encounter your problem. Emacs 27 is planned to use it, though, so we should see how to best fix the problem. As you say, it's a serious bug in your filesystem. It strikes me that it is likely to affect programs other than Emacs, so it should be high priority to fix regardless of what we do in Emacs. Some questions: What is the NFS fileserver (NetApp, etc.)? What's the blocksize on the remote file system? Does copy_file_range work correctly when the size is a multiple of 32*1024? If so, perhaps we could tweak Emacs to use copy_file_range for most of the file, and use read+write only for the trailing <32 KiB. > When I have time I'll try to reproduce the problem with a quick C++ > routine using copy_file_range. To save you some time, attached is a quick C routine that attempts to reproduce the problem. Does it reproduce the problem for you? If so, you can use it in your bug report to Red Hat. Also, can you strace the failing Emacs? Something like this: strace -o trace.log emacs -Q -batch -eval '(copy-file "a" "b" t t)' and then look at the relevant part of trace.log. --------------6901160BFFB496FA0EBDA546 Content-Type: text/x-csrc; charset=UTF-8; name="cfrbug.c" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="cfrbug.c" #define _GNU_SOURCE #include #include #include int main (void) { char buf[36368]; char abuf[sizeof buf]; memset (abuf, 'a', sizeof abuf); char bbuf[sizeof buf]; memset (bbuf, 'b', sizeof bbuf); int src = open ("src", O_WRONLY | O_CREAT | O_TRUNC, 0666); if (src < 0) return 2; if (write (src, abuf, sizeof buf) != sizeof buf) return 3; if (close (src) != 0) return 4; int dst = open ("dst", O_WRONLY | O_CREAT | O_TRUNC, 0666); if (dst < 0) return 5; if (write (dst, bbuf, sizeof buf) != sizeof buf) return 6; if (close (dst) != 0) return 7; src = open ("src", O_RDONLY); if (src < 0) return 8; dst = open ("dst", O_WRONLY); if (dst < 0) return 9; if (copy_file_range (src, 0, dst, 0, sizeof buf, 0) != sizeof buf) return 10; if (close (src) != 0) return 11; if (close (dst) != 0) return 12; dst = open ("dst", O_RDONLY); if (dst < 0) return 13; if (read (dst, buf, sizeof buf) != sizeof buf) return 14; if (memcmp (buf, abuf, sizeof buf) != 0) return 15; return 0; } --------------6901160BFFB496FA0EBDA546--