From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Paul Eggert Newsgroups: gmane.emacs.bugs Subject: bug#36940: tests slowness and failure after recent Tramp changes Date: Sun, 25 Aug 2019 13:36:46 -0700 Organization: UCLA Computer Science Department Message-ID: References: <57e233fe-19e9-6eac-7fd5-7e181879d4df@cs.ucla.edu> <87h86uh2l6.fsf@gmx.de> <1bec46f5-01e7-5ea4-5562-501185d80ecf@cs.ucla.edu> <87v9vaf4sk.fsf@gmx.de> <87r25xf1dn.fsf@gmx.de> <8736ibkbne.fsf@gmx.de> <87mughfj96.fsf@gmx.de> <470b50ab-6658-e3cb-cd59-916c398c442f@cs.ucla.edu> <87h86of1tm.fsf@gmx.de> <87v9v0avhm.fsf@gmx.de> <8736hryoi1.fsf@gmx.de> <8736hpxy87.fsf@gmx.de> <661631ef-57ff-a907-d69c-b78aebfa68b9@cs.ucla.edu> <87y2zhw6eq.fsf@gmx.de> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------A76796BFFCF3F51395F3812D" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="46886"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 Cc: Stefan Kangas , 36940@debbugs.gnu.org To: Michael Albinus Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sun Aug 25 22:37:18 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1i1zGE-000C1n-BM for geb-bug-gnu-emacs@m.gmane.org; Sun, 25 Aug 2019 22:37:14 +0200 Original-Received: from localhost ([::1]:47712 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i1zGC-0007jX-Hf for geb-bug-gnu-emacs@m.gmane.org; Sun, 25 Aug 2019 16:37:12 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:49690) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i1zG3-0007hI-CX for bug-gnu-emacs@gnu.org; Sun, 25 Aug 2019 16:37:04 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1i1zG2-00030S-2T for bug-gnu-emacs@gnu.org; Sun, 25 Aug 2019 16:37:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:36320) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1i1zG1-00030L-U8 for bug-gnu-emacs@gnu.org; Sun, 25 Aug 2019 16:37:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1i1zG1-0002Xk-QB for bug-gnu-emacs@gnu.org; Sun, 25 Aug 2019 16:37:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 25 Aug 2019 20:37:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36940 X-GNU-PR-Package: emacs Original-Received: via spool by 36940-submit@debbugs.gnu.org id=B36940.15667654179765 (code B ref 36940); Sun, 25 Aug 2019 20:37:01 +0000 Original-Received: (at 36940) by debbugs.gnu.org; 25 Aug 2019 20:36:57 +0000 Original-Received: from localhost ([127.0.0.1]:45141 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i1zFw-0002XP-Vr for submit@debbugs.gnu.org; Sun, 25 Aug 2019 16:36:57 -0400 Original-Received: from zimbra.cs.ucla.edu ([131.179.128.68]:39204) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i1zFu-0002X7-FU for 36940@debbugs.gnu.org; Sun, 25 Aug 2019 16:36:55 -0400 Original-Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 31220160057; Sun, 25 Aug 2019 13:36:48 -0700 (PDT) Original-Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id Gxp4GIpvzweH; Sun, 25 Aug 2019 13:36:47 -0700 (PDT) Original-Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 34418160079; Sun, 25 Aug 2019 13:36:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Original-Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id zYrwuKjr3Kj8; Sun, 25 Aug 2019 13:36:47 -0700 (PDT) Original-Received: from [192.168.1.9] (cpe-23-242-74-103.socal.res.rr.com [23.242.74.103]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id F1DE5160057; Sun, 25 Aug 2019 13:36:46 -0700 (PDT) In-Reply-To: <87y2zhw6eq.fsf@gmx.de> Content-Language: en-US X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:165855 Archived-At: This is a multi-part message in MIME format. --------------A76796BFFCF3F51395F3812D Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Michael Albinus wrote: > At which places in the code do you believe such errors will happen? The > floating number will be converted in tramp-convert-file-attributes, I > cannot see how a rounding error could happen there. I think the rounding error doesn't happen there, as that code accurately converts the Lisp float to an integer or to a cons of integers. The rounding error happens in functions like tramp-send-command-and-read when the Emacs Lisp reader scans the string generated by the Perl code, and converts the string to a Lisp float. For example, suppose the inode number is 2**63 - 512 and the Perl code generates "9223372036854775296.0", which is exact. When the Emacs Lisp reader converts this to double it must round since there is no exact double representation, and rounding yields 2**63, i.e., 9223372036854775808.0, which is off by 512. A simple fix is to change the Perl code to omit the trailing ".0", e.g., "9223372036854775296" rather than "9223372036854775296.0". This will cause the Emacs Lisp reader in master to return an exact integer instead of a float, thus avoiding the rounding error. This fix will still suffer from the same rounding errors in older Emacs releases where the reader returns a float for integers out of fixnum range, but it shouldn't hurt those older releases and it should fix the problem in master. Please see attached patch. I haven't tested or installed the patch as I don't use Tramp, but you should be able to test it with something like this: $ echo '' | dd obs=1 seek=9007199254740992 of=file 0+1 records in 1+0 records out 1 byte (1 B) copied, 0.015429 seconds, 0.1 kB/s $ ls -l file -rw-rw-r-- 1 eggert faculty 9007199254740993 2019-08-25 13:28 file assuming your filesystem supports files containing more than 2**53 bytes - since there's a similar bug for file sizes. > I haven't seen any code in Emacs which needs this slot. It's used by ede--inode-for-dir, eshell-shuffle-files, ls-lisp-format, find-lisp-format, nnmaildir--group-maxnum, nnmaildir--new-number. Typical uses are to determine whether two directory entries identify the same file, e.g., (equal (file-attribute-inode-number file1) (file-attribute-inode-number file2)) or to use it in a hash table. As I mentioned above, a similar bug occurs for the file size slot: files containing more than 2**53 bytes have rounding errors in their sizes. The attached patch should fix that too. --------------A76796BFFCF3F51395F3812D Content-Type: text/x-patch; name="0001-Fix-Tramp-rounding-of-file-sizes-and-inode-numbers.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0001-Fix-Tramp-rounding-of-file-sizes-and-inode-numbers.patc"; filename*1="h" >From b8242feb8a2b70ce28a72d3976d2c34ce664d0b3 Mon Sep 17 00:00:00 2001 From: Paul Eggert Date: Sun, 25 Aug 2019 12:27:54 -0700 Subject: [PATCH] Fix Tramp rounding of file sizes and inode numbers * lisp/net/tramp-sh.el (tramp-perl-file-attributes) (tramp-perl-directory-files-and-attributes): Format file sizes and inode numbers with "%u" instead of "%u.0", to avoid rounding errors when absolute values exceed 2**53 (Bug#36940#94). This fixes the problem for Emacs 27 and later, and doesn't hurt in earlier Emacs. --- lisp/net/tramp-sh.el | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lisp/net/tramp-sh.el b/lisp/net/tramp-sh.el index 1f7c8f6e49..542bd1c234 100644 --- a/lisp/net/tramp-sh.el +++ b/lisp/net/tramp-sh.el @@ -669,7 +669,7 @@ tramp-perl-file-attributes $uid = ($ARGV[1] eq \"integer\") ? $stat[4] : \"\\\"\" . getpwuid($stat[4]) . \"\\\"\"; $gid = ($ARGV[1] eq \"integer\") ? $stat[5] : \"\\\"\" . getgrgid($stat[5]) . \"\\\"\"; printf( - \"(%%s %%u %%s %%s (%%u %%u) (%%u %%u) (%%u %%u) %%u.0 %%u t %%u.0 -1)\\n\", + \"(%%s %%u %%s %%s (%%u %%u) (%%u %%u) (%%u %%u) %%u %%u t %%u -1)\\n\", $type, $stat[3], $uid, @@ -719,7 +719,7 @@ tramp-perl-directory-files-and-attributes $gid = ($ARGV[1] eq \"integer\") ? $stat[5] : \"\\\"\" . getgrgid($stat[5]) . \"\\\"\"; $filename =~ s/\"/\\\\\"/g; printf( - \"(\\\"%%s\\\" %%s %%u %%s %%s (%%u %%u) (%%u %%u) (%%u %%u) %%u.0 %%u t %%u.0 -1)\\n\", + \"(\\\"%%s\\\" %%s %%u %%s %%s (%%u %%u) (%%u %%u) (%%u %%u) %%u %%u t %%u -1)\\n\", $filename, $type, $stat[3], -- 2.17.1 --------------A76796BFFCF3F51395F3812D--