bug#36940: tests slowness and failure after recent Tramp changes

[Paul Eggert <eggert@cs.ucla.edu>]

[-- Attachment #1: Type: text/plain, Size: 2437 bytes --]

Michael Albinus wrote:

> At which places in the code do you believe such errors will happen? The
> floating number will be converted in tramp-convert-file-attributes, I
> cannot see how a rounding error could happen there.

I think the rounding error doesn't happen there, as that code accurately 
converts the Lisp float to an integer or to a cons of integers. The rounding 
error happens in functions like tramp-send-command-and-read when the Emacs Lisp 
reader scans the string generated by the Perl code, and converts the string to a 
Lisp float.

For example, suppose the inode number is 2**63 - 512 and the Perl code generates 
"9223372036854775296.0", which is exact. When the Emacs Lisp reader converts 
this to double it must round since there is no exact double representation, and 
rounding yields 2**63, i.e., 9223372036854775808.0, which is off by 512.

A simple fix is to change the Perl code to omit the trailing ".0", e.g., 
"9223372036854775296" rather than "9223372036854775296.0". This will cause the 
Emacs Lisp reader in master to return an exact integer instead of a float, thus 
avoiding the rounding error. This fix will still suffer from the same rounding 
errors in older Emacs releases where the reader returns a float for integers out 
of fixnum range, but it shouldn't hurt those older releases and it should fix 
the problem in master.

Please see attached patch. I haven't tested or installed the patch as I don't 
use Tramp, but you should be able to test it with something like this:

$ echo '' | dd obs=1 seek=9007199254740992 of=file
0+1 records in
1+0 records out
1 byte (1 B) copied, 0.015429 seconds, 0.1 kB/s
$ ls -l file
-rw-rw-r-- 1 eggert faculty 9007199254740993 2019-08-25 13:28 file

assuming your filesystem supports files containing more than 2**53 bytes - since 
there's a similar bug for file sizes.

> I haven't seen any code in Emacs which needs this slot.

It's used by ede--inode-for-dir, eshell-shuffle-files, ls-lisp-format, 
find-lisp-format, nnmaildir--group-maxnum, nnmaildir--new-number. Typical uses 
are to determine whether two directory entries identify the same file, e.g., 
(equal (file-attribute-inode-number file1) (file-attribute-inode-number file2)) 
or to use it in a hash table.

As I mentioned above, a similar bug occurs for the file size slot: files 
containing more than 2**53 bytes have rounding errors in their sizes. The 
attached patch should fix that too.

[-- Attachment #2: 0001-Fix-Tramp-rounding-of-file-sizes-and-inode-numbers.patch --]
[-- Type: text/x-patch, Size: 1661 bytes --]

From b8242feb8a2b70ce28a72d3976d2c34ce664d0b3 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Sun, 25 Aug 2019 12:27:54 -0700
Subject: [PATCH] Fix Tramp rounding of file sizes and inode numbers

* lisp/net/tramp-sh.el (tramp-perl-file-attributes)
(tramp-perl-directory-files-and-attributes):
Format file sizes and inode numbers with "%u" instead of "%u.0",
to avoid rounding errors when absolute values exceed 2**53
(Bug#36940#94).  This fixes the problem for Emacs 27 and later,
and doesn't hurt in earlier Emacs.
---
 lisp/net/tramp-sh.el | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lisp/net/tramp-sh.el b/lisp/net/tramp-sh.el
index 1f7c8f6e49..542bd1c234 100644
--- a/lisp/net/tramp-sh.el
+++ b/lisp/net/tramp-sh.el
@@ -669,7 +669,7 @@ tramp-perl-file-attributes
 $uid = ($ARGV[1] eq \"integer\") ? $stat[4] : \"\\\"\" . getpwuid($stat[4]) . \"\\\"\";
 $gid = ($ARGV[1] eq \"integer\") ? $stat[5] : \"\\\"\" . getgrgid($stat[5]) . \"\\\"\";
 printf(
-    \"(%%s %%u %%s %%s (%%u %%u) (%%u %%u) (%%u %%u) %%u.0 %%u t %%u.0 -1)\\n\",
+    \"(%%s %%u %%s %%s (%%u %%u) (%%u %%u) (%%u %%u) %%u %%u t %%u -1)\\n\",
     $type,
     $stat[3],
     $uid,
@@ -719,7 +719,7 @@ tramp-perl-directory-files-and-attributes
     $gid = ($ARGV[1] eq \"integer\") ? $stat[5] : \"\\\"\" . getgrgid($stat[5]) . \"\\\"\";
     $filename =~ s/\"/\\\\\"/g;
     printf(
-        \"(\\\"%%s\\\" %%s %%u %%s %%s (%%u %%u) (%%u %%u) (%%u %%u) %%u.0 %%u t %%u.0 -1)\\n\",
+        \"(\\\"%%s\\\" %%s %%u %%s %%s (%%u %%u) (%%u %%u) (%%u %%u) %%u %%u t %%u -1)\\n\",
         $filename,
         $type,
         $stat[3],
-- 
2.17.1