bug#33781: tramp tests abort on hydra

bug#33781: tramp tests abort on hydra

[Glenn Morris]

Package: emacs
Version: 27.0.50
Severity: minor

Since Emacs hydra jobs were updated to Nix OS 18.09, tramp tests are
aborting in tramp-test19-directory-files-and-attributes.

Ref eg https://hydra.nixos.org/build/85966679

bug#33781: tramp tests abort on hydra

[Michael Albinus]

Glenn Morris <rgm@gnu.org> writes:

Hi Glenn,

> Since Emacs hydra jobs were updated to Nix OS 18.09, tramp tests are
> aborting in tramp-test19-directory-files-and-attributes.
>
> Ref eg https://hydra.nixos.org/build/85966679

I'm very sorry that I had to ignore the hydra problems (not only this
one, but also file-notifications) last days. I hope the days until XMas
allow me to work on them.

Best regards, Michael.

bug#33781: tramp tests abort on hydra

[Glenn Morris]

I installed nix locally and reproduced the issue.
AFAICS it is due to

(setq attr (directory-files-and-attributes tmp-name2))

throwing an error with

tmp-name2 =
#("/mock:AAAA.BBBBBBBB.CCC:/AAAA_data1/scratch/UUUUUUU/software64/nix/tmp/tramp-testqSsjHd/bla" 6 23 (tramp-default t))

AFAICS from the tramp debug buffer, the problem is a failure to find
"sed" on the remote system. It should be using
/nix/store/ny5p32137wfyzdm485xfdck21w1gyl3g-gnused-4.5/bin/sed, in much
the same way as it using
/nix/store/wm8va53fh5158ipi0ic9gir64hrvqv1z-coreutils-8.29/bin/ls .

;; Emacs: 27.0.50 Tramp: 2.4.1-pre -*- mode: outline; -*-
16:53:21.240431 tramp-send-command (6) # test -d /AAAA_data1/scratch/UUUUUUU/software64/nix/tmp/tramp-testRd0FTK/bla 2>/dev/null; echo tramp_exit_status $?
16:53:21.241805 tramp-wait-for-regexp (6) # 
tramp_exit_status 0
///4edae1a3102909a43b8d4c6e6e62ec10#$
16:53:21.248310 tramp-do-directory-files-and-attributes-with-stat (5) # directory-files-and-attributes with stat: /AAAA_data1/scratch/UUUUUUU/software64/nix/tmp/tramp-testRd0FTK/bla
16:53:21.249681 tramp-send-command (6) # cd /AAAA_data1/scratch/UUUUUUU/software64/nix/tmp/tramp-testRd0FTK/bla && echo "("; (/nix/store/wm8va53fh5158ipi0ic9gir64hrvqv1z-coreutils-8.29/bin/ls --color=never --quoting-style=literal --show-control-chars -a | xargs env QUOTING_STYLE=locale /nix/store/wm8va53fh5158ipi0ic9gir64hrvqv1z-coreutils-8.29/bin/stat -c '(/////%n///// (/////%N/////) %h %u %g %X %Y %Z %s /////%A///// t %i -1)' -- 2>/dev/null | sed -e 's/"/\\"/g' -e 's/\/\/\/\/\//"/g'); echo ")" 2>/dev/null; echo tramp_exit_status $?
16:53:21.257525 tramp-wait-for-regexp (6) # 
(
sh: sed: command not found
)
tramp_exit_status 0
///4edae1a3102909a43b8d4c6e6e62ec10#$
16:53:21.279389 tramp-send-command (6) # cd / && rm -rf /AAAA_data1/scratch/UUUUUUU/software64/nix/tmp/tramp-testRd0FTK 2>/dev/null; echo tramp_exit_status $?
16:53:21.286694 tramp-wait-for-regexp (6) # 
tramp_exit_status 0
///4edae1a3102909a43b8d4c6e6e62ec10#$

bug#33781: tramp tests abort on hydra

[Glenn Morris]

[-- Attachment #1: Type: text/plain, Size: 27 bytes --]


Full debug log attached.


[-- Attachment #2: DEBUG.xz --]
[-- Type: application/octet-stream, Size: 11632 bytes --]

bug#33781: tramp tests abort on hydra

[Glenn Morris]

On further poking around, I think the issue might be that remote PATH is
getting truncated to ~ 4096 characters, which happens to lose sed.

At least, this is what I seem to see if I add

echo \$PATH > /tmp/FOO

to the start of the shell command in
tramp-do-directory-files-and-attributes-with-stat.

bug#33781: tramp tests abort on hydra

[Glenn Morris]

Glenn Morris wrote:

> On further poking around, I think the issue might be that remote PATH is
> getting truncated to ~ 4096 characters, which happens to lose sed.

Time will tell, but I think I have worked around this by simply putting
sed earlier in the PATH on hydra. (I wonder if this was causing other
test failures.)

I did not investigate the cause of the 4096 truncation, but perhaps it
is https://debbugs.gnu.org/12440, 6149, etc.

bug#33781: tramp tests abort on hydra

[Michael Albinus]

Glenn Morris <rgm@gnu.org> writes:

Hi Glenn,

> Since Emacs hydra jobs were updated to Nix OS 18.09, tramp tests are
> aborting in tramp-test19-directory-files-and-attributes.

The first problem was, that `tramp-remote-path' has lost its autoload
cookie. Requiring tramp-sh is a valid workaround, but the better
solution is to bring the autoload cookie back into place. Which I did.

Best regards, Michael.

bug#33781: tramp tests abort on hydra

[Michael Albinus]

[-- Attachment #1: Type: text/plain, Size: 7302 bytes --]

Glenn Morris <rgm@gnu.org> writes:

Hi Glenn,

>> On further poking around, I think the issue might be that remote PATH is
>> getting truncated to ~ 4096 characters, which happens to lose sed.
>
> Time will tell, but I think I have worked around this by simply putting
> sed earlier in the PATH on hydra. (I wonder if this was causing other
> test failures.)

Thanks. I'm wondering, why there are so many and so long directory names
in $PATH. Tramp takes the setting over from the current environment, and
does (taken from your full log):

17:30:30.324763 tramp-send-command (6) # PATH=/nix/store/bqp3d4yhpx6h60fv8jww5id0nkm44034-patchelf-0.9/bin:/nix/store/63j8mi5vjyxfc89rhjknwxaxqg3a2sx2-paxctl-0.9/bin:/nix/store/iw94llkj05wgaz268mlzvgx8jkbi1ss0-gcc-wrapper-7.3.0/bin:/nix/store/vv4r320p5yd1k01kld62q1lppjxcswhb-gcc-7.3.0/bin:/nix/store/5hir9w0mnrw86yv7sf4qghsard0ccwl1-glibc-2.27-bin/bin:/nix/store/wm8va53fh5158ipi0ic9gir64hrvqv1z-coreutils-8.29/bin:/nix/store/6yz7851vibc1xjxpiyfzqqi2ksbv6qah-binutils-wrapper-2.30/bin:/nix/store/h0lbngpv6ln56hjj59i6l77vxq25flbz-binutils-2.30/bin:/nix/store/9748iw15rqyd8s5i716hzkh8h6kx4bli-gcc-wrapper-4.8.5/bin:/nix/store/aph39agmihcibk3rp57mlzb27nzfqx15-gcc-4.8.5/bin:/nix/store/gc7p17ps5h3vlrbflhqps7ypnjfhnic5-bazaar-2.7.0/bin:/nix/store/5qm63py6n4bqhfmyb5nbvzf545arymwz-python-2.7.15/bin:/nix/store/l2rdymv3d1zhv1303ni3zx1z58vm20yx-python2.7-setuptools-40.2.0/bin:/nix/store/q9ajsngf2w3r10dg7b1148wzq1avp4gj-perl-5.28.0/bin:/nix/store/rk6df3iamxmiywq9rlqy72fhvwqhk1qk-gnupg-2.2.11/bin:/nix/store/3zal9zfqb41fsbmwl5ba7sxh3h2ngvxi-git-2.18.1/bin:/nix/store/g9r3l1l839xsxinmkfnsf682dza0gnsy-mercurial-4.7/bin:/nix/store/fxx73g9gwwcngwb1mlaikzsh1i7w7ykq-python2.7-dulwich-0.19.6/bin:/nix/store/4p88zybzrsg5vgm09jwylzyw5v9vb6xs-libjpeg-turbo-1.5.3-bin/bin:/nix/store/26lgqf0ja6rx8dnz972a3f56vfxmmmv5-xz-5.2.4-bin/bin:/nix/store/a72yvnxq9k0g0s9m9kv1s6yrjfg4p4xb-libtiff-4.0.10-bin/bin:/nix/store/kk9hay3dci9rk1aq5fdzznjp01yjbkiw-lcms2-2.9-bin/bin:/nix/store/1icn0kv4czsx6sxp970rzlp0yc3ycspv-ruby-2.5.3/bin:/nix/store/llk4irv6aajfd2hgn3ss3np9dn4wsb0j-m17n-lib-1.8.0/bin:/nix/store/mkhxzd28sc5q94smdlbbk8ldhp0wlw1c-libotf-0.9.16/bin:/nix/store/4i17crc8xvqr94vl0z1c2pd7kfxv11vf-texinfo-6.5/bin:/nix/store/9pfkx28kzpagdh24xkwdsrndim6dilks-ncurses-6.1-20181027-dev/bin:/nix/store/9kr7pv6m220gk5518qvkbxdbwq5gzb80-ncurses-6.1-20181027/bin:/nix/store/dj3sdmwgfqbnn09bdf6spzw5p58ck0jq-pkg-config-0.29.2/bin:/nix/store/57crgngllpc1dh2y90dc874q5d0z4iiy-freetype-2.9-dev/bin:/nix/store/1iih7pgc7krhis13zaq8ajdcb2hd10d9-bzip2-1.0.6.0.1-bin/bin:/nix/store/gvaadn6sby7mv35fb6jq97vz10rd59vs-libpng-apng-1.6.35-dev/bin:/nix/store/ijvpmbvy04g216604w4isj13z0wjdisz-fontconfig-2.12.6-bin/bin:/nix/store/mcx1dlzirfrf0b86qh4bfg3skn2p35p2-libXpm-3.5.12-bin/bin:/nix/store/zj5kabjc3bmlzwc9yk0g214h59br65m8-gtk+3-3.22.30-dev/bin:/nix/store/4ps4yzv9lw772xqkb7sswp0rgmdp3rmn-expat-2.2.6-dev/bin:/nix/store/0g0icfkz98f3hcb5sin35z2ciq1nsfv1-glib-2.56.0-dev/bin:/nix/store/32ayyxv62j400sim448lg1z73fnwlqmq-gettext-0.19.8.1/bin:/nix/store/583lm0wzlv16ivdad5qhq8lrfxxbwcqn-cairo-1.15.12-dev/bin:/nix/store/g3mf4r9viyhkavxbm7n5na2a9mbg2l0w-harfbuzz-1.8.8-dev/bin:/nix/store/z4sr1ydi9gl0gy466vkymmnapcgl7b46-graphite2-1.3.6/bin:/nix/store/32pva7lryj1s9bcc1alpj5jhxxzijh63-fribidi-1.0.5/bin:/nix/store/9mkblf5hgygbm7pvv8ajmn0sar9w5yrd-pango-1.42.1-bin/bin:/nix/store/2nalwmak6fk5bkyjdslzbkhp20ppdcab-gdk-pixbuf-2.36.12-dev/bin:/nix/store/gfs3wc7w03ddviif75z8a7j6nmd4kjga-jasper-2.0.14-bin/bin:/nix/store/6jjaknw6p8hqi7pzwinl2j5na3wabi21-gdk-pixbuf-2.36.12/bin:/nix/store/1nnplnv2xn6gh1m66sjcvpw5gm8x0j0n-libdrm-2.4.93-bin/bin:/nix/store/x5nz4k8m7gdgl2rjvbm5863vp1mxq1gk-wayland-1.15.0/bin:/nix/store/pwd7kz3rqj33mjh6n872nyisxwa294kl-cups-2.2.6-dev/bin:/nix/store/1vf8f6s28kqn3lvgc41yqgrsd4sw0czc-cups-2.2.6/bin:/nix/store/w27j17i2w6ns4axphr3zzlzxwfx0f9sh-gtk+3-3.22.30/bin:/nix/store/dscmmkbs6g4r0019vcx9114q1hmzc3rb-librsvg-2.42.4/bin:/nix/store/g29v2vm4g95h1z3x1c3b48m6jd9m5dzx-acl-2.2.52-bin/bin:/nix/store/nrfgkm89c628affm7fw84kvjja8jqi5c-dbus-1.12.10-lib/bin:/nix/store/xm7bh9mmd9yyz328fx150dp43k37ir42-dbus-1.12.10/bin:/nix/store/2ar3wn4xwwvzyxi34p12kh97hbq71n6h-nettle-3.4-dev/bin:/nix/store/002kgy8h44cbqg7312mcir5w2k6fsssr-gnutls-3.6.2-bin/bin:/nix/store/p2s4s69k2m1bajza76712pgw31p48brc-gpm-1.20.7/bin:/nix/store/pcrb9ip5q7yc3rk7ccab15apjy9sjggb-libselinux-2.7-bin/bin:/nix/store/mjpq0xm1zmvr9ybfwrz3dv22znrgr8nc-imagemagick-6.9.9-34-dev/bin:/nix/store/kr0djjvisbh2n1444i8fs9cv43v96m5d-fftw-double-3.3.8-dev/bin:/nix/store/kfalxsmcvr5qrayx35vpmdk6zb8n4lr6-libwebp-1.0.0/bin:/nix/store/10v9wrfsb8dj7qw1s2d77rizxw2szzp6-imagemagick-6.9.9-34/bin:/nix/store/yzq4c2z3i4nj2wxyjxh1brih4kxbdcf5-libungif-4.1.4/bin:/nix/store/g8w4qdqxpnkn297672fr4kzyr366cvmw-libxml2-2.9.8-dev/bin:/nix/store/bnnsb7r8c6ddmp9f6w18wddb1abbcq3c-libxml2-2.9.8-bin/bin:/nix/store/sp77lj21zk7ys4jl2y09vaj59kc8s9m7-lcov-1.13/bin:/nix/store/g5dlpwd44kd75i71nwzii8w4bp4inxwk-findutils-4.6.0/bin:/nix/store/2wdj4rznh37jd44lxz9nn6ymnbqs9axp-diffutils-3.6/bin:/nix/store/ny5p32137wfyzdm485xfdck21w1gyl3g-gnused-4.5/bin:/nix/store/9f89z51na7w931aja8lqlmhqny9h16cj-gnugrep-3.1/bin:/nix/store/zdadpaj5z8k9ifkgsc16yvfjn4wdv0r2-gawk-4.2.1/bin:/nix/store/lxbaya2mzjc3xzbcsykrcbvcf5vxvfia-gnutar-1.30/bin:/nix/store/ix13jm9a1jfkcg1fs3skagyz4s1w77r6-gzip-1.9/bin:/nix/store/ql3azyviwyc6w73h6prk1j2945skfapf-gnumake-4.2.1/bin:/nix/store/czx8vkrb9jdgjyz8qfksh10vrnqa723l-bash-4.4-p23/bin:/nix/store/61m8xswm9xxwb3ddw1za2cxvcikgk656-patch-2.7.6/bin:/home/UUUUUUU/.nix-profile/bin:/home/UUUUUUU/work/xmm/scripts:/home/UUUUUUU/work/chandra/scripts:/home/UUUUUUU/bin:/data/soft/local/x86_64/bin:/data/soft/local/bin:/scratch/UUUUUUU/software64/xfce4.12/bin:/scratch/UUUUUUU/software64/xfce4.12/sbin:/usr/lib64/openmpi/bin:/opt/rh/rh-git29/root/usr/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/scratch/UUUUUUU/software64/bin:/scratch/UUUUUUU/software64/sbin:/scratch/UUUUUUU/software/bin:/scratch/UUUUUUU/software/sbin:/data/soft/cxc/ciao-install/Linux64/ciao-4.10/bin:/data/soft/cxc/ciao-install/Linux64/ciao-4.10/ots/bin:/data/soft/cxc/ciao-install/Linux64/ciao-4.10/contrib/bin:/data/soft/heasoft/6.25/x86_64/heasoft/x86_64-pc-linux-gnu-libc2.17/bin:/data/soft/sas/xmmsas_20180620_1732/bin:/data/soft/casa/current/bin:/data/soft/funtools/current/x86_64/bin:/data/soft/marx/current/bin:/data/soft/marx/current/lib/marx:/data/soft/scisim/4.0.4/bin:/data/soft/wcstools/current/bin:/data/soft/eclipse/5.0.0/bin:/data/soft/zhtools/3.0.4/bin:/data/soft/imcat/20030419/bin/Linux:/data/soft/imcat/20030419/bin/scripts:/data/soft/cuda/current/bin:/data/soft/simx/2.4.1/bin:/data/soft/starlink/2018A/bin/startcl:/data/soft/starlink/2018A/bin:/data/soft/starlink/2018A/java/jre/bin:/data/soft/starlink/2018A/java/bin:/data/soft/starlink/2018A/starjava/bin:/bin:/sbin; export PATH

Doesn't there exist default directories like /bin, /usr/bin etc which
could be used instead? And are all those directories needed? I doubt,
for example, that we need /nix/store/gc7p17ps5h3vlrbflhqps7ypnjfhnic5-bazaar-2.7.0/bin

> I did not investigate the cause of the 4096 truncation, but perhaps it
> is https://debbugs.gnu.org/12440, 6149, etc.

Perhaps. Maybe we shall set tramp-chunksize, for example


[-- Attachment #2: Type: text/plain, Size: 461 bytes --]

diff --git a/test/lisp/net/tramp-tests.el b/test/lisp/net/tramp-tests.el
index 57b67a0bfe..80939934e0 100644
--- a/test/lisp/net/tramp-tests.el
+++ b/test/lisp/net/tramp-tests.el
@@ -107,6 +107,7 @@ tramp-test-temporary-file-directory
 
 ;; This should happen on hydra only.
 (when (getenv "EMACS_HYDRA_CI")
+  (customize-set-variable 'tramp-chunksize 3072)
   (add-to-list 'tramp-remote-path 'tramp-own-remote-path))
 
 (defvar tramp--test-enabled-checked nil

[-- Attachment #3: Type: text/plain, Size: 59 bytes --]


Could you pls check this setting?

Best regards, Michael.

bug#33781: tramp tests abort on hydra

[Glenn Morris]

Michael Albinus wrote:

> The first problem was, that `tramp-remote-path' has lost its autoload
> cookie. Requiring tramp-sh is a valid workaround, but the better
> solution is to bring the autoload cookie back into place. Which I did.

That's a shame, I always like to see variables get un-autoloaded.

bug#33781: tramp tests abort on hydra

[Glenn Morris]

Michael Albinus wrote:

>> Time will tell, but I think I have worked around this by simply putting
>> sed earlier in the PATH on hydra. (I wonder if this was causing other
>> test failures.)

It did work, and also fixed part of bug#33735.
Now tramp-test32-environment-variables fails, see eg
https://hydra.nixos.org/build/85984005

> Thanks. I'm wondering, why there are so many and so long directory names
> in $PATH. 

This is how Nix works. Each package has its own directory.
This is not specific to the hydra instance.

> Doesn't there exist default directories like /bin, /usr/bin etc which
> could be used instead?

They exist but don't contain the binaries.

> And are all those directories needed? 

Yes.

> I doubt, for example, that we
> need /nix/store/gc7p17ps5h3vlrbflhqps7ypnjfhnic5-bazaar-2.7.0/bin

This is how we get bzr so that we can run the vc-bzr tests.

>  (when (getenv "EMACS_HYDRA_CI")
> +  (customize-set-variable 'tramp-chunksize 3072)
>    (add-to-list 'tramp-remote-path 'tramp-own-remote-path))

This doesn't seem to help. Nor did 30720.
(Debugging these incredibly slow tests is painful.)

bug#33781: tramp tests abort on hydra

[Michael Albinus]

Glenn Morris <rgm@gnu.org> writes:

> Michael Albinus wrote:
>
>> The first problem was, that `tramp-remote-path' has lost its autoload
>> cookie. Requiring tramp-sh is a valid workaround, but the better
>> solution is to bring the autoload cookie back into place. Which I did.
>
> That's a shame, I always like to see variables get un-autoloaded.

Same here. Well, the variables go to tramp-loaddefs.el, which is less
annoying than loaddefs.el, isn't it?

The problem is setting a user option. If you want to set just a value,
it works fine:

--8<---------------cut here---------------start------------->8---
# emacs -Q --eval '(customize-set-variable (quote tramp-verbose) 6)'
--8<---------------cut here---------------end--------------->8---

This doesn't work, if you need the initial value of the user option in
order to compute a new value:

--8<---------------cut here---------------start------------->8---
# emacs -Q --eval '(customize-set-variable (quote tramp-remote-path) (add-to-list (quote tramp-remote-path) (quote tramp-own-remote-path)))'

=> customize-set-variable: Symbol’s value as variable is void: tramp-remote-path
--8<---------------cut here---------------end--------------->8---

If I would know a trick how to apply the latter one, I could get rid of
the respective autoloads.

Best regards, Michael.

bug#33781: tramp tests abort on hydra

[Michael Albinus]

Glenn Morris <rgm@gnu.org> writes:

Hi Glenn,

>>> Time will tell, but I think I have worked around this by simply putting
>>> sed earlier in the PATH on hydra. (I wonder if this was causing other
>>> test failures.)
>
> It did work, and also fixed part of bug#33735.
> Now tramp-test32-environment-variables fails, see eg
> https://hydra.nixos.org/build/85984005

I will check.

>>  (when (getenv "EMACS_HYDRA_CI")
>> +  (customize-set-variable 'tramp-chunksize 3072)
>>    (add-to-list 'tramp-remote-path 'tramp-own-remote-path))
>
> This doesn't seem to help. Nor did 30720.

Hmm, maybe this is not the problem. It could be, that the shell command
is too long, when setting $PATH. What does

--8<---------------cut here---------------start------------->8---
# getconf LINE_MAX
--8<---------------cut here---------------end--------------->8---

return?

> (Debugging these incredibly slow tests is painful.)

Well, I prefer traces. See tramp--test-instrument-test-case and
tramp--test-message, and how they are applied for example in
tramp-test42-asynchronous-requests.

Best regards, Michael.

bug#33781: tramp tests abort on hydra

[Glenn Morris]

Michael Albinus wrote:

> This doesn't work, if you need the initial value of the user option in
> order to compute a new value:
>
> --8<---------------cut here---------------start------------->8---
> # emacs -Q --eval '(customize-set-variable (quote tramp-remote-path) (add-to-list (quote tramp-remote-path) (quote tramp-own-remote-path)))'
>
> => customize-set-variable: Symbol's value as variable is void: tramp-remote-path
> --8<---------------cut here---------------end--------------->8---

This is a generic issue. Personally I don't see it as justification for
autoloading a new defcustom. Removing long-established autoloads from
existing variables is a different story.

bug#33781: tramp tests abort on hydra

[Glenn Morris]

Michael Albinus wrote:

> # getconf LINE_MAX

This returns 2048.

Anyway, I don't think this issue will be specific to hydra.nixos.org.
I hope eg that you can reproduce it on your own systems using an
artificially lengthened PATH=/some/long/dummy/thing/.../$PATH
such that the directory with "sed" is beyond 4096 characters from the
start of PATH.

bug#33781: tramp tests abort on hydra

[Michael Albinus]

Glenn Morris <rgm@gnu.org> writes:

Hi Glenn,

>> # getconf LINE_MAX
>
> This returns 2048.
>
> Anyway, I don't think this issue will be specific to hydra.nixos.org.
> I hope eg that you can reproduce it on your own systems using an
> artificially lengthened PATH=/some/long/dummy/thing/.../$PATH
> such that the directory with "sed" is beyond 4096 characters from the
> start of PATH.

Yes, I could reproduce it. The problem is, that there is an upper limit
to the length of the $PATH variable, to be checked by

# getconf PATH_MAX /

On my Ubuntu system this is 4096; I guess there is a similar result on
hydra. Until now, Tramp didn't check this limit, and it isn't a problem
but on hydra. I've added a test for this, and Tramp fires a warning, if
the intended $PATH exceeds the limit.

But this isn't satisfactory. Tramp shall raise an error in this
case. But this would result in complete failing tests on hydra involving
Tramp.

Is there a chance, that we could shorten the $PATH in the Makefile for
tests on hydra, to be shorter than 4096 characters? According to what I
see in the Tramp logs, I would say that the following directories aren't
needed for Tramp tests (to be tested, of course):

/nix/store/bqp3d4yhpx6h60fv8jww5id0nkm44034-patchelf-0.9/bin
/nix/store/63j8mi5vjyxfc89rhjknwxaxqg3a2sx2-paxctl-0.9/bin
/nix/store/iw94llkj05wgaz268mlzvgx8jkbi1ss0-gcc-wrapper-7.3.0/bin
/nix/store/vv4r320p5yd1k01kld62q1lppjxcswhb-gcc-7.3.0/bin
/nix/store/5hir9w0mnrw86yv7sf4qghsard0ccwl1-glibc-2.27-bin/bin
/nix/store/wm8va53fh5158ipi0ic9gir64hrvqv1z-coreutils-8.29/bin
/nix/store/9748iw15rqyd8s5i716hzkh8h6kx4bli-gcc-wrapper-4.8.5/bin
/nix/store/aph39agmihcibk3rp57mlzb27nzfqx15-gcc-4.8.5/bin
/nix/store/gc7p17ps5h3vlrbflhqps7ypnjfhnic5-bazaar-2.7.0/bin
/nix/store/rk6df3iamxmiywq9rlqy72fhvwqhk1qk-gnupg-2.2.11/bin
/nix/store/g9r3l1l839xsxinmkfnsf682dza0gnsy-mercurial-4.7/bin
/nix/store/4p88zybzrsg5vgm09jwylzyw5v9vb6xs-libjpeg-turbo-1.5.3-bin/bin
/nix/store/a72yvnxq9k0g0s9m9kv1s6yrjfg4p4xb-libtiff-4.0.10-bin/bin
/nix/store/kk9hay3dci9rk1aq5fdzznjp01yjbkiw-lcms2-2.9-bin/bin
/nix/store/1icn0kv4czsx6sxp970rzlp0yc3ycspv-ruby-2.5.3/bin
/nix/store/mkhxzd28sc5q94smdlbbk8ldhp0wlw1c-libotf-0.9.16/bin
/nix/store/4i17crc8xvqr94vl0z1c2pd7kfxv11vf-texinfo-6.5/bin
/nix/store/57crgngllpc1dh2y90dc874q5d0z4iiy-freetype-2.9-dev/bin
/nix/store/gvaadn6sby7mv35fb6jq97vz10rd59vs-libpng-apng-1.6.35-dev/bin
/nix/store/ijvpmbvy04g216604w4isj13z0wjdisz-fontconfig-2.12.6-bin/bin
/nix/store/mcx1dlzirfrf0b86qh4bfg3skn2p35p2-libXpm-3.5.12-bin/bin
/nix/store/4ps4yzv9lw772xqkb7sswp0rgmdp3rmn-expat-2.2.6-dev/bin
/nix/store/g3mf4r9viyhkavxbm7n5na2a9mbg2l0w-harfbuzz-1.8.8-dev/bin
/nix/store/z4sr1ydi9gl0gy466vkymmnapcgl7b46-graphite2-1.3.6/bin
/nix/store/32pva7lryj1s9bcc1alpj5jhxxzijh63-fribidi-1.0.5/bin
/nix/store/9mkblf5hgygbm7pvv8ajmn0sar9w5yrd-pango-1.42.1-bin/bin
/nix/store/gfs3wc7w03ddviif75z8a7j6nmd4kjga-jasper-2.0.14-bin/bin
/nix/store/1nnplnv2xn6gh1m66sjcvpw5gm8x0j0n-libdrm-2.4.93-bin/bin
/nix/store/pwd7kz3rqj33mjh6n872nyisxwa294kl-cups-2.2.6-dev/bin
/nix/store/1vf8f6s28kqn3lvgc41yqgrsd4sw0czc-cups-2.2.6/bin
/nix/store/dscmmkbs6g4r0019vcx9114q1hmzc3rb-librsvg-2.42.4/bin
/nix/store/g29v2vm4g95h1z3x1c3b48m6jd9m5dzx-acl-2.2.52-bin/bin
/nix/store/mjpq0xm1zmvr9ybfwrz3dv22znrgr8nc-imagemagick-6.9.9-34-dev/bin
/nix/store/kr0djjvisbh2n1444i8fs9cv43v96m5d-fftw-double-3.3.8-dev/bin
/nix/store/kfalxsmcvr5qrayx35vpmdk6zb8n4lr6-libwebp-1.0.0/bin
/nix/store/10v9wrfsb8dj7qw1s2d77rizxw2szzp6-imagemagick-6.9.9-34/bin
/nix/store/yzq4c2z3i4nj2wxyjxh1brih4kxbdcf5-libungif-4.1.4/bin
/nix/store/sp77lj21zk7ys4jl2y09vaj59kc8s9m7-lcov-1.13/bin
/home/UUUUUUU/.nix-profile/bin
/home/UUUUUUU/work/xmm/scripts
/home/UUUUUUU/work/chandra/scripts
/home/UUUUUUU/bin
/scratch/UUUUUUU/software64/xfce4.12/bin
/scratch/UUUUUUU/software64/xfce4.12/sbin
/usr/lib64/openmpi/bin
/opt/rh/rh-git29/root/usr/bin
/usr/lib64/qt-3.3/bin
/scratch/UUUUUUU/software64/bin
/scratch/UUUUUUU/software64/sbin
/scratch/UUUUUUU/software/bin
/scratch/UUUUUUU/software/sbin
/data/soft/cxc/ciao-install/Linux64/ciao-4.10/bin
/data/soft/cxc/ciao-install/Linux64/ciao-4.10/ots/bin
/data/soft/cxc/ciao-install/Linux64/ciao-4.10/contrib/bin
/data/soft/heasoft/6.25/x86_64/heasoft/x86_64-pc-linux-gnu-libc2.17/bin
/data/soft/sas/xmmsas_20180620_1732/bin
/data/soft/casa/current/bin
/data/soft/funtools/current/x86_64/bin
/data/soft/marx/current/bin
/data/soft/marx/current/lib/marx
/data/soft/scisim/4.0.4/bin
/data/soft/wcstools/current/bin
/data/soft/eclipse/5.0.0/bin
/data/soft/zhtools/3.0.4/bin
/data/soft/imcat/20030419/bin/Linux
/data/soft/imcat/20030419/bin/scripts
/data/soft/cuda/current/bin
/data/soft/simx/2.4.1/bin
/data/soft/starlink/2018A/bin/startcl
/data/soft/starlink/2018A/bin
/data/soft/starlink/2018A/java/jre/bin
/data/soft/starlink/2018A/java/bin
/data/soft/starlink/2018A/starjava/bin

This $PATH shortening shall be applied for filenotify-tests.el,
shadowfile-tests.el, tramp-archive-tests.el and tramp-tests.el.

Best regards, Michael.

bug#33781: tramp tests abort on hydra

[Andreas Schwab]

On Dez 19 2018, Michael Albinus <michael.albinus@gmx.de> wrote:

> Yes, I could reproduce it. The problem is, that there is an upper limit
> to the length of the $PATH variable, to be checked by
>
> # getconf PATH_MAX /

PATH_MAX has nothing to do with $PATH.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#33781: tramp tests abort on hydra

[Glenn Morris]

Michael Albinus wrote:

> Yes, I could reproduce it. The problem is, that there is an upper limit
> to the length of the $PATH variable, to be checked by
>
> # getconf PATH_MAX /

I don't think this is it. If it was, I would not have been able to build
Emacs, because the build process would not have been able to find "sed".
The problem is specific to Tramp, when it tries to pass long values to
the remote environment. This makes me think of bug#24531.

As a test, perhaps you could change tramp-set-remote-path to write
PATH=...
to a buffer, write that buffer to a file, and then source that file in
the remote environment. This avoids passing long strings to subprocesses.

bug#33781: tramp tests abort on hydra

[Glenn Morris]

Michael Albinus wrote:

> Is there a chance, that we could shorten the $PATH in the Makefile for
> tests on hydra, to be shorter than 4096 characters?

I imagine that would be possible, with some effort.
(BTW anything in PATH that did not begin /nix/store was just a
bleed-over from my own non-Nix environment, and won't be present on
hydra.)
But this would just be hiding a real problem that is not specific to hydra.
As I said, it's a feature of Nix/Guix installations in general.

Ref eg https://nixos.wiki/wiki/Nix_vs._Linux_Standard_Base

bug#33781: tramp tests abort on hydra

[Michael Albinus]

Andreas Schwab <schwab@linux-m68k.org> writes:

>> Yes, I could reproduce it. The problem is, that there is an upper limit
>> to the length of the $PATH variable, to be checked by
>>
>> # getconf PATH_MAX /
>
> PATH_MAX has nothing to do with $PATH.

Yes, PATH_MAX by definition is the maximum length of a file path. But I
still have the feeling it is also involved in setting the $PATH
environment variable. As Glenn did report, there are problems on hydra
when setting $PATH to a string longer than 4096 characters. It doesn't
seem to be related to the shell limit of input lines; Glenn reported
that 'getconf LINE_MAX' returns 2048.

> Andreas.

Best regards, Michael.

bug#33781: tramp tests abort on hydra

[Michael Albinus]

Glenn Morris <rgm@gnu.org> writes:

Hi Glenn,

> As a test, perhaps you could change tramp-set-remote-path to write
> PATH=...
> to a buffer, write that buffer to a file, and then source that file in
> the remote environment. This avoids passing long strings to subprocesses.

I gave it a try, pushed to master. There is also a new test
tramp-test34-remote-path, which works for me locally.

Let's see how it evolves on hydra.

Best regards, Michael.

bug#33781: tramp tests abort on hydra

[Andreas Schwab]

On Dez 20 2018, Michael Albinus <michael.albinus@gmx.de> wrote:

> Yes, PATH_MAX by definition is the maximum length of a file path. But I
> still have the feeling it is also involved in setting the $PATH
> environment variable. As Glenn did report, there are problems on hydra
> when setting $PATH to a string longer than 4096 characters. It doesn't
> seem to be related to the shell limit of input lines; Glenn reported
> that 'getconf LINE_MAX' returns 2048.

4096 is your typical page size, which is also connected to the pipe or
pty buffer.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#33781: tramp tests abort on hydra

[Glenn Morris]

Great job, thanks for fixing it!

Michael Albinus wrote:

> Yes, PATH_MAX by definition is the maximum length of a file path. But I
> still have the feeling it is also involved in setting the $PATH
> environment variable. As Glenn did report, there are problems on hydra
> when setting $PATH to a string longer than 4096 characters.

No, I did not report that. PATHs longer than 4096 work fine on
hydra.nixos.org, and on my local machine. What did not work was tramp
trying to pass a long string to a process. It is an Emacs-specific problem.

> It doesn't seem to be related to the shell limit of input lines; Glenn
> reported that 'getconf LINE_MAX' returns 2048.

I think LINE_MAX has no relevance here. Eg:

getconf LINE_MAX               # -> 2048
echo $(seq 1 10000)  | wc -w   # -> 10000
echo $(seq 1 400000) | wc -c   # -> 2688895
echo $(seq 1 500000)           # -> bash: /usr/bin/echo: Argument list too long

bug#33781: tramp tests abort on hydra

[Michael Albinus]

Andreas Schwab <schwab@linux-m68k.org> writes:

>> Yes, PATH_MAX by definition is the maximum length of a file path. But I
>> still have the feeling it is also involved in setting the $PATH
>> environment variable. As Glenn did report, there are problems on hydra
>> when setting $PATH to a string longer than 4096 characters. It doesn't
>> seem to be related to the shell limit of input lines; Glenn reported
>> that 'getconf LINE_MAX' returns 2048.
>
> 4096 is your typical page size, which is also connected to the pipe or
> pty buffer.

Is there a way to determine the value on a system?

> Andreas.

Best regards, Michael.

bug#33781: tramp tests abort on hydra

[Michael Albinus]

Glenn Morris <rgm@gnu.org> writes:

Hi Glenn,

> What did not work was tramp trying to pass a long string to a
> process. It is an Emacs-specific problem.

I'm not sure about. You've tested with tramp-chunksize less than 4096,
and this didn't work either. So it doesn't look like the
process-send-string bug.

Anyway, let's be happy that it works, finally. Thanks for your patience!

Best regards, Michael.

bug#33781: tramp tests abort on hydra

[Andreas Schwab]

On Dez 20 2018, Michael Albinus <michael.albinus@gmx.de> wrote:

> Andreas Schwab <schwab@linux-m68k.org> writes:
>
>>> Yes, PATH_MAX by definition is the maximum length of a file path. But I
>>> still have the feeling it is also involved in setting the $PATH
>>> environment variable. As Glenn did report, there are problems on hydra
>>> when setting $PATH to a string longer than 4096 characters. It doesn't
>>> seem to be related to the shell limit of input lines; Glenn reported
>>> that 'getconf LINE_MAX' returns 2048.
>>
>> 4096 is your typical page size, which is also connected to the pipe or
>> pty buffer.
>
> Is there a way to determine the value on a system?

getconf PIPE_BUF

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#33781: tramp tests abort on hydra

[Michael Albinus]

Andreas Schwab <schwab@linux-m68k.org> writes:

Hi Andreas,

>>> 4096 is your typical page size, which is also connected to the pipe or
>>> pty buffer.
>>
>> Is there a way to determine the value on a system?
>
> getconf PIPE_BUF

Thanks, I've applied it in tramp-sh.el.

> Andreas.

Best regards, Michael.