Tramp with global-auto-revert-mode.

Tramp with global-auto-revert-mode.

[Luc Teirlinck]

I start:

emacs -q --eval '(progn (blink-cursor-mode 0) (global-auto-revert-mode 1))'

I visit a file ~/streams.texi on a remote Solaris2.8 machine, using:
C-x C-f /ssh:USER@HOST:~/streams.texi.  (This file is essentially
identical to lispref/streams.texi).  Works perfectly.

Now I do the same for ~/sequences.texi.  (Essentially identical to
lispref/sequences.texi).  Tramp complains about "Invalid base64 data",
so it fails.  I try again.  Emacs crashes.

I start Emacs under gdb.  This time I visit sequences.texi first.  No
problem, no invalid base64 data.  Then I try to visit streams.texi.
Now this time Tramp claims streams.texi is the one with invalid data.
I try again.  Emacs crashes.

Without the (global-auto-revert-mode 1), the crash does not occur, nor
is there any mention of invalid base64 data.

The (blink-cursor-mode 0) is only there because without it, my nervous
system crashes, long before Emacs does.

[bash2.05b.0 ~ 3 1] cd /home/teirllm/emacscvsdir/emacs/src
[bash2.05b.0 ~/emacscvsdir/emacs/src 3 2] gdb emacs-21.3.50.1
GNU gdb Red Hat Linux 7.x (5.0rh-15) (MI_OUT)
Copyright 2001 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...
DISPLAY = :0
TERM = xterm
Breakpoint 1 at 0x80dfc42: file emacs.c, line 433.
Breakpoint 2 at 0x80c4554: file xterm.c, line 7838.
(gdb)  run -q --eval '(progn (blink-cursor-mode 0) (global-auto-revert-mode 1))' 
Starting program: /home/teirllm/emacscvsdir/emacs/src/emacs-21.3.50.1 -q --eval '(progn (blink-cursor-mode 0) (global-auto-revert-mode 1))'

Breakpoint 1, abort () at emacs.c:433
433	     kill (getpid (), SIGABRT);
(gdb)

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Here is the complete backtrace of the crash:

===File ~/bt================================================
#0  abort () at emacs.c:433
#1  0x081272ed in mark_object (arg=1217377132) at alloc.c:5034
#2  0x08127352 in mark_object (arg=-1466694512) at alloc.c:5051
#3  0x08127352 in mark_object (arg=-1466695680) at alloc.c:5051
#4  0x08125dba in mark_memory (start=0xbffe63c0, end=0xbffff57c)
    at alloc.c:3700
#5  0x081261ed in mark_stack () at alloc.c:4055
#6  0x081267ea in Fgarbage_collect () at alloc.c:4429
#7  0x081393f6 in Ffuncall (nargs=2, args=0xbffe6550) at eval.c:2666
#8  0x0816134c in Fbyte_code (bytestr=1753791512, vector=-2005954448, 
    maxdepth=5) at bytecode.c:689
#9  0x08139afe in funcall_lambda (fun=-2005954240, nargs=5, 
    arg_vector=0xbffe6674) at eval.c:2913
#10 0x081396cd in Ffuncall (nargs=6, args=0xbffe6670) at eval.c:2783
#11 0x0816134c in Fbyte_code (bytestr=1753791848, vector=-2004302960, 
    maxdepth=10) at bytecode.c:689
#12 0x08139afe in funcall_lambda (fun=-2004302760, nargs=5, 
    arg_vector=0xbffe67a4) at eval.c:2913
#13 0x081396cd in Ffuncall (nargs=6, args=0xbffe67a0) at eval.c:2783
#14 0x0816134c in Fbyte_code (bytestr=1750793136, vector=-2007006080, 
    maxdepth=11) at bytecode.c:689
#15 0x08139afe in funcall_lambda (fun=-2007001216, nargs=2, 
    arg_vector=0xbffe68d8) at eval.c:2913
#16 0x081396cd in Ffuncall (nargs=3, args=0xbffe68d4) at eval.c:2783
#17 0x0816134c in Fbyte_code (bytestr=1749507240, vector=-2007216216, 
    maxdepth=4) at bytecode.c:689
#18 0x08139afe in funcall_lambda (fun=-2006137376, nargs=1, 
    arg_vector=0xbffe6aa8) at eval.c:2913
#19 0x081396cd in Ffuncall (nargs=2, args=0xbffe6aa4) at eval.c:2783
#20 0x08138e89 in Fapply (nargs=2, args=0xbffe6aa4) at eval.c:2179
#21 0x08139536 in Ffuncall (nargs=3, args=0xbffe6aa0) at eval.c:2707
#22 0x0816134c in Fbyte_code (bytestr=1750929032, vector=-2008962568, 
    maxdepth=4) at bytecode.c:689
#23 0x08139afe in funcall_lambda (fun=-2007165128, nargs=2, 
    arg_vector=0xbffe6c78) at eval.c:2913
#24 0x081396cd in Ffuncall (nargs=3, args=0xbffe6c74) at eval.c:2783
#25 0x08138e89 in Fapply (nargs=3, args=0xbffe6c74) at eval.c:2179
#26 0x08139536 in Ffuncall (nargs=4, args=0xbffe6c70) at eval.c:2707
#27 0x0816134c in Fbyte_code (bytestr=1750929000, vector=-2006774704, 
    maxdepth=5) at bytecode.c:689
#28 0x08139afe in funcall_lambda (fun=-2007165288, nargs=2, 
    arg_vector=0xbffe6db4) at eval.c:2913
#29 0x081396cd in Ffuncall (nargs=3, args=0xbffe6db0) at eval.c:2783
#30 0x08139332 in call2 (fn=675581880, arg1=675175800, arg2=1753896232)
    at eval.c:2542
#31 0x081089e7 in Ffile_readable_p (filename=1754131912) at fileio.c:3096
#32 0x081395ca in Ffuncall (nargs=2, args=0xbffe6ea0) at eval.c:2726
#33 0x0816134c in Fbyte_code (bytestr=1749155536, vector=-2006000696, 
    maxdepth=4) at bytecode.c:689
#34 0x08138c3d in Feval (form=-1470614408) at eval.c:2084
#35 0x08137ab9 in Fcondition_case (args=-1466641064) at eval.c:1280
#36 0x081617f6 in Fbyte_code (bytestr=1749155568, vector=-2005992288, 
    maxdepth=3) at bytecode.c:870
#37 0x08139afe in funcall_lambda (fun=-2005992168, nargs=0, 
    arg_vector=0xbffe7204) at eval.c:2913
#38 0x081396cd in Ffuncall (nargs=1, args=0xbffe7200) at eval.c:2783
#39 0x0816134c in Fbyte_code (bytestr=1749162128, vector=-2005992104, 
    maxdepth=4) at bytecode.c:689
#40 0x08138c3d in Feval (form=-1470614256) at eval.c:2084
#41 0x08137ab9 in Fcondition_case (args=-1466641272) at eval.c:1280
#42 0x081617f6 in Fbyte_code (bytestr=1749162336, vector=-2005991920, 
    maxdepth=3) at bytecode.c:870
#43 0x08139afe in funcall_lambda (fun=-2005996208, nargs=0, 
---Type <return> to continue, or q <return> to quit---
    arg_vector=0xbffe7628) at eval.c:2913
#44 0x081396cd in Ffuncall (nargs=1, args=0xbffe7624) at eval.c:2783
#45 0x08138e60 in Fapply (nargs=2, args=0xbffe7624) at eval.c:2175
#46 0x08139536 in Ffuncall (nargs=3, args=0xbffe7620) at eval.c:2707
#47 0x0816134c in Fbyte_code (bytestr=1747232900, vector=-2010863448, 
    maxdepth=4) at bytecode.c:689
#48 0x08138c3d in Feval (form=-1473992588) at eval.c:2084
#49 0x08137ab9 in Fcondition_case (args=-1466641288) at eval.c:1280
#50 0x081617f6 in Fbyte_code (bytestr=1747232660, vector=-2010863600, 
    maxdepth=5) at bytecode.c:870
#51 0x08139afe in funcall_lambda (fun=-2010863760, nargs=1, 
    arg_vector=0xbffe79b4) at eval.c:2913
#52 0x081396cd in Ffuncall (nargs=2, args=0xbffe79b0) at eval.c:2783
#53 0x08139316 in call1 (fn=675096024, arg1=-2005996080) at eval.c:2520
#54 0x080e6b62 in timer_check (do_it_now=1) at keyboard.c:4404
#55 0x0816733d in wait_reading_process_input (time_limit=1, microsecs=0, 
    read_kbd=0, do_display=0) at process.c:4127
#56 0x08166ae9 in Faccept_process_output (process=-2006691680, timeout=1, 
    timeout_msecs=675044336) at process.c:3774
#57 0x081395fa in Ffuncall (nargs=3, args=0xbffe7cc0) at eval.c:2733
#58 0x0816134c in Fbyte_code (bytestr=1753791640, vector=-2004303512, 
    maxdepth=8) at bytecode.c:689
#59 0x08139afe in funcall_lambda (fun=-2004303232, nargs=0, 
    arg_vector=0xbffe7de4) at eval.c:2913
#60 0x081396cd in Ffuncall (nargs=1, args=0xbffe7de0) at eval.c:2783
#61 0x0816134c in Fbyte_code (bytestr=1753791848, vector=-2004302960, 
    maxdepth=10) at bytecode.c:689
#62 0x08139afe in funcall_lambda (fun=-2004302760, nargs=5, 
    arg_vector=0xbffe7f14) at eval.c:2913
#63 0x081396cd in Ffuncall (nargs=6, args=0xbffe7f10) at eval.c:2783
#64 0x0816134c in Fbyte_code (bytestr=1750793136, vector=-2007006080, 
    maxdepth=11) at bytecode.c:689
#65 0x08139afe in funcall_lambda (fun=-2007001216, nargs=2, 
    arg_vector=0xbffe8048) at eval.c:2913
#66 0x081396cd in Ffuncall (nargs=3, args=0xbffe8044) at eval.c:2783
#67 0x0816134c in Fbyte_code (bytestr=1749507240, vector=-2007216216, 
    maxdepth=4) at bytecode.c:689
#68 0x08139afe in funcall_lambda (fun=-2006137376, nargs=1, 
    arg_vector=0xbffe8218) at eval.c:2913
#69 0x081396cd in Ffuncall (nargs=2, args=0xbffe8214) at eval.c:2783
#70 0x08138e89 in Fapply (nargs=2, args=0xbffe8214) at eval.c:2179
#71 0x08139536 in Ffuncall (nargs=3, args=0xbffe8210) at eval.c:2707
#72 0x0816134c in Fbyte_code (bytestr=1750929032, vector=-2008962568, 
    maxdepth=4) at bytecode.c:689
#73 0x08139afe in funcall_lambda (fun=-2007165128, nargs=2, 
    arg_vector=0xbffe83e8) at eval.c:2913
#74 0x081396cd in Ffuncall (nargs=3, args=0xbffe83e4) at eval.c:2783
#75 0x08138e89 in Fapply (nargs=3, args=0xbffe83e4) at eval.c:2179
#76 0x08139536 in Ffuncall (nargs=4, args=0xbffe83e0) at eval.c:2707
#77 0x0816134c in Fbyte_code (bytestr=1750929000, vector=-2006774704, 
    maxdepth=5) at bytecode.c:689
#78 0x08139afe in funcall_lambda (fun=-2007165288, nargs=2, 
    arg_vector=0xbffe8524) at eval.c:2913
#79 0x081396cd in Ffuncall (nargs=3, args=0xbffe8520) at eval.c:2783
#80 0x08139332 in call2 (fn=675581880, arg1=675175800, arg2=1754438176)
    at eval.c:2542
#81 0x081089e7 in Ffile_readable_p (filename=1754131912) at fileio.c:3096
#82 0x081395ca in Ffuncall (nargs=2, args=0xbffe8610) at eval.c:2726
#83 0x0816134c in Fbyte_code (bytestr=1749155536, vector=-2006000696, 
    maxdepth=4) at bytecode.c:689
#84 0x08138c3d in Feval (form=-1470614408) at eval.c:2084
#85 0x08137ab9 in Fcondition_case (args=-1466636800) at eval.c:1280
#86 0x081617f6 in Fbyte_code (bytestr=1749155568, vector=-2005992288, 
---Type <return> to continue, or q <return> to quit---
    maxdepth=3) at bytecode.c:870
#87 0x08139afe in funcall_lambda (fun=-2005992168, nargs=0, 
    arg_vector=0xbffe8974) at eval.c:2913
#88 0x081396cd in Ffuncall (nargs=1, args=0xbffe8970) at eval.c:2783
#89 0x0816134c in Fbyte_code (bytestr=1749162128, vector=-2005992104, 
    maxdepth=4) at bytecode.c:689
#90 0x08138c3d in Feval (form=-1470614256) at eval.c:2084
#91 0x08137ab9 in Fcondition_case (args=-1466637008) at eval.c:1280
#92 0x081617f6 in Fbyte_code (bytestr=1749162336, vector=-2005991920, 
    maxdepth=3) at bytecode.c:870
#93 0x08139afe in funcall_lambda (fun=-2005996208, nargs=0, 
    arg_vector=0xbffe8d98) at eval.c:2913
#94 0x081396cd in Ffuncall (nargs=1, args=0xbffe8d94) at eval.c:2783
#95 0x08138e60 in Fapply (nargs=2, args=0xbffe8d94) at eval.c:2175
#96 0x08139536 in Ffuncall (nargs=3, args=0xbffe8d90) at eval.c:2707
#97 0x0816134c in Fbyte_code (bytestr=1747232900, vector=-2010863448, 
    maxdepth=4) at bytecode.c:689
#98 0x08138c3d in Feval (form=-1473992588) at eval.c:2084
#99 0x08137ab9 in Fcondition_case (args=-1466637024) at eval.c:1280
#100 0x081617f6 in Fbyte_code (bytestr=1747232660, vector=-2010863600, 
    maxdepth=5) at bytecode.c:870
#101 0x08139afe in funcall_lambda (fun=-2010863760, nargs=1, 
    arg_vector=0xbffe9124) at eval.c:2913
#102 0x081396cd in Ffuncall (nargs=2, args=0xbffe9120) at eval.c:2783
#103 0x08139316 in call1 (fn=675096024, arg1=-2005996080) at eval.c:2520
#104 0x080e6b62 in timer_check (do_it_now=1) at keyboard.c:4404
#105 0x0816733d in wait_reading_process_input (time_limit=1, microsecs=0, 
    read_kbd=0, do_display=0) at process.c:4127
#106 0x08166ae9 in Faccept_process_output (process=-2006691680, timeout=1, 
    timeout_msecs=675044336) at process.c:3774
#107 0x081395fa in Ffuncall (nargs=3, args=0xbffe9430) at eval.c:2733
#108 0x0816134c in Fbyte_code (bytestr=1753791640, vector=-2004303512, 
    maxdepth=8) at bytecode.c:689
#109 0x08139afe in funcall_lambda (fun=-2004303232, nargs=0, 
    arg_vector=0xbffe9554) at eval.c:2913
#110 0x081396cd in Ffuncall (nargs=1, args=0xbffe9550) at eval.c:2783
#111 0x0816134c in Fbyte_code (bytestr=1753791848, vector=-2004302960, 
    maxdepth=10) at bytecode.c:689
#112 0x08139afe in funcall_lambda (fun=-2004302760, nargs=6, 
    arg_vector=0xbffe9688) at eval.c:2913
#113 0x081396cd in Ffuncall (nargs=7, args=0xbffe9684) at eval.c:2783
#114 0x0816134c in Fbyte_code (bytestr=1753792024, vector=-2004302696, 
    maxdepth=8) at bytecode.c:689
#115 0x08139afe in funcall_lambda (fun=-2004302528, nargs=9, 
    arg_vector=0xbffe97a4) at eval.c:2913
#116 0x081396cd in Ffuncall (nargs=10, args=0xbffe97a0) at eval.c:2783
#117 0x0816134c in Fbyte_code (bytestr=1750976296, vector=-2006980272, 
    maxdepth=11) at bytecode.c:689
#118 0x08139afe in funcall_lambda (fun=-2006813984, nargs=1, 
    arg_vector=0xbffe9998) at eval.c:2913
#119 0x081396cd in Ffuncall (nargs=2, args=0xbffe9994) at eval.c:2783
#120 0x08138e89 in Fapply (nargs=2, args=0xbffe9994) at eval.c:2179
#121 0x08139536 in Ffuncall (nargs=3, args=0xbffe9990) at eval.c:2707
#122 0x0816134c in Fbyte_code (bytestr=1750929032, vector=-2008962568, 
    maxdepth=4) at bytecode.c:689
#123 0x08139afe in funcall_lambda (fun=-2007165128, nargs=2, 
    arg_vector=0xbffe9b68) at eval.c:2913
#124 0x081396cd in Ffuncall (nargs=3, args=0xbffe9b64) at eval.c:2783
#125 0x08138e89 in Fapply (nargs=3, args=0xbffe9b64) at eval.c:2179
#126 0x08139536 in Ffuncall (nargs=4, args=0xbffe9b60) at eval.c:2707
#127 0x0816134c in Fbyte_code (bytestr=1750929000, vector=-2006774704, 
    maxdepth=5) at bytecode.c:689
#128 0x08139afe in funcall_lambda (fun=-2007165288, nargs=2, 
---Type <return> to continue, or q <return> to quit---
    arg_vector=0xbffe9c84) at eval.c:2913
#129 0x081396cd in Ffuncall (nargs=3, args=0xbffe9c80) at eval.c:2783
#130 0x0816134c in Fbyte_code (bytestr=1746721772, vector=-2011374580, 
    maxdepth=4) at bytecode.c:689
#131 0x08139afe in funcall_lambda (fun=-2011374648, nargs=1, 
    arg_vector=0xbffe9d94) at eval.c:2913
#132 0x081396cd in Ffuncall (nargs=2, args=0xbffe9d90) at eval.c:2783
#133 0x0816134c in Fbyte_code (bytestr=1750976728, vector=-2007274168, 
    maxdepth=9) at bytecode.c:689
#134 0x08139afe in funcall_lambda (fun=-2007273920, nargs=5, 
    arg_vector=0xbffe9ec4) at eval.c:2913
#135 0x081396cd in Ffuncall (nargs=6, args=0xbffe9ec0) at eval.c:2783
#136 0x08138f9e in Fapply (nargs=2, args=0xbffe9fa4) at eval.c:2231
#137 0x08139536 in Ffuncall (nargs=3, args=0xbffe9fa0) at eval.c:2707
#138 0x0816134c in Fbyte_code (bytestr=1750929032, vector=-2008962568, 
    maxdepth=4) at bytecode.c:689
#139 0x08139afe in funcall_lambda (fun=-2007165128, nargs=6, 
    arg_vector=0xbffea0b4) at eval.c:2913
#140 0x081396cd in Ffuncall (nargs=7, args=0xbffea0b0) at eval.c:2783
#141 0x08138f9e in Fapply (nargs=3, args=0xbffea194) at eval.c:2231
#142 0x08139536 in Ffuncall (nargs=4, args=0xbffea190) at eval.c:2707
#143 0x0816134c in Fbyte_code (bytestr=1750929000, vector=-2006774704, 
    maxdepth=5) at bytecode.c:689
#144 0x08139afe in funcall_lambda (fun=-2007165288, nargs=6, 
    arg_vector=0xbffea2d4) at eval.c:2913
#145 0x081396cd in Ffuncall (nargs=7, args=0xbffea2d0) at eval.c:2783
#146 0x081393a2 in call6 (fn=675581880, arg1=675177144, arg2=1754430920, 
    arg3=675044384, arg4=675044336, arg5=675044336, arg6=675044336)
    at eval.c:2640
#147 0x0810974a in Finsert_file_contents (filename=1754430920, 
    visit=675044384, beg=675044336, end=675044336, replace=675044336)
    at fileio.c:3719
#148 0x08139650 in Ffuncall (nargs=3, args=0xbfffea40) at eval.c:2759
#149 0x0816134c in Fbyte_code (bytestr=1746734708, vector=-2011361640, 
    maxdepth=3) at bytecode.c:689
#150 0x08138c3d in Feval (form=-1474490780) at eval.c:2084
#151 0x08137ab9 in Fcondition_case (args=-1466648624) at eval.c:1280
#152 0x081617f6 in Fbyte_code (bytestr=1746734104, vector=-2011362084, 
    maxdepth=4) at bytecode.c:870
#153 0x08139afe in funcall_lambda (fun=-2011362352, nargs=6, 
    arg_vector=0xbfffeda4) at eval.c:2913
#154 0x081396cd in Ffuncall (nargs=7, args=0xbfffeda0) at eval.c:2783
#155 0x0816134c in Fbyte_code (bytestr=1746731576, vector=-2011364140, 
    maxdepth=8) at bytecode.c:689
#156 0x08139afe in funcall_lambda (fun=-2011364876, nargs=4, 
    arg_vector=0xbfffeec4) at eval.c:2913
#157 0x081396cd in Ffuncall (nargs=5, args=0xbfffeec0) at eval.c:2783
#158 0x0816134c in Fbyte_code (bytestr=1746725796, vector=-2011370544, 
    maxdepth=6) at bytecode.c:689
#159 0x08139afe in funcall_lambda (fun=-2011370644, nargs=2, 
    arg_vector=0xbfffefe4) at eval.c:2913
#160 0x081396cd in Ffuncall (nargs=3, args=0xbfffefe0) at eval.c:2783
#161 0x08138f9e in Fapply (nargs=2, args=0xbffff070) at eval.c:2231
#162 0x081392df in apply1 (fn=675120712, arg=-1466675088) at eval.c:2487
#163 0x08135371 in Fcall_interactively (function=675120712, 
    record_flag=675044336, keys=-2009253376) at callint.c:406
#164 0x080ecb6a in Fcommand_execute (cmd=675120712, record_flag=675044336, 
    keys=675044336, special=675044336) at keyboard.c:9670
#165 0x080e315d in command_loop_1 () at keyboard.c:1727
#166 0x08137baa in internal_condition_case (bfun=0x80e2424 <command_loop_1>, 
    handlers=675105248, hfun=0x80e2030 <cmd_error>) at eval.c:1333
#167 0x080e22fa in command_loop_2 () at keyboard.c:1264
#168 0x0813774b in internal_catch (tag=675099264, 
---Type <return> to continue, or q <return> to quit---
    func=0x80e22dc <command_loop_2>, arg=675044336) at eval.c:1094
#169 0x080e2288 in command_loop () at keyboard.c:1243
#170 0x080e1e13 in recursive_edit_1 () at keyboard.c:959
#171 0x080e1f2b in Frecursive_edit () at keyboard.c:1015
#172 0x080e0e73 in main (argc=4, argv=0xbffff814) at emacs.c:1692
#173 0x40317627 in __libc_start_main (main=0x80e0194 <main>, argc=4, 
    ubp_av=0xbffff814, init=0x804e1b0 <_init>, fini=0x817f698 <_fini>, 
    rtld_fini=0x4000dcd4 <_dl_fini>, stack_end=0xbffff80c)
    at ../sysdeps/generic/libc-start.c:129
(gdb)
============================================================

Re: Tramp with global-auto-revert-mode.

[Kim F. Storm]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> Here is the complete backtrace of the crash:

Always include an xbacktrace please!


Some observations:

> #31 0x081089e7 in Ffile_readable_p (filename=1754131912) at fileio.c:3096

Calls a tramp handler?

> #53 0x08139316 in call1 (fn=675096024, arg1=-2005996080) at eval.c:2520
> #54 0x080e6b62 in timer_check (do_it_now=1) at keyboard.c:4404
> #55 0x0816733d in wait_reading_process_input (time_limit=1, microsecs=0,
>     read_kbd=0, do_display=0) at process.c:4127
> #56 0x08166ae9 in Faccept_process_output (process=-2006691680, timeout=1,
>     timeout_msecs=675044336) at process.c:3774

We are inside a timer handler here:

That timer handler (at some point) calls accept-process-output, which
calls wait_reading_process_input, which may run timers (and does so).

> #81 0x081089e7 in Ffile_readable_p (filename=1754131912) at fileio.c:3096

We are inside a timer handler here:

I suppose this calls a tramp handler.

> #104 0x080e6b62 in timer_check (do_it_now=1) at keyboard.c:4404
> #105 0x0816733d in wait_reading_process_input (time_limit=1, microsecs=0,
>     read_kbd=0, do_display=0) at process.c:4127
> #106 0x08166ae9 in Faccept_process_output (process=-2006691680, timeout=1,
>     timeout_msecs=675044336) at process.c:3774

Tramp handler calls accept-process-output, which calls
wait_reading_process_input, which may run timers (and does so).

> #147 0x0810974a in Finsert_file_contents (filename=1754430920,
>     visit=675044384, beg=675044336, end=675044336, replace=675044336)
>     at fileio.c:3719

I suppose this calls a tramp handler.

> #164 0x080ecb6a in Fcommand_execute (cmd=675120712, record_flag=675044336,
>     keys=675044336, special=675044336) at keyboard.c:9670

I suppose you did C-x C-f ?



So we in three levels of tramp handlers and two levels of
accept-process-output, wait_reading_process_input, and timer_check.

I just installed a change to make wait_reading_process_input reentrant
which may or may not be relevant to this crash.

--
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Kim Storm wrote:

   Always include an xbacktrace please!

(gdb) xbacktrace
Invalid number -2147483630 of repetitions.

   > #31 0x081089e7 in Ffile_readable_p (filename=1754131912) at fileio.c:3096

   Calls a tramp handler?

Yes.

   We are inside a timer handler here:

   That timer handler (at some point) calls accept-process-output, which
   calls wait_reading_process_input, which may run timers (and does so).

   > #81 0x081089e7 in Ffile_readable_p (filename=1754131912) at fileio.c:3096

   We are inside a timer handler here:

   I suppose this calls a tramp handler.

global-autorevert-mode is active.  It calls file-readable-p on a
remote file.  This call gets handled by Tramp.

   > #147 0x0810974a in Finsert_file_contents (filename=1754430920,
   >     visit=675044384, beg=675044336, end=675044336, replace=675044336)
   >     at fileio.c:3719

   I suppose this calls a tramp handler.

Yes.

   I suppose you did C-x C-f ?

Yes.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Kim F. Storm]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> Kim Storm wrote:
> 
>    Always include an xbacktrace please!
> 
> (gdb) xbacktrace
> Invalid number -2147483630 of repetitions.

Well :-(

> 
>    We are inside a timer handler here:
> 
>    I suppose this calls a tramp handler.
> 
> global-autorevert-mode is active.  It calls file-readable-p on a
> remote file.  This call gets handled by Tramp.

Does this indicate that with global-autorevert-mode on, there is a
higher probability that tramp (or other things activated through a
timer handler) will be called recusively?

What is the value of Vtimer_list and Vtimer_idle_list?

Do:

(gdb) p Vtimer_list
(gdb) pr
(gdb) p Vtimer_idle_list
(gdb) pr

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Kim Storm wrote:

   Does this indicate that with global-autorevert-mode on, there is a
   higher probability that tramp (or other things activated through a
   timer handler) will be called recusively?

I do not see why global-auto-revert-mode should increase the
probability of Tramp being recalled recursively.  It certainly
guarantees that Tramp will be called very often.
global-auto-revert-mode runs by default every five seconds and
invoking Tramp on a slow connection (like mine) can easily take more
than five seconds.  I believe that I will put in a user option
allowing to disable reverting of remote files by
global-auto-revert-mode.  If one has a slow connection, it just causes
too much trouble, regardless of whether it is responsible for this
crash.

   What is the value of Vtimer_list and Vtimer_idle_list?

   Do:

   (gdb) p Vtimer_list
   (gdb) pr
   (gdb) p Vtimer_idle_list
   (gdb) pr

(gdb) p Vtimer_list
$8 = -1466641304
(gdb) pr
([])
(gdb) p Vtimer_idle_list
$9 = 675044336
(gdb) pr
nil

That does not seem right.  Could this data have been corrupted?  Maybe
I am misunderstanding this.  Several timers and idle-timers should
have been active.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Kim Storm wrote:

   So we in three levels of tramp handlers and two levels of
   accept-process-output, wait_reading_process_input, and timer_check.

   I just installed a change to make wait_reading_process_input reentrant
   which may or may not be relevant to this crash.

Apparently it is relevant of sorts.

Note that Lars Hansen has reports and backtraces of crashes using
Tramp _without_ global-auto-revert-mode being enabled.  Maybe these do
not happen any more after your change.  Mine definitely do not happen
any more but for strange reasons.

Before I visited one large file using Tramp-ssh, without problems.
Then visiting a second large file got an "Invalid base64 data" error
(just because it is the second file).  Tried again.  Crash.

Now when I visit one file using Tramp-ssh, Emacs freezes.  In my
personal Emacs, I can still quit with C-g because I put in calls to
with-local-quit.  In CVS Emacs, this is an outright freeze.  My
with-local-quits do not help very much, because it refreezes 0-5
seconds later.

I can not check whether the crash still occurs after your changes,
because I can no longer carry out the recipe to produce the crash.  I
wonder why I was able to do so _before_ your change.  The cause for the
freezing bug seems obvious:  It is now completely clear that
global-auto-revert-mode should not try to revert remote files over a
slow connection. 

I plan to disable auto-reverting of remote files using
global-auto-revert-mode, with a user option to re-enable it for people
with fast connections that are permanently on-line.  For other people,
there just seems to be too much ugliness involved.  I believe I will
do this tomorrow.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Stefan Monnier]

> I do not see why global-auto-revert-mode should increase the
> probability of Tramp being recalled recursively.  It certainly

Because it runs Tramp from a timer.  Timers can be run whenever Emacs is
waiting, i.e. it can happen when Tramp is waiting for the other end's
prompt to come up.


        Stefan

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Stefan Monnier wrote:

   > I do not see why global-auto-revert-mode should increase the
   > probability of Tramp being recalled recursively.  It certainly

   Because it runs Tramp from a timer.  Timers can be run whenever Emacs is
   waiting, i.e. it can happen when Tramp is waiting for the other end's
   prompt to come up.

So is this sufficient of a problem to just disable auto-reverting of
remote files completely?  The problem I am experiencing now is
definitely bad enough that it should be disabled by default.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

>From my previous message:

   Before I visited one large file using Tramp-ssh, without problems.
   Then visiting a second large file got an "Invalid base64 data" error
   (just because it is the second file).  Tried again.  Crash.

   Now when I visit one file using Tramp-ssh, Emacs freezes.  In my
   personal Emacs, I can still quit with C-g because I put in calls to
   with-local-quit.  In CVS Emacs, this is an outright freeze.  My
   with-local-quits do not help very much, because it refreezes 0-5
   seconds later.

The situation I described above seems to have been due to a temporary
deterioration of my connection.  Everything is back as before now.
The crash still occurs, exactly as described above.  Kim's recent
change seems to have had no effect on the situation.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

I plan to commit the following shortly.  The problems with
auto-reverting remote files can occasionally become so bad as to make
Emacs unusable.  The crash is not the only problem.  Maybe one could
discuss setting the default back to t.  But people with a slow
connection definitely need a way to disable auto-reverting of remote
files.  So the option seems to be definitely needed.

===File ~/auto-revert-diff==================================
*** autorevert.el	04 Apr 2004 19:50:59 -0500	1.29
--- autorevert.el	13 May 2004 20:45:22 -0500	
***************
*** 185,190 ****
--- 185,199 ----
    :group 'auto-revert
    :type 'boolean)
  
+ (defcustom global-auto-revert-remote-files nil
+   "When non-nil, Global Auto-Revert Mode reverts remote files.
+ Setting this non-nil can be dangerous.  If you have a slow
+ connection, or are not permanently on-line, freezes and other
+ problems can result."
+   :group 'auto-revert
+   :type 'boolean
+   :version "21.4")
+ 
  (defcustom global-auto-revert-ignore-modes '()
    "List of major modes Global Auto-Revert Mode should not check."
    :group 'auto-revert
***************
*** 311,316 ****
--- 320,326 ----
    (unless (buffer-modified-p)
      (let ((buffer (current-buffer)) revert eob eoblist)
        (or (and buffer-file-name
+ 	       (or auto-revert-mode global-auto-revert-remote-files)
  	       (file-readable-p buffer-file-name)
  	       (not (verify-visited-file-modtime buffer))
  	       (setq revert t))
============================================================

Re: Tramp with global-auto-revert-mode.

[Stefan Monnier]

>> I do not see why global-auto-revert-mode should increase the
>> probability of Tramp being recalled recursively.  It certainly

>    Because it runs Tramp from a timer.  Timers can be run whenever Emacs is
>    waiting, i.e. it can happen when Tramp is waiting for the other end's
>    prompt to come up.

> So is this sufficient of a problem to just disable auto-reverting of
> remote files completely?  The problem I am experiencing now is
> definitely bad enough that it should be disabled by default.

Let's first try to *fix* bugs.  Hiding them should be left for
a last-resort when all else fails.


        Stefan

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Stefan Monnier wrote:

   Let's first try to *fix* bugs.  Hiding them should be left for
   a last-resort when all else fails.

OK, I will wait and _not_ commit the patch I proposed right now.  But
I still wonder how it can possibly make sense to carry out every five
seconds an operation that usually takes longer than five seconds.
Probably the crash can be fixed, but the crash is not the only bug.
There are freezes.  `with-local-quit' does not really help.  Things
refreeze 0-5 seconds later.  I just come to have a relatively
prolonged period where I not only could not use Emacs any more, I had
a hard time to do C-x C-c and then press `y' quickly enough to kill
Emacs.  My patch is motivated by these freezes.  It is _not_ intended
to "hide" crashes.

I believe there should at least be an option to disable auto-reverting
of remote files.  I also believe disabling should be the default,
although that might be less clear.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

>From my previous message:

   Probably the crash can be fixed, but the crash is not the only bug.

I meant:

   Probably the crash can be fixed, but the crash is not the only
   _problem_.

I believe that the freezes are not an Emacs or Tramp bug, but the
inevitable result of a slow connection.  Hence we need a user option
to deal with them.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Stefan Monnier]

> I believe that the freezes are not an Emacs or Tramp bug, but the
> inevitable result of a slow connection.  Hence we need a user option
> to deal with them.

If C-g does not get you out of the hang, it's a bug.
As I said, the with-local-quit should be put in Tramp.
You might be right that auto-reverting remote files needs to be turned off
or customizable, but the problems you're experiencing could also be
triggered by other packages than auto-revert, so we should fix them.

Among all the problems, one of them is actually in autorevert.el:
the auto-revert timer should not be re-run if it hasn't finished running
yet.  One way to do that is to not use a repeating timer, but instead to
reset the timer manually at the end of auto-revert-buffers.  Another is to
bind auto-revert-running to t in auto-revert-buffers and check it before
running, as in the patch below.  A third option is to do this kind of
re-entrance-check directly in the C code.


        Stefan


--- orig/lisp/autorevert.el
+++ mod/lisp/autorevert.el
@@ -1,6 +1,6 @@
 ;;; autorevert.el --- revert buffers when files on disk change
 
-;; Copyright (C) 1997, 1998, 1999, 2001 Free Software Foundation, Inc.
+;; Copyright (C) 1997, 98, 1999, 2001, 2004  Free Software Foundation, Inc.
 
 ;; Author: Anders Lindgren <andersl@andersl.com>
 ;; Keywords: convenience
@@ -343,7 +343,7 @@
       ;; do want to reset the mode for VC, so we do it manually.
       (when (or revert auto-revert-check-vc-info)
 	(vc-find-file-hook)))))
-
+(defvar auto-revert-running nil)
 (defun auto-revert-buffers ()
   "Revert buffers as specified by Auto-Revert and Global Auto-Revert Mode.
 
@@ -367,9 +367,13 @@
 This function is also responsible for removing buffers no longer in
 Auto-Revert mode from `auto-revert-buffer-list', and for canceling
 the timer when no buffers need to be checked."
+  (if auto-revert-running
+      ;; Don't re-run if we're not finished executing the previous run.
+      nil
   (let ((bufs (if global-auto-revert-mode
 		  (buffer-list)
 		auto-revert-buffer-list))
+	(auto-revert-running t)
 	(remaining '())
 	(new '()))
     ;; Partition `bufs' into two halves depending on whether or not
@@ -405,7 +409,7 @@
     (when (and (not global-auto-revert-mode)
 	       (null auto-revert-buffer-list))
       (cancel-timer auto-revert-timer)
-      (setq auto-revert-timer nil))))
+      (setq auto-revert-timer nil)))))
 
 
 ;; The end:

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Stefan Monnier wrote:

   If C-g does not get you out of the hang, it's a bug.
   As I said, the with-local-quit should be put in Tramp.

I put in with-local-quits in my private Emacs.  C-g does get me out of
the hang, but for 0 to 5 seconds.  That is no big help.

I never was able to check the problem with more than one file, because
trying to open a second remote file produces a crash.  With
global-auto-revert-remote-files set to nil I opened five remote files
and worked with them without problems.  Then I tried to set
global-auto-revert-remote-files to t.  Immediate freeze.

To be able to use Tramp efficiently, there should be no problems
having ten or more remote files open at the same time.

I will try out your patch tomorrow.  From looking at it, it is
probably going to be an improvement, but I doubt that it is going to
solve all my problems.  I will check.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

>From my previous message:

   I never was able to check the problem with more than one file, because
   trying to open a second remote file produces a crash.  With
   global-auto-revert-remote-files set to nil I opened five remote files
   and worked with them without problems.  Then I tried to set
   global-auto-revert-remote-files to t.  Immediate freeze.

Well, immediate freeze after entering my password.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Stefan Monnier]

> I put in with-local-quits in my private Emacs.  C-g does get me out of
> the hang, but for 0 to 5 seconds.  That is no big help.

I see.  A misbehavior, but not a clearcut bug indeed.

> I never was able to check the problem with more than one file, because
> trying to open a second remote file produces a crash.

That one is a bug ;-)

> I will try out your patch tomorrow.  From looking at it, it is
> probably going to be an improvement, but I doubt that it is going to
> solve all my problems.  I will check.

I don't think it'll fix your problems.  But if we want to support remote
files with auto-revert, we'll need something like that patch (after we've
fixed the crashes).


        Stefan

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

My proposed patch had an obvious bug.  Here is a corrected version,
although we will probably need additional or different changes.

===File ~/auto-revert-diff==================================
*** autorevert.el	04 Apr 2004 19:50:59 -0500	1.29
--- autorevert.el	14 May 2004 00:12:18 -0500	
***************
*** 185,190 ****
--- 185,199 ----
    :group 'auto-revert
    :type 'boolean)
  
+ (defcustom global-auto-revert-remote-files nil
+   "When non-nil, Global Auto-Revert Mode reverts remote files.
+ Setting this non-nil can be dangerous.  If you have a slow
+ connection, or are not permanently on-line, freezes and other
+ problems can result."
+   :group 'auto-revert
+   :type 'boolean
+   :version "21.4")
+ 
  (defcustom global-auto-revert-ignore-modes '()
    "List of major modes Global Auto-Revert Mode should not check."
    :group 'auto-revert
***************
*** 311,316 ****
--- 320,328 ----
    (unless (buffer-modified-p)
      (let ((buffer (current-buffer)) revert eob eoblist)
        (or (and buffer-file-name
+ 	       (or (not (file-remote-p buffer-file-name))
+ 		   auto-revert-mode
+ 		   global-auto-revert-remote-files)
  	       (file-readable-p buffer-file-name)
  	       (not (verify-visited-file-modtime buffer))
  	       (setq revert t))
============================================================

Re: Tramp with global-auto-revert-mode.

[Kai Grossjohann]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I do not see why global-auto-revert-mode should increase the
>> probability of Tramp being recalled recursively.  It certainly
>
> Because it runs Tramp from a timer.  Timers can be run whenever Emacs is
> waiting, i.e. it can happen when Tramp is waiting for the other end's
> prompt to come up.

Eeek.

There is nothing in Tramp that anticipates such reentrant calls.

What should Tramp do with such calls?

Kai

Re: Tramp with global-auto-revert-mode.

[David Kastrup]

Kai Grossjohann <kai@emptydomain.de> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
> >> I do not see why global-auto-revert-mode should increase the
> >> probability of Tramp being recalled recursively.  It certainly
> >
> > Because it runs Tramp from a timer.  Timers can be run whenever Emacs is
> > waiting, i.e. it can happen when Tramp is waiting for the other end's
> > prompt to come up.
> 
> Eeek.
> 
> There is nothing in Tramp that anticipates such reentrant calls.
> 
> What should Tramp do with such calls?

Work properly if they are on a different connection.  Queue them if
they are on the same connection.  You don't need an explicit queue for
that, you can just sleep with accept-process-output and get woken up
at a time when there might nothing else be pending anymore.  I don't
think a timer gets rerun while it is sleeping on that.

When the code continues, you probably should check whether something
else that was scheduled beforehand has already done the job.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Tramp with global-auto-revert-mode.

[Stefan Monnier]

> Work properly if they are on a different connection.  Queue them if
> they are on the same connection.

Can't queue 'em.

Scenario:

        file-exist-p -> Tramp -> accept-process-output
        <<Now process filters and sentinels and timers can be run>>
        One of those async thingies calls
        file-readable-p -> Tramp -> ????

If it's on the same connection Tramp is stuck: its connection is either
busy waiting for some answer from the remote host (which is OK: we can
wait) or busy waiting for accept-process-output to return so Emacs can
react to the remote process's output; but accept-process-output can't return
before file-readable-p is finished.  Sending new commands to perform
file-readable-p canbe tricky because the remote shell might be in any
intermediate state, so we'd need to detect this state, save it and restore
it when done.  Or else open up a new connection.

Note that the above also shows that async code is not only run when you call
accept-process-output or sit-for or sleep-for or read-kbd-sequence: it can
also be run when you call file-exists-p.  I.e.

     (let ((default-directory "/a"))
       (if (file-exists-p default-directory) default-directory))

can return "/b" (if one of the process filters changes default-directory).


        Stefan

Re: Tramp with global-auto-revert-mode.

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > Work properly if they are on a different connection.  Queue them if
> > they are on the same connection.
> 
> Can't queue 'em.
> 
> Scenario:
> 
>         file-exist-p -> Tramp -> accept-process-output
>         <<Now process filters and sentinels and timers can be run>>
>         One of those async thingies calls
>         file-readable-p -> Tramp -> ????
> 
> If it's on the same connection Tramp is stuck: its connection is either
> busy waiting for some answer from the remote host (which is OK: we can
> wait) or busy waiting for accept-process-output to return so Emacs can
> react to the remote process's output; but accept-process-output can't return
> before file-readable-p is finished.

So file-readable-p calls accept-process-output, too, which means it is
effectively queued.  Eventually output arrives.  The problem is that
the output is for a different operation, but the way "concurrency"
works in Emacs, the "async thingy" will be the one that is woken up.
So whoever calls accept-process-output must be prepared to deal with
preprocessing the accepted output sufficiently so that it can use the
connection itself.  It would appear that pushing the output into some
FIFO should do the trick.  Everybody that initiates an operation
expecting output will call accept-process-output, and then first
check whether there is something in the FIFO for it.

Maybe I am too stupid to understand the implications.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Tramp with global-auto-revert-mode.

[Stefan Monnier]

[...]
> Maybe I am too stupid to understand the implications.

I think the problem is just sufficiently complex that everybody is "too
stupid to understand the implications", unless we devise a clear set of
guidelines/invariants.


        Stefan

Re: Tramp with global-auto-revert-mode.

[Kim F. Storm]

David Kastrup <dak@gnu.org> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
> > > Work properly if they are on a different connection.  Queue them if
> > > they are on the same connection.
> > 
> > Can't queue 'em.
> > 
> > Scenario:
> > 
> >         file-exist-p -> Tramp -> accept-process-output
> >         <<Now process filters and sentinels and timers can be run>>
> >         One of those async thingies calls
> >         file-readable-p -> Tramp -> ????
> > 
> > If it's on the same connection Tramp is stuck: its connection is either
> > busy waiting for some answer from the remote host (which is OK: we can
> > wait) or busy waiting for accept-process-output to return so Emacs can
> > react to the remote process's output; but accept-process-output can't return
> > before file-readable-p is finished.
> 
> So file-readable-p calls accept-process-output, too, which means it is
> effectively queued.  Eventually output arrives.  The problem is that
> the output is for a different operation, but the way "concurrency"
> works in Emacs, the "async thingy" will be the one that is woken up.
> So whoever calls accept-process-output must be prepared to deal with
> preprocessing the accepted output sufficiently so that it can use the
> connection itself.  It would appear that pushing the output into some
> FIFO should do the trick.  Everybody that initiates an operation
> expecting output will call accept-process-output, and then first
> check whether there is something in the FIFO for it.

How does it know where the output from the previous operation ends
(or have arrived at all), i.e. how does it know when the output to its
own command starts?

A simple solution [but probably fully acceptable in practice] would
be to have a global remote-busy-p flag that could be tested by
async handlers doing file operations - and simply do nothing (in the
present activation) if non-nil, i.e.

        (unless remote-busy-p
         ...).

Then tramp, ange-ftp, etc. could do

 (let ((remote-busy-p t))
        ...)

-- 
Kim F. Storm  http://www.cua.dk

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Stefan Monnier wrote:

   > I will try out your patch tomorrow.  From looking at it, it is
   > probably going to be an improvement, but I doubt that it is going to
   > solve all my problems.  I will check.

   I don't think it'll fix your problems.  But if we want to support remote
   files with auto-revert, we'll need something like that patch (after we've
   fixed the crashes).

As you already expected, it did not fix my problems.  I actually now
seem to get much more frequent freezes and crashes.  (Tramp is currently
outright unusable for me with auto-reverting of remote files enabled.)
This can not be due to your patch, however.  Maybe setting
tramp-chunksize to 500 slowed things down, or maybe my connection
got temporarily worse again.

Of course, Lars reported crashes using Tramp and he is _not_ using
auto-revert.  I have never seen any of my problems when auto-reverting
of remote files is disabled or when auto-revert is not active.


Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[David Kastrup]

storm@cua.dk (Kim F. Storm) writes:

> David Kastrup <dak@gnu.org> writes:
> 
> > Stefan Monnier <monnier@iro.umontreal.ca> writes:
> > 
> > > > Work properly if they are on a different connection.  Queue
> > > > them if they are on the same connection.
> > > 
> > > Can't queue 'em.
> > > 
> > > Scenario:
> > > 
> > >         file-exist-p -> Tramp -> accept-process-output
> > >         <<Now process filters and sentinels and timers can be run>>
> > >         One of those async thingies calls
> > >         file-readable-p -> Tramp -> ????
> > > 
> > > If it's on the same connection Tramp is stuck: its connection is
> > > either busy waiting for some answer from the remote host (which
> > > is OK: we can wait) or busy waiting for accept-process-output to
> > > return so Emacs can react to the remote process's output; but
> > > accept-process-output can't return before file-readable-p is
> > > finished.
> > 
> > So file-readable-p calls accept-process-output, too, which means
> > it is effectively queued.  Eventually output arrives.  The problem
> > is that the output is for a different operation, but the way
> > "concurrency" works in Emacs, the "async thingy" will be the one
> > that is woken up.  So whoever calls accept-process-output must be
> > prepared to deal with preprocessing the accepted output
> > sufficiently so that it can use the connection itself.  It would
> > appear that pushing the output into some FIFO should do the trick.
> > Everybody that initiates an operation expecting output will call
> > accept-process-output, and then first check whether there is
> > something in the FIFO for it.
> 
> How does it know where the output from the previous operation ends
> (or have arrived at all), i.e. how does it know when the output to
> its own command starts?

The output is yours if nobody else requests it.  In short, you work
this in the following manner:

(if process-is-busy
  (push my-request process-input-queue)
  (while (not (member my-request process-output-queue))
    (work-on-the-queues)))

And work-on-the-queues basically loops around accept-process-output.
The filter function correlates the output with the currently running
command in the remote shell, writes the results into the
process-output-queue and moves the next material from the front of
process-input-queue into the shell.

> be to have a global remote-busy-p flag that could be tested by
> async handlers doing file operations - and simply do nothing (in the
> present activation) if non-nil, i.e.
> 
>         (unless remote-busy-p
>          ...).

Do nothing?  How is the task accomplished then?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Tramp with global-auto-revert-mode.

[Kai Grossjohann]

David Kastrup <dak@gnu.org> writes:

> Kai Grossjohann <kai@emptydomain.de> writes:
>
>> What should Tramp do with [reentrant] calls?
>
> Work properly if they are on a different connection.

Okay, that can be made to work.

> Queue them if they are on the same connection.  You don't need an
> explicit queue for that, you can just sleep with
> accept-process-output and get woken up at a time when there might
> nothing else be pending anymore.  I don't think a timer gets rerun
> while it is sleeping on that.

So user does C-x C-f on a large file using an inline method.  Tramp
starts to transfer the base64 data across the wire...  And then the
timer kicks in.  Tramp notices it is already busy in that buffer, so
the timer waits... and waits... and waits...  It could be quite a
while until the large file is transferred.

(I don't see how accept-process-output can wake me up when there is
no output, btw.  But, as everybody else here, I'm too stupid to see
the consequences :-)  However, the difference is that you seem to
have an idea how to solve it, and I don't.)

Kai

Re: Tramp with global-auto-revert-mode.

[Kai Grossjohann]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Sending new commands to perform file-readable-p canbe tricky because
> the remote shell might be in any intermediate state, so we'd need to
> detect this state, save it and restore it when done.  Or else open
> up a new connection.

Opening a new connection actually seems the most promising approach to
this humble poster.  Managing multiple connections is already on the
todo list -- it could be used for compilation buffers, for instance.
So just opening a new connection on a reentrant call would be fairly
easy to do, once the infrastructure is in place.

Kai

Re: Tramp with global-auto-revert-mode.

[Kim F. Storm]

David Kastrup <dak@gnu.org> writes:

> > How does it know where the output from the previous operation ends
> > (or have arrived at all), i.e. how does it know when the output to
> > its own command starts?
>
> The output is yours if nobody else requests it.  In short, you work
> this in the following manner:
>
> (if process-is-busy
>   (push my-request process-input-queue)
>   (while (not (member my-request process-output-queue))
>     (work-on-the-queues)))
>
> And work-on-the-queues basically loops around accept-process-output.
> The filter function correlates the output with the currently running
> command in the remote shell, writes the results into the
> process-output-queue and moves the next material from the front of
> process-input-queue into the shell.

It might be the way for Tramp to do this -- Kai ?


>
> > be to have a global remote-busy-p flag that could be tested by
> > async handlers doing file operations - and simply do nothing (in the
> > present activation) if non-nil, i.e.
> >
> >         (unless remote-busy-p
> >          ...).
>
> Do nothing?  How is the task accomplished then?

I was referring to repeating timers running async handlers related to
things which are not really required to be run on every activation --
such as auto-revert.


A completely different approach would be to NOT run timers from a
timer handler (which may happen because timers are too slow to
execute).

To me, this spells troubles anyway...

I have installed a change to avoid running timers inside a timer.
There may be problems with that change, but let's see if it has
a positive effect on the current problems.

--
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Tramp with global-auto-revert-mode.

[Kai Grossjohann]

storm@cua.dk (Kim F. Storm) writes:

> How does it know where the output from the previous operation ends
> (or have arrived at all), i.e. how does it know when the output to its
> own command starts?

I think that David is suggesting that the timer does not start its
own command until the command-in-process is finished.  So it goes
like this:

- User invokes Tramp in some manner.

- Tramp sends a command to the remote host and enters a loop calling
  accept-process-output until the command is complete.  (Tramp knows
  to look for a shell prompt to see that the command is complete.)

- This loop is interrupted by a timer.  The timer sees that such a
  loop is in progress already.

- The timer enters its own accept-process-output loop, fetching
  output from the unknown command until the command completes (shell
  prompt arrives).  It stashes the output away someplace safe.

- Now the timer is free to do its own thing: it sends a command to
  the remote end, uses the accept-process-output loop to fetch the
  result, is happy.

- Just before returning, the timer fetches the output from the safe
  place and inserts it into the connection buffer.  Then the timer
  sends a no-op command to the remote end (but does *not* invoke
  accept-process-output) and returns.

- This is the point in time where the "master" Emacs process gets to
  do something.  It's still inside accept-process-output.  The no-op
  command sent by the timer produces output (a shell prompt) which
  the now-awakened accept-process-output is happy to receive, and it
  wakes up.

- The master accept-process-output loop sees nothing amiss and just
  blissfully processes the buffer contents as if nothing strange
  happened.

I hope I got that right.

It seems rather fragile to me.

Kai

Re: Tramp with global-auto-revert-mode.

[David Kastrup]

storm@cua.dk (Kim F. Storm) writes:

> David Kastrup <dak@gnu.org> writes:
> 
> > > be to have a global remote-busy-p flag that could be tested by
> > > async handlers doing file operations - and simply do nothing (in the
> > > present activation) if non-nil, i.e.
> > >
> > >         (unless remote-busy-p
> > >          ...).
> >
> > Do nothing?  How is the task accomplished then?
> 
> I was referring to repeating timers running async handlers related
> to things which are not really required to be run on every
> activation -- such as auto-revert.

Ah, ok.  But this sounds like one might have some sort of outstanding
operations (like saving buffers, auto-reverting or something) where a
later entry into the queue would just take a look of whether somebody
else is doing something similar or conflicting or overriding or
whatever, try to merge the operations somehow and consider both
finished together when they get done.  For example, if we have
creation/auto-save of a file pending in the queue, and then comes
deletion after it, we can just consider the creation/save finished.
I don't think one needs to bother about atomicity of requests: only
operations from different threads/timers/whatever can manage to
actually enter stuff into the queues, so there was no guarantee that
they were entered in a specific order, and you can just work them off
in any old order you like.

> A completely different approach would be to NOT run timers from a
> timer handler (which may happen because timers are too slow to
> execute).
> 
> To me, this spells troubles anyway...
> 
> I have installed a change to avoid running timers inside a timer.
> There may be problems with that change, but let's see if it has a
> positive effect on the current problems.

If a timer does something like accept-process-output, stopping other
timers would probably be a mistake.

We probably should not run one and the same timer while it has not
yet finished, but freeze different ones?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Tramp with global-auto-revert-mode.

[Richard Stallman]

    (gdb) xbacktrace
    Invalid number -2147483630 of repetitions.

Can you investigate the bug in xbacktrace?
Fixing that is very important.  If you can't find
the first bug, but you can fix xbacktrace to operate
correctly in this case, it will be an important contribution.

Meanwhile, you can find out what Lisp functions are running in this
case using the old-fashioned method, by going to Ffuncall frames and
typing

    p *args
    xsymbol

    Tramp handler calls accept-process-output, which calls
    wait_reading_process_input, which may run timers (and does so).

Re: Tramp with global-auto-revert-mode.

[Richard Stallman]

       Because it runs Tramp from a timer.  Timers can be run whenever Emacs is
       waiting, i.e. it can happen when Tramp is waiting for the other end's
       prompt to come up.

    So is this sufficient of a problem to just disable auto-reverting of
    remote files completely?  The problem I am experiencing now is
    definitely bad enough that it should be disabled by default.

I think that calling Tramp from within a timer is in itself a bug.
One way to prevent that is not to auto-revert remote files.  That
might the right thing.  But it is not the only way to avoid this
problem.

The timer could send messages to a subprocess (such as the one
that Tramp uses anyway), and when the answers come back,
the process handler could take appropriate action.

One would probably want a longer interval between rechecking of a
remote file, not just a few seconds.  A minute, or several minutes, is
what I would think of.  And while a recheck is in progress, it should
know this and avoid starting another recheck of the same file.

Just that last change might be enough to prevent this problem.

Of course, we suspect there is a bug at the C level and we
would like to fix that as well.

Re: Tramp with global-auto-revert-mode.

[Richard Stallman]

I think that Finsert_file_contents does not allow quitting from within
read.  The reason is that there was no way to make things consistent
after a quit.  It's possible that that problem was in the case of NFS.

Re: Tramp with global-auto-revert-mode.

[Kim F. Storm]

David Kastrup <dak@gnu.org> writes:

> > 
> > I have installed a change to avoid running timers inside a timer.
> > There may be problems with that change, but let's see if it has a
> > positive effect on the current problems.
> 
> If a timer does something like accept-process-output, stopping other
> timers would probably be a mistake.

But it seems quite bogus for a timer to do that ...  as it blocks
interactive use.

IMO, timers should be "fast".


> 
> We probably should not run one and the same timer while it has not
> yet finished, but freeze different ones?

That would be easy (it can be done in timer-event-handler).

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Tramp with global-auto-revert-mode.

[Stefan Monnier]

>> If a timer does something like accept-process-output, stopping other
>> timers would probably be a mistake.
> But it seems quite bogus for a timer to do that ...  as it blocks
> interactive use.
> IMO, timers should be "fast".

Agreed, but accept-process-output is not always slow and we don't always get
to choose.  I've hacked my Emacs to signal an error if accept-process-output
is called (without timeout) with inhibit-quit set to non-nil and discovered
that some package (IIRC, flyspell) does exactly that.  Of course, ispell
is a local process that always responds immeditely, right?
Anyway: how could flyspell do its job without calling 
accept-process-output from a timer or a post-command-hook?


        Stefan

Re: Tramp with global-auto-revert-mode.

[Michael Albinus]

Kai Grossjohann <kai.grossjohann@gmx.net> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>> Sending new commands to perform file-readable-p canbe tricky because
>> the remote shell might be in any intermediate state, so we'd need to
>> detect this state, save it and restore it when done.  Or else open
>> up a new connection.
>
> Opening a new connection actually seems the most promising approach to
> this humble poster.  Managing multiple connections is already on the
> todo list -- it could be used for compilation buffers, for instance.
> So just opening a new connection on a reentrant call would be fairly
> easy to do, once the infrastructure is in place.

I have the feeling that at first we need to have a locking mechanism
in Tramp for "atomic" file operations. This would prevent us from
mixing different commands send to the remote side, which seems to be
the cause of the trouble. File operations will be handled one at a
time from a queue. Obvious disadvantage is delay in response.

Optimization (like multiple connections as mentioned by Kai) would be
the next step. But I doubt it is usefull for such operations like
`file-readable-p', given the overhead of a new connection.

> Kai

Best regards, Michael.

Re: Tramp with global-auto-revert-mode.

[David Kastrup]

storm@cua.dk (Kim F. Storm) writes:

> David Kastrup <dak@gnu.org> writes:
> > 
> > We probably should not run one and the same timer while it has not
> > yet finished, but freeze different ones?
> 
> That would be easy (it can be done in timer-event-handler).

Uh, that was a rhetorical question, short for "are you really sure
that freezing different timers would be a good idea?".

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Kim Storm wrote:

   I have installed a change to avoid running timers inside a timer.
   There may be problems with that change, but let's see if it has
   a positive effect on the current problems.

Still a crash.

Still:

(gdb) xbacktrace
Invalid number -2147483627 of repetitions.

It is a while ago that I have used GDB on a regular basis and I am not
terribly familiar with debugging Emacs under GDB.

Given the C backtrace below, obtained using bt, I do:

(gdb) up 9
#9  0x0813cb75 in Ffuncall (nargs=1, args=0xbffe7980) at eval.c:2783
2783		  val = funcall_lambda (fun, numargs, args + 1);
(gdb) p *args
$4 = 143080985
(gdb) pr


Am I doing something stupid?  I was expecting to see the name of the
Lisp function.

(gdb) p *args
$7 = 143080985
(gdb) xtype
Lisp_Symbol
(gdb) xsymbol
$8 = (struct Lisp_Symbol *) 0x8873e18
Argument to arithmetic operation not a number or boolean.
(gdb) 


I guess I must be doing something wrong.

In absence of better, here is the C backtrace:

===File ~/newbt=============================================
(gdb) bt
#0  abort () at emacs.c:433
#1  0x0812a5a5 in mark_object (arg=143721618) at alloc.c:5034
#2  0x0812a606 in mark_object (arg=143467901) at alloc.c:5051
#3  0x0812a606 in mark_object (arg=143458309) at alloc.c:5051
#4  0x081294ba in mark_memory (start=0xbffe7720, end=0xbffff57c)
    at alloc.c:3781
#5  0x08129525 in mark_stack () at alloc.c:4055
#6  0x08129aea in Fgarbage_collect () at alloc.c:4429
#7  0x08165a5d in Fbyte_code (bytestr=143174243, vector=143176372, maxdepth=64)
    at bytecode.c:522
#8  0x0813cf9f in funcall_lambda (fun=143176652, nargs=0, 
    arg_vector=0xbffe7984) at eval.c:2913
#9  0x0813cb75 in Ffuncall (nargs=1, args=0xbffe7980) at eval.c:2783
#10 0x08165c80 in Fbyte_code (bytestr=143174451, vector=143177972, maxdepth=80)
    at bytecode.c:689
#11 0x0813cf9f in funcall_lambda (fun=143178172, nargs=5, 
    arg_vector=0xbffe7ac8) at eval.c:2913
#12 0x0813cb75 in Ffuncall (nargs=6, args=0xbffe7ac4) at eval.c:2783
#13 0x08165c80 in Fbyte_code (bytestr=140897419, vector=140909636, maxdepth=96)
    at bytecode.c:689
#14 0x0813cf9f in funcall_lambda (fun=140296844, nargs=1, 
    arg_vector=0xbffe7c04) at eval.c:2913
#15 0x0813cb75 in Ffuncall (nargs=2, args=0xbffe7c00) at eval.c:2783
#16 0x08165c80 in Fbyte_code (bytestr=140897275, vector=140707364, maxdepth=56)
    at bytecode.c:689
#17 0x0813cf9f in funcall_lambda (fun=141544356, nargs=1, 
    arg_vector=0xbffe7df8) at eval.c:2913
#18 0x0813cb75 in Ffuncall (nargs=2, args=0xbffe7df4) at eval.c:2783
#19 0x0813c30e in Fapply (nargs=2, args=0xbffe7df4) at eval.c:2179
#20 0x0813c9f6 in Ffuncall (nargs=3, args=0xbffe7df0) at eval.c:2707
#21 0x08165c80 in Fbyte_code (bytestr=143098715, vector=140246412, maxdepth=32)
    at bytecode.c:689
#22 0x0813cf9f in funcall_lambda (fun=141529212, nargs=2, 
    arg_vector=0xbffe7fd8) at eval.c:2913
#23 0x0813cb75 in Ffuncall (nargs=3, args=0xbffe7fd4) at eval.c:2783
#24 0x0813c30e in Fapply (nargs=3, args=0xbffe7fd4) at eval.c:2179
#25 0x0813c9f6 in Ffuncall (nargs=4, args=0xbffe7fd0) at eval.c:2707
#26 0x08165c80 in Fbyte_code (bytestr=143098683, vector=141549828, maxdepth=40)
    at bytecode.c:689
#27 0x0813cf9f in funcall_lambda (fun=140695124, nargs=2, 
    arg_vector=0xbffe8124) at eval.c:2913
#28 0x0813cb75 in Ffuncall (nargs=3, args=0xbffe8120) at eval.c:2783
#29 0x0813c7e6 in call2 (fn=138315697, arg1=138312169, arg2=143110907)
    at eval.c:2542
#30 0x08111f8b in Ffile_attributes (filename=143497499, id_format=138193553)
    at dired.c:920
#31 0x0813ca96 in Ffuncall (nargs=2, args=0xbffe8270) at eval.c:2729
#32 0x08165c80 in Fbyte_code (bytestr=143082251, vector=141550372, maxdepth=96)
    at bytecode.c:689
#33 0x0813cf9f in funcall_lambda (fun=141550580, nargs=1, 
    arg_vector=0xbffe8478) at eval.c:2913
#34 0x0813cb75 in Ffuncall (nargs=2, args=0xbffe8474) at eval.c:2783
#35 0x0813c30e in Fapply (nargs=2, args=0xbffe8474) at eval.c:2179
#36 0x0813c9f6 in Ffuncall (nargs=3, args=0xbffe8470) at eval.c:2707
#37 0x08165c80 in Fbyte_code (bytestr=143098715, vector=140246412, maxdepth=32)
    at bytecode.c:689
#38 0x0813cf9f in funcall_lambda (fun=141529212, nargs=2, 
    arg_vector=0xbffe8658) at eval.c:2913
#39 0x0813cb75 in Ffuncall (nargs=3, args=0xbffe8654) at eval.c:2783
#40 0x0813c30e in Fapply (nargs=3, args=0xbffe8654) at eval.c:2179
#41 0x0813c9f6 in Ffuncall (nargs=4, args=0xbffe8650) at eval.c:2707
#42 0x08165c80 in Fbyte_code (bytestr=143098683, vector=141549828, maxdepth=40)
    at bytecode.c:689
---Type <return> to continue, or q <return> to quit---
#43 0x0813cf9f in funcall_lambda (fun=140695124, nargs=2, 
    arg_vector=0xbffe87a4) at eval.c:2913
#44 0x0813cb75 in Ffuncall (nargs=3, args=0xbffe87a0) at eval.c:2783
#45 0x0813c7e6 in call2 (fn=138315697, arg1=138327513, arg2=141535284)
    at eval.c:2542
#46 0x0810efcc in Fverify_visited_file_modtime (buf=141535284) at fileio.c:5593
#47 0x0813ca83 in Ffuncall (nargs=2, args=0xbffe88a0) at eval.c:2726
#48 0x08165c80 in Fbyte_code (bytestr=140057011, vector=141540396, maxdepth=32)
    at bytecode.c:689
#49 0x0813cf9f in funcall_lambda (fun=141540636, nargs=0, 
    arg_vector=0xbffe89c4) at eval.c:2913
#50 0x0813cb75 in Ffuncall (nargs=1, args=0xbffe89c0) at eval.c:2783
#51 0x08165c80 in Fbyte_code (bytestr=140056819, vector=141540700, maxdepth=32)
    at bytecode.c:689
#52 0x0813cf9f in funcall_lambda (fun=141540892, nargs=0, 
    arg_vector=0xbffe8ba8) at eval.c:2913
#53 0x0813cb75 in Ffuncall (nargs=1, args=0xbffe8ba4) at eval.c:2783
#54 0x0813c2ea in Fapply (nargs=2, args=0xbffe8ba4) at eval.c:2175
#55 0x0813c9f6 in Ffuncall (nargs=3, args=0xbffe8ba0) at eval.c:2707
#56 0x08165c80 in Fbyte_code (bytestr=136673299, vector=136673340, maxdepth=32)
    at bytecode.c:689
#57 0x0813c0ca in Feval (form=136673285) at eval.c:2084
#58 0x0813aef5 in Fcondition_case (args=144192805) at eval.c:1280
#59 0x0816617e in Fbyte_code (bytestr=136673051, vector=136673180, maxdepth=40)
    at bytecode.c:870
#60 0x0813cf9f in funcall_lambda (fun=136673012, nargs=1, 
    arg_vector=0xbffe8f54) at eval.c:2913
#61 0x0813cb75 in Ffuncall (nargs=2, args=0xbffe8f50) at eval.c:2783
#62 0x0813c7ca in call1 (fn=138245265, arg1=141541020) at eval.c:2520
#63 0x080e84cc in timer_check (do_it_now=1) at keyboard.c:4411
#64 0x0816bf39 in wait_reading_process_input (time_limit=1, microsecs=0, 
    read_kbd=0, do_display=0) at process.c:4128
#65 0x0816b678 in Faccept_process_output (process=143197484, timeout=8, 
    timeout_msecs=138193553) at process.c:3774
#66 0x0813caab in Ffuncall (nargs=3, args=0xbffe9360) at eval.c:2733
#67 0x08165c80 in Fbyte_code (bytestr=143174243, vector=143176372, maxdepth=64)
    at bytecode.c:689
#68 0x0813cf9f in funcall_lambda (fun=143176652, nargs=0, 
    arg_vector=0xbffe9494) at eval.c:2913
#69 0x0813cb75 in Ffuncall (nargs=1, args=0xbffe9490) at eval.c:2783
#70 0x08165c80 in Fbyte_code (bytestr=143174451, vector=143177972, maxdepth=80)
    at bytecode.c:689
#71 0x0813cf9f in funcall_lambda (fun=143178172, nargs=6, 
    arg_vector=0xbffe95d8) at eval.c:2913
#72 0x0813cb75 in Ffuncall (nargs=7, args=0xbffe95d4) at eval.c:2783
#73 0x08165c80 in Fbyte_code (bytestr=143176955, vector=143178268, maxdepth=64)
    at bytecode.c:689
#74 0x0813cf9f in funcall_lambda (fun=143178436, nargs=9, 
    arg_vector=0xbffe9704) at eval.c:2913
#75 0x0813cb75 in Ffuncall (nargs=10, args=0xbffe9700) at eval.c:2783
#76 0x08165c80 in Fbyte_code (bytestr=143095083, vector=143096220, maxdepth=88)
    at bytecode.c:689
#77 0x0813cf9f in funcall_lambda (fun=140582636, nargs=1, 
    arg_vector=0xbffe9908) at eval.c:2913
#78 0x0813cb75 in Ffuncall (nargs=2, args=0xbffe9904) at eval.c:2783
#79 0x0813c30e in Fapply (nargs=2, args=0xbffe9904) at eval.c:2179
#80 0x0813c9f6 in Ffuncall (nargs=3, args=0xbffe9900) at eval.c:2707
#81 0x08165c80 in Fbyte_code (bytestr=143098715, vector=140246412, maxdepth=32)
    at bytecode.c:689
#82 0x0813cf9f in funcall_lambda (fun=141529212, nargs=2, 
    arg_vector=0xbffe9ae8) at eval.c:2913
#83 0x0813cb75 in Ffuncall (nargs=3, args=0xbffe9ae4) at eval.c:2783
#84 0x0813c30e in Fapply (nargs=3, args=0xbffe9ae4) at eval.c:2179
---Type <return> to continue, or q <return> to quit---
#85 0x0813c9f6 in Ffuncall (nargs=4, args=0xbffe9ae0) at eval.c:2707
#86 0x08165c80 in Fbyte_code (bytestr=143098683, vector=141549828, maxdepth=40)
    at bytecode.c:689
#87 0x0813cf9f in funcall_lambda (fun=140695124, nargs=2, 
    arg_vector=0xbffe9c14) at eval.c:2913
#88 0x0813cb75 in Ffuncall (nargs=3, args=0xbffe9c10) at eval.c:2783
#89 0x08165c80 in Fbyte_code (bytestr=136139755, vector=136139788, maxdepth=32)
    at bytecode.c:689
#90 0x0813cf9f in funcall_lambda (fun=136139716, nargs=1, 
    arg_vector=0xbffe9d34) at eval.c:2913
#91 0x0813cb75 in Ffuncall (nargs=2, args=0xbffe9d30) at eval.c:2783
#92 0x08165c80 in Fbyte_code (bytestr=143095523, vector=143096572, maxdepth=72)
    at bytecode.c:689
#93 0x0813cf9f in funcall_lambda (fun=140734892, nargs=5, 
    arg_vector=0xbffe9e74) at eval.c:2913
#94 0x0813cb75 in Ffuncall (nargs=6, args=0xbffe9e70) at eval.c:2783
#95 0x0813c428 in Fapply (nargs=2, args=0xbffe9f54) at eval.c:2231
#96 0x0813c9f6 in Ffuncall (nargs=3, args=0xbffe9f50) at eval.c:2707
#97 0x08165c80 in Fbyte_code (bytestr=143098715, vector=140246412, maxdepth=32)
    at bytecode.c:689
#98 0x0813cf9f in funcall_lambda (fun=141529212, nargs=6, 
    arg_vector=0xbffea074) at eval.c:2913
#99 0x0813cb75 in Ffuncall (nargs=7, args=0xbffea070) at eval.c:2783
#100 0x0813c428 in Fapply (nargs=3, args=0xbffea154) at eval.c:2231
#101 0x0813c9f6 in Ffuncall (nargs=4, args=0xbffea150) at eval.c:2707
#102 0x08165c80 in Fbyte_code (bytestr=143098683, vector=141549828, 
    maxdepth=40) at bytecode.c:689
#103 0x0813cf9f in funcall_lambda (fun=140695124, nargs=6, 
    arg_vector=0xbffea2a4) at eval.c:2913
#104 0x0813cb75 in Ffuncall (nargs=7, args=0xbffea2a0) at eval.c:2783
#105 0x0813c856 in call6 (fn=138315697, arg1=138327465, arg2=143685203, 
    arg3=138193601, arg4=138193553, arg5=138193553, arg6=138193553)
    at eval.c:2640
#106 0x0810be1f in Finsert_file_contents (filename=143685203, visit=138193601, 
    beg=138193553, end=138193553, replace=138193553) at fileio.c:3719
#107 0x0813caf9 in Ffuncall (nargs=3, args=0xbfffea00) at eval.c:2759
#108 0x08165c80 in Fbyte_code (bytestr=136153099, vector=136153140, 
    maxdepth=24) at bytecode.c:689
#109 0x0813c0ca in Feval (form=136153085) at eval.c:2084
#110 0x0813aef5 in Fcondition_case (args=143801229) at eval.c:1280
#111 0x0816617e in Fbyte_code (bytestr=136152483, vector=136152684, 
    maxdepth=32) at bytecode.c:870
#112 0x0813cf9f in funcall_lambda (fun=136152412, nargs=6, 
    arg_vector=0xbfffed84) at eval.c:2913
#113 0x0813cb75 in Ffuncall (nargs=7, args=0xbfffed80) at eval.c:2783
#114 0x08165c80 in Fbyte_code (bytestr=136149907, vector=136150580, 
    maxdepth=64) at bytecode.c:689
#115 0x0813cf9f in funcall_lambda (fun=136149836, nargs=4, 
    arg_vector=0xbfffeeb4) at eval.c:2913
#116 0x0813cb75 in Ffuncall (nargs=5, args=0xbfffeeb0) at eval.c:2783
#117 0x08165c80 in Fbyte_code (bytestr=136143923, vector=136143972, 
    maxdepth=48) at bytecode.c:689
#118 0x0813cf9f in funcall_lambda (fun=136143868, nargs=2, 
    arg_vector=0xbfffefe4) at eval.c:2913
#119 0x0813cb75 in Ffuncall (nargs=3, args=0xbfffefe0) at eval.c:2783
#120 0x0813c428 in Fapply (nargs=2, args=0xbffff070) at eval.c:2231
#121 0x0813c793 in apply1 (fn=138299777, arg=143493061) at eval.c:2487
#122 0x081385d5 in Fcall_interactively (function=138299777, 
    record_flag=138193553, keys=138250412) at callint.c:406
#123 0x080ee60d in Fcommand_execute (cmd=138299777, record_flag=138193553, 
    keys=138193553, special=138193553) at keyboard.c:9679
#124 0x080e4963 in command_loop_1 () at keyboard.c:1727
#125 0x0813afe6 in internal_condition_case (bfun=0x80e3a90 <command_loop_1>, 
---Type <return> to continue, or q <return> to quit---
    handlers=138254481, hfun=0x80e3674 <cmd_error>) at eval.c:1333
#126 0x080e394e in command_loop_2 () at keyboard.c:1264
#127 0x0813ab57 in internal_catch (tag=138248505, 
    func=0x80e3930 <command_loop_2>, arg=138193553) at eval.c:1094
#128 0x080e38dc in command_loop () at keyboard.c:1243
#129 0x080e345b in recursive_edit_1 () at keyboard.c:959
#130 0x080e356f in Frecursive_edit () at keyboard.c:1015
#131 0x080e24ab in main (argc=4, argv=0xbffff814) at emacs.c:1692
#132 0x40317627 in __libc_start_main (main=0x80e17d4 <main>, argc=4, 
    ubp_av=0xbffff814, init=0x804e1b0 <_init>, fini=0x818462c <_fini>, 
    rtld_fini=0x4000dcd4 <_dl_fini>, stack_end=0xbffff80c)
    at ../sysdeps/generic/libc-start.c:129
============================================================

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

>From my previous message:

   (gdb) up 9
   #9  0x0813cb75 in Ffuncall (nargs=1, args=0xbffe7980) at eval.c:2783
   2783		  val = funcall_lambda (fun, numargs, args + 1);
   (gdb) p *args
   $4 = 143080985
   (gdb) pr


   Am I doing something stupid?  I was expecting to see the name of the
   Lisp function.

I believe I see know.  I was not really reading well enough.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Kim F. Storm]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> >> If a timer does something like accept-process-output, stopping other
> >> timers would probably be a mistake.
> > But it seems quite bogus for a timer to do that ...  as it blocks
> > interactive use.
> > IMO, timers should be "fast".
> 
> Agreed, but accept-process-output is not always slow and we don't always get
> to choose.  I've hacked my Emacs to signal an error if accept-process-output
> is called (without timeout) with inhibit-quit set to non-nil and discovered
> that some package (IIRC, flyspell) does exactly that.  Of course, ispell
> is a local process that always responds immeditely, right?
> Anyway: how could flyspell do its job without calling 
> accept-process-output from a timer or a post-command-hook?

timer-event-handler sets inhibit-quit to t before calling the timer's
function, so if that function calls accept-process-output, it should
explicitly set inhibit-quit to nil -- but how does that affect the 
timer system ?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Tramp with global-auto-revert-mode.

[Kim F. Storm]

David Kastrup <dak@gnu.org> writes:

> storm@cua.dk (Kim F. Storm) writes:
> 
> > David Kastrup <dak@gnu.org> writes:
> > > 
> > > We probably should not run one and the same timer while it has not
> > > yet finished, but freeze different ones?
> > 
> > That would be easy (it can be done in timer-event-handler).
> 
> Uh, that was a rhetorical question, short for "are you really sure
> that freezing different timers would be a good idea?".
> 

Doh!  I was referring to the first part of that sentence:

> > > We probably should not run one and the same timer while it has not
> > > yet finished

That would be easy.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Tramp with global-auto-revert-mode.

[Kim F. Storm]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > I believe that the freezes are not an Emacs or Tramp bug, but the
> > inevitable result of a slow connection.  Hence we need a user option
> > to deal with them.
> 
> If C-g does not get you out of the hang, it's a bug.
> As I said, the with-local-quit should be put in Tramp.
> You might be right that auto-reverting remote files needs to be turned off
> or customizable, but the problems you're experiencing could also be
> triggered by other packages than auto-revert, so we should fix them.
> 
> Among all the problems, one of them is actually in autorevert.el:
> the auto-revert timer should not be re-run if it hasn't finished running
> yet.  One way to do that is to not use a repeating timer, but instead to
> reset the timer manually at the end of auto-revert-buffers.  Another is to
> bind auto-revert-running to t in auto-revert-buffers and check it before
> running, as in the patch below.  A third option is to do this kind of
> re-entrance-check directly in the C code.

The C code actually does this, but the lisp handler breaks this.
I will install a patch shortly.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

>From my previous message:

   I believe I see know.  I was not really reading well enough.

I am afraid that I still do not get it.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Stefan Monnier]

>> Agreed, but accept-process-output is not always slow and we don't always get
>> to choose.  I've hacked my Emacs to signal an error if accept-process-output
>> is called (without timeout) with inhibit-quit set to non-nil and discovered
>> that some package (IIRC, flyspell) does exactly that.  Of course, ispell
>> is a local process that always responds immeditely, right?
>> Anyway: how could flyspell do its job without calling 
>> accept-process-output from a timer or a post-command-hook?

> timer-event-handler sets inhibit-quit to t before calling the timer's
> function, so if that function calls accept-process-output, it should
> explicitly set inhibit-quit to nil -- but how does that affect the 
> timer system ?

It doesn't except that it's while debugging such
blocking-with-quit-inhibited bugs that I discovered the above example of
a perfectly valid call to accept-process-output from a piece of code that
needs to be run from a timer or a post-command-hook.


        Stefan

Re: Tramp with global-auto-revert-mode.

[Kim F. Storm]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> (gdb) p *args
> $7 = 143080985
> (gdb) xtype
> Lisp_Symbol
> (gdb) xsymbol
> $8 = (struct Lisp_Symbol *) 0x8873e18
> Argument to arithmetic operation not a number or boolean.
> (gdb) 

Try: p *$8

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Tramp with global-auto-revert-mode.

[David Kastrup]

Kai Grossjohann <kai.grossjohann@gmx.net> writes:

> storm@cua.dk (Kim F. Storm) writes:
> 
> > How does it know where the output from the previous operation ends
> > (or have arrived at all), i.e. how does it know when the output to its
> > own command starts?
> 
> I think that David is suggesting that the timer does not start its
> own command until the command-in-process is finished.  So it goes
> like this:
> 
> - User invokes Tramp in some manner.
> 
> - Tramp sends a command to the remote host and enters a loop calling
>   accept-process-output until the command is complete.  (Tramp knows
>   to look for a shell prompt to see that the command is complete.)
> 
> - This loop is interrupted by a timer.  The timer sees that such a
>   loop is in progress already.
> 
> - The timer enters its own accept-process-output loop, fetching
>   output from the unknown command until the command completes (shell
>   prompt arrives).  It stashes the output away someplace safe.
> 
> - Now the timer is free to do its own thing: it sends a command to
>   the remote end, uses the accept-process-output loop to fetch the
>   result, is happy.
> 
> - Just before returning, the timer fetches the output from the safe
>   place and inserts it into the connection buffer.  Then the timer
>   sends a no-op command to the remote end (but does *not* invoke
>   accept-process-output) and returns.
> 
> - This is the point in time where the "master" Emacs process gets to
>   do something.  It's still inside accept-process-output.  The no-op
>   command sent by the timer produces output (a shell prompt) which
>   the now-awakened accept-process-output is happy to receive, and it
>   wakes up.

Uh, much too complicated.  accept-process-output does _not_ read any
process-output.  It just puts a process to sleep until something has
been processed.  Since it is not processing the output itself (but
relies on a filter function or the default buffer insertion to do so),
we don't need any do-nothing commands or whatsoever.  When output
arrives, the currently active filter function will be called with it
and _all_ accept-process-output calling threads will get woken up
again.  So they will need to check whether their respective work has
already been done or aborted by the filter routine.

> I hope I got that right.

Too complicated, I guess.

> It seems rather fragile to me.

Same here.  The filter routine just should just mark the last command
sent to the remote shell as finished, and then see whether any
job/command is still pending.  If yes, it sends it.

Every accept-process-output checks whether its job is finished, in
which case it returns, has been dequeued by force (closed connection
whatever) in which case a signal is probably thrown, or is either
still in queue or currently in progress.  In which case
accept-process-output is called again.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

I am just blundering along, I guess.

I got function names to print out correctly at higher up frames, but
then I went back to that particular frame and did as you suggested:

(gdb) down 83
#0  abort () at emacs.c:433
433   kill (getpid (), SIGABRT);
(gdb) up 9
#9  0x0813cb75 in Ffuncall (nargs=1, args=0xbffe7980) at eval.c:2783
2783		  val = funcall_lambda (fun, numargs, args + 1);
(gdb) p *args
$24 = 143080985
(gdb) pr

(gdb) p *args
$25 = 143080985
(gdb) xtype
Lisp_Symbol
(gdb) xsymbol
$26 = (struct Lisp_Symbol *) 0x8873e18
Argument to arithmetic operation not a number or boolean.
(gdb) p *$26
$27 = {
  gcmarkbit = 1, 
  indirect_variable = 0, 
  constant = 0, 
  interned = 2, 
  xname = 141437083, 
  value = 138193577, 
  function = 143176652, 
  plist = 138193553, 
  next = 0x8453fb8
}

Then I instinctively typed:

(gdb) pr

Should have known better:

Program received signal SIGSEGV, Segmentation fault.
0x0814c829 in print_object (obj=17, printcharfun=138277665,
escapeflag=1)
    at print.c:1607
1607	register unsigned char *p = SDATA (SYMBOL_NAME (obj));
The program being debugged was signaled while in a function called
from GDB.
GDB remains in the frame where the signal was received.
To change this behavior use "set unwindonsignal on"
Evaluation of the expression containing the function (debug_print)
will be abandoned.

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Kim Storm wrote:

   The C code actually does this, but the lisp handler breaks this.
   I will install a patch shortly.

I now realize that I should have read etc/DEBUG more carefully before
impatiently starting to go ahead.  I believe I now understand what I
should have done.  Anyway, I will first see whether the crash still
occurs after your latest changes to timer.el and, if so, try to
produce a Lisp backtrace for that one.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Kim Storm wrote:

   The C code actually does this, but the lisp handler breaks this.
   I will install a patch shortly.

Interesting.  Now I was able to load four remote files, without any
problems.  No crash, no base64 error messages.  Then I do 
M-x customize-group RET auto-revert to check whether I did not
accidentally change some option.  Crash.

Here is a C backtace.  I will try to produce a Lisp backtrace
manually.  This time I will not try any more to come up with the name
of a function object that has no name.  I hope xbacktrace does not try
to that too, but anyway, it again does not work.

===File ~/crash3============================================
(gdb) run -q --eval '(progn (blink-cursor-mode 0) (global-auto-revert-mode 1))'
Starting program: /home/teirllm/emacscvsdir/emacs/src/emacs-21.3.50.1 -q --eval '(progn (blink-cursor-mode 0) (global-auto-revert-mode 1))'

Breakpoint 1, abort () at emacs.c:433
433	  kill (getpid (), SIGABRT);
(gdb) xbacktrace
Invalid number -2147483625 of repetitions.
(gdb) bt
#0  abort () at emacs.c:433
#1  0x0812a575 in mark_object (arg=144520658) at alloc.c:5034
#2  0x0812a5d6 in mark_object (arg=144315413) at alloc.c:5051
#3  0x0812a5d6 in mark_object (arg=144310277) at alloc.c:5051
#4  0x0812948a in mark_memory (start=0xbfffc780, end=0xbffff57c)
    at alloc.c:3781
#5  0x081294f5 in mark_stack () at alloc.c:4055
#6  0x08129aba in Fgarbage_collect () at alloc.c:4429
#7  0x0813c887 in Ffuncall (nargs=3, args=0xbfffc918) at eval.c:2666
#8  0x08165c50 in Fbyte_code (bytestr=143196355, vector=143198068, maxdepth=64)
    at bytecode.c:689
#9  0x0813cf6f in funcall_lambda (fun=143198276, nargs=1, 
    arg_vector=0xbfffca44) at eval.c:2913
#10 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffca40) at eval.c:2783
#11 0x08165c50 in Fbyte_code (bytestr=143219675, vector=143221980, maxdepth=48)
    at bytecode.c:689
#12 0x0813cf6f in funcall_lambda (fun=143222156, nargs=1, 
    arg_vector=0xbfffcb74) at eval.c:2913
#13 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffcb70) at eval.c:2783
#14 0x08165c50 in Fbyte_code (bytestr=143100619, vector=140058860, maxdepth=24)
    at bytecode.c:689
#15 0x0813cf6f in funcall_lambda (fun=140058084, nargs=1, 
    arg_vector=0xbfffcc94) at eval.c:2913
#16 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffcc90) at eval.c:2783
#17 0x08165c50 in Fbyte_code (bytestr=143100651, vector=140057732, maxdepth=40)
    at bytecode.c:689
#18 0x0813cf6f in funcall_lambda (fun=140821156, nargs=3, 
    arg_vector=0xbfffcde4) at eval.c:2913
#19 0x0813cb45 in Ffuncall (nargs=4, args=0xbfffcde0) at eval.c:2783
#20 0x0813c7d2 in call3 (fn=138573993, arg1=138326689, arg2=144357203, 
    arg3=140947419) at eval.c:2565
#21 0x0810951f in Fexpand_file_name (name=144357203, 
    default_directory=140947419) at fileio.c:1705
#22 0x0810ae74 in Ffile_readable_p (filename=144357203) at fileio.c:3090
#23 0x0813ca53 in Ffuncall (nargs=2, args=0xbfffcf20) at eval.c:2726
#24 0x08165c50 in Fbyte_code (bytestr=138319363, vector=141527212, maxdepth=40)
    at bytecode.c:689
#25 0x0813cf6f in funcall_lambda (fun=141527404, nargs=1, 
    arg_vector=0xbfffd054) at eval.c:2913
#26 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffd050) at eval.c:2783
#27 0x08165c50 in Fbyte_code (bytestr=143400275, vector=143770820, maxdepth=56)
    at bytecode.c:689
#28 0x0813cf6f in funcall_lambda (fun=143709916, nargs=1, 
    arg_vector=0xbfffd184) at eval.c:2913
#29 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffd180) at eval.c:2783
#30 0x08165c50 in Fbyte_code (bytestr=143400291, vector=143308492, maxdepth=48)
    at bytecode.c:689
#31 0x0813cf6f in funcall_lambda (fun=143628260, nargs=4, 
    arg_vector=0xbfffd2b4) at eval.c:2913
#32 0x0813cb45 in Ffuncall (nargs=5, args=0xbfffd2b0) at eval.c:2783
#33 0x08165c50 in Fbyte_code (bytestr=143579067, vector=143742476, maxdepth=48)
    at bytecode.c:689
#34 0x0813cf6f in funcall_lambda (fun=143361260, nargs=1, 
    arg_vector=0xbfffd464) at eval.c:2913
#35 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffd460) at eval.c:2783
#36 0x0813c2ba in Fapply (nargs=3, args=0xbfffd460) at eval.c:2175
#37 0x0814609f in Fwidget_apply (nargs=2, args=0xbfffd4f4) at fns.c:3519
#38 0x0813c9c6 in Ffuncall (nargs=3, args=0xbfffd4f0) at eval.c:2707
#39 0x08165c50 in Fbyte_code (bytestr=143523147, vector=144170052, maxdepth=72)
    at bytecode.c:689
#40 0x0813cf6f in funcall_lambda (fun=143300308, nargs=1, 
    arg_vector=0xbfffd6b4) at eval.c:2913
#41 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffd6b0) at eval.c:2783
---Type <return> to continue, or q <return> to quit---
#42 0x0813c2ba in Fapply (nargs=3, args=0xbfffd6b0) at eval.c:2175
#43 0x0814609f in Fwidget_apply (nargs=2, args=0xbfffd744) at fns.c:3519
#44 0x0813c9c6 in Ffuncall (nargs=3, args=0xbfffd740) at eval.c:2707
#45 0x08165c50 in Fbyte_code (bytestr=143285115, vector=143742332, maxdepth=56)
    at bytecode.c:689
#46 0x0813cf6f in funcall_lambda (fun=143629564, nargs=11, 
    arg_vector=0xbfffd874) at eval.c:2913
#47 0x0813cb45 in Ffuncall (nargs=12, args=0xbfffd870) at eval.c:2783
#48 0x08165c50 in Fbyte_code (bytestr=143578795, vector=143367724, 
    maxdepth=136) at bytecode.c:689
#49 0x0813cf6f in funcall_lambda (fun=143478244, nargs=1, 
    arg_vector=0xbfffda54) at eval.c:2913
#50 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffda50) at eval.c:2783
#51 0x0813c2ba in Fapply (nargs=3, args=0xbfffda50) at eval.c:2175
#52 0x0814609f in Fwidget_apply (nargs=2, args=0xbfffdae4) at fns.c:3519
#53 0x0813c9c6 in Ffuncall (nargs=3, args=0xbfffdae0) at eval.c:2707
#54 0x08165c50 in Fbyte_code (bytestr=143523147, vector=144170052, maxdepth=72)
    at bytecode.c:689
#55 0x0813cf6f in funcall_lambda (fun=143300308, nargs=1, 
    arg_vector=0xbfffdca4) at eval.c:2913
#56 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffdca0) at eval.c:2783
#57 0x0813c2ba in Fapply (nargs=3, args=0xbfffdca0) at eval.c:2175
#58 0x0814609f in Fwidget_apply (nargs=2, args=0xbfffdd34) at fns.c:3519
#59 0x0813c9c6 in Ffuncall (nargs=3, args=0xbfffdd30) at eval.c:2707
#60 0x08165c50 in Fbyte_code (bytestr=143285115, vector=143742332, maxdepth=56)
    at bytecode.c:689
#61 0x0813cf6f in funcall_lambda (fun=143629564, nargs=12, 
    arg_vector=0xbfffde64) at eval.c:2913
#62 0x0813cb45 in Ffuncall (nargs=13, args=0xbfffde60) at eval.c:2783
#63 0x08165c50 in Fbyte_code (bytestr=143832635, vector=143834484, 
    maxdepth=104) at bytecode.c:689
#64 0x0813cf6f in funcall_lambda (fun=143834676, nargs=1, 
    arg_vector=0xbfffdfd4) at eval.c:2913
#65 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffdfd0) at eval.c:2783
#66 0x0813c79a in call1 (fn=143834676, arg1=139739749) at eval.c:2520
#67 0x08145264 in mapcar1 (leni=12, vals=0xbfffe050, fn=143834676, 
    seq=143939629) at fns.c:2962
#68 0x08145347 in Fmapcar (function=143834676, sequence=143939629)
    at fns.c:3022
#69 0x0813ca66 in Ffuncall (nargs=3, args=0xbfffe110) at eval.c:2729
#70 0x08165c50 in Fbyte_code (bytestr=143832059, vector=143834740, 
    maxdepth=144) at bytecode.c:689
#71 0x0813cf6f in funcall_lambda (fun=143835340, nargs=1, 
    arg_vector=0xbfffe2f4) at eval.c:2913
#72 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffe2f0) at eval.c:2783
#73 0x0813c2ba in Fapply (nargs=3, args=0xbfffe2f0) at eval.c:2175
#74 0x0814609f in Fwidget_apply (nargs=2, args=0xbfffe384) at fns.c:3519
#75 0x0813c9c6 in Ffuncall (nargs=3, args=0xbfffe380) at eval.c:2707
#76 0x08165c50 in Fbyte_code (bytestr=143523147, vector=144170052, maxdepth=72)
    at bytecode.c:689
#77 0x0813cf6f in funcall_lambda (fun=143300308, nargs=1, 
    arg_vector=0xbfffe544) at eval.c:2913
#78 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffe540) at eval.c:2783
#79 0x0813c2ba in Fapply (nargs=3, args=0xbfffe540) at eval.c:2175
#80 0x0814609f in Fwidget_apply (nargs=2, args=0xbfffe5d4) at fns.c:3519
#81 0x0813c9c6 in Ffuncall (nargs=3, args=0xbfffe5d0) at eval.c:2707
#82 0x08165c50 in Fbyte_code (bytestr=143523163, vector=143308404, maxdepth=32)
    at bytecode.c:689
#83 0x0813cf6f in funcall_lambda (fun=143629436, nargs=9, 
    arg_vector=0xbfffe6f4) at eval.c:2913
#84 0x0813cb45 in Ffuncall (nargs=10, args=0xbfffe6f0) at eval.c:2783
#85 0x08165c50 in Fbyte_code (bytestr=143386083, vector=143794796, maxdepth=80)
    at bytecode.c:689
---Type <return> to continue, or q <return> to quit---
#86 0x0813cf6f in funcall_lambda (fun=143794940, nargs=1, 
    arg_vector=0xbfffe854) at eval.c:2913
#87 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffe850) at eval.c:2783
#88 0x0813c79a in call1 (fn=143794940, arg1=143939725) at eval.c:2520
#89 0x08145264 in mapcar1 (leni=1, vals=0xbfffe8d0, fn=143794940, 
    seq=143939717) at fns.c:2962
#90 0x08145347 in Fmapcar (function=143794940, sequence=143939717)
    at fns.c:3022
#91 0x0813ca66 in Ffuncall (nargs=3, args=0xbfffe970) at eval.c:2729
#92 0x08165c50 in Fbyte_code (bytestr=143303011, vector=143229492, maxdepth=80)
    at bytecode.c:689
#93 0x0813cf6f in funcall_lambda (fun=143795260, nargs=2, 
    arg_vector=0xbfffeab4) at eval.c:2913
#94 0x0813cb45 in Ffuncall (nargs=3, args=0xbfffeab0) at eval.c:2783
#95 0x08165c50 in Fbyte_code (bytestr=143303203, vector=143793340, maxdepth=24)
    at bytecode.c:689
#96 0x0813cf6f in funcall_lambda (fun=143793476, nargs=3, 
    arg_vector=0xbfffebd4) at eval.c:2913
#97 0x0813cb45 in Ffuncall (nargs=4, args=0xbfffebd0) at eval.c:2783
#98 0x08165c50 in Fbyte_code (bytestr=143363443, vector=143788396, maxdepth=48)
    at bytecode.c:689
#99 0x0813cf6f in funcall_lambda (fun=143790948, nargs=1, 
    arg_vector=0xbfffed84) at eval.c:2913
#100 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffed80) at eval.c:2783
#101 0x0813c2de in Fapply (nargs=2, args=0xbfffed80) at eval.c:2179
#102 0x0813c763 in apply1 (fn=138503777, arg=143949915) at eval.c:2487
#103 0x081385a5 in Fcall_interactively (function=138503777, 
    record_flag=138193537, keys=138250348) at callint.c:406
#104 0x080ee5dd in Fcommand_execute (cmd=138503777, record_flag=138193537, 
    keys=138193489, special=138193489) at keyboard.c:9670
#105 0x080ee84e in Fexecute_extended_command (prefixarg=138193489)
    at keyboard.c:9781
#106 0x0813ca53 in Ffuncall (nargs=2, args=0xbffff070) at eval.c:2726
#107 0x081394c9 in Fcall_interactively (function=138250769, 
    record_flag=17274186, keys=138250348) at callint.c:862
#108 0x080ee5dd in Fcommand_execute (cmd=138250769, record_flag=138193489, 
    keys=138193489, special=138193489) at keyboard.c:9670
#109 0x080e4963 in command_loop_1 () at keyboard.c:1727
#110 0x0813afb6 in internal_condition_case (bfun=0x80e3a90 <command_loop_1>, 
    handlers=138254417, hfun=0x80e3674 <cmd_error>) at eval.c:1333
#111 0x080e394e in command_loop_2 () at keyboard.c:1264
#112 0x0813ab27 in internal_catch (tag=138248441, 
    func=0x80e3930 <command_loop_2>, arg=138193489) at eval.c:1094
#113 0x080e38dc in command_loop () at keyboard.c:1243
#114 0x080e345b in recursive_edit_1 () at keyboard.c:959
#115 0x080e356f in Frecursive_edit () at keyboard.c:1015
#116 0x080e24ab in main (argc=4, argv=0xbffff814) at emacs.c:1692
#117 0x40317627 in __libc_start_main (main=0x80e17d4 <main>, argc=4, 
    ubp_av=0xbffff814, init=0x804e1b0 <_init>, fini=0x81845fc <_fini>, 
    rtld_fini=0x4000dcd4 <_dl_fini>, stack_end=0xbffff80c)
    at ../sysdeps/generic/libc-start.c:129
(gdb)
============================================================

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

This time I have again trouble reading an argument and maybe this time
that is not just due to my failure to read etc/DEBUG carefully enough.
We get at:

#7  (match-string 6 "/ssh:raven.dms.auburn.edu:/home/teirllm/")
#10 #[(name) "" [] 8 ("" . 211858)]
#13 #[(filename) "" [] 6 ("" . 2640)]
#16 #[(filename) "" [] 3 ("" . 119402)]
#19 (tramp-file-name-handler expand-file-name "" "/ssh:raven.dms.auburn.edu:/home/teirllm/")
#23 (file-readable-p ???

(gdb) up 4
#23 0x0813ca53 in Ffuncall (nargs=2, args=0xbfffcf20) at eval.c:2726
2726		    val = (*XSUBR (fun)->function) (internal_args[0]);
(gdb) p *args
$33 = 138327097
(gdb) pr
file-readable-p
(gdb) p args[1]
$40 = 144357203
(gdb) xtype
Lisp_String
(gdb) xstring
$41 = (struct Lisp_String *) 0x89ab750
Invalid number -2147483598 of repetitions.

Could this be relevant to the bug in xbacktrace?

Remember:

(gdb) xbacktrace
Invalid number -2147483625 of repetitions.

After that, I absent-mindedly produced a SIGSEGV again.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

This time I was able to load ten large remote files and do
M-x customize-group.  I only got the crash when, after ages, I did
M-x customize-browse and clicked on "Editing".  Immediate crash.

Looks like an easier backtrace.  At least _some_ part of xbacktrace is
still functional, apparently.  I might wait till tomorrow to try to
translate the bt into Lisp.  Right now I would probably manage to
segfault it again.

===File ~/latestbt==========================================
(gdb) xbacktrace
"require"
"custom-load-symbol"
Invalid number -2147483630 of repetitions.
(gdb) bt
#0  abort () at emacs.c:433
#1  0x0812a575 in mark_object (arg=144569074) at alloc.c:5034
#2  0x0812a5d6 in mark_object (arg=145360869) at alloc.c:5051
#3  0x0812a5d6 in mark_object (arg=145358853) at alloc.c:5051
#4  0x0812948a in mark_memory (start=0xbfffd790, end=0xbffff57c)
    at alloc.c:3781
#5  0x081294f5 in mark_stack () at alloc.c:4055
#6  0x08129aba in Fgarbage_collect () at alloc.c:4429
#7  0x0813bdb9 in Feval (form=145267357) at eval.c:1977
#8  0x0814fb75 in readevalloop (readcharfun=138273097, stream=0x88b0090, 
    sourcename=136026155, evalfun=0x813bd28 <Feval>, printflag=0, 
    unibyte=138193489, readfun=138193489) at lread.c:1376
#9  0x0814eec5 in Fload (file=136026155, noerror=138193489, 
    nomessage=138193537, nosuffix=138193489, must_suffix=17274192)
    at lread.c:914
#10 0x08145de9 in Frequire (feature=138502713, filename=138193489, 
    noerror=138193489) at fns.c:3417
#11 0x0813c09a in Feval (form=136045605) at eval.c:2084
#12 0x0813aec5 in Fcondition_case (args=144832669) at eval.c:1280
#13 0x0816614e in Fbyte_code (bytestr=136045163, vector=136045316, maxdepth=40)
    at bytecode.c:870
#14 0x0813cf6f in funcall_lambda (fun=136045124, nargs=1, 
    arg_vector=0xbfffdea4) at eval.c:2913
#15 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffdea0) at eval.c:2783
#16 0x08165c50 in Fbyte_code (bytestr=143329475, vector=143121692, maxdepth=24)
    at bytecode.c:689
#17 0x0813cf6f in funcall_lambda (fun=143558988, nargs=1, 
    arg_vector=0xbfffdfc4) at eval.c:2913
#18 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffdfc0) at eval.c:2783
#19 0x08165c50 in Fbyte_code (bytestr=143398043, vector=143312572, 
    maxdepth=144) at bytecode.c:689
#20 0x0813cf6f in funcall_lambda (fun=143732732, nargs=1, 
    arg_vector=0xbfffe1a4) at eval.c:2913
#21 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffe1a0) at eval.c:2783
#22 0x0813c2ba in Fapply (nargs=3, args=0xbfffe1a0) at eval.c:2175
#23 0x0814609f in Fwidget_apply (nargs=2, args=0xbfffe234) at fns.c:3519
#24 0x0813c9c6 in Ffuncall (nargs=3, args=0xbfffe230) at eval.c:2707
#25 0x08165c50 in Fbyte_code (bytestr=143419323, vector=144229652, maxdepth=72)
    at bytecode.c:689
#26 0x0813cf6f in funcall_lambda (fun=143773748, nargs=1, 
    arg_vector=0xbfffe3f4) at eval.c:2913
#27 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffe3f0) at eval.c:2783
#28 0x0813c2ba in Fapply (nargs=3, args=0xbfffe3f0) at eval.c:2175
#29 0x0814609f in Fwidget_apply (nargs=2, args=0xbfffe484) at fns.c:3519
#30 0x0813c9c6 in Ffuncall (nargs=3, args=0xbfffe480) at eval.c:2707
#31 0x08165c50 in Fbyte_code (bytestr=143419307, vector=144229612, maxdepth=32)
    at bytecode.c:689
#32 0x0813cf6f in funcall_lambda (fun=143766900, nargs=9, 
    arg_vector=0xbfffe5a4) at eval.c:2913
#33 0x0813cb45 in Ffuncall (nargs=10, args=0xbfffe5a0) at eval.c:2783
#34 0x08165c50 in Fbyte_code (bytestr=143347427, vector=143757276, maxdepth=80)
    at bytecode.c:689
#35 0x0813cf6f in funcall_lambda (fun=143756860, nargs=1, 
    arg_vector=0xbfffe704) at eval.c:2913
#36 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffe700) at eval.c:2783
#37 0x0813c79a in call1 (fn=143756860, arg1=143568245) at eval.c:2520
#38 0x08145264 in mapcar1 (leni=1, vals=0xbfffe780, fn=143756860, 
    seq=143568253) at fns.c:2962
#39 0x08145347 in Fmapcar (function=143756860, sequence=143568253)
    at fns.c:3022
#40 0x0813ca66 in Ffuncall (nargs=3, args=0xbfffe820) at eval.c:2729
#41 0x08165c50 in Fbyte_code (bytestr=140741795, vector=143776468, maxdepth=80)
    at bytecode.c:689
---Type <return> to continue, or q <return> to quit---
#42 0x0813cf6f in funcall_lambda (fun=143755740, nargs=2, 
    arg_vector=0xbfffe964) at eval.c:2913
#43 0x0813cb45 in Ffuncall (nargs=3, args=0xbfffe960) at eval.c:2783
#44 0x08165c50 in Fbyte_code (bytestr=140741971, vector=143753868, maxdepth=48)
    at bytecode.c:689
#45 0x0813cf6f in funcall_lambda (fun=143726996, nargs=3, 
    arg_vector=0xbfffea94) at eval.c:2913
#46 0x0813cb45 in Ffuncall (nargs=4, args=0xbfffea90) at eval.c:2783
#47 0x08165c50 in Fbyte_code (bytestr=143592315, vector=143183804, maxdepth=48)
    at bytecode.c:689
#48 0x0813cf6f in funcall_lambda (fun=140707652, nargs=1, 
    arg_vector=0xbfffebc4) at eval.c:2913
#49 0x0813cb45 in Ffuncall (nargs=2, args=0xbfffebc0) at eval.c:2783
#50 0x08165c50 in Fbyte_code (bytestr=143579563, vector=143754380, maxdepth=24)
    at bytecode.c:689
#51 0x0813cf6f in funcall_lambda (fun=143752596, nargs=2, 
    arg_vector=0xbfffed64) at eval.c:2913
#52 0x0813cb45 in Ffuncall (nargs=3, args=0xbfffed60) at eval.c:2783
#53 0x0813c2de in Fapply (nargs=3, args=0xbfffed60) at eval.c:2179
#54 0x0814609f in Fwidget_apply (nargs=3, args=0xbfffedf4) at fns.c:3519
#55 0x0813c9c6 in Ffuncall (nargs=4, args=0xbfffedf0) at eval.c:2707
#56 0x08165c50 in Fbyte_code (bytestr=144629011, vector=144225468, maxdepth=32)
    at bytecode.c:689
#57 0x0813cf6f in funcall_lambda (fun=143558636, nargs=2, 
    arg_vector=0xbfffef14) at eval.c:2913
#58 0x0813cb45 in Ffuncall (nargs=3, args=0xbfffef10) at eval.c:2783
#59 0x08165c50 in Fbyte_code (bytestr=144629027, vector=144404340, maxdepth=32)
    at bytecode.c:689
#60 0x0813cf6f in funcall_lambda (fun=143522236, nargs=1, 
    arg_vector=0xbffff064) at eval.c:2913
#61 0x0813cb45 in Ffuncall (nargs=2, args=0xbffff060) at eval.c:2783
#62 0x081394c9 in Fcall_interactively (function=138579001, 
    record_flag=17274186, keys=138250348) at callint.c:862
#63 0x080ee5dd in Fcommand_execute (cmd=138579001, record_flag=138193489, 
    keys=138193489, special=138193489) at keyboard.c:9670
#64 0x080e4963 in command_loop_1 () at keyboard.c:1727
#65 0x0813afb6 in internal_condition_case (bfun=0x80e3a90 <command_loop_1>, 
    handlers=138254417, hfun=0x80e3674 <cmd_error>) at eval.c:1333
#66 0x080e394e in command_loop_2 () at keyboard.c:1264
#67 0x0813ab27 in internal_catch (tag=138248441, 
    func=0x80e3930 <command_loop_2>, arg=138193489) at eval.c:1094
#68 0x080e38dc in command_loop () at keyboard.c:1243
#69 0x080e345b in recursive_edit_1 () at keyboard.c:959
#70 0x080e356f in Frecursive_edit () at keyboard.c:1015
#71 0x080e24ab in main (argc=4, argv=0xbffff814) at emacs.c:1692
#72 0x40317627 in __libc_start_main (main=0x80e17d4 <main>, argc=4, 
    ubp_av=0xbffff814, init=0x804e1b0 <_init>, fini=0x81845fc <_fini>, 
    rtld_fini=0x4000dcd4 <_dl_fini>, stack_end=0xbffff80c)
    at ../sysdeps/generic/libc-start.c:129
(gdb) 
============================================================

Re: Tramp with global-auto-revert-mode.

[Andreas Schwab]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> (gdb) xstring
> $41 = (struct Lisp_String *) 0x89ab750
> Invalid number -2147483598 of repetitions.

What are the contents of the pointer?

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Andreas Schwab wrote:

   Luc Teirlinck <teirllm@dms.auburn.edu> writes:

   > (gdb) xstring
   > $41 = (struct Lisp_String *) 0x89ab750
   > Invalid number -2147483598 of repetitions.

   What are the contents of the pointer?

I should have thought of that:

(gdb) p *$41
$75 = {
  size = -2147483598, 
  size_byte = -1, 
  intervals = 0x0, 
  data = 0x89a8984
"/usr/local/share/emacs/21.3.50/lisp/mh-e/down.jpeg"
}

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

I do not have a file "/usr/local/share/emacs/21.3.50/lisp/mh-e/down.jpeg"
by the way.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Kai Grossjohann]

David Kastrup <dak@gnu.org> writes:

> Uh, much too complicated.

I'm afraid I don't understand you: Do you mean that just the last
step is too complicated, or that the whole procedure that I described
is too complicated?

At some point, I thought I understood what you were suggesting, but
now I'm not sure anymore.

Let me try to describe my understanding with less detail, then please
tell me if I got it at least at this level.

    Whenever Tramp is invoked, it looks to see if the connection
    buffer is busy.  If it is, it knows that Tramp is interrupting
    itself, so to speak.

    If this is the case (Tramp is interrupting itself), Tramp "takes
    over" the command in progress, fetches all output from it and
    stashes it someplace safe.

    Then it does its own thing.

    Afterwards, it gets the output from the safe place, inserts it
    into the connection buffer, and makes it appear to the
    "interrupted" "master" Tramp as if nothing had happened in the
    meantime.

You were also talking about a queue of commands, which I'm not seeing.
So this would be an indication that I misunderstood you.

(The above does imply a stack of commands, corresponding to
interruptions of interruptions.)

> When output arrives, the currently active filter function will be
> called with it and _all_ accept-process-output calling threads will
> get woken up again.  So they will need to check whether their
> respective work has already been done or aborted by the filter
> routine.

Hm.  If more than one thread of execution will be woken up at the
same time, then surely the above procedure won't work.

> I wrote:
>
>> It seems rather fragile to me.
>
> Same here.  The filter routine just should just mark the last command
> sent to the remote shell as finished, and then see whether any
> job/command is still pending.  If yes, it sends it.

Maybe once I understand your proposal I will understand that it is
less fragile...

Kai

Re: Tramp with global-auto-revert-mode.

[Kai Grossjohann]

Michael Albinus <michael.albinus@gmx.de> writes:

> Optimization (like multiple connections as mentioned by Kai) would be
> the next step. But I doubt it is usefull for such operations like
> `file-readable-p', given the overhead of a new connection.

I was thinking of using multiple connections not as an optimization
regarding speed, but more of an optimization regarding complexity.
It seems to me it would be much easier to implement.

My plan was to keep the additional connections open; so speaking of
opening a connection as overhead for `file-readable-p' is a little
bit misleading: it would only happen seldom.  Given the frequency in
which auto-revert is doing things, I expect that in effect a
connection would be dedicated to auto-revert.

Kai

Re: Tramp with global-auto-revert-mode.

[Andreas Schwab]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> (gdb) p *$41
> $75 = {
>   size = -2147483598, 
>   size_byte = -1, 
>   intervals = 0x0, 
>   data = 0x89a8984
> "/usr/local/share/emacs/21.3.50/lisp/mh-e/down.jpeg"
> }

-2147483598 is 50 | ARRAY_MARK_FLAG.  I have fixed the gdb makros to mask
off ARRAY_MARK_FLAG from vector sizes.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Tramp with global-auto-revert-mode.

[David Kastrup]

Kai Grossjohann <kai.grossjohann@gmx.net> writes:

> David Kastrup <dak@gnu.org> writes:
> 
> > Uh, much too complicated.
> 
> I'm afraid I don't understand you: Do you mean that just the last
> step is too complicated, or that the whole procedure that I described
> is too complicated?

The whole procedure.

> At some point, I thought I understood what you were suggesting, but
> now I'm not sure anymore.
> 
> Let me try to describe my understanding with less detail, then please
> tell me if I got it at least at this level.
> 
>     Whenever Tramp is invoked, it looks to see if the connection
>     buffer is busy.  If it is, it knows that Tramp is interrupting
>     itself, so to speak.
> 
>     If this is the case (Tramp is interrupting itself), Tramp "takes
>     over" the command in progress, fetches all output from it and
>     stashes it someplace safe.

Look, you are talking all the time about "Tramp" as if it was a
sentient being.  It isn't.

"Tramp" consists of two entirely different pieces: the user level
routines (of which several can be running at once in different
"threads" or Emacs' equivalence to it) and the filter routine.  Those
are basically independent from each other.

In order avoid unnecessary lockup in process-send-string, and
problems with identifying responses and input, we might not just
indiscriminately call process-send-string when there are already
outstanding commands.

Ok, here is what the filter routine does when it is called: it
collects the stuff from the output until it has a completely reply to
the currently sent command available.  If it has, it takes the
request and marks it as completed (tacking the results to the
request).

[Entry point for getting a command on the way:]
It then takes a look whether there are still outstanding commands in
the queue.  If there are, it takes the next one from the queue and
sends it through, marking it as being in progress.

That's all.  The filter routine never changes, it does just that.
There is only one filter routine at work at most at any time.

Now for the user level stuff: it knows it needs to get commands
through.  So it makes a request data structure and tacks it to the end
of the current queue (or, if the command is particularly urgent, like
when we are doing autorevert checking, to the _front_ of the current
queue) and then calls accept-process-output on the process repeatedly
until the command finally is marked as being processed.  Then it
takes the results and returns.

That is all.

>     Then it does its own thing.
> 
>     Afterwards, it gets the output from the safe place, inserts it
>     into the connection buffer, and makes it appear to the
>     "interrupted" "master" Tramp as if nothing had happened in the
>     meantime.
> 
> You were also talking about a queue of commands, which I'm not
> seeing.  So this would be an indication that I misunderstood you.

Probably.  You don't just want to stuff the pipe with
process-send-string indiscriminately, it may stall at one time, and
then a different thread might feel tempted to stuff its material
right in the wrong place.  So at any time, at most one thread may be
allowed to send strings.  If there is already a reason for the filter
routine to run (because some command is in the process of being
transmitted), then the requesting process will _not_ write to the
tty, but meekly queue his request into a request queue and hope that
the filter routine will at some time be friendly enough to process it.

> (The above does imply a stack of commands, corresponding to
> interruptions of interruptions.)

Quite.  Since Emacs does not have true concurrency, "deeper" commands
in the "user stack" will not get resumed or finished until "higher"
commands are completed.  Which they eventually will, since the filter
routine will cater for everything in the queue.  For stuff like
autorevert it would be a good idea to check whether the queue already
contains some request for the same thing.  If it does, no need to add
another one.  Similar things hold for fetching a particular directory
into the directory buffer, although here the resulting information
_is_ needed, so one needs to call accept-process-output until the
information has arrived, even though it was requested by a different
thread/layer/task/whatyouwantocallit.

> > When output arrives, the currently active filter function will be
> > called with it and _all_ accept-process-output calling threads
> > will get woken up again.  So they will need to check whether their
> > respective work has already been done or aborted by the filter
> > routine.
> 
> Hm.  If more than one thread of execution will be woken up at the
> same time, then surely the above procedure won't work.

Emacs can't do anything "at the same time".  We don't have
concurrency.  It has a stack of outstanding commands started from the
main loop (and a list of things it may call from the main loop).  It
works this stack off top to bottom.

Even if Emacs had separate stacks for the main loops, it would not
schedule between them except when control is explicitly yielded (can
happen with I/O).

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Tramp with global-auto-revert-mode.

[Kai Grossjohann]

David Kastrup <dak@gnu.org> writes:

> Kai Grossjohann <kai.grossjohann@gmx.net> writes:
>
>> David Kastrup <dak@gnu.org> writes:
>> 
>> > Uh, much too complicated.
>> 
>> I'm afraid I don't understand you: Do you mean that just the last
>> step is too complicated, or that the whole procedure that I described
>> is too complicated?
>
> The whole procedure.

Okay.  I agree.  Though our ideas of simplicity are different -- I
was thinking of using multiple connection buffers.

>> At some point, I thought I understood what you were suggesting, but
>> now I'm not sure anymore.
>> 
>> Let me try to describe my understanding with less detail, then please
>> tell me if I got it at least at this level.
>> 
>>     Whenever Tramp is invoked, it looks to see if the connection
>>     buffer is busy.  If it is, it knows that Tramp is interrupting
>>     itself, so to speak.
>> 
>>     If this is the case (Tramp is interrupting itself), Tramp "takes
>>     over" the command in progress, fetches all output from it and
>>     stashes it someplace safe.
>
> Look, you are talking all the time about "Tramp" as if it was a
> sentient being.  It isn't.

It must be!  It even changed names multiple times, this tells us it
knows what it likes.

> "Tramp" consists of two entirely different pieces: the user level
> routines (of which several can be running at once in different
> "threads" or Emacs' equivalence to it) and the filter routine.  Those
> are basically independent from each other.

What is a filter routine?  If you are talking about process filters,
in the sense of set-process-filter, then there are no filter routines
in Tramp.

Are you suggesting to change Tramp such that there is a filter
routine?

[time passes]  Oh, I see that Michael added a process filter in one
part of Tramp, to support async shell commands.

> In order avoid unnecessary lockup in process-send-string, and
> problems with identifying responses and input, we might not just
> indiscriminately call process-send-string when there are already
> outstanding commands.

Yes.

> Ok, here is what the filter routine does when it is called: it
> collects the stuff from the output until it has a completely reply to
> the currently sent command available.  If it has, it takes the
> request and marks it as completed (tacking the results to the
> request).
>
> [Entry point for getting a command on the way:]
> It then takes a look whether there are still outstanding commands in
> the queue.  If there are, it takes the next one from the queue and
> sends it through, marking it as being in progress.
>
> That's all.  The filter routine never changes, it does just that.
> There is only one filter routine at work at most at any time.
>
> Now for the user level stuff: it knows it needs to get commands
> through.  So it makes a request data structure and tacks it to the end
> of the current queue (or, if the command is particularly urgent, like
> when we are doing autorevert checking, to the _front_ of the current
> queue) and then calls accept-process-output on the process repeatedly
> until the command finally is marked as being processed.  Then it
> takes the results and returns.
>
> That is all.

Okay, this is much clearer.

Kai

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Andreas Schwab wrote:

   -2147483598 is 50 | ARRAY_MARK_FLAG.  I have fixed the gdb makros to mask
   off ARRAY_MARK_FLAG from vector sizes.

Wonderful.  Now xbacktrace works again.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Since Andreas fixed xbacktrace, I can now include an xbackrace from a
new crash.  The crash occurred while loading a remote file using
Tramp-ssh.  Everything in the xbacktrace seems expected, given that.
I actually manually looked at the Lisp functions in the crash I produced
yesterday and there the results were more surprising.  All involved
functions were "innocent looking" custom and widget stuff, apparently
no suspicious Tramp stuff in sight.  Of course, if the crash would be
due to memory corruption, then gc can detect it at any time.

===File ~/xbt===============================================
(gdb) run -q --eval '(progn (blink-cursor-mode 0) (global-auto-revert-mode 1))'
Starting program: /home/teirllm/emacscvsdir/emacs/src/emacs-21.3.50.1 -q --eval '(progn (blink-cursor-mode 0) (global-auto-revert-mode 1))'

Breakpoint 1, abort () at emacs.c:434
434	  kill (getpid (), SIGABRT);
(gdb) xbacktrace
"tramp-wait-for-output"
"tramp-send-command-and-check"
"tramp-barf-unless-okay"
"tramp-handle-file-local-copy"
"apply"
"tramp-sh-file-name-handler"
"apply"
"tramp-file-name-handler"
"file-local-copy"
"tramp-handle-insert-file-contents"
"apply"
"tramp-sh-file-name-handler"
"apply"
"tramp-file-name-handler"
"insert-file-contents"
"byte-code"
"find-file-noselect-1"
"find-file-noselect"
"find-file"
"call-interactively"
(gdb) bt
#0  abort () at emacs.c:434
#1  0x0812a575 in mark_object (arg=142231922) at alloc.c:5034
#2  0x0812a5d6 in mark_object (arg=142409733) at alloc.c:5051
#3  0x0812948a in mark_memory (start=0xbffe91d0, end=0xbffff57c)
    at alloc.c:3781
#4  0x081294f5 in mark_stack () at alloc.c:4055
#5  0x08129aba in Fgarbage_collect () at alloc.c:4429
#6  0x0813c887 in Ffuncall (nargs=2, args=0xbffe9360) at eval.c:2666
#7  0x08165c50 in Fbyte_code (bytestr=143174307, vector=143176292, maxdepth=64)
    at bytecode.c:689
#8  0x0813cf6f in funcall_lambda (fun=143176572, nargs=0, 
    arg_vector=0xbffe9494) at eval.c:2913
#9  0x0813cb45 in Ffuncall (nargs=1, args=0xbffe9490) at eval.c:2783
#10 0x08165c50 in Fbyte_code (bytestr=143176667, vector=143177892, maxdepth=80)
    at bytecode.c:689
#11 0x0813cf6f in funcall_lambda (fun=143178092, nargs=6, 
    arg_vector=0xbffe95d8) at eval.c:2913
#12 0x0813cb45 in Ffuncall (nargs=7, args=0xbffe95d4) at eval.c:2783
#13 0x08165c50 in Fbyte_code (bytestr=143176843, vector=143178156, maxdepth=64)
    at bytecode.c:689
#14 0x0813cf6f in funcall_lambda (fun=143178324, nargs=9, 
    arg_vector=0xbffe9704) at eval.c:2913
#15 0x0813cb45 in Ffuncall (nargs=10, args=0xbffe9700) at eval.c:2783
#16 0x08165c50 in Fbyte_code (bytestr=143095763, vector=143098012, maxdepth=88)
    at bytecode.c:689
#17 0x0813cf6f in funcall_lambda (fun=140697732, nargs=1, 
    arg_vector=0xbffe9908) at eval.c:2913
#18 0x0813cb45 in Ffuncall (nargs=2, args=0xbffe9904) at eval.c:2783
#19 0x0813c2de in Fapply (nargs=2, args=0xbffe9904) at eval.c:2179
#20 0x0813c9c6 in Ffuncall (nargs=3, args=0xbffe9900) at eval.c:2707
#21 0x08165c50 in Fbyte_code (bytestr=143099195, vector=140318948, maxdepth=32)
    at bytecode.c:689
#22 0x0813cf6f in funcall_lambda (fun=140318692, nargs=2, 
    arg_vector=0xbffe9ae8) at eval.c:2913
#23 0x0813cb45 in Ffuncall (nargs=3, args=0xbffe9ae4) at eval.c:2783
#24 0x0813c2de in Fapply (nargs=3, args=0xbffe9ae4) at eval.c:2179
#25 0x0813c9c6 in Ffuncall (nargs=4, args=0xbffe9ae0) at eval.c:2707
#26 0x08165c50 in Fbyte_code (bytestr=143099163, vector=140369356, maxdepth=40)
    at bytecode.c:689
#27 0x0813cf6f in funcall_lambda (fun=140905228, nargs=2, 
    arg_vector=0xbffe9c14) at eval.c:2913
#28 0x0813cb45 in Ffuncall (nargs=3, args=0xbffe9c10) at eval.c:2783
#29 0x08165c50 in Fbyte_code (bytestr=136139723, vector=136139756, maxdepth=32)
    at bytecode.c:689
#30 0x0813cf6f in funcall_lambda (fun=136139684, nargs=1, 
    arg_vector=0xbffe9d34) at eval.c:2913
#31 0x0813cb45 in Ffuncall (nargs=2, args=0xbffe9d30) at eval.c:2783
#32 0x08165c50 in Fbyte_code (bytestr=143096195, vector=143098364, maxdepth=72)
    at bytecode.c:689
#33 0x0813cf6f in funcall_lambda (fun=140805684, nargs=5, 
    arg_vector=0xbffe9e74) at eval.c:2913
#34 0x0813cb45 in Ffuncall (nargs=6, args=0xbffe9e70) at eval.c:2783
#35 0x0813c3f8 in Fapply (nargs=2, args=0xbffe9f54) at eval.c:2231
#36 0x0813c9c6 in Ffuncall (nargs=3, args=0xbffe9f50) at eval.c:2707
#37 0x08165c50 in Fbyte_code (bytestr=143099195, vector=140318948, maxdepth=32)
    at bytecode.c:689
#38 0x0813cf6f in funcall_lambda (fun=140318692, nargs=6, 
    arg_vector=0xbffea074) at eval.c:2913
#39 0x0813cb45 in Ffuncall (nargs=7, args=0xbffea070) at eval.c:2783
#40 0x0813c3f8 in Fapply (nargs=3, args=0xbffea154) at eval.c:2231
#41 0x0813c9c6 in Ffuncall (nargs=4, args=0xbffea150) at eval.c:2707
#42 0x08165c50 in Fbyte_code (bytestr=143099163, vector=140369356, maxdepth=40)
    at bytecode.c:689
---Type <return> to continue, or q <return> to quit---
#43 0x0813cf6f in funcall_lambda (fun=140905228, nargs=6, 
    arg_vector=0xbffea2a4) at eval.c:2913
#44 0x0813cb45 in Ffuncall (nargs=7, args=0xbffea2a0) at eval.c:2783
#45 0x0813c826 in call6 (fn=138971825, arg1=138327433, arg2=142420219, 
    arg3=138193569, arg4=138193521, arg5=138193521, arg6=138193521)
    at eval.c:2640
#46 0x0810bdef in Finsert_file_contents (filename=142420219, visit=138193569, 
    beg=138193521, end=138193521, replace=138193521) at fileio.c:3719
#47 0x0813cac9 in Ffuncall (nargs=3, args=0xbfffea00) at eval.c:2759
#48 0x08165c50 in Fbyte_code (bytestr=136153067, vector=136153108, maxdepth=24)
    at bytecode.c:689
#49 0x0813c09a in Feval (form=136153053) at eval.c:2084
#50 0x0813aec5 in Fcondition_case (args=148227829) at eval.c:1280
#51 0x0816614e in Fbyte_code (bytestr=136152451, vector=136152652, maxdepth=32)
    at bytecode.c:870
#52 0x0813cf6f in funcall_lambda (fun=136152380, nargs=6, 
    arg_vector=0xbfffed84) at eval.c:2913
#53 0x0813cb45 in Ffuncall (nargs=7, args=0xbfffed80) at eval.c:2783
#54 0x08165c50 in Fbyte_code (bytestr=136149875, vector=136150548, maxdepth=64)
    at bytecode.c:689
#55 0x0813cf6f in funcall_lambda (fun=136149804, nargs=4, 
    arg_vector=0xbfffeeb4) at eval.c:2913
#56 0x0813cb45 in Ffuncall (nargs=5, args=0xbfffeeb0) at eval.c:2783
#57 0x08165c50 in Fbyte_code (bytestr=136143891, vector=136143940, maxdepth=48)
    at bytecode.c:689
#58 0x0813cf6f in funcall_lambda (fun=136143836, nargs=2, 
    arg_vector=0xbfffefe4) at eval.c:2913
#59 0x0813cb45 in Ffuncall (nargs=3, args=0xbfffefe0) at eval.c:2783
#60 0x0813c3f8 in Fapply (nargs=2, args=0xbffff070) at eval.c:2231
#61 0x0813c763 in apply1 (fn=138309881, arg=142376469) at eval.c:2487
#62 0x081385a5 in Fcall_interactively (function=138309881, 
    record_flag=138193521, keys=138250380) at callint.c:406
#63 0x080ee5dd in Fcommand_execute (cmd=138309881, record_flag=138193521, 
    keys=138193521, special=138193521) at keyboard.c:9670
#64 0x080e4963 in command_loop_1 () at keyboard.c:1727
#65 0x0813afb6 in internal_condition_case (bfun=0x80e3a90 <command_loop_1>, 
    handlers=138254449, hfun=0x80e3674 <cmd_error>) at eval.c:1333
#66 0x080e394e in command_loop_2 () at keyboard.c:1264
#67 0x0813ab27 in internal_catch (tag=138248473, 
    func=0x80e3930 <command_loop_2>, arg=138193521) at eval.c:1094
#68 0x080e38dc in command_loop () at keyboard.c:1243
#69 0x080e345b in recursive_edit_1 () at keyboard.c:959
#70 0x080e356f in Frecursive_edit () at keyboard.c:1015
#71 0x080e24ab in main (argc=4, argv=0xbffff814) at emacs.c:1693
#72 0x40317627 in __libc_start_main (main=0x80e17d4 <main>, argc=4, 
    ubp_av=0xbffff814, init=0x804e1b0 <_init>, fini=0x81845fc <_fini>, 
    rtld_fini=0x4000dcd4 <_dl_fini>, stack_end=0xbffff80c)
    at ../sysdeps/generic/libc-start.c:129
============================================================

Re: Tramp with global-auto-revert-mode.

[David Kastrup]

Kai Grossjohann <kai.grossjohann@gmx.net> writes:

> David Kastrup <dak@gnu.org> writes:
> 
> > Kai Grossjohann <kai.grossjohann@gmx.net> writes:
> >
> >> David Kastrup <dak@gnu.org> writes:
> >> 
> >> > Uh, much too complicated.
> >> 
> >> I'm afraid I don't understand you: Do you mean that just the last
> >> step is too complicated, or that the whole procedure that I described
> >> is too complicated?
> >
> > The whole procedure.
> 
> Okay.  I agree.  Though our ideas of simplicity are different -- I
> was thinking of using multiple connection buffers.

Those have to be set up and initialized.  You know that Tramp
initialization takes a _lot_ of time.  If we have some timer-related
events, it is likely that we will get an indefinite stack of sessions
all occupied with initializing a connection.

> > Look, you are talking all the time about "Tramp" as if it was a
> > sentient being.  It isn't.
> 
> It must be!  It even changed names multiple times, this tells us it
> knows what it likes.

Pffft.  With that sort of reasoning, XEmacs aka Lucid Emacs aka
Emacs19-to-be-well-at-some-time-we-thought-so would be three times as
intelligent as Emacs.  Don't bother commenting on that: it's no fun
starting a flame war if we are all supposed to be on the same side.

> > "Tramp" consists of two entirely different pieces: the user level
> > routines (of which several can be running at once in different
> > "threads" or Emacs' equivalence to it) and the filter routine.
> > Those are basically independent from each other.
> 
> What is a filter routine?  If you are talking about process filters,
> in the sense of set-process-filter, then there are no filter
> routines in Tramp.

Oh, I see.  Well, it doesn't matter, strictly speaking.  Whoever calls
accept-process-output should feel qualified, when being woken up, to
do the work of the non-existent process-filter, and if after that its
own task has not been finished (or even started), call
accept-process-output again.

> Are you suggesting to change Tramp such that there is a filter
> routine?

Whether you call the routine by an actual process-filter, or some task
having called accept-process-filter does it, does not make _much_ of a
difference.  However, a separate process filter can be called at
slightly more times than an accept-process-filter task can be woken
up, (because the process filter starts without a personal stack frame,
and thus need not be "on top" of the stack), so the process filter
approach would probably provide a lower latency.

> > Ok, here is what the filter routine does when it is called: it
> > collects the stuff from the output until it has a completely reply to
> > the currently sent command available.  If it has, it takes the
> > request and marks it as completed (tacking the results to the
> > request).
> >
> > [Entry point for getting a command on the way:]
> > It then takes a look whether there are still outstanding commands in
> > the queue.  If there are, it takes the next one from the queue and
> > sends it through, marking it as being in progress.
> >
> > That's all.  The filter routine never changes, it does just that.
> > There is only one filter routine at work at most at any time.
> >
> > Now for the user level stuff: it knows it needs to get commands
> > through.  So it makes a request data structure and tacks it to the end
> > of the current queue (or, if the command is particularly urgent, like
> > when we are doing autorevert checking, to the _front_ of the current
> > queue) and then calls accept-process-output on the process repeatedly
> > until the command finally is marked as being processed.  Then it
> > takes the results and returns.

Of course, if the shell is idle at the time that a command has been
entered into the queue, the user level routine should manually call
the above "Entry point for getting..." before relapsing into
accept-process-output, or nothing will get done, ever.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Tramp with global-auto-revert-mode.

[Eli Zaretskii]

> Date: Sat, 15 May 2004 08:51:28 -0500 (CDT)
> From: Luc Teirlinck <teirllm@dms.auburn.edu>
> 
> I do not have a file "/usr/local/share/emacs/21.3.50/lisp/mh-e/down.jpeg"
> by the way.

That doesn't necessarily tell anything of value: Emacs could, for
example, cons such a string while looking for a `down' icon, whereby
it tries several extensions that indicate image files.

Re: Tramp with global-auto-revert-mode.

[Richard Stallman]

I think it is best to prevent recursive calls to Tramp.
One way to do this is to turn off auto-revert mode
while inside of any Tramp operation.

Re: Tramp with global-auto-revert-mode.

[Richard Stallman]

    Agreed, but accept-process-output is not always slow and we don't always get
    to choose.  I've hacked my Emacs to signal an error if accept-process-output
    is called (without timeout) with inhibit-quit set to non-nil and discovered
    that some package (IIRC, flyspell) does exactly that.  Of course, ispell
    is a local process that always responds immeditely, right?
    Anyway: how could flyspell do its job without calling 
    accept-process-output from a timer or a post-command-hook?

flyspell has to do this, so I think that proves this is legitimate.
It could perhaps temporarily turn off all timers while doing this,
so that they would not run until later.

However, I think it makes more sense to say that timers have to run
fast.  That is a natural and easily understood requirement.

If auto-revert is implemented by telling a post-command-hook or idle hook
to do the job, then its timer could run fast.

Re: Tramp with global-auto-revert-mode.

[Richard Stallman]

    #0  abort () at emacs.c:433
    #1  0x0812a575 in mark_object (arg=144520658) at alloc.c:5034
    #2  0x0812a5d6 in mark_object (arg=144315413) at alloc.c:5051
    #3  0x0812a5d6 in mark_object (arg=144310277) at alloc.c:5051
    #4  0x0812948a in mark_memory (start=0xbfffc780, end=0xbffff57c)
	at alloc.c:3781
    #5  0x081294f5 in mark_stack () at alloc.c:4055

Can you figure out which slot, in which stack frame,
is currently being examined within mark_stack?  That would
probably lead us to the C level bug that directly causes
these crashes.

If we make changes at the Lisp level, the C-level bug may no
longer happen often in this way, but it will still remain and
may cause occasional crashes on other occasions.  So please try
to debug it now while the Lisp code still triggers it.

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Richard Stallman wrote:

   Can you figure out which slot, in which stack frame,
   is currently being examined within mark_stack?  That would
   probably lead us to the C level bug that directly causes
   these crashes.

I am not familiar at all with how gc checks for memory corruption.
Here is just a try.  I use the latest crash I got, after Andreas'
bugfix.

#0  abort () at emacs.c:434
#1  0x0812a575 in mark_object (arg=142231922) at alloc.c:5034
#2  0x0812a5d6 in mark_object (arg=142409733) at alloc.c:5051
#3  0x0812948a in mark_memory (start=0xbffe91d0, end=0xbffff57c)
    at alloc.c:3781
#4  0x081294f5 in mark_stack () at alloc.c:4055
#5  0x08129aba in Fgarbage_collect () at alloc.c:4429

The abort at alloc.c, line 5034 is the default clause of a switch
form.  The abort occurs if XMISCTYPE (obj) is none of the listed
possibilities.

(gdb) up 1
#1  0x0812a575 in mark_object (arg=142231922) at alloc.c:5034
5034		    abort ();
(gdb) p obj
$3 = 142231922
(gdb) p *obj
$4 = 1230503937
(gdb) xtype
Lisp_Symbol
(gdb) xsymbol
$5 = (struct Lisp_Symbol *) 0x49580000
Argument to arithmetic operation not a number or boolean.
(gdb) p *$5
$6 = {
  gcmarkbit = 0, 
  indirect_variable = 0, 
  constant = 0, 
  interned = 0, 
  xname = 0, 
  value = 0, 
  function = 0, 
  plist = 0, 
  next = 0x0
}

So unless I am confused, this is what the obj that causes the abort
points to.

Some further info, for whatever it is worth:

(gdb) frame 1
#1  0x0812a575 in mark_object (arg=142231922) at alloc.c:5034
5034		  abort ();
(gdb) p obj
$12 = 142231922
(gdb) xtype
Lisp_Misc
Lisp_Misc_Free
(gdb) xmiscfree 
$13 = (struct Lisp_Free *) 0x87a4970

I do not know whether I am doing the right things here.  I am not used
to debugging bugs in gc.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Maybe some additions to my previous answer.

Reminder:

(gdb) bt
#0  abort () at emacs.c:434
#1  0x0812a575 in mark_object (arg=142231922) at alloc.c:5034
#2  0x0812a5d6 in mark_object (arg=142409733) at alloc.c:5051
#3  0x0812948a in mark_memory (start=0xbffe91d0, end=0xbffff57c)
    at alloc.c:3781
#4  0x081294f5 in mark_stack () at alloc.c:4055
#5  0x08129aba in Fgarbage_collect () at alloc.c:4429

(gdb) frame 5                         
#5  0x08129aba in Fgarbage_collect () at alloc.c:4429
4429	         mark_stack ();
(gdb) p (char *) stack_base + i     
$41 = 0xbffff9e2 "AME=Default"
(gdb) p (char *) stack_base + i - 15
$42 = 0xbffff9d3 "GNOME_SESSION_NAME=Default"
(gdb) 

This is one of my environment variables.

>From this, through all kinds of stuff I do not understand we arrive
three frames down at:

gdb) frame 2
#2  0x0812a5d6 in mark_object (arg=142409733) at alloc.c:5051
5051		  mark_object (ptr->car);
(gdb) p ptr
$83 = (struct Lisp_Cons *) 0x87d0000
(gdb) p *ptr
$84 = {
  car = 142231922, 
  cdr = -16
}
(gdb) p ptr->car
$85 = 142231922
(gdb) p *$85
$86 = 1230503937
(gdb) xtype
Lisp_Symbol
(gdb) xsymbol
$87 = (struct Lisp_Symbol *) 0x49580000
Argument to arithmetic operation not a number or boolean.
(gdb) p *$87
$88 = {
  gcmarkbit = 0, 
  indirect_variable = 0, 
  constant = 0, 
  interned = 0, 
  xname = 0, 
  value = 0, 
  function = 0, 
  plist = 0, 
  next = 0x0
}
(gdb) p ptr->cdr
$89 = -16
(gdb) xtype
Lisp_Int
(gdb) xint
$90 = -2
(gdb) 

The ptr->car then seems to produce the abort in frame 1, as I pointed
out in my previous message.  But then again, maybe nothing I am trying
to do makes any sense.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

>From my previous message:

   (gdb) p (char *) stack_base + i     
   $41 = 0xbffff9e2 "AME=Default"

I get gc aborting on that _exact_ same string, starting with the exact
same `A' in _every single one_ of my many crashes.

   (gdb) p (char *) stack_base + i - 15
   $42 = 0xbffff9d3 "GNOME_SESSION_NAME=Default"
   (gdb) 

   This is one of my environment variables.

How can this make sense?  I hope nothing in Emacs makes assumptions
about some maximum size of environment variables (or whatever).  I am
using GNU/Linux.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Eli Zaretskii]

> Date: Sat, 15 May 2004 15:44:44 -0500 (CDT)
> From: Luc Teirlinck <teirllm@dms.auburn.edu>
> 
> I am not familiar at all with how gc checks for memory corruption.

Did you look in etc/DEBUG?  It has a section about debugging crashes
in GC.  You need to use the last_marked[] array to guide you through
the recursive invocations of mark_object and its ilk, and find out
which part of what Lisp data structure is being marked when the
disaster strikes.

Re: Tramp with global-auto-revert-mode.

[Andreas Schwab]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

>>From my previous message:
>
>    (gdb) p (char *) stack_base + i     
>    $41 = 0xbffff9e2 "AME=Default"
>
> I get gc aborting on that _exact_ same string, starting with the exact
> same `A' in _every single one_ of my many crashes.

That does not say anything, it's just the highest stack address the
garbage collector looks at.  It doesn't have to be exact, just above the
highest address that could contain live Lisp pointers, which appears to be
the case here.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Tramp with global-auto-revert-mode.

[Kai Grossjohann]

David Kastrup <dak@gnu.org> writes:

> Kai Grossjohann <kai.grossjohann@gmx.net> writes:
>
>> David Kastrup <dak@gnu.org> writes:
>> 
>> > Kai Grossjohann <kai.grossjohann@gmx.net> writes:
>> >
>> >> David Kastrup <dak@gnu.org> writes:
>> >> 
>> >> > Uh, much too complicated.
>> >> 
>> >> I'm afraid I don't understand you: Do you mean that just the last
>> >> step is too complicated, or that the whole procedure that I described
>> >> is too complicated?
>> >
>> > The whole procedure.
>> 
>> Okay.  I agree.  Though our ideas of simplicity are different -- I
>> was thinking of using multiple connection buffers.
>
> Those have to be set up and initialized.  You know that Tramp
> initialization takes a _lot_ of time.  If we have some timer-related
> events, it is likely that we will get an indefinite stack of sessions
> all occupied with initializing a connection.

I was thinking of letting the additional connection buffer stay
around.  I agree, however, that bootstrapping is somewhat difficult:
how do I make it so that the second buffer appears quickly enough for
auto-revert to be of some use...

>> > "Tramp" consists of two entirely different pieces: the user level
>> > routines (of which several can be running at once in different
>> > "threads" or Emacs' equivalence to it) and the filter routine.
>> > Those are basically independent from each other.
>> 
>> What is a filter routine?  If you are talking about process filters,
>> in the sense of set-process-filter, then there are no filter
>> routines in Tramp.
>
> Oh, I see.  Well, it doesn't matter, strictly speaking.  Whoever calls
> accept-process-output should feel qualified, when being woken up, to
> do the work of the non-existent process-filter,

That work is to wait for the next shell prompt.  If it isn't there,
call accept-process-output again.

> and if after that its own task has not been finished (or even
> started), call accept-process-output again.

Yes.

Kai

Re: Tramp with global-auto-revert-mode.

[Kai Grossjohann]

Richard Stallman <rms@gnu.org> writes:

> I think it is best to prevent recursive calls to Tramp.
> One way to do this is to turn off auto-revert mode
> while inside of any Tramp operation.

Okay, so I suggest to extend Tramp such that it looks to see if the
buffer is busy, and if so, it signals an error.

Additionally, global-auto-revert-mode could be changed so that
auto-reverting is off for remote files.

WDYT?

Kai

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Eli Zaretskii wrote:

   Did you look in etc/DEBUG?  It has a section about debugging crashes
   in GC.  You need to use the last_marked[] array to guide you through
   the recursive invocations of mark_object and its ilk, and find out
   which part of what Lisp data structure is being marked when the
   disaster strikes.

I forgot to read that.  However, I have trouble making sense of the
contents of last_marked and I do not really understand the gc code.
Using the very latest crash I produced, I get the _impression_ that gc
is examining some structure related to the
*tramp/ssh raven.dms.auburn.edu* buffer, but that is just a guess.

(gdb) run -q --eval '(progn (blink-cursor-mode 0) (global-auto-revert-mode 1))'
Starting program: /home/teirllm/emacscvsdir/emacs/src/emacs-21.3.50.1 -q --eval '(progn (blink-cursor-mode 0) (global-auto-revert-mode 1))'

Breakpoint 1, abort () at emacs.c:434
434	     kill (getpid (), SIGABRT);
(gdb) bt
#0  abort () at emacs.c:434
#1  0x0812a575 in mark_object (arg=145714794) at alloc.c:5034
#2  0x0812a5d6 in mark_object (arg=148693005) at alloc.c:5051
#3  0x0812948a in mark_memory (start=0xbffe91d0, end=0xbffff57c)
    at alloc.c:3781
#4  0x081294f5 in mark_stack () at alloc.c:4055
#5  0x08129aba in Fgarbage_collect () at alloc.c:4429

I omit the rest of the bt-output, since it is unrelated to gc.

(gdb) p last_marked_index
$197 = 18
(gdb) p last_marked[17]
$198 = 145714794
(gdb) pr
#<misc free cell>
(gdb) p last_marked[16]
$199 = 148693005
(gdb) pr
(#<misc free cell> . -37)
(gdb) p last_marked[15]
$200 = 143097017
(gdb) pr

(gdb) xtype
Lisp_Symbol
(gdb) xsymbol
$201 = (struct Lisp_Symbol *) 0x8877cb8
Argument to arithmetic operation not a number or boolean.
(gdb) p *$201
$202 = {
  gcmarkbit = 1, 
  indirect_variable = 0, 
  constant = 0, 
  interned = 2, 
  xname = 143098427, 
  value = 138193545, 
  function = 138193545, 
  plist = 138193521, 
  next = 0x86621b0
}
(gdb) p $202->xname
$203 = 143098427
(gdb) pr
""
(gdb) xstring
$204 = (struct Lisp_String *) 0x8878238
"cygwin-mount-map-drive-hook-function"
(gdb) p last_marked[14]
$205 = 148054205
(gdb) pr
()
(gdb) p last_marked[13]
$206 = 143096993
(gdb) pr

(gdb) xtype
Lisp_Symbol
(gdb) xsymbol
$207 = (struct Lisp_Symbol *) 0x8877ca0
Argument to arithmetic operation not a number or boolean.
(gdb) p *$207
$208 = {
  gcmarkbit = 1, 
  indirect_variable = 0, 
  constant = 0, 
  interned = 2, 
  xname = 143098411, 
  value = 138193545, 
  function = 138193545, 
  plist = 138193521, 
  next = 0x8636688
}
(gdb) p $208->xname
$209 = 143098411
(gdb) pr
""
(gdb) xstring
$210 = (struct Lisp_String *) 0x8878228
"cygwin-mount-name-hook-function"
(gdb) p last_marked[12]
$211 = 148054197
(gdb) pr
( )
(gdb) p last_marked[11]
$212 = 138971849
(gdb) pr
tramp-completion-file-name-handler
(gdb) p last_marked[10]
$213 = 148054029
(gdb) pr
(tramp-completion-file-name-handler  )
(gdb) p last_marked[9]
$214 = 145715130
(gdb) pr
#<marker at 185421 in *tramp/ssh raven.dms.auburn.edu*>
(gdb) p last_marked[8]
$215 = 920
(gdb) pr 
115
(gdb) p last_marked[7]
$216 = 148538453
(gdb) pr
(115)
(gdb) p last_marked[6]
$217 = 936
(gdb) pr
117
(gdb) p last_marked[5]
$218 = 148538277
(gdb) pr
(117)
(gdb) p last_marked[4]
$219 = 880
(gdb) pr
110
(gdb) p last_marked[3]
$220 = 148538285
(gdb) pr
(110 117)
(gdb) p last_marked[2]
$221 = 920
(gdb) pr
115
(gdb) p last_marked[1]
$222 = 148538293
(gdb) pr
(115 110 117)
(gdb) p last_marked[0]
$223 = 840
(gdb) pr
105

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

>From my previous message:

   (gdb) p $202->xname
   $203 = 143098427
   (gdb) pr
   ""
   (gdb) xstring
   $204 = (struct Lisp_String *) 0x8878238
   "cygwin-mount-map-drive-hook-function"

Is this normal or bug?  I know that pr sometimes is going to give _no_
answer and then you have to do the "xtype" stuff, but a wrong answer?

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

For whatever it is worth, the entries 10, 11, 13 and 15 in
last_marked, close before the abort caused by 17, refer to
tramp-completion-file-name-handler, "cygwin-mount-name-hook-function"
and "cygwin-mount-map-drive-hook-function".  During the execution of
`tramp-run-real-handler' or `tramp-completion-run-real-handler', these
three functions are elements of the list to which
`inhibit-file-name-handlers' is bound with a let-binding.  I do not
know whether this is meaningful, but it is a bizarre coincidence if not.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Kim F. Storm]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> For whatever it is worth, the entries 10, 11, 13 and 15 in
> last_marked, close before the abort caused by 17, refer to
> tramp-completion-file-name-handler, "cygwin-mount-name-hook-function"
> and "cygwin-mount-map-drive-hook-function".  During the execution of
> `tramp-run-real-handler' or `tramp-completion-run-real-handler', these
> three functions are elements of the list to which
> `inhibit-file-name-handlers' is bound with a let-binding.  I do not
> know whether this is meaningful, but it is a bizarre coincidence if not.

I also made emacs crash with tramp + global-auto-revert-mode.

I haven't had time to dig much into this, but it traps in the
same way as you have observed -- the offending object seems to
result from the scan for lisp-object pointers on the stack.

One thing I noticed is that tramps seems to enable undo in its
work buffers -- if that's really true, it looks like a bug to me...

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Kai Grossjohann wrote:

   Okay, so I suggest to extend Tramp such that it looks to see if the
   buffer is busy, and if so, it signals an error.

   Additionally, global-auto-revert-mode could be changed so that
   auto-reverting is off for remote files.

   WDYT?

That looks good to me.  Disabling auto-reverting of remote files is
trivial, it is just adding a single call to `file-remote-p'.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Kai Grossjohann]

storm@cua.dk (Kim F. Storm) writes:

> One thing I noticed is that tramps seems to enable undo in its
> work buffers -- if that's really true, it looks like a bug to me...

Yes.  I forgot to turn it off.  Perhaps it was needed for debugging
ages ago.

Kai

Re: Tramp with global-auto-revert-mode.

[Eli Zaretskii]

> Date: Sun, 16 May 2004 13:11:03 -0500 (CDT)
> From: Luc Teirlinck <teirllm@dms.auburn.edu>
> 
> I have trouble making sense of the contents of last_marked

You need to look backwards in last_marked starting with
last_marked_index, like you did, but to look at the GC code as you
go.

The part of GC where your session crashed examines live Lisp objects
and marks all of the components of each object (so that the following
sweep step will not garbage-collect what is still in use).  Some Lisp
data structures are marked by looping inside the same invocation of
mark_object; others cause mark_object to be invoked recursively.  By
contrast, the objects in last_marked[] array are recorded
chronologically as they are marked, and the recursive invocations are
not visible there.  So, to find out which Lisp data structure was
being marked, you need to identify what object in last_marked[]
corresponds to each recursive invocation of mark_object that you see
in the backtrace.  That would hopefully give a hint as to where to
look for the villain(s).

Does that help?

Re: Tramp with global-auto-revert-mode.

[Richard Stallman]

    (gdb) frame 5                         
    #5  0x08129aba in Fgarbage_collect () at alloc.c:4429
    4429	         mark_stack ();
    (gdb) p (char *) stack_base + i     
    $41 = 0xbffff9e2 "AME=Default"
    (gdb) p (char *) stack_base + i - 15
    $42 = 0xbffff9d3 "GNOME_SESSION_NAME=Default"
    (gdb) 

What is the value of i?  What is stack_base?

Anywya, they indicate where the stack begins.  That doesn't tell us
where the problem is.  Please also look in the mark_memory frame and
see what address it is actually operating on.

That address should be part of some stack frame.
Which stack frame is it?  You may need to type
`info frame 1', `info frame 2', and so on in order to
see the addresses of various stack frames and find
out which one contains the address in question.

Re: Tramp with global-auto-revert-mode.

[Kim F. Storm]

storm@cua.dk (Kim F. Storm) writes:

> I also made emacs crash with tramp + global-auto-revert-mode.
> 

And I have studied the cause a little further and found at least one
reason for the crash -- trying to mark a Lisp_Misc_Free object, most
likely resulting from freeing a marker which is still present on
some undo list.

I have installed a fix for this; I don't know if it has any
bad effects, but at least it solves the crash for me.

Now, tramp is hanging in accept-process-output when I try to
open a remote file, but it can be interrupted with C-g.
Solving that problem is a different issue ...

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

None of my crashes will still occur after Kim's recent change, because
they all were caused by Lisp_Misc_Free objects.  The garbage collector
will ignore these now.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Kim Storm wrote:

   And I have studied the cause a little further and found at least one
   reason for the crash -- trying to mark a Lisp_Misc_Free object, most
   likely resulting from freeing a marker which is still present on
   some undo list.

Unless I am misunderstanding, it would seem that freeing a marker
which is still referenced on an undo list is a bug.  Is disabling undo
in Tramp work buffers sufficient to avoid this kind of problem or is
there still a bug in the C code?

   I have installed a fix for this; I don't know if it has any
   bad effects, but at least it solves the crash for me.

Clearly any bad effects will only occur in situations were Emacs would
have crashed earlier without your fix.  An obvious bad effect is the
one you mentioned yourself in the comment to your fix: unpredictable
behavior if the user does `undo'.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Kim F. Storm]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> Kim Storm wrote:
> 
>    And I have studied the cause a little further and found at least one
>    reason for the crash -- trying to mark a Lisp_Misc_Free object, most
>    likely resulting from freeing a marker which is still present on
>    some undo list.
> 
> Unless I am misunderstanding, it would seem that freeing a marker
> which is still referenced on an undo list is a bug.  Is disabling undo
> in Tramp work buffers sufficient to avoid this kind of problem or is
> there still a bug in the C code?

In any case, the problem seems to be aggrevated by tramp as it has a
marker which was put on the undo-list MANY MANY times ...   But I
guess it could happen with any code involving markers (which may explain
some of the less frequent crashes in mark_object.)

> 
>    I have installed a fix for this; I don't know if it has any
>    bad effects, but at least it solves the crash for me.
> 
> Clearly any bad effects will only occur in situations were Emacs would
> have crashed earlier without your fix.  An obvious bad effect is the
> one you mentioned yourself in the comment to your fix: unpredictable
> behavior if the user does `undo'.

If the marker isn't reused, it boils down to checking that undo on
a Lisp_Misc_Free object is a noop.

My latest fix didn't mark the Lisp_Misc_Free cell as used -- I just
committed another change to do that.

It seems that primitive-undo just ignores a Lisp_Misc_Free object,
so there doesn't seem to be a reason to worry about undefined behaviour.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Tramp with global-auto-revert-mode.

[Richard Stallman]

    > I think it is best to prevent recursive calls to Tramp.
    > One way to do this is to turn off auto-revert mode
    > while inside of any Tramp operation.

    Okay, so I suggest to extend Tramp such that it looks to see if the
    buffer is busy, and if so, it signals an error.

We could do that, but we don't want attempts to auto-revert to cause
errors.  How about turning off auto-revert mode while inside Tramp?

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Richard Stallman wrote:

   We could do that, but we don't want attempts to auto-revert to cause
   errors.  How about turning off auto-revert mode while inside Tramp?

Tramp can take a long time on slow connections.  It typically takes me
40-45 seconds to visit sequences.texi using Tramp with ssh, when
auto-revert-mode is disabled.  It sometimes takes longer than that,
and sequences.texi is not _that_ long of a file.  With a default-value
for auto-revert-interval of five seconds the delay in reverting might
be confusing to the user.

If we turn off auto-reverting of remote files altogether, there is no
problem.  Then there will be no attempts to auto-revert.  

If not, then I believe that we at least need an option to disable
auto-reverting of remote files, say `global-auto-revert-remote-files'.
If we go for the option, then we could also say that auto-reverting of
a remote file would be enabled, even if `global-auto-revert-remote-files'
is nil, if auto-revert-mode (as opposed to global-auto-revert-mode) is
enabled in the buffer (since then the user explicitly asked for it).
This would be consistent with `global-auto-revert-non-file-buffers'.
If Tramp bound an internal variable `tramp-active' or whatever to t,
auto-revert-mode could unconditionally disable auto-reverting of
remote files if that variable is non-nil.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Stefan Monnier]

>> I also made emacs crash with tramp + global-auto-revert-mode.

> And I have studied the cause a little further and found at least one
> reason for the crash -- trying to mark a Lisp_Misc_Free object, most
> likely resulting from freeing a marker which is still present on
> some undo list.

> I have installed a fix for this; I don't know if it has any
> bad effects, but at least it solves the crash for me.

Well, it makes it "legal" to have a live Lisp_Misc_free object, which is at
least odd and might lead to further problems.

But more importantly, Fgarbage_collect has code that traverses the undo
lists and removes entries that contain unmarked markers, which should make
sure the problem you describe doesn't happen.  Obviously that code is not
100% successful at it, but rather than hide the problem we might want to
track it down further to see what's really going on.

Looking at the code, I see one case where things could go awry:
1) mark buffer -> mark the undo_list, skipping markers
2) coming from somewhere else, mark a cons cell who happens to also be
   on the undo_list we marked in step 1 and whose car is a marker:
   because the cons cells were already marked, we won't look at the marker.
   Also the code in Fgarbage_collect will remove the cons cell from the
   undo_list but that will not remove it from the other list, so it will
   stay and be turned into a reachable Lisp_Misc_Free.

Another thing I notice is that we do mark_stack and xg_mark_data after
cleaning up the undo_lists, so if any undo_list gets marked from mark_stack
or xg_mark_data we will leave the unmarked markers and turn them info
Lisp_Misc_Free.  I've just installed a fix for this last problem (although
it's not a very likely scenario so I don't expect it fixes the problem
we're seeing).

I don't have a fix for the first yet problem yet (which is also somewhat
unlikely, but less so, so maybe it is the problem we're seeing).


        Stefan

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Stefan Monnier wrote:

   I've just installed a fix for this last problem (although
   it's not a very likely scenario so I don't expect it fixes the problem
   we're seeing).

It does not.  I still get the same data type corruption crashes.
This time I had a hard time producing one however (which does not
necessarily mean anything).

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Kim F. Storm]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> Stefan Monnier wrote:
> 
>    I've just installed a fix for this last problem (although
>    it's not a very likely scenario so I don't expect it fixes the problem
>    we're seeing).
> 
> It does not.  I still get the same data type corruption crashes.
> This time I had a hard time producing one however (which does not
> necessarily mean anything).

Please try again with my latest changes to GC.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Tramp with global-auto-revert-mode.

[Luc Teirlinck]

Kim Storm wrote:

   Please try again with my latest changes to GC.

This time I indeed do not seem able to produce this type of crash anymore.   

Tramp is still unusable with auto-revert-mode and I see a variety of
strange bugs when trying to visit a second remote file using Tramp,
for instance a copy of the modeline temporarily appearing inside the
minibuffer.  I suspect that all these bugs are related to recursive
Tramp invocations, which should be avoided.  I guess we have up till
now avoided actually doing something about these problems, not to
interfere with the gc debugging.

Should everybody agree that the problems with gc are fixed, then we
could make the necessary changes in autorevert.el and tramp.el to get
rid of the recursive invocation bugs.

Sincerely,

Luc.

Re: Tramp with global-auto-revert-mode.

[Kim F. Storm]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> Kim Storm wrote:
> 
>    Please try again with my latest changes to GC.
> 
> This time I indeed do not seem able to produce this type of crash anymore.   
> 
> Tramp is still unusable with auto-revert-mode and I see a variety of
> strange bugs when trying to visit a second remote file using Tramp,
> for instance a copy of the modeline temporarily appearing inside the
> minibuffer.  I suspect that all these bugs are related to recursive
> Tramp invocations, which should be avoided.  I guess we have up till
> now avoided actually doing something about these problems, not to
> interfere with the gc debugging.
> 
> Should everybody agree that the problems with gc are fixed, then we
> could make the necessary changes in autorevert.el and tramp.el to get
> rid of the recursive invocation bugs.

I have experienced various problems with tramp itself while debugging
the GC problems -- they certainly need to be fixed before release.

I have a good confidence that my latest fixes to GC do fix the
reported problems with bad Misc object types.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Tramp with global-auto-revert-mode.

[Michael Albinus]

storm@cua.dk (Kim F. Storm) writes:

> I have experienced various problems with tramp itself while debugging
> the GC problems -- they certainly need to be fixed before release.

Could you, please, be a little bit more verbose? I guess you mean
something else but the autorevert problem.

Best regards, Michael.