tramp no longer recognizes when remote files changed

tramp no longer recognizes when remote files changed

[Sascha Wilde]

Hi *,

when a buffer visits a file that gets changes outside of Emacs the
user is warned the next time he tries to alter the buffer in question:

  foo changed on disk; really edit the buffer? (y, n, r or C-h)

This extremely useful feature used to work with tramp and remote
files, too.  Now some time (days?) ago it stopped working, if I edit a
remote file that changed in the meantime on disk I get no warning, and
if I save the buffer all changes made outside of my Emacs session are
silently lost.

Cheers
sascha
-- 
Sascha Wilde : "There are 10 types of people in the world. 
             : Those who understand binary and those who don't."

Re: tramp no longer recognizes when remote files changed

[Michael Albinus]

Sascha Wilde <wilde@sha-bang.de> writes:

> Hi *,

Hi,

> when a buffer visits a file that gets changes outside of Emacs the
> user is warned the next time he tries to alter the buffer in question:
>
>   foo changed on disk; really edit the buffer? (y, n, r or C-h)
>
> This extremely useful feature used to work with tramp and remote
> files, too.  Now some time (days?) ago it stopped working, if I edit a
> remote file that changed in the meantime on disk I get no warning, and
> if I save the buffer all changes made outside of my Emacs session are
> silently lost.

That's the price ...

Tramp 2.1, brought to Emacs trunk some weeks ago, caches file
information in order to speed up. It uses cached information for
example when `file-attributes' is called, and cannot recognize
external file changes therefore.

Maybe one could introduce an option whether such caches shall be used,
or not. Tramp checks already `file-precious-flag' when it copies files
(if set, Tramp applies "cksum" on both the source and target files in
order to recognize errors). It could also stop using cached
information in such a case - slowing down performance ...

> Cheers
> sascha

Best regards, Michael.

Re: tramp no longer recognizes when remote files changed

[Sascha Wilde]

Michael Albinus <michael.albinus@gmx.de> wrote:
> Sascha Wilde <wilde@sha-bang.de> writes:
>>   foo changed on disk; really edit the buffer? (y, n, r or C-h)
>>
>> This extremely useful feature used to work with tramp and remote
>> files, too.  Now some time (days?) ago it stopped working,
[...]

> That's the price ...
>
> Tramp 2.1, brought to Emacs trunk some weeks ago, caches file
> information in order to speed up.

Yes, I recognized the, sometimes tremendous speed up, and indeed it
made me happy -- but if _this_ is the price, it is much to high (IMO).

I often remotely edit files which intentionally change outside Emacs,
and the risk of loosing data with the new tramp version is high and a
major step backwards compared to the situation before.

[...]
> Maybe one could introduce an option whether such caches shall be used,
> or not.

Yes, I would highly appreciate that!

I don't know, but I would guess that some of the optimization could be
even kept.  For example when opening an remote file for the first time
the new version is much faster, and at this point of time no
changes would be needed to satisfy my needs, right?

Cheers
sascha
-- 
Sascha Wilde
... mein Opa [...]  würde an dieser Stelle zu Dir sagen: Junge, such Dir 
ne Frau, bau Dir ein Haus, mach ein Kind und laß' die Finger von dem Zeug,
das Du gerade machst. -- Michael Winklhofer in d.a.e.auktionshaeuser

Re: tramp no longer recognizes when remote files changed

[Michael Albinus]

Sascha Wilde <wilde@sha-bang.de> writes:

>> Maybe one could introduce an option whether such caches shall be used,
>> or not.
>
> Yes, I would highly appreciate that!

Could you, please, try the appended patch? You must set
`file-precious-flag' to t.

> Cheers
> sascha

Best regards, Michael.

*** /home/albinus/src/tramp/lisp/tramp-cache.el	2007-07-27 11:14:10.000000000 +0200
--- /home/albinus/src/emacs/lisp/net/tramp-cache.el	2007-07-10 20:45:30.000000000 +0200
***************
*** 108,124 ****
  
  (defun tramp-set-file-property (vec file property value)
    "Set the PROPERTY of FILE to VALUE, in the cache context of VEC.
! It is not set when `file-precious-flag' is non-nil.  Returns VALUE."
!   (unless file-precious-flag
!     ;; Unify localname.
!     (setq vec (copy-sequence vec))
!     (aset vec 3 (directory-file-name file))
!     (let ((hash (or (gethash vec tramp-cache-data)
! 		    (puthash vec (make-hash-table :test 'equal)
! 			     tramp-cache-data))))
!       (puthash property value hash)
!       (tramp-message vec 8 "%s %s %s" file property value)))
!   value)
  
  (defun tramp-flush-file-property (vec file)
    "Remove all properties of FILE in the cache context of VEC."
--- 108,123 ----
  
  (defun tramp-set-file-property (vec file property value)
    "Set the PROPERTY of FILE to VALUE, in the cache context of VEC.
! Returns VALUE."
!   ;; Unify localname.
!   (setq vec (copy-sequence vec))
!   (aset vec 3 (directory-file-name file))
!   (let ((hash (or (gethash vec tramp-cache-data)
! 		  (puthash vec (make-hash-table :test 'equal)
! 			   tramp-cache-data))))
!     (puthash property value hash)
!     (tramp-message vec 8 "%s %s %s" file property value)
!     value))
  
  (defun tramp-flush-file-property (vec file)
    "Remove all properties of FILE in the cache context of VEC."

Re: tramp no longer recognizes when remote files changed

[Sascha Wilde]

Michael Albinus <michael.albinus@gmx.de> wrote:
> Sascha Wilde <wilde@sha-bang.de> writes:
>
>>> Maybe one could introduce an option whether such caches shall be used,
>>> or not.
>>
>> Yes, I would highly appreciate that!
>
> Could you, please, try the appended patch? You must set
> `file-precious-flag' to t.

This seems to fix it!  Thanks a lot!

sascha
-- 
Sascha Wilde
To become a Jedi, use Emacs you have to.

Re: tramp no longer recognizes when remote files changed

[Stefan Monnier]

>>> Maybe one could introduce an option whether such caches shall be used,
>>> or not.
>> Yes, I would highly appreciate that!
> Could you, please, try the appended patch? You must set
> `file-precious-flag' to t.

How 'bout an idle hook that flushes the cache after 0.5s of idleness?


        Stefan

Re: tramp no longer recognizes when remote files changed

[martin rudalics]

>>>>Maybe one could introduce an option whether such caches shall be used,
>>>>or not.
>>>
>>>Yes, I would highly appreciate that!
>>
>>Could you, please, try the appended patch? You must set
>>`file-precious-flag' to t.
> 
> 
> How 'bout an idle hook that flushes the cache after 0.5s of idleness?

How 'bout checking whether the file has changed when saving the buffer?

Re: tramp no longer recognizes when remote files changed

[Richard Stallman]

    Tramp 2.1, brought to Emacs trunk some weeks ago, caches file
    information in order to speed up. It uses cached information for
    example when `file-attributes' is called, and cannot recognize
    external file changes therefore.

Does it make sense to block use of the cached data specifically in the
case of checking whether a file has changed for C-x C-f?

What are the main occasions on which the cache is useful?

    > Could you, please, try the appended patch? You must set
    > `file-precious-flag' to t.

    This seems to fix it!  Thanks a lot!

`file-precious-flag' does other things.  It should not be reused to
control whether Tramp checks for a changed file.

Having a parameter for users to set is not a very good solution.
First let's look for a way to make Tramp do the right combination
of things by default.  Michael, can you try?

Re: tramp no longer recognizes when remote files changed

[Michael Albinus]

Richard Stallman <rms@gnu.org> writes:

> What are the main occasions on which the cache is useful?

I've made a test: Run 1000x (file-attributes "/ssh:ford:/etc/hosts").
"ford" is another host in my LAN.

With enabled cache, it takes 1.821823 sec. Without cache, it takes
60.103737 sec. And it is a nearby host.

A simple (find-file "/ssh:ford:/etc/hosts") shows in the profiler the
following file name handler based functions:

expand-file-name                          127
file-exists-p                             42
file-attributes                           23
file-name-directory                       23
file-directory-p                          11
directory-file-name                       8
substitute-in-file-name                   7
file-name-nondirectory                    5
file-modes                                3
file-readable-p                           2
file-truename                             2
make-auto-save-file-name                  2
file-local-copy                           1
file-symlink-p                            1
file-writable-p                           1
insert-file-contents                      1
set-visited-file-modtime                  1

See f.e. the large number of `file-exists-p' calls. This is mainly due
to vc, there are repeated tests for existence of

/
/etc
/etc/hosts
/etc/RCS/hosts,v
/etc/hosts,v
/etc/RCS/hosts
/etc/SCCS/s.hosts
/etc/s.hosts
/etc/.bzr
/.bzr
/etc/.hg
/.hg
/etc/{arch}/=tagging-method
/{arch}/=tagging-method
/etc/MCVS/CVS
/MCVS/CVS

> Having a parameter for users to set is not a very good solution.
> First let's look for a way to make Tramp do the right combination
> of things by default.  Michael, can you try?

Tramp flushes the cached data in `verify-visited-file-modtime' now. That
seems to be sufficient.

Sascha, could you, please, test it?

Best regards, Michael.

Re: tramp no longer recognizes when remote files changed

[Richard Stallman]

    Tramp flushes the cached data in `verify-visited-file-modtime' now. That
    seems to be sufficient.

That sounds plausible.  It could be that users will report a couple of
other problems, and we might want to make a couple of other operations
discard the cached data.  I assume you'll keep an eye out to see
if this needs to be done.

Re: tramp no longer recognizes when remote files changed

[Michael Albinus]

Richard Stallman <rms@gnu.org> writes:

>     Tramp flushes the cached data in `verify-visited-file-modtime' now. That
>     seems to be sufficient.
>
> That sounds plausible.  It could be that users will report a couple of
> other problems, and we might want to make a couple of other operations
> discard the cached data.  I assume you'll keep an eye out to see
> if this needs to be done.

For sure. Tramp flushes already the cache in in a lot of other file
name primitives; `verify-visited-file-modtime' has been simply
overlooked by me. I thought I do it there also.

Best regards, Michael.

Re: tramp no longer recognizes when remote files changed

[Sascha Wilde]

Michael Albinus <michael.albinus@gmx.de> wrote:

> Tramp flushes the cached data in `verify-visited-file-modtime' now. That
> seems to be sufficient.
>
> Sascha, could you, please, test it?

Just gave it a short try:
- used Emacs to edit remotehost:foo
- sshed to remotehost and issued echo bar>>foo
- back in Emacs trying to edit the foo buffer warns that the file
  changed on disk
That's the desired behavior.  Thanks!

sascha
-- 
Sascha Wilde : xedit? Das sieht zwar wie vi aus als könne es nix, aber im
             : Gegensatz zu vi kann es wirklich nix und nix ist noch geschönt!
             : (Michael Core in dafc)