Best way to check file modification time?

Best way to check file modification time?

[Joost Kremers]

Hi all,

I'm dealing with files that I do not visit directly, instead I read
their contents into a temp buffer and parse them. At a later point, I
write the new contents into a temp buffer and save them to the original
file, overwriting the previous version.

I would like to check whether the original file may have been
overwritten by some other process (possibly by Emacs itself if the user
visits the same file in a buffer, but also by external programs).

Right now, I'm essentially doing the following:

```
(setq my-file-modtime (nth 5 (file-attributes my-file)))
(my-parse-function file)
```

where `my-parse-function' reads the contents of FILE into a temp buffer
and does the parsing. IOW, there is a function call between saving the
mod time and reading the file.

Saving is basically the same: the mod time of the file on disk is read
again and checked against the saved mod time, then if that checks out, a
function is called that creates the temp buffer, writes out the
information and saves the file.

Now, it seems to me that there may be too much time between checking the
mod time and reading/saving the file, such that it is in principle
possible that the relevant file gets overwritten at just the wrong moment.

I can easily move the mod time checking to a position in the code that's
closer to the actual reading or saving of the file, but I suspect that
only reduces the problem, it doesn't eliminate it altogether. So I'm
wondering if there's a Right Way™ to do this. If not, any advice on how
to reduce the risks as much as possible would be great!

TIA


-- 
Joost Kremers                                   joostkremers@fastmail.fm
Selbst in die Unterwelt dringt durch Spalten Licht
EN:SiS(9)

Re: Best way to check file modification time?

[Barry Margolin]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2316 bytes --]

In article <db0g80F14poU1@mid.individual.net>,
 Joost Kremers <joost.m.kremers@gmail.com> wrote:

> Hi all,
> 
> I'm dealing with files that I do not visit directly, instead I read
> their contents into a temp buffer and parse them. At a later point, I
> write the new contents into a temp buffer and save them to the original
> file, overwriting the previous version.
> 
> I would like to check whether the original file may have been
> overwritten by some other process (possibly by Emacs itself if the user
> visits the same file in a buffer, but also by external programs).
> 
> Right now, I'm essentially doing the following:
> 
> ```
> (setq my-file-modtime (nth 5 (file-attributes my-file)))
> (my-parse-function file)
> ```
> 
> where `my-parse-function' reads the contents of FILE into a temp buffer
> and does the parsing. IOW, there is a function call between saving the
> mod time and reading the file.
> 
> Saving is basically the same: the mod time of the file on disk is read
> again and checked against the saved mod time, then if that checks out, a
> function is called that creates the temp buffer, writes out the
> information and saves the file.
> 
> Now, it seems to me that there may be too much time between checking the
> mod time and reading/saving the file, such that it is in principle
> possible that the relevant file gets overwritten at just the wrong moment.
> 
> I can easily move the mod time checking to a position in the code that's
> closer to the actual reading or saving of the file, but I suspect that
> only reduces the problem, it doesn't eliminate it altogether. So I'm
> wondering if there's a Right Way’Ñ¢ to do this. If not, any advice on how
> to reduce the risks as much as possible would be great!

The same window exists when visiting files normally. There's no way to 
get the mod time and read a file atomically.

Instead of saving the file's modification time, remember the time that 
you started reading the file. Then when you're about to save it, see if 
the mod time is newer than that.

This can potentially run into a problem if the file is on a server and 
there's clock skew between the client and server, but that's a common 
problem.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Re: Best way to check file modification time?

[Stefan Monnier]

>> I can easily move the mod time checking to a position in the code that's
>> closer to the actual reading or saving of the file, but I suspect that
>> only reduces the problem, it doesn't eliminate it altogether. So I'm
>> wondering if there's a Right Way Ѣ to do this. If not, any advice on how
>> to reduce the risks as much as possible would be great!

You can't solve this without locking.
The mod-time check has to be done before writing (since writing is
conditional on the result of the mod-time check), you have a race:

   User A checks mod-time
   User B checks mod-time
   User A writes
   User B writes

And this applies no matter how/when the mod-time was originally acquired.


        Stefan

Re: Best way to check file modification time?

[Joost Kremers]

Stefan Monnier wrote:
>>> I can easily move the mod time checking to a position in the code that's
>>> closer to the actual reading or saving of the file, but I suspect that
>>> only reduces the problem, it doesn't eliminate it altogether. So I'm
>>> wondering if there's a Right Way Ѣ to do this. If not, any advice on how
>>> to reduce the risks as much as possible would be great!
>
> You can't solve this without locking.

Which, if I understand the Elisp manual correctly, can only be done for
files being visited (which I don't do in order to avoid the overhead of
setting up the major mode and any minor modes the user may have
installed), and only works to lock out Emacs.

> The mod-time check has to be done before writing (since writing is
> conditional on the result of the mod-time check), you have a race:
>
>    User A checks mod-time
>    User B checks mod-time
>    User A writes
>    User B writes
>
> And this applies no matter how/when the mod-time was originally acquired.

Yes, I figured that. I was just wondering if there were a standard way
of dealing with such cases, since AFAIU it is possible to put a lock on
a file on the OS level. But Emacs doesn't do that, right?

Thanks for the answer, at least I know how to proceed now.



-- 
Joost Kremers                                   joostkremers@fastmail.fm
Selbst in die Unterwelt dringt durch Spalten Licht
EN:SiS(9)

Re: Best way to check file modification time?

[Stefan Monnier]

> Yes, I figured that. I was just wondering if there were a standard way
> of dealing with such cases, since AFAIU it is possible to put a lock on
> a file on the OS level. But Emacs doesn't do that, right?

Locking in general is difficult, and on top of that locking in POSIX is
a big mess.  So you have to restrict the problem by considering which
other programs might interfere (and what kind of locking they might
use), as well how much effort you're willing to expend and what amount
of breakage you're willing to live with.


        Stefan

Re: Best way to check file modification time?

[Joost Kremers]

Barry Margolin wrote:
> Instead of saving the file's modification time, remember the time that 
> you started reading the file. Then when you're about to save it, see if 
> the mod time is newer than that.

Ok, that sounds like a good suggestion. Thanks.

> This can potentially run into a problem if the file is on a server and 
> there's clock skew between the client and server, but that's a common 
> problem.

Yeah, I already figured it would not be possible to exclude the risk
completely. Such is life.


-- 
Joost Kremers                                   joostkremers@fastmail.fm
Selbst in die Unterwelt dringt durch Spalten Licht
EN:SiS(9)