bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Tim Van Holder]

Started noticing this last Thursday or Friday on a bzr trunk build.
After committing from vc-dir (CVS backend), the committed files were not
automatically disappearing from the vc-dir list, requiring an explicit
[g] instead.
I notice today that the files themselves are left in a "modified" state
as far as VC is concerned ("CVS:1.x" in modeline). [C-x v u] is needed
to restore the "unchanged" state ("CVS-1.x" in the modeline).
While not a huge problem, this is an inconvenience (and definitely
suggests a bug somewhere); for one thing, it inhibits the creation of a
backup file when such a committed file is modified again, making a
revert hit the server.

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Glenn Morris]

Tim Van Holder wrote:

> Started noticing this last Thursday or Friday on a bzr trunk build.
> After committing from vc-dir (CVS backend), the committed files were not
> automatically disappearing from the vc-dir list, requiring an explicit
> [g] instead.

I've never noticed committed files automatically disappear from vc-dir.
They change to status "up-to-date", and disappear if one presses "x".
AFAIK, this is always how vc-dir has behaved.

> I notice today that the files themselves are left in a "modified" state
> as far as VC is concerned ("CVS:1.x" in modeline).

I cannot reproduce this.

Please give a recipe starting from emacs -Q, and include the setup
information from M-x report-emacs-bug.

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Tim Van Holder]

On 25 January 2011 01:48, Glenn Morris <rgm@gnu.org> wrote:
> Tim Van Holder wrote:
>
>> Started noticing this last Thursday or Friday on a bzr trunk build.
>> After committing from vc-dir (CVS backend), the committed files were not
>> automatically disappearing from the vc-dir list, requiring an explicit
>> [g] instead.
>
> I've never noticed committed files automatically disappear from vc-dir.
> They change to status "up-to-date", and disappear if one presses "x".
> AFAIK, this is always how vc-dir has behaved.

Fair enough, I described this incorrectly. As it stands, they're
remaining as "edited", not changing to "up-to-date".

>> I notice today that the files themselves are left in a "modified" state
>> as far as VC is concerned ("CVS:1.x" in modeline).
>
> I cannot reproduce this.

I think it was only for multi-file commits, and when the file isn't
loaded in a buffer.
If it's in a buffer, I think that buffer's modeline updates to "-",
but closing the file and revisiting it shows the ":" again.
Running a "cvs update" outside of emacs also has no effect so it looks
like it's state kept internally by VC (if it was (re)checking
CVS/Entries it would/should see it as unmodified again).

I'll re-bootstrap emacs to ensure the compiled lisp files are properly
in sync, and I'll try to set up a scenario with emacs -Q.

Info from report-emacs-bug:

In GNU Emacs 24.0.50.2 (i686-pc-linux-gnu, GTK+ Version 2.8.20)
 of 2011-01-24 on leeloo
Windowing system distributor `The Cygwin/X Project', version 11.0.10503000
configured using `configure  '--with-x''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  show-paren-mode: t
  pc-selection-mode: t
  display-time-mode: t
  delete-selection-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Tim Van Holder]

On 25 January 2011 09:00, Tim Van Holder <tim.vanholder@gmail.com> wrote:
> I'll re-bootstrap emacs to ensure the compiled lisp files are properly
> in sync, and I'll try to set up a scenario with emacs -Q.

After bootstrap, a commit on 2 files in "added" state changed them to
"edited" state (instead of up-to-date). So there is _some_ state
changing still happening.

I tried a further multi-file commit with emacs -Q; this did change the
state to up-to-date, so it looks like it's an interaction with some
setting somewhere. I'll try to hunt it down.

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Tim Van Holder]

[-- Attachment #1: Type: text/plain, Size: 1587 bytes --]

On 25 January 2011 10:44, Tim Van Holder <tim.vanholder@gmail.com> wrote:
> On 25 January 2011 09:00, Tim Van Holder <tim.vanholder@gmail.com> wrote:
>> I'll re-bootstrap emacs to ensure the compiled lisp files are properly
>> in sync, and I'll try to set up a scenario with emacs -Q.
>
> After bootstrap, a commit on 2 files in "added" state changed them to
> "edited" state (instead of up-to-date). So there is _some_ state
> changing still happening.
>
> I tried a further multi-file commit with emacs -Q; this did change the
> state to up-to-date, so it looks like it's an interaction with some
> setting somewhere. I'll try to hunt it down.

This is very odd. With "emacs -q --no-site-file --no-site-lisp" (i.e.
-Q but without the --no-splash) it still doesn't work, but with "emacs
-q --no-splash" it does, so the --no-splash by itself has a positive
effect.
"emacs --no-splash" also works, but only if I disable loading of my
customize settings.
By a process of elimination, it turns out that the
  '(display-time-mode t)
line in (custom-set-variables) is the culprit. With that removed,
things work, even without --no-splash.

I'm attaching a tarball with a dummy CVS repos; my tests used:

  mkdir -p /wherever/you/want
  cd /wherever/you/want
  tar xzvf /somewhere/emacs-7905.tar.gz
  ./reset   # resets the dummy repos and checks out a sandbox with two
newly-added files
  emacs <command line options>
     [C-x v d] sandbox
     n
     M
     v
     foo[C-c C-c]

And then the two files are either edited or up-to-date, depending on
whether the issue presents itself or not.

[-- Attachment #2: emacs-7905.tar.gz --]
[-- Type: application/x-gzip, Size: 12361 bytes --]

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Glenn Morris]

Tim Van Holder wrote:

> By a process of elimination, it turns out that the
>   '(display-time-mode t)
> line in (custom-set-variables) is the culprit. With that removed,
> things work, even without --no-splash.
>
> I'm attaching a tarball with a dummy CVS repos; my tests used:

Thanks for the great test case, but I'm afraid I still cannot
reproduce it if I use a .emacs that just has a customized
display-time-mode (together with --no-site-file).

Does anyone else see this problem?

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Tim Van Holder]

On 26 January 2011 05:25, Glenn Morris <rgm@gnu.org> wrote:
> Tim Van Holder wrote:
>
>> By a process of elimination, it turns out that the
>>   '(display-time-mode t)
>> line in (custom-set-variables) is the culprit. With that removed,
>> things work, even without --no-splash.
>>
>> I'm attaching a tarball with a dummy CVS repos; my tests used:
>
> Thanks for the great test case, but I'm afraid I still cannot
> reproduce it if I use a .emacs that just has a customized
> display-time-mode (together with --no-site-file).
>
> Does anyone else see this problem?

That's odd - using the testcase, if I do

  $ emacs -Q
  [C-x v d]sandbox
  n
  M
  [M-x]display-time-mode
  v
  foo[C-c C-c]

then the files go to "edited" for me (without the display-time-mode
they become "up-to-date").
Would it help for me to edebug particular VC defuns to see where
exactly things go wrong?

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Tim Van Holder]

On 26 January 2011 10:44, Tim Van Holder <tim.vanholder@gmail.com> wrote:
> On 26 January 2011 05:25, Glenn Morris <rgm@gnu.org> wrote:
>> Tim Van Holder wrote:
>>
>>> By a process of elimination, it turns out that the
>>>   '(display-time-mode t)
>>> line in (custom-set-variables) is the culprit. With that removed,
>>> things work, even without --no-splash.
>>>
>>> I'm attaching a tarball with a dummy CVS repos; my tests used:
>>
>> Thanks for the great test case, but I'm afraid I still cannot
>> reproduce it if I use a .emacs that just has a customized
>> display-time-mode (together with --no-site-file).
>>
>> Does anyone else see this problem?
>
> That's odd - using the testcase, if I do
>
>  $ emacs -Q
>  [C-x v d]sandbox
>  n
>  M
>  [M-x]display-time-mode
>  v
>  foo[C-c C-c]
>
> then the files go to "edited" for me (without the display-time-mode
> they become "up-to-date").
> Would it help for me to edebug particular VC defuns to see where
> exactly things go wrong?

It seems to run deeper than just commits. I'm seeing now that any
CVS-controlled file I visit is marked as "CVS:1.x" (i.e. edited), even
files in a freshly checked out sandbox. [C-x v u] clears that state,
but saving the file, closing it and then revisiting it puts it in
edited state again.
I'm reasonably sure that didn't happen before; however, files are only
showing up in vc-dir if they are actually changed, so I may just have
missed this. This behaviour does not go away when I disable
display-time-mode, however, so it could be a completely separate
issue.

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Glenn Morris]

Tim Van Holder wrote:

>> Would it help for me to edebug particular VC defuns to see where
>> exactly things go wrong?
>
> It seems to run deeper than just commits. I'm seeing now that any
> CVS-controlled file I visit is marked as "CVS:1.x" (i.e. edited), even
> files in a freshly checked out sandbox. [C-x v u] clears that state,
> but saving the file, closing it and then revisiting it puts it in
> edited state again.

Well that is all very wacky. Does eg Emacs 23.2 work right on that
system at present with CVS? (This is a Cygwin build?)

You could try edebugging vc-cvs-state, vc-cvs-state-heuristic, and
vc-cvs-parse-entry. And the vc-cvs-*-dir-* functions if you want to
check vc-dir behaviour.

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Tim Van Holder]

On 27 January 2011 08:25, Glenn Morris <rgm@gnu.org> wrote:
> Tim Van Holder wrote:
>
>>> Would it help for me to edebug particular VC defuns to see where
>>> exactly things go wrong?
>>
>> It seems to run deeper than just commits. I'm seeing now that any
>> CVS-controlled file I visit is marked as "CVS:1.x" (i.e. edited), even
>> files in a freshly checked out sandbox. [C-x v u] clears that state,
>> but saving the file, closing it and then revisiting it puts it in
>> edited state again.
>
> Well that is all very wacky. Does eg Emacs 23.2 work right on that
> system at present with CVS? (This is a Cygwin build?)

I've been running from CVS HEAD (and now bzr trunk) for years now; the
latest "installed" version is 23.0.60.1 (an old CVS HEAD build).
A commit from [C-x v d] there has the "(added)" tag correctly
disappearing, but it's still the old VC (vc-dired-mode instead of
vc-dir), so I'm not sure it matters much.
Main problem with the system is that the hardware has no kernel
support anymore, so I cannot deploy official packages anymore (they
all require a kernel upgrade); if absolutely necessary I could build
23.2 from sources to test its behaviour.

It's not a cygwin build by the way - it's a debian linux build, but
I'm running it from a PuTTY session on Windows, which then tunnels the
X connection to the cygwin X windows.
It's definitely an exotic setup (there are some serious problems with
the recent changes to the clipboard handling), but I don't think
that's likely to affect VC (the sandboxes are all on the linux
machine's file system, not on a samba share).

> You could try edebugging vc-cvs-state, vc-cvs-state-heuristic, and
> vc-cvs-parse-entry. And the vc-cvs-*-dir-* functions if you want to
> check vc-dir behaviour.

Think I see the problem being caused by vc-cvs-state-heuristic; it
uses the file's checkout property:
  (vc-file-getprop file 'vc-checkout-time)
But this seems to always return 0 when I try to open a CVS-controlled
file, which means vc-cvs-state-heuristic will only report either
'added or 'edited.

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Glenn Morris]

Tim Van Holder wrote:

> Think I see the problem being caused by vc-cvs-state-heuristic;

This doesn't make sense to me, because vc-dir supposedly never uses that.
But if so, setting vc-cvs-stay-local to nil should make everything work.

> it uses the file's checkout property:
>   (vc-file-getprop file 'vc-checkout-time)
> But this seems to always return 0 when I try to open a CVS-controlled
> file, which means vc-cvs-state-heuristic will only report either
> 'added or 'edited.

Maybe edebugging vc-cvs-parse-entry will show why this is so.
Perhaps the CVS/Entries file has an unexpected format.

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Tim Van Holder]

On 28 January 2011 19:16, Glenn Morris <rgm@gnu.org> wrote:
> Tim Van Holder wrote:
>
>> Think I see the problem being caused by vc-cvs-state-heuristic;
>
> This doesn't make sense to me, because vc-dir supposedly never uses that.
> But if so, setting vc-cvs-stay-local to nil should make everything work.

I checked the file-is-considered-edited-on-open behaviour, and there
vc-cvs-state-heuristic seems to be the culprit; as I mentioned, vc-dir
did not consider all files as modified.
Also, vc-dir correctly identifies the state after a 'g', so vc-dir
seems to work correctly in and of itself. I'm assuming the
update-status-in-vc-dir behaviour after a commit relies on local
operations, so if it expects to run vc-cvs-state-heuristic on the
newly-committed files to get their new state, it could be the culprit
there too.

>> it uses the file's checkout property:
>>   (vc-file-getprop file 'vc-checkout-time)
>> But this seems to always return 0 when I try to open a CVS-controlled
>> file, which means vc-cvs-state-heuristic will only report either
>> 'added or 'edited.
>
> Maybe edebugging vc-cvs-parse-entry will show why this is so.

Aha - it looks like it's a timezone thing. vc-cvs-parse-entry does
everything right, except that it tries to compare the file's mtime
with the time from CVS/Entries, assuming UTC as timezone for it (which
is correct for CVS/Entries but apparently not for mtime).
In my case, they differ by an hour:

$ date; date --utc
Mon Jan 31 09:50:50 CET 2011
Mon Jan 31 08:50:50 UTC 2011
$ grep foo CVS/Entries
/foo/1.297.2.24/Mon Jan 31 08:30:58 2011//Tmytag
$ stat foo
  File: `foo'
  Size: 75579           Blocks: 160        IO Block: 4096   regular file
Device: 808h/2056d      Inode: 8142863     Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1000/     tim)   Gid: ( 1000/     tim)
Access: 2011-01-31 09:37:57.000000000 +0100
Modify: 2011-01-31 09:30:58.000000000 +0100
Change: 2011-01-31 09:31:00.000000000 +0100

As a result, checkout-time is set to 0 (and state to edited).

Running "TZ=UTC emacs" shows the problem going away (both the vc-dir
state change and the file-edited-on-open).
So perhaps mtime needs decode-time applied, and then the zone argument
applied to the time info (I would expect there to be a function for
that already, but I didn't see one offhand).

> Perhaps the CVS/Entries file has an unexpected format.

Doubtful, since this also affected fresh clean sandboxes (like the one
in the sample).

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Andreas Schwab]

Tim Van Holder <tim.vanholder@gmail.com> writes:

> $ grep foo CVS/Entries
> /foo/1.297.2.24/Mon Jan 31 08:30:58 2011//Tmytag
> $ stat foo
> Modify: 2011-01-31 09:30:58.000000000 +0100

Those are the same time stamp.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Tim Van Holder]

On 31 January 2011 11:31, Andreas Schwab <schwab@linux-m68k.org> wrote:
> Tim Van Holder <tim.vanholder@gmail.com> writes:
>
>> $ grep foo CVS/Entries
>> /foo/1.297.2.24/Mon Jan 31 08:30:58 2011//Tmytag
>> $ stat foo
>> Modify: 2011-01-31 09:30:58.000000000 +0100
>
> Those are the same time stamp.
>
> Andreas.

Yes, but they end up not comparing as equal in VC.

Given a similar file but with CVS/Entries having "Tue Jan 11 15:18:16
2011" and stat giving "2011-01-11 16:18:16 +0100", edebugging
vc-cvs-parse-entry gets:

"time" as "Tue Jan 11 15:18:16 2011" (from matching its CVS/Entries line).
"mtime" as (19756 29880) (from "(nth 5 (file-attributes <file>))")
"parsed-time" as (16 18 15 11 1 2011 2 nil 0) (from parse-time on
value of "time")

It then has a check
    (equal mtime (apply 'encode-time parsed-time))
which returns nil; the apply yields (19756 26280), which is indeed not
the same as (19756 29880).

Checking the behaviour of (encode-time) I see that
  (encode-time 16 18 15 11 1 2011 2 nil 3600)
  (encode-time 16 18 15 11 1 2011 2 nil 0)
both return the same value "(19756 26280)" (which decodes as "(16 18
15 11 1 2011 2 nil 3600)").
This strikes me as odd since encode-time explicitly says that it
treats the last argument as timezone offset, suggesting that it cares
about it.

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Andreas Schwab]

Tim Van Holder <tim.vanholder@gmail.com> writes:

> Checking the behaviour of (encode-time) I see that
>   (encode-time 16 18 15 11 1 2011 2 nil 3600)
>   (encode-time 16 18 15 11 1 2011 2 nil 0)
> both return the same value "(19756 26280)"

You should find out why.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Tim Van Holder]

On 31 January 2011 15:10, Andreas Schwab <schwab@linux-m68k.org> wrote:
> Tim Van Holder <tim.vanholder@gmail.com> writes:
>
>> Checking the behaviour of (encode-time) I see that
>>   (encode-time 16 18 15 11 1 2011 2 nil 3600)
>>   (encode-time 16 18 15 11 1 2011 2 nil 0)
>> both return the same value "(19756 26280)"
>
> You should find out why.
>
> Andreas.

I tried treating a zone of 0 the same way as a nil zone (i.e. set TZ
to UTZ0), in case it was the XXX-0:00:00 TZ setting that was not
working:

      if (EQ (zone, Qt) || (INTEGERP (zone) && XINT (zone) == 0))
	tzstring = "UTC0";

but that made no difference: encode-time completely ignores the zone
setting (whether nil, t, 0, 3600, 17, 36000, "CET", "UTC", "FOO").

In order to do the (apparent) right thing, encode-time seems to rely
on mktime() to take $TZ into account (otherwise there is no point in
saving/restoring the environment with a specific TZ value).
However, lib/mktime.c (which is in use by my emacs build) does not
seem to refer to any timezone info at all - neither $TZ nor tm_zone is
referenced; it seems to rely on localtime() to deal with that. I guess
that's not the case for my localtime() (from glibc 2.3.6).

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Glenn Morris]

Tim Van Holder wrote:

>>>   (encode-time 16 18 15 11 1 2011 2 nil 3600)
>>>   (encode-time 16 18 15 11 1 2011 2 nil 0)
>>> both return the same value "(19756 26280)"
[...]
> In order to do the (apparent) right thing, encode-time seems to rely
> on mktime() to take $TZ into account (otherwise there is no point in
> saving/restoring the environment with a specific TZ value).
> However, lib/mktime.c (which is in use by my emacs build) does not
> seem to refer to any timezone info at all - neither $TZ nor tm_zone is
> referenced; it seems to rely on localtime() to deal with that. I guess
> that's not the case for my localtime() (from glibc 2.3.6).

FWIW I built the current trunk on a system with glibc 2.3.4, and the
two above encode-times return two different values. So whatever it is,
it's not related to the glibc version.

bug#7905: 24.0.50; VC not updating file status properly anymore after commit from vc-dir

[Tim Van Holder]

On 1 February 2011 22:45, Glenn Morris <rgm@gnu.org> wrote:
> Tim Van Holder wrote:
>
>>>>   (encode-time 16 18 15 11 1 2011 2 nil 3600)
>>>>   (encode-time 16 18 15 11 1 2011 2 nil 0)
>>>> both return the same value "(19756 26280)"
> [...]
>> In order to do the (apparent) right thing, encode-time seems to rely
>> on mktime() to take $TZ into account (otherwise there is no point in
>> saving/restoring the environment with a specific TZ value).
>> However, lib/mktime.c (which is in use by my emacs build) does not
>> seem to refer to any timezone info at all - neither $TZ nor tm_zone is
>> referenced; it seems to rely on localtime() to deal with that. I guess
>> that's not the case for my localtime() (from glibc 2.3.6).
>
> FWIW I built the current trunk on a system with glibc 2.3.4, and the
> two above encode-times return two different values. So whatever it is,
> it's not related to the glibc version.

I ran some separate tests and glibc's mktime and localtime seem to
properly take time zone info into account, regardless of whether it
came from $TZ, setenv("TZ", ...) or set_time_zone_rule() as copied
from editfns.c. So things _should_ be working fine in encode-time.
So it looks like it's an interaction inside emacs breaking things.
Whatever it is, it seems to be triggered by display-time-mode. Before
that is run, everything works fine. After running it, all
CVS-controlled files are edited as far as VC knows. After running it
again (disabling the actual display-time-mode display), things do NOT
return to normal. But I don't immediately see anything in time.el that
would suggest a problem.
It's particularly annoying that noone seems to be able to reproduce
it... I give up - I'll just disable display-time-mode.