corrupted ,v files break cvs-to-git mirror

corrupted ,v files break cvs-to-git mirror

[Jim Meyering]

This morning I noticed that my semi-automatic cvs-to-git
emacs repo-mirroring service got stuck in an infinite loop.
I tracked it down (through git-cvsimport and cvsps) to a corrupt ,v file:

    $ rlog /emacs/.state/cvsrepo/emacs/emacs/lisp/obsolete/swedish.el,v \
      |grep revision|head -25
    total revisions: 21;    selected revisions: 21
    revision 1.6
    revision 1.5
    revision 1.4
    revision 1.3
    revision 1.2
    revision 1.1   <<-- Huh?!?  This should be 1.13, not a duplicate of "1.1".
    revision 1.12
    revision 1.11
    revision 1.10
    revision 1.9
    revision 1.8
    revision 1.7
    revision 1.6
    revision 1.5
    revision 1.4
    revision 1.3
    revision 1.2
    revision 1.1
    Initial revision
    revision 1.8.14.5
    revision 1.8.14.4
    revision 1.8.14.3
    revision 1.8.14.2
    revision 1.8.14.1

I've never seen duplicate revision numbers like that before,
and I've been around the block a few time with cvs.
That seems like a serious problem.

For example, how will "cvs diff -r 1.6" work on this file, now?
Either way, it is guaranteed to be wrong some of the time.
If you tag the latest (1.6) version, which will it select?
If a tag for an older version of emacs pointed at the original
1.6 revision, does it still?  FYI, a couple of experiments suggest
that at least to rcsdiff (and probably cvs, too) the older shadowed
version numbers are not usable anymore.  E.g., rcsdiff -u -r1.11 -r1.12
on that ,v file gives an empty diff.  The 1.1 -> 1.2 diff selects the
most recent one, not the original.

Does anyone here have any idea how that happened?
FYI, here's the second "1.1" log entry, from "cvs log" output:
[What does "Moved" mean? ]

    ----------------------------
    revision 1.1
    date: 2005/08/23 20:05:32;  author: jhd;  state: Exp;
    Moved from international.  Old log:
    ;;; swedish.el --- miscellaneous functions for dealing with Swedish.
    ----------------------------
    revision 1.12
    date: 2005/07/04 17:22:27;  author: lute;  state: Exp;  lines: +2 -2
    Update FSF's address.
    ----------------------------

I hope it's easily fixed.

Re: corrupted ,v files break cvs-to-git mirror

[Miles Bader]

Jim Meyering <jim@meyering.net> writes:
> Does anyone here have any idea how that happened?
> FYI, here's the second "1.1" log entry, from "cvs log" output:
> [What does "Moved" mean? ]
>
>     ----------------------------
>     revision 1.1
>     date: 2005/08/23 20:05:32;  author: jhd;  state: Exp;
>     Moved from international.  Old log:
>     ;;; swedish.el --- miscellaneous functions for dealing with Swedish.
>     ----------------------------
>     revision 1.12
>     date: 2005/07/04 17:22:27;  author: lute;  state: Exp;  lines: +2 -2
>     Update FSF's address.
>     ----------------------------
>
> I hope it's easily fixed.

It looks as if the stuff past "Moved old log:" are _not_ actual CVS
revisions, but all part of the _log message_ for revision 1.1 -- i.e.,
the person who did the "move" just copied the entire CVS log of from the
old location verbatim, and stuffed into his log entry.

So if that's the case probably CVS isn't confused, because it should do
proper escaping of log messages, but any tools which try to parse the
output of "cvs log" will be very confused (as I guess your tool was).

I guess you can fix it just by  giving revision 1.1 a differnt log
message without all the annoying stuff (using "cvs admin -m...").

-Miles

-- 
(\(\
(^.^)
(")")
*This is the cute bunny virus, please copy this into your sig so it can spread.

Re: corrupted ,v files break cvs-to-git mirror

[Jan Djärv]

Miles Bader skrev:
> Jim Meyering <jim@meyering.net> writes:
>> Does anyone here have any idea how that happened?
>> FYI, here's the second "1.1" log entry, from "cvs log" output:
>> [What does "Moved" mean? ]
>>
>>     ----------------------------
>>     revision 1.1
>>     date: 2005/08/23 20:05:32;  author: jhd;  state: Exp;
>>     Moved from international.  Old log:
>>     ;;; swedish.el --- miscellaneous functions for dealing with Swedish.
>>     ----------------------------
>>     revision 1.12
>>     date: 2005/07/04 17:22:27;  author: lute;  state: Exp;  lines: +2 -2
>>     Update FSF's address.
>>     ----------------------------
>>
>> I hope it's easily fixed.
> 
> It looks as if the stuff past "Moved old log:" are _not_ actual CVS
> revisions, but all part of the _log message_ for revision 1.1 -- i.e.,
> the person who did the "move" just copied the entire CVS log of from the
> old location verbatim, and stuffed into his log entry.

Yes, I did that.  How else can we preserve the CVS log when moving files?  But 
I guess the whole file can be removed, it was obsolete in emacs 22.1, so for 
23.1 it should be OK to remove, no?

	Jan D.

Re: corrupted ,v files break cvs-to-git mirror

[David Kastrup]

Jan Djärv <jan.h.d@swipnet.se> writes:

> Miles Bader skrev:
>> Jim Meyering <jim@meyering.net> writes:
>>> Does anyone here have any idea how that happened?
>>> FYI, here's the second "1.1" log entry, from "cvs log" output:
>>> [What does "Moved" mean? ]
>>>
>>>     ----------------------------
>>>     revision 1.1
>>>     date: 2005/08/23 20:05:32;  author: jhd;  state: Exp;
>>>     Moved from international.  Old log:
>>>     ;;; swedish.el --- miscellaneous functions for dealing with Swedish.
>>>     ----------------------------
>>>     revision 1.12
>>>     date: 2005/07/04 17:22:27;  author: lute;  state: Exp;  lines: +2 -2
>>>     Update FSF's address.
>>>     ----------------------------
>>>
>>> I hope it's easily fixed.
>>
>> It looks as if the stuff past "Moved old log:" are _not_ actual CVS
>> revisions, but all part of the _log message_ for revision 1.1 -- i.e.,
>> the person who did the "move" just copied the entire CVS log of from the
>> old location verbatim, and stuffed into his log entry.
>
> Yes, I did that.  How else can we preserve the CVS log when moving
> files?

Start off the new file with a suitably increased revision number.
Starting from 1.1 again is just asking for trouble.

-- 
David Kastrup

Re: corrupted ,v files break cvs-to-git mirror

[Jan Djärv]

David Kastrup skrev:
> Jan Djärv <jan.h.d@swipnet.se> writes:
>> Yes, I did that.  How else can we preserve the CVS log when moving
>> files?
> 
> Start off the new file with a suitably increased revision number.
> Starting from 1.1 again is just asking for trouble.


Okay, I didn't know you could do that.  I'll keep that in mind.

	Jan D.

Re: corrupted ,v files break cvs-to-git mirror

[Andreas Schwab]

Jan Djärv <jan.h.d@swipnet.se> writes:

> Yes, I did that.  How else can we preserve the CVS log when moving files?

The log is still available in the removed file.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: corrupted ,v files break cvs-to-git mirror

[Stefan Monnier]

>>> Does anyone here have any idea how that happened?
>>> FYI, here's the second "1.1" log entry, from "cvs log" output:
>>> [What does "Moved" mean? ]
>>> 
>>> ----------------------------
>>> revision 1.1
>>> date: 2005/08/23 20:05:32;  author: jhd;  state: Exp;
>>> Moved from international.  Old log:
>>> ;;; swedish.el --- miscellaneous functions for dealing with Swedish.
>>> ----------------------------
>>> revision 1.12
>>> date: 2005/07/04 17:22:27;  author: lute;  state: Exp;  lines: +2 -2
>>> Update FSF's address.
>>> ----------------------------
>>> 
>>> I hope it's easily fixed.
>> 
>> It looks as if the stuff past "Moved old log:" are _not_ actual CVS
>> revisions, but all part of the _log message_ for revision 1.1 -- i.e.,
>> the person who did the "move" just copied the entire CVS log of from the
>> old location verbatim, and stuffed into his log entry.

> Yes, I did that.  How else can we preserve the CVS log when moving files?

All you need to put in the log is where to find the rest of the history, by
saying something like "moved from ../foo/bar".


        Stefan

Re: corrupted ,v files break cvs-to-git mirror

[Jim Meyering]

Jan Djärv <jan.h.d@swipnet.se> wrote:
...
> Yes, I did that.  How else can we preserve the CVS log when moving
> files?  But I guess the whole file can be removed, it was obsolete in
> emacs 22.1, so for 23.1 it should be OK to remove, no?

Dare I say it? :-)

Use better software for version-control.

For example, git can detect renames like that automatically,
based on similarity of removed and added-elsewhere content.
When you rename or copy a file, you can see it in logs and diff
output, e.g., like this:

[here, I renamed "r" to "s" and moved part of
"r"s data to the new file, "t", along the way.
Hence the "similarity index 68%" for r->s. ]

    $ git-log -C -M -p -1
    commit 55f682db0aa88af169d8812f366348a8b05f70e0
    Author: Jim Meyering <jim@meyering.net>
    Date:   Thu Aug 30 09:42:59 2007 +0200

        .

    diff --git a/r b/s
    similarity index 68%
    rename from r
    rename to s
    index 6006ca4..748f1f3 100644
    --- a/r
    +++ b/s
    @@ -27,15 +27,3 @@
     27
     28
     29
    -30
    -31
    -32
    -33
    -34
    -35
    -36
    -37
    -38
    -39
    -40
    -41
    diff --git a/t b/t
    new file mode 100644
    index 0000000..36763a7
    --- /dev/null
    +++ b/t
    @@ -0,0 +1,12 @@
    +30
    +31
    +32
    +33
    +34
    +35
    +36
    +37
    +38
    +39
    +40
    +41

Re: corrupted ,v files break cvs-to-git mirror

[Jim Meyering]

Miles Bader <miles.bader@necel.com> wrote:
> Jim Meyering <jim@meyering.net> writes:
>> Does anyone here have any idea how that happened?
>> FYI, here's the second "1.1" log entry, from "cvs log" output:
>> [What does "Moved" mean? ]
>>
>>     ----------------------------
>>     revision 1.1
>>     date: 2005/08/23 20:05:32;  author: jhd;  state: Exp;
>>     Moved from international.  Old log:
>>     ;;; swedish.el --- miscellaneous functions for dealing with Swedish.
>>     ----------------------------
>>     revision 1.12
>>     date: 2005/07/04 17:22:27;  author: lute;  state: Exp;  lines: +2 -2
>>     Update FSF's address.
>>     ----------------------------
>>
>> I hope it's easily fixed.
>
> It looks as if the stuff past "Moved old log:" are _not_ actual CVS
> revisions, but all part of the _log message_ for revision 1.1 -- i.e.,
> the person who did the "move" just copied the entire CVS log of from the
> old location verbatim, and stuffed into his log entry.

Yep.  I should have looked directly at the ,v file.

> So if that's the case probably CVS isn't confused, because it should do
> proper escaping of log messages, but any tools which try to parse the
> output of "cvs log" will be very confused (as I guess your tool was).

Yes.  For cvsps to do its job properly, it would need an rlog-style
log-listing tool, with an option to produce unambiguous (i.e., NUL-separated)
blocks of per-revision data.

> I guess you can fix it just by  giving revision 1.1 a differnt log
> message without all the annoying stuff (using "cvs admin -m...").

I've already modified the ,v files in a mirror hierarchy and
confirmed that it avoids the infloop.  If I hear no objection
in the next few hours, I'll update the official ,v files on savannah.

lisp/obsolete/iso-swed.el has the same problem.

emacs.git is in sync again [Re: corrupted , v files break cvs-to-git mirror

[Jim Meyering]

Jim Meyering <jim@meyering.net> wrote:
> I've already modified the ,v files in a mirror hierarchy and
> confirmed that it avoids the infloop.  If I hear no objection
> in the next few hours, I'll update the official ,v files on savannah.

Done.  I've left the original ,v files as backups, just in case.
They'll be removed in 2 days.

The git mirror is working again:

    git://git.sv.gnu.org/emacs.git
    http://git.sv.gnu.org/r/emacs.git