Everyone, please stop making my life more difficult

Everyone, please stop making my life more difficult

[Eric S. Raymond]

While running my latest repository conversion, I found this comment

    revno: 116811.1.662
    fixes bug: http://debbugs.gnu.org/18384
    committer: Eli Zaretskii <eliz@gnu.org>
    branch nick: emacs-24
    timestamp: Tue 2014-09-02 18:16:42 +0300
    message:
      Fix bug #18384 with incorrect reporting of row number by posn-col-row.
      
       lisp/subr.el (posn-col-row): Revert the change from commit
       r99634.2.576 eliz@gnu.org-20101113210758-8ml5kibjtza5ysmb, which
       was inadvertently merged from emacs-23 release branch in r102428
       monnier@iro.umontreal.ca-20101118035414-yvlg7k7dk4k4l3q, and
       introduced an off-by-one error in the reported row when there is a
       header line.
      
       src/dispnew.c (buffer_posn_from_coords): Fix an off-by-one error in
       the reported row in the case of a window with a header line, by
       improving on the fix committed in r106022
       eliz@gnu.org-20111008105850-ht4tvsayohvr1kjc.

There are two things about this comment that make my life
unnecessarily more difficult.

1) People are continuing to embed bzr revision numbers like
   r99634.2.576 and r102428 in comments.  Do not do this.  
   They're soon going to cease being meaningful, and every time
   I have to run another conversion to clean these out it costs
   me eight to ten hours.

2) "eliz@gnu.org-20101113210758-8ml5kibjtza5ysmb" Um.  What the *hell*?
   This is not the time to be inventing some random new magic-cookie
   commit reference format that my tools don't know how to recognize.
   That should have been 2010-11-13T21:07:58Z!eliz@gnu.org (I think).

Please stop this nonsense.  Now I'm going to have to write a new comment
scanner to find these monster turds, and it's going to take me hours
of work to make sure I've cleaned them all out.

The correct format for a commit reference that will remain
intelligible across every VCS, forever, is this:

<RFC3339-date>!<email-address>

Example: 2010-11-13T21:07:58Z!eliz@gnu.org

That is what everyone should be using, now and in the future, to avoid
causing unnecessary problems for this repository conversion and any 
we may have to do in the future.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Rifles, muskets, long-bows and hand-grenades are inherently democratic
weapons.  A complex weapon makes the strong stronger, while a simple
weapon -- so long as there is no answer to it -- gives claws to the
weak.
        -- George Orwell, "You and the Atom Bomb", 1945

Re: Everyone, please stop making my life more difficult

[Eli Zaretskii]

> From: esr@thyrsus.com (Eric S. Raymond)
> Date: Fri, 12 Sep 2014 00:36:52 -0400 (EDT)
> 
> While running my latest repository conversion, I found this comment
> 
>     revno: 116811.1.662
>     fixes bug: http://debbugs.gnu.org/18384
>     committer: Eli Zaretskii <eliz@gnu.org>
>     branch nick: emacs-24
>     timestamp: Tue 2014-09-02 18:16:42 +0300
>     message:
>       Fix bug #18384 with incorrect reporting of row number by posn-col-row.
>       
>        lisp/subr.el (posn-col-row): Revert the change from commit
>        r99634.2.576 eliz@gnu.org-20101113210758-8ml5kibjtza5ysmb, which
>        was inadvertently merged from emacs-23 release branch in r102428
>        monnier@iro.umontreal.ca-20101118035414-yvlg7k7dk4k4l3q, and
>        introduced an off-by-one error in the reported row when there is a
>        header line.
>       
>        src/dispnew.c (buffer_posn_from_coords): Fix an off-by-one error in
>        the reported row in the case of a window with a header line, by
>        improving on the fix committed in r106022
>        eliz@gnu.org-20111008105850-ht4tvsayohvr1kjc.
> 
> There are two things about this comment that make my life
> unnecessarily more difficult.
> 
> 1) People are continuing to embed bzr revision numbers like
>    r99634.2.576 and r102428 in comments.  Do not do this.  

There's no easy way to refer to an old commit in an unequivocal way.

>    They're soon going to cease being meaningful, and every time
>    I have to run another conversion to clean these out it costs
>    me eight to ten hours.

I thought you already have machinery in place to do that
automatically.

> 2) "eliz@gnu.org-20101113210758-8ml5kibjtza5ysmb" Um.  What the *hell*?

It's a bzr revision-id.  You can use it in "bzr log" or any other bzr
command that accepts a revision spec.

>    This is not the time to be inventing some random new magic-cookie
>    commit reference format that my tools don't know how to recognize.
>    That should have been 2010-11-13T21:07:58Z!eliz@gnu.org (I think).

I was certain you already deal with this.  If not, please tell what
information you need to convert this automatically.

> The correct format for a commit reference that will remain
> intelligible across every VCS, forever, is this:
> 
> <RFC3339-date>!<email-address>
> 
> Example: 2010-11-13T21:07:58Z!eliz@gnu.org

How can I produce this automatically given the information I have
about the revision, i.e. its number and, possibly, revision-id?
Is it possible to write a command to do this?

Re: Everyone, please stop making my life more difficult

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 1049 bytes --]

() esr@thyrsus.com (Eric S. Raymond)
() Fri, 12 Sep 2014 00:36:52 -0400 (EDT)

   That is what everyone should be using, now and in the future,
   to avoid causing unnecessary problems for this repository
   conversion and any we may have to do in the future.

I think it's good to encounter anomalies, figure out how to
handle them, and encode that handling, rather sooner than later.

Also, a run taking 8-10 hours indicates that another code
enhancement opportunity would be to re-slice the process from
one-shot-batch to incremental-piecemeal.  A bonus side-effect of
checkpointed state is that other tools can be brought to bear
(w/ a sane data design, which i presume).

All this is more difficult, true, but still not as difficult as
getting all pertinent programmers to perform perfectly... :-D

-- 
Thien-Thi Nguyen
   GPG key: 4C807502
   (if you're human and you know it)
      read my lisp: (responsep (questions 'technical)
                               (not (via 'mailing-list)))
                     => nil

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Everyone, please stop making my life more difficult

[David Kastrup]

Thien-Thi Nguyen <ttn@gnu.org> writes:

> () esr@thyrsus.com (Eric S. Raymond)
> () Fri, 12 Sep 2014 00:36:52 -0400 (EDT)
>
>    That is what everyone should be using, now and in the future,
>    to avoid causing unnecessary problems for this repository
>    conversion and any we may have to do in the future.
>
> I think it's good to encounter anomalies, figure out how to
> handle them, and encode that handling, rather sooner than later.
>
> Also, a run taking 8-10 hours indicates that another code
> enhancement opportunity would be to re-slice the process from
> one-shot-batch to incremental-piecemeal.  A bonus side-effect of
> checkpointed state is that other tools can be brought to bear
> (w/ a sane data design, which i presume).

Uh, we are talking about something which is supposed to be done once,
ever.  This is not mirroring.

-- 
David Kastrup

Re: Everyone, please stop making my life more difficult

[Eli Zaretskii]

> From: esr@thyrsus.com (Eric S. Raymond)
> Date: Fri, 12 Sep 2014 00:36:52 -0400 (EDT)
> 
> The correct format for a commit reference that will remain
> intelligible across every VCS, forever, is this:
> 
> <RFC3339-date>!<email-address>
> 
> Example: 2010-11-13T21:07:58Z!eliz@gnu.org

Btw, what is <email-address> here?  Is that the author or the
committer?

Re: Everyone, please stop making my life more difficult

[Eric S. Raymond]

Eli Zaretskii <eliz@gnu.org>:
> > From: esr@thyrsus.com (Eric S. Raymond)
> > Date: Fri, 12 Sep 2014 00:36:52 -0400 (EDT)
> > 
> > The correct format for a commit reference that will remain
> > intelligible across every VCS, forever, is this:
> > 
> > <RFC3339-date>!<email-address>
> > 
> > Example: 2010-11-13T21:07:58Z!eliz@gnu.org
> 
> Btw, what is <email-address> here?  Is that the author or the
> committer?

The committer.

This choice helps avoid collisions.  It is unlikely that one
committer will ever manage two commits in the same clock second.
Slightly less ulikely that two different committers at different
consoles will check in a patch from a different author.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Everyone, please stop making my life more difficult

[Eric S. Raymond]

Eric S. Raymond <esr@thyrsus.com>:
> > Btw, what is <email-address> here?  Is that the author or the
> > committer?
> 
> The committer.
> 
> This choice helps avoid collisions.  It is unlikely that one
> committer will ever manage two commits in the same clock second.
> Slightly less ulikely that two different committers at different
> consoles will check in a patch from a different author.

I meant "a different author who happens to be the same person".
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Everyone, please stop making my life more difficult

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 523 bytes --]

() David Kastrup <dak@gnu.org>
() Fri, 12 Sep 2014 09:25:14 +0200

   Uh, we are talking about something which is supposed to
   be done once, ever.  This is not mirroring.

Right.  The key word here is "supposed".  I'm trying to
substitute grouchy optimism w/ cheerful pessimism, FWIW.

-- 
Thien-Thi Nguyen
   GPG key: 4C807502
   (if you're human and you know it)
      read my lisp: (responsep (questions 'technical)
                               (not (via 'mailing-list)))
                     => nil

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Everyone, please stop making my life more difficult

[Eric S. Raymond]

Eli Zaretskii <eliz@gnu.org>:
> > 1) People are continuing to embed bzr revision numbers like
> >    r99634.2.576 and r102428 in comments.  Do not do this.  
> 
> There's no easy way to refer to an old commit in an unequivocal way.

Its date and committer is pretty unambiguous.  Better yet is a
reference to text in the old commit's summary line, when that's unique
(which it usually is) as that conveys information about what is being
worked on and can easly be searched for.

> >    They're soon going to cease being meaningful, and every time
> >    I have to run another conversion to clean these out it costs
> >    me eight to ten hours.
> 
> I thought you already have machinery in place to do that
> automatically.

Semi-automatically, for the short-form revision numbers.

The fundamental problems here are (a) that the Emacs repository is
large enough to make search and replace on these things a very
expensive proposition, and (b) people cite these in variant forms,
such as "r3333", "3333", and "revno:3333".

Thus, I can't simply take an exhaustive list of all possible revision
IDs in all their possible variant forms and massage it into
search-replace commands. If I did that, a conversion run would take
weeks or months, mostly searching for IDs that are never cited. Also,
there are a few cases where a revision number was typoed, and it's
possible to deduce  what digit was added or garbled.

Thus, to get the job done in finite time. I have to maintain a list of
all actually existing reference cookies and update it whenever I
become aware that somebody adds a new one.

To give you an idea of the scale of the task, implementing the
machinery and assembling the first ID list took me eight solid weeks
of work.  The ID list is large enough that, with read and write
overhead for the repo, applying it now takes over 10 hours - during
which my main and fastest machine is continuously so loaded that the
regression test suite on one of my other projects tends to fail due to
kernel race conditions in the pty layer.

And it's worse than that.  Merging in a new set of reference cookies
usually takes me two or three runs, because for various technical reasons
too painful to describe the reference-patching commands have to be manually
corrected occasionally (this is most likely to happen on branch merges).

So the overhead of fixing these things is *large*.  I only try it every
couple of weeks, at which point it's usually about a three-day process. It'll
be longer this time.

> > 2) "eliz@gnu.org-20101113210758-8ml5kibjtza5ysmb" Um.  What the *hell*?
> 
> It's a bzr revision-id.  You can use it in "bzr log" or any other bzr
> command that accepts a revision spec.

Well, at least that makes it less arbitrary.  But still soon to become
meaningless, and I still have to fix it.
 
> >    This is not the time to be inventing some random new magic-cookie
> >    commit reference format that my tools don't know how to recognize.
> >    That should have been 2010-11-13T21:07:58Z!eliz@gnu.org (I think).
> 
> I was certain you already deal with this.  If not, please tell what
> information you need to convert this automatically.

One I have the location of one of these, it's a simple regex
transformation to turn it into a <RFC3339-date>!<committer-email>
portable revision stamp.  The expensive part will be finding them all.
And I can't do the replacement mechanically, because this: a reference
pair consisting of a short revision ID and one of these long-form ones
would get turned into two duplicate version stamps.

Avoiding that is going to take wearying handhacking, and I am unhappy
that it is required.  I didn't ask everyone to start using portable revision
stamps months ago because I like hearing myself talk - there have been
real consequences because people didn't, and now I'm going to have to
write more machinery to just to be sure I've *found* all these
goddamned long-form IDs so I can patch them out manually.  With
conversion overhead making my tests take forever I figure I'm looking
at maybe five days of intermittent boring shit-work to cope with this.

This has been the nastiest repository conversion ever.  The data is a
jungle, developer cooperation has been spotty at best, and nobody will
schedule a date for the pain to be *over*.

> > The correct format for a commit reference that will remain
> > intelligible across every VCS, forever, is this:
> > 
> > <RFC3339-date>!<email-address>
> > 
> > Example: 2010-11-13T21:07:58Z!eliz@gnu.org
> 
> How can I produce this automatically given the information I have
> about the revision, i.e. its number and, possibly, revision-id?

bzr log on the revision ID should give you the committer date and email.

> Is it possible to write a command to do this?

I know how to do it using a script wrapper around git log.  The
central bit of magic is this:

git log -1 --pretty='%ci!%ce' $1

The output from that needs trivial editing in the date part.  I don't
know how to do the equivalent in bzr log.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Everyone, please stop making my life more difficult

[Eli Zaretskii]

> Date: Fri, 12 Sep 2014 05:55:43 -0400
> From: "Eric S. Raymond" <esr@thyrsus.com>
> Cc: emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org>:
> > > 1) People are continuing to embed bzr revision numbers like
> > >    r99634.2.576 and r102428 in comments.  Do not do this.  
> > 
> > There's no easy way to refer to an old commit in an unequivocal way.
> 
> Its date and committer is pretty unambiguous.

But less than useful when you need to look up that commit by that
spec.  AFAIK, we don't have VC commands that can look up a commit
given such a spec, so it requires tedious manual work.

> > > 2) "eliz@gnu.org-20101113210758-8ml5kibjtza5ysmb" Um.  What the *hell*?
> > 
> > It's a bzr revision-id.  You can use it in "bzr log" or any other bzr
> > command that accepts a revision spec.
> 
> Well, at least that makes it less arbitrary.  But still soon to become
> meaningless, and I still have to fix it.

You already have other such references in the logs, so I was quite
sure that problem was solved.

These revision-ids are in fixed format, so there should be no need for
heuristics in finding the corresponding commit, nor do I see a need
for a list of conversions when the revision is referenced by such an
ID.  This is why I added them in the first place in this case: to make
your job easier, not harder.

> One I have the location of one of these, it's a simple regex
> transformation to turn it into a <RFC3339-date>!<committer-email>
> portable revision stamp.  The expensive part will be finding them all.

With revision-ids, it's not supposed to be expensive.

> And I can't do the replacement mechanically, because this: a reference
> pair consisting of a short revision ID and one of these long-form ones
> would get turned into two duplicate version stamps.

You can replace the "dotted" revno with nothing in this case.

> Avoiding that is going to take wearying handhacking, and I am unhappy
> that it is required.  I didn't ask everyone to start using portable revision
> stamps months ago because I like hearing myself talk - there have been
> real consequences because people didn't, and now I'm going to have to
> write more machinery to just to be sure I've *found* all these
> goddamned long-form IDs so I can patch them out manually.  With
> conversion overhead making my tests take forever I figure I'm looking
> at maybe five days of intermittent boring shit-work to cope with this.

Why do you run this conversion regularly at all?  Why not do it once
on the D-Day?

> > Is it possible to write a command to do this?
> 
> I know how to do it using a script wrapper around git log.  The
> central bit of magic is this:
> 
> git log -1 --pretty='%ci!%ce' $1
> 
> The output from that needs trivial editing in the date part.

This produces the local time, while we need UTC (a.k.a. "Zulu").  Is
there a way to have a UTC time?

Re: Everyone, please stop making my life more difficult

[Gregor Zattler]

Hi Eli,
* Eli Zaretskii <eliz@gnu.org> [12. Sep. 2014]:
>> I know how to do it using a script wrapper around git log.  The
>> central bit of magic is this:
>> 
>> git log -1 --pretty='%ci!%ce' $1
>> 
>> The output from that needs trivial editing in the date part.
> 
> This produces the local time, while we need UTC (a.k.a. "Zulu").  Is
> there a way to have a UTC time?

How about:

#!/bin/sh
echo $(date -u --rfc-3339=seconds -d "$(git log -1 --pretty='%ci' ${1})"|sed -e 's/ /T/' -e 's/+00:00//')Z$(git log -1 --pretty='!%ce' "${1}")



Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

Re: Everyone, please stop making my life more difficult

[Eric S. Raymond]

Eli Zaretskii <eliz@gnu.org>:
> > Its date and committer is pretty unambiguous.
> 
> But less than useful when you need to look up that commit by that
> spec.  AFAIK, we don't have VC commands that can look up a commit
> given such a spec, so it requires tedious manual work.

Yeah, that's on my to-do list.  Along with a minor mode to chase bzr IDs
found in mailing-list archives.

I could be working on this stuff more if the conversion itself weren't
such a huge time sink.

> These revision-ids are in fixed format, so there should be no need for
> heuristics in finding the corresponding commit, nor do I see a need
> for a list of conversions when the revision is referenced by such an
> ID.  This is why I added them in the first place in this case: to make
> your job easier, not harder.

Not successful.  I requested portable revision stamps for good reasons.

> Why do you run this conversion regularly at all?  Why not do it once
> on the D-Day?

Because (a) if I let the new IDs pile up it that high it will be just
too daunting a task to face, and (b) I'm still trying to make
conversions available on gitorious so people can inspect them, get
used to what the full conversion looks like, and critique it.

> > git log -1 --pretty='%ci!%ce' $1
> > 
> > The output from that needs trivial editing in the date part.
> 
> This produces the local time, while we need UTC (a.k.a. "Zulu").  Is
> there a way to have a UTC time?

Somebody else posted one.  I'll put that in my toolkit and use it as 
the basis for the VC-mode changes.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Everyone, please stop making my life more difficult

[Harald Hanche-Olsen]

[Eli Zaretskii <eliz@gnu.org> (2014-09-12 10:33:48 UTC)]

> This produces the local time, while we need UTC (a.k.a. "Zulu").  Is
> there a way to have a UTC time?

AFAIK, the easiest and most reliable way to do that is to set TZ=UTC
in the environment.

– Harald

Re: Everyone, please stop making my life more difficult

[Andreas Schwab]

"Eric S. Raymond" <esr@thyrsus.com> writes:

> This choice helps avoid collisions.  It is unlikely that one
> committer will ever manage two commits in the same clock second.

That's pretty easy if you are rebasing.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Everyone, please stop making my life more difficult

[Eric S. Raymond]

Andreas Schwab <schwab@linux-m68k.org>:
> "Eric S. Raymond" <esr@thyrsus.com> writes:
> 
> > This choice helps avoid collisions.  It is unlikely that one
> > committer will ever manage two commits in the same clock second.
> 
> That's pretty easy if you are rebasing.

Explain?
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Everyone, please stop making my life more difficult

[David Kastrup]

"Eric S. Raymond" <esr@thyrsus.com> writes:

> Andreas Schwab <schwab@linux-m68k.org>:
>> "Eric S. Raymond" <esr@thyrsus.com> writes:
>> 
>> > This choice helps avoid collisions.  It is unlikely that one
>> > committer will ever manage two commits in the same clock second.
>> 
>> That's pretty easy if you are rebasing.
>
> Explain?

Any repository with some history:

git checkout -b experiment
git rebase --no-ff HEAD~10

commit 0b3e31e53cb267354531cad632dad684cce0f32e
Author:     David Kastrup <dak@gnu.org>
AuthorDate: Fri Sep 12 10:48:20 2014 +0200
Commit:     David Kastrup <dak@gnu.org>
CommitDate: Fri Sep 12 14:15:53 2014 +0200

    Let Lyric_combine_music_iterator use SyllableEvent

commit 1ae99e9abdd8cedd94b64a2760c5a7c2e3a58102
Author:     David Kastrup <dak@gnu.org>
AuthorDate: Thu Sep 11 23:08:25 2014 +0200
Commit:     David Kastrup <dak@gnu.org>
CommitDate: Fri Sep 12 14:15:53 2014 +0200

    Create syllable stream event class

commit 290e4780ed48a88454c35b4620a59484f0db88ec
Author:     David Kastrup <dak@gnu.org>
AuthorDate: Mon Sep 8 22:58:52 2014 +0200
Commit:     David Kastrup <dak@gnu.org>
CommitDate: Fri Sep 12 14:15:53 2014 +0200

    wip

commit 7955b06802bc8f1b2fc6e63529e0c9f6a64f1804
Author:     David Kastrup <dak@gnu.org>
AuthorDate: Thu Sep 11 19:35:49 2014 +0200
Commit:     David Kastrup <dak@gnu.org>
CommitDate: Fri Sep 12 14:15:53 2014 +0200

    slur-proto-engraver: Rename {start,stop}_events_ to slur_{start,stop}_events
    
    Since we are going to borrow code into the Syllable_translator, its
    nicer to make the borrowing more obvious by using the same naming.
    
    And the Syllable_translator has to deal with more events than slurs.

commit b2a2c42568429409ca2bc3f0d0a3cd2ace432590
Author:     David Kastrup <dak@gnu.org>
AuthorDate: Thu Sep 4 20:48:06 2014 +0200
Commit:     David Kastrup <dak@gnu.org>
CommitDate: Fri Sep 12 14:15:53 2014 +0200

    Issue 3855: programming error with \markup \fill-line { \vspace #10 }

commit 79eabf8df2d85d995bc800b289e4e09feffd8fa4
Author:     David Kastrup <dak@gnu.org>
AuthorDate: Thu Sep 4 17:23:51 2014 +0200
Commit:     David Kastrup <dak@gnu.org>
CommitDate: Fri Sep 12 14:15:53 2014 +0200

    Issue 4092: Limit tuplet bracket length to actual tuplet length


Any questions?

-- 
David Kastrup

Re: Everyone, please stop making my life more difficult

[Eli Zaretskii]

> Date: Fri, 12 Sep 2014 12:49:29 +0200
> From: Gregor Zattler <telegraph@gmx.net>
> 
> echo $(date -u --rfc-3339=seconds -d "$(git log -1 --pretty='%ci' ${1})"|sed -e 's/ /T/' -e 's/+00:00//')Z$(git log -1 --pretty='!%ce' "${1}")

I'm pretty shocked that Git doesn't provide a way to show times in
UTC.

Re: Everyone, please stop making my life more difficult

[Eli Zaretskii]

> Date: Fri, 12 Sep 2014 07:38:21 -0400
> From: "Eric S. Raymond" <esr@thyrsus.com>
> Cc: emacs-devel@gnu.org
> 
> > These revision-ids are in fixed format, so there should be no need for
> > heuristics in finding the corresponding commit, nor do I see a need
> > for a list of conversions when the revision is referenced by such an
> > ID.  This is why I added them in the first place in this case: to make
> > your job easier, not harder.
> 
> Not successful.

Our cooperation is not exactly a success story anyway.  At least I
tried.

> I requested portable revision stamps for good reasons.

I don't think you can expect that, until we have VC commands to
produce them and look up commits using them.  I see exactly zero such
references in all of our ChangeLog files as of now, so it's not like
I'm the odd one out.

I'm pretty sure that once we switch to Git, everybody will use Git sha
values instead of bzr revnos.

> > Why do you run this conversion regularly at all?  Why not do it once
> > on the D-Day?
> 
> Because (a) if I let the new IDs pile up it that high it will be just
> too daunting a task to face

I think you are wrong on this one, but it's your call.

> and (b) I'm still trying to make conversions available on gitorious
> so people can inspect them, get used to what the full conversion
> looks like, and critique it.

Did you get any critique lately?

Re: Everyone, please stop making my life more difficult

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Fri, 12 Sep 2014 12:49:29 +0200
>> From: Gregor Zattler <telegraph@gmx.net>
>> 
>> echo $(date -u --rfc-3339=seconds -d "$(git log -1 --pretty='%ci'
>> ${1})"|sed -e 's/ /T/' -e 's/+00:00//')Z$(git log -1 --pretty='!%ce'
>> "${1}")
>
> I'm pretty shocked that Git doesn't provide a way to show times in
> UTC.

TZ=UTC git log --date=local ...

-- 
David Kastrup

Re: Everyone, please stop making my life more difficult

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Date: Fri, 12 Sep 2014 15:19:35 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Date: Fri, 12 Sep 2014 12:49:29 +0200
> >> From: Gregor Zattler <telegraph@gmx.net>
> >> 
> >> echo $(date -u --rfc-3339=seconds -d "$(git log -1 --pretty='%ci'
> >> ${1})"|sed -e 's/ /T/' -e 's/+00:00//')Z$(git log -1 --pretty='!%ce'
> >> "${1}")
> >
> > I'm pretty shocked that Git doesn't provide a way to show times in
> > UTC.
> 
> TZ=UTC git log --date=local ...

Thanks, but that's not exactly Git provided option (and is
non-portable).

Re: Everyone, please stop making my life more difficult

[Andreas Schwab]

"Eric S. Raymond" <esr@thyrsus.com> writes:

> Andreas Schwab <schwab@linux-m68k.org>:
>> "Eric S. Raymond" <esr@thyrsus.com> writes:
>> 
>> > This choice helps avoid collisions.  It is unlikely that one
>> > committer will ever manage two commits in the same clock second.
>> 
>> That's pretty easy if you are rebasing.
>
> Explain?

Rebasing takes only a fraction of a second.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Everyone, please stop making my life more difficult

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: David Kastrup <dak@gnu.org>
>> Date: Fri, 12 Sep 2014 15:19:35 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Date: Fri, 12 Sep 2014 12:49:29 +0200
>> >> From: Gregor Zattler <telegraph@gmx.net>
>> >> 
>> >> echo $(date -u --rfc-3339=seconds -d "$(git log -1 --pretty='%ci'
>> >> ${1})"|sed -e 's/ /T/' -e 's/+00:00//')Z$(git log -1 --pretty='!%ce'
>> >> "${1}")
>> >
>> > I'm pretty shocked that Git doesn't provide a way to show times in
>> > UTC.
>> 
>> TZ=UTC git log --date=local ...
>
> Thanks, but that's not exactly Git provided option (and is
> non-portable).

git show --date=raw

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Everyone, please stop making my life more difficult

[Paul Eggert]

Harald Hanche-Olsen wrote:
> the easiest and most reliable way to do that is to set TZ=UTC
> in the environment.

TZ='UTC' works on systems that have the tz database, but for portability 
to strict-POSIX platforms you'll need something like TZ='UTC0'.

Re: Everyone, please stop making my life more difficult

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: David Kastrup <dak@gnu.org>
>> Date: Fri, 12 Sep 2014 15:19:35 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Date: Fri, 12 Sep 2014 12:49:29 +0200
>> >> From: Gregor Zattler <telegraph@gmx.net>
>> >> 
>> >> echo $(date -u --rfc-3339=seconds -d "$(git log -1 --pretty='%ci'
>> >> ${1})"|sed -e 's/ /T/' -e 's/+00:00//')Z$(git log -1 --pretty='!%ce'
>> >> "${1}")
>> >
>> > I'm pretty shocked that Git doesn't provide a way to show times in
>> > UTC.
>> 
>> TZ=UTC git log --date=local ...
>
> Thanks, but that's not exactly Git provided option (and is
> non-portable).

Of course the format %ct is in UTC.  It's just the raw number.

For formatting times, POSIX provides/prescribes strftime.  All the
formats in strftime are in local time with the exception of %s (the raw
second number in UTC), %z (the numeric timezone) and %Z (the timezone
name).

Git does offer the --date=raw option that will produce the ilk of

commit 85e17384aceddcb761f48f95ee2244a60f9e8408
Author: David Kastrup <dak@gnu.org>
Date:   1410511700 +0200

    Let Lyric_combine_music_iterator use SyllableEvent

I am somewhat surprised that you should state that setting TZ to UTC is
not going to be portable.  Why?

-- 
David Kastrup

Re: Everyone, please stop making my life more difficult

[Sam Steingold]

> * Andreas Schwab <fpujno@yvahk-z68x.bet> [2014-09-12 13:47:44 +0200]:
>
> "Eric S. Raymond" <esr@thyrsus.com> writes:
>
>> This choice helps avoid collisions.  It is unlikely that one
>> committer will ever manage two commits in the same clock second.
>
> That's pretty easy if you are rebasing.

Are you saying that rebasing changes the timestamp?!
I.e., I make a commit X today and tomorrow rebase (= remove X; pull from
upstream; reapply X) it.
Is X now dated today or tomorrow?
If the answer is "tomorrow", this looks like a bug.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1265
http://www.childpsy.net/ http://www.memritv.org http://honestreporting.com
http://americancensorship.org http://iris.org.il http://think-israel.org
If you need a helping hand, just remember that you already have two.

Re: Everyone, please stop making my life more difficult

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: David Kastrup <dak@gnu.org>,  emacs-devel@gnu.org
> Date: Fri, 12 Sep 2014 15:55:17 +0200
> 
> git show --date=raw

With --date=raw, you still need something like 'date' or similar to
produce the YYYY-MM-DDTHH:MM:SS time we need.

Re: Everyone, please stop making my life more difficult

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Fri, 12 Sep 2014 15:57:02 +0200
> 
> Of course the format %ct is in UTC.  It's just the raw number.

Right, so it will need a converter.

> For formatting times, POSIX provides/prescribes strftime.  All the
> formats in strftime are in local time with the exception of %s (the raw
> second number in UTC), %z (the numeric timezone) and %Z (the timezone
> name).
> 
> Git does offer the --date=raw option that will produce the ilk of
> 
> commit 85e17384aceddcb761f48f95ee2244a60f9e8408
> Author: David Kastrup <dak@gnu.org>
> Date:   1410511700 +0200
> 
>     Let Lyric_combine_music_iterator use SyllableEvent

I'm just surprised that something that's trivially available in bzr
(and evidently hg as well, but I'm not sure about the latter) isn't
available with git, that's all.  --date accepts 6 different values (in
addition to 'default'), but not UTC.

> I am somewhat surprised that you should state that setting TZ to UTC is
> not going to be portable.  Why?

Windows runtime doesn't support such a value of TZ.  (msysGit works
around that by using a private timezone database, but setting TZ
inside Emacs to a non-supported value, or doing that in a script that
might invoke programs not belonging to the msysGit package, is playing
with fire.)

Re: Everyone, please stop making my life more difficult

[David Kastrup]

Sam Steingold <sds@gnu.org> writes:

>> * Andreas Schwab <fpujno@yvahk-z68x.bet> [2014-09-12 13:47:44 +0200]:
>>
>> "Eric S. Raymond" <esr@thyrsus.com> writes:
>>
>>> This choice helps avoid collisions.  It is unlikely that one
>>> committer will ever manage two commits in the same clock second.
>>
>> That's pretty easy if you are rebasing.
>
> Are you saying that rebasing changes the timestamp?!

The commit timestamp?  Most certainly.

> I.e., I make a commit X today and tomorrow rebase (= remove X; pull from
> upstream; reapply X) it.
> Is X now dated today or tomorrow?
> If the answer is "tomorrow", this looks like a bug.

If the commit timestamp were anything but "tomorrow" it would be a bug.
The whole point of a commit timestamp is to track who created a commit
when.  Since a rebase creates commits, obviously the commit timestamp
will correspond to the time of the rebase, and the commit author will
correspond to the person doing the rebase.

Which is exactly why using "commit author/timestamp" is not going to cut
it as a unique identifier when rebases come into play.

-- 
David Kastrup

Re: Everyone, please stop making my life more difficult

[Yuri Khan]

On Fri, Sep 12, 2014 at 9:36 PM, David Kastrup <dak@gnu.org> wrote:

> If the commit timestamp were anything but "tomorrow" it would be a bug.
> The whole point of a commit timestamp is to track who created a commit
> when.  Since a rebase creates commits, obviously the commit timestamp
> will correspond to the time of the rebase, and the commit author will
> correspond to the person doing the rebase.
>
> Which is exactly why using "commit author/timestamp" is not going to cut
> it as a unique identifier when rebases come into play.

Neither does author/timestamp uniquely identify a commit in a
rebaseful commit graph. But at least the commits having the same
author/timestamp all conceptually represent the same change.

Re: Everyone, please stop making my life more difficult

[Barry Warsaw]

[-- Attachment #1: Type: text/plain, Size: 1591 bytes --]

On Sep 12, 2014, at 05:55 AM, Eric S. Raymond wrote:

>> > 2) "eliz@gnu.org-20101113210758-8ml5kibjtza5ysmb" Um.  What the *hell*?
>> 
>> It's a bzr revision-id.  You can use it in "bzr log" or any other bzr
>> command that accepts a revision spec.
>
>Well, at least that makes it less arbitrary.  But still soon to become
>meaningless, and I still have to fix it.

Almost every $vcs conversion I've ever been a part of (going back to RCS days)
has always kept the old repository around in read-only mode for long enough
time that people no longer care.  I'm currently involved in some svn->git
transitions and we're debating about history and revision reference
preservation[*].  I think the general consensus for us is to keep the old svn
repos around read-only so that *if* such history is required (and it rarely
is, at least IME), the user could go to the svn repo to look something up.
Yes, it's less convenient, but preserving this is also a lot of work for a
rare use case.

I've also heard that git has a way to refer to "external" history, i.e. from
git into svn, so that from git's point of view, you can traverse into logs for
svn history.  But I could be wrong and no nothing more about this than
hearsay.

Maybe Emacs's conversion has different requirements.  I certainly don't mean
to derail the current conversion efforts, just relating previous experience.

Cheers,
-Barry

[*] I may soon be looking at some bzr->git transitions, and I'm wondering if
Eric's code would be generally useful, or is to focused on the vagaries of
Emacs's repo.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Everyone, please stop making my life more difficult

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: David Kastrup <dak@gnu.org>,  emacs-devel@gnu.org
> Date: Fri, 12 Sep 2014 15:55:17 +0200
> 
> git show --date=raw

This shows the author's timestamp, not the committer's one, right?

Re: Everyone, please stop making my life more difficult

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andreas Schwab <schwab@linux-m68k.org>
>> Cc: David Kastrup <dak@gnu.org>,  emacs-devel@gnu.org
>> Date: Fri, 12 Sep 2014 15:55:17 +0200
>> 
>> git show --date=raw
>
> This shows the author's timestamp, not the committer's one, right?

It accepts the same options.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Everyone, please stop making my life more difficult

[Sam Steingold]

> * David Kastrup <qnx@tah.bet> [2014-09-12 16:36:59 +0200]:
>
> Sam Steingold <sds@gnu.org> writes:
>
>>> * Andreas Schwab <fpujno@yvahk-z68x.bet> [2014-09-12 13:47:44 +0200]:
>>>
>>> "Eric S. Raymond" <esr@thyrsus.com> writes:
>>>
>>>> This choice helps avoid collisions.  It is unlikely that one
>>>> committer will ever manage two commits in the same clock second.
>>>
>>> That's pretty easy if you are rebasing.
>>
>> Are you saying that rebasing changes the timestamp?!
>
> The commit timestamp?  Most certainly.
>
>> I.e., I make a commit X today and tomorrow rebase (= remove X; pull from
>> upstream; reapply X) it.
>> Is X now dated today or tomorrow?
>> If the answer is "tomorrow", this looks like a bug.
>
> If the commit timestamp were anything but "tomorrow" it would be a
> bug.  The whole point of a commit timestamp is to track who created a
> commit when.  Since a rebase creates commits, obviously the commit
> timestamp will correspond to the time of the rebase, and the commit
> author will correspond to the person doing the rebase.

I thought that rebase _moves_ commits, not _creates_ them anew.
Just like "mv" does not change ctime of a file,
rebase must not change the commit time.


-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1265
http://www.childpsy.net/ http://think-israel.org http://openvotingconsortium.org
http://mideasttruth.com http://memri.org http://ffii.org
Lisp: Serious empowerment.

Re: Everyone, please stop making my life more difficult

[Sam Steingold]

> * Yuri Khan <lhev.i.xuna@tznvy.pbz> [2014-09-12 21:44:24 +0700]:
>
> On Fri, Sep 12, 2014 at 9:36 PM, David Kastrup <dak@gnu.org> wrote:
>
>> If the commit timestamp were anything but "tomorrow" it would be a bug.
>> The whole point of a commit timestamp is to track who created a commit
>> when.  Since a rebase creates commits, obviously the commit timestamp
>> will correspond to the time of the rebase, and the commit author will
>> correspond to the person doing the rebase.
>>
>> Which is exactly why using "commit author/timestamp" is not going to cut
>> it as a unique identifier when rebases come into play.
>
> Neither does author/timestamp uniquely identify a commit in a
> rebaseful commit graph. But at least the commits having the same
> author/timestamp all conceptually represent the same change.

Not at all - if we accept the notion that rebase changes timestamp.
E.g., if I work for a week making 20 separate unrelated commits on
different days and then rebase, then all the 20 commits will have the
same timestamp.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1265
http://www.childpsy.net/ http://mideasttruth.com http://islamexposedonline.com
http://think-israel.org http://www.dhimmitude.org http://memri.org
You won't get smarter by calling me a fool.

Re: Everyone, please stop making my life more difficult

[Phillip Lord]

Sam Steingold <sds@gnu.org> writes:

>> * Andreas Schwab <fpujno@yvahk-z68x.bet> [2014-09-12 13:47:44 +0200]:
>> That's pretty easy if you are rebasing.
>
> Are you saying that rebasing changes the timestamp?!
> I.e., I make a commit X today and tomorrow rebase (= remove X; pull from
> upstream; reapply X) it.
> Is X now dated today or tomorrow?
> If the answer is "tomorrow", this looks like a bug.


Rebasing changes history, which includes timestamps. It's why it's such
a dangerous command, and some do argue strongly that its a bug.

Pity it's so useful really.

Phil

Re: Everyone, please stop making my life more difficult

[Harald Hanche-Olsen]

[Sam Steingold <sds@gnu.org> (2014-09-12 15:21:21 UTC)]

> I thought that rebase _moves_ commits, not _creates_ them anew.
> Just like "mv" does not change ctime of a file,
> rebase must not change the commit time.

Rebase needs to create new commits for two reasons:

1. Each commit contains a reference to its parent(s), and hence
   indirectly to its entire prehistory, and
2. When you rebase, you are essentially applying diffs corresponding
   to the commits you are “moving” to other commits, so the rebased
   commit will refer to files that are different from the ones before
   the rebase.

– Harald

Re: Everyone, please stop making my life more difficult

[David Kastrup]

Sam Steingold <sds@gnu.org> writes:

>> * David Kastrup <qnx@tah.bet> [2014-09-12 16:36:59 +0200]:
>>
>> Sam Steingold <sds@gnu.org> writes:
>>
>>> I.e., I make a commit X today and tomorrow rebase (= remove X; pull from
>>> upstream; reapply X) it.
>>> Is X now dated today or tomorrow?
>>> If the answer is "tomorrow", this looks like a bug.
>>
>> If the commit timestamp were anything but "tomorrow" it would be a
>> bug.  The whole point of a commit timestamp is to track who created a
>> commit when.  Since a rebase creates commits, obviously the commit
>> timestamp will correspond to the time of the rebase, and the commit
>> author will correspond to the person doing the rebase.

Replace "commit author" with "committer" here since using "author" is
confusing in this context:

commit 85e17384aceddcb761f48f95ee2244a60f9e8408
Author:     David Kastrup <dak@gnu.org>
AuthorDate: Fri Sep 12 10:48:20 2014 +0200
Commit:     David Kastrup <dak@gnu.org>
CommitDate: Fri Sep 12 14:18:26 2014 +0200

    Let Lyric_combine_music_iterator use SyllableEvent

> I thought that rebase _moves_ commits, not _creates_ them anew.
> Just like "mv" does not change ctime of a file,
> rebase must not change the commit time.

Then you lack a fundamental understanding of what constitutes a "commit"
in Git's terms.  A commit is identified by its commit hash, and what is
hashed is the entire commit metadata including the ancestors of a
commit, its own commit date and committer and the corresponding tree.

Rebasing creates new commits with a new identity and commit hash,
retaining a substantial amount of headers _not_ including committer and
commit date.  And obviously also not generally including the same
ancestry or the rebase would be pretty pointless.

The committer is the person responsible for a commit being in a
repository (mirroring, and non-rebasing push or pull are just
duplications and neither change commit ids not commit metadata).  The
author is the one who is more or less considered responsible for
creating its content.

-- 
David Kastrup

Re: Everyone, please stop making my life more difficult

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andreas Schwab <schwab@linux-m68k.org>
>> Cc: David Kastrup <dak@gnu.org>,  emacs-devel@gnu.org
>> Date: Fri, 12 Sep 2014 15:55:17 +0200
>> 
>> git show --date=raw
>
> This shows the author's timestamp, not the committer's one, right?

It shows what you tell it to show elsewhere.  For example, with

git show --pretty=fuller --date=raw

it shows both.  With --date=raw --pretty='format:...' it shows the
fields specified in the format.

-- 
David Kastrup

Re: Everyone, please stop making my life more difficult

[Eric S. Raymond]

Andreas Schwab <schwab@linux-m68k.org>:
> >> That's pretty easy if you are rebasing.
> >
> > Explain?
> 
> Rebasing takes only a fraction of a second.

And actually changes the timestamps, which I would not have expected -
as someone else observed, I would have expected that to move the commits
wuthout changing the metadata. 

That is unfortunate.  I'd mumble that someone could have pointed this
out six months ago, but it wouldn't have helped because there's no
better alternative. Looking across VCS metadata, the only things you
can always get are committer name and committer data. Sometimes author
name and author are available, but it is not even guaranteed that the
distinction is portable.  It wasn't in darcs last I checked.

I will document this as a weakness of the technique that we're stuck
with.  In order for it to create a problem, commits on a branch would
have to be rebased after someone has already referred to a commit on
the branch.  It is pretty unlikely that this would happen in a public
repository, but not impssible.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Everyone, please stop making my life more difficult

[David Kastrup]

phillip.lord@newcastle.ac.uk (Phillip Lord) writes:

> Sam Steingold <sds@gnu.org> writes:
>
>>> * Andreas Schwab <fpujno@yvahk-z68x.bet> [2014-09-12 13:47:44 +0200]:
>>> That's pretty easy if you are rebasing.
>>
>> Are you saying that rebasing changes the timestamp?!
>> I.e., I make a commit X today and tomorrow rebase (= remove X; pull from
>> upstream; reapply X) it.
>> Is X now dated today or tomorrow?
>> If the answer is "tomorrow", this looks like a bug.
>
>
> Rebasing changes history, which includes timestamps. It's why it's such
> a dangerous command, and some do argue strongly that its a bug.

And some argue strongly that spiders are bugs, but if you are planning
on getting anywhere in a discussion, they tend to contribute rather
little of use.

-- 
David Kastrup

Re: Everyone, please stop making my life more difficult

[David Kastrup]

Sam Steingold <sds@gnu.org> writes:

>> * Yuri Khan <lhev.i.xuna@tznvy.pbz> [2014-09-12 21:44:24 +0700]:
>>
>> On Fri, Sep 12, 2014 at 9:36 PM, David Kastrup <dak@gnu.org> wrote:
>>
>>> If the commit timestamp were anything but "tomorrow" it would be a bug.
>>> The whole point of a commit timestamp is to track who created a commit
>>> when.  Since a rebase creates commits, obviously the commit timestamp
>>> will correspond to the time of the rebase, and the commit author will
>>> correspond to the person doing the rebase.
>>>
>>> Which is exactly why using "commit author/timestamp" is not going to cut
>>> it as a unique identifier when rebases come into play.
>>
>> Neither does author/timestamp uniquely identify a commit in a
>> rebaseful commit graph. But at least the commits having the same
>> author/timestamp all conceptually represent the same change.
>
> Not at all - if we accept the notion that rebase changes timestamp.
> E.g., if I work for a week making 20 separate unrelated commits on
> different days and then rebase, then all the 20 commits will have the
> same timestamp.

Git has different timestamps for author and committer.  Rebasing does
not change author timestamp and identity.

-- 
David Kastrup

Re: Everyone, please stop making my life more difficult

[David Kastrup]

"Eric S. Raymond" <esr@thyrsus.com> writes:

> Andreas Schwab <schwab@linux-m68k.org>:
>> >> That's pretty easy if you are rebasing.
>> >
>> > Explain?
>> 
>> Rebasing takes only a fraction of a second.
>
> And actually changes the timestamps, which I would not have expected -
> as someone else observed, I would have expected that to move the commits
> wuthout changing the metadata. 
>
> That is unfortunate.  I'd mumble that someone could have pointed this
> out six months ago, but it wouldn't have helped because there's no
> better alternative. Looking across VCS metadata, the only things you
> can always get are committer name and committer data. Sometimes author
> name and author are available, but it is not even guaranteed that the
> distinction is portable.  It wasn't in darcs last I checked.
>
> I will document this as a weakness of the technique that we're stuck
> with.  In order for it to create a problem, commits on a branch would
> have to be rebased after someone has already referred to a commit on
> the branch.  It is pretty unlikely that this would happen in a public
> repository, but not impssible.

In order for it to create a problem, commits on a branch would have to
be rebased.  Because at that point of time, commit author+commit date no
longer uniquely identify a commit.

Rebasing again loses the information you use for identifying a commit.
This may or may be another problem.  But the problem of
commitdate+committer not providing uniqie identification for any commit
in a rebased branch is a separate one.

-- 
David Kastrup

Re: Everyone, please stop making my life more difficult

[Eric S. Raymond]

Yuri Khan <yuri.v.khan@gmail.com>:
> Neither does author/timestamp uniquely identify a commit in a
> rebaseful commit graph. But at least the commits having the same
> author/timestamp all conceptually represent the same change.

That is an interesting point.

We are still at a point where I could, theoretically, change this.
I could run the through the revision-ID mapping replacing committer-based
version stamps with author-based version stamps.

I am not sure this would be a good idea, though. It wouldn't actually
eliminate the multiple-target problem in searches using revision stamps 
as keys; the best it would do is support an argument that the ambiguity
might not matter as much.  Maybe - usually when a branch is rebased 
on to trunk it's because someone is treating *that* as a conceptual 
unit too, representing a feature landing.  

And we really aren't guaranteed that future VCSes will have the
distinction; darcs, or something else like it based on a patch
algebra, could still win.

For now, I ask everyone to generate committer-based version stamps.  If
we switch, I'll do it in one big operation.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Everyone, please stop making my life more difficult

[Eric S. Raymond]

[-- Attachment #1: Type: text/plain, Size: 481 bytes --]

Barry Warsaw <barry@python.org>:
> [*] I may soon be looking at some bzr->git transitions, and I'm wondering if
> Eric's code would be generally useful, or is to focused on the vagaries of
> Emacs's repo.

I think you will find that reposurgeon is very useful.  But the scripts I
built for the Emacs conversion are very specific to the weird situation
here - five layers of VCSes in the history and so forth.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 173 bytes --]

Re: Everyone, please stop making my life more difficult

[Andreas Schwab]

"Eric S. Raymond" <esr@thyrsus.com> writes:

> And actually changes the timestamps, which I would not have expected -
> as someone else observed, I would have expected that to move the commits
> wuthout changing the metadata. 

This is impossible, since the parents are part of the metadata.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Everyone, please stop making my life more difficult

[Eric S. Raymond]

Eli Zaretskii <eliz@gnu.org>:
> > I requested portable revision stamps for good reasons.
> 
> I don't think you can expect that, until we have VC commands to
> produce them and look up commits using them.

Which is why that's on my to-do list.  A good alternative is to 
refer to a commit by a quote from its summary line; in fact this
is usually best for a human reader.

> I'm pretty sure that once we switch to Git, everybody will use Git sha
> values instead of bzr revnos.

A request for better behavior is already in the developer guidelines.
You could be helpful by setting a positive precedent.

> > and (b) I'm still trying to make conversions available on gitorious
> > so people can inspect them, get used to what the full conversion
> > looks like, and critique it.
> 
> Did you get any critique lately?

No.  But whether I do or not, I feel a responsibility to the developer
community to make that possible.  The right thing must be done and
seen to be done.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Everyone, please stop making my life more difficult

[Eric S. Raymond]

Andreas Schwab <schwab@linux-m68k.org>:
> "Eric S. Raymond" <esr@thyrsus.com> writes:
> 
> > And actually changes the timestamps, which I would not have expected -
> > as someone else observed, I would have expected that to move the commits
> > wuthout changing the metadata. 
> 
> This is impossible, since the parents are part of the metadata.

I didn't expect ito leave hashes unaltered, just the visible headers.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Everyone, please stop making my life more difficult

[David Kastrup]

"Eric S. Raymond" <esr@thyrsus.com> writes:

> Andreas Schwab <schwab@linux-m68k.org>:
>> "Eric S. Raymond" <esr@thyrsus.com> writes:
>> 
>> > And actually changes the timestamps, which I would not have expected -
>> > as someone else observed, I would have expected that to move the commits
>> > wuthout changing the metadata. 
>> 
>> This is impossible, since the parents are part of the metadata.
>
> I didn't expect ito leave hashes unaltered, just the visible headers.

A plain

    git log

will not list the committer and commit date.  So a rebase _does_ leave
the visible headers unaltered.  Pretty much the whole point for
distinguishing author and committer and the respective dates is to
distinguish the origin of a change and the person responsible for its
state in the repository.

-- 
David Kastrup

Re: Everyone, please stop making my life more difficult

[Stefan Monnier]

I can't say you guys send so much time on an issue that seems
so irrelevant.

If you want a precise and reliable reference, then use the
human-incompatible revid.  In text such as ChangeLog and commit messages
(i.e. for human consumption), please use something human can understand,
typically author + date.  The fact that it can be ambiguous is not
a serious concern, AFAIK, and human should/will be able to figure out
what was meant.


        Stefan

Re: Everyone, please stop making my life more difficult

[Stefan Monnier]

>>> * Andreas Schwab <fpujno@yvahk-z68x.bet> [2014-09-12 13:47:44 +0200]:
>>> That's pretty easy if you are rebasing.
>> Are you saying that rebasing changes the timestamp?!
>> I.e., I make a commit X today and tomorrow rebase (= remove X; pull from
>> upstream; reapply X) it.
>> Is X now dated today or tomorrow?
>> If the answer is "tomorrow", this looks like a bug.
> Rebasing changes history, which includes timestamps.

If the answer is "tomorrow" than history was *not* changed (the old
commits still exist, they're just not referenced any more).  It's if the
answer is "today" that the history would be changed (by creating commit
"tomorrow" which pretend that they already existed "today").


        Stefan

Re: Everyone, please stop making my life more difficult

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: dak@gnu.org,  emacs-devel@gnu.org
> Date: Fri, 12 Sep 2014 17:21:09 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Andreas Schwab <schwab@linux-m68k.org>
> >> Cc: David Kastrup <dak@gnu.org>,  emacs-devel@gnu.org
> >> Date: Fri, 12 Sep 2014 15:55:17 +0200
> >> 
> >> git show --date=raw
> >
> > This shows the author's timestamp, not the committer's one, right?
> 
> It accepts the same options.

But the documentation says that only %ad is affected by --date, and in
my testing indeed %ci is not affected.

Re: Everyone, please stop making my life more difficult

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Date: Fri, 12 Sep 2014 17:53:34 +0200
> 
> >> git show --date=raw
> >
> > This shows the author's timestamp, not the committer's one, right?
> 
> It shows what you tell it to show elsewhere.  For example, with
> 
> git show --pretty=fuller --date=raw
> 
> it shows both.  With --date=raw --pretty='format:...' it shows the
> fields specified in the format.

When I use --format='%ci', it doesn't seem to be affected by
--date=raw, which is consistent with the documentation as I understand
it.

Re: Everyone, please stop making my life more difficult

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andreas Schwab <schwab@linux-m68k.org>
>> Cc: dak@gnu.org,  emacs-devel@gnu.org
>> Date: Fri, 12 Sep 2014 17:21:09 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> From: Andreas Schwab <schwab@linux-m68k.org>
>> >> Cc: David Kastrup <dak@gnu.org>,  emacs-devel@gnu.org
>> >> Date: Fri, 12 Sep 2014 15:55:17 +0200
>> >> 
>> >> git show --date=raw
>> >
>> > This shows the author's timestamp, not the committer's one, right?
>> 
>> It accepts the same options.
>
> But the documentation says that only %ad is affected by --date, and in
> my testing indeed %ci is not affected.

Of course it isn't.  But %cd is.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Everyone, please stop making my life more difficult

[Stephen J. Turnbull]

Sam Steingold writes:

 > I thought that rebase _moves_ commits, not _creates_ them anew.

It's easy to that this must be so.  The data of a commit, in all of
the DAG-based VCSes, not just git, contains the parent of that commit.
Rebase by definition changes the parent of at least one commit, and
therefore must create new a commit to store it.  I'm sure you can
handle the recursive proof for the descendants of that commit.

OTOH, rebase merely moves *trees*; those (and the blobs that they
point to in turn) are not rewritten.

Re: Everyone, please stop making my life more difficult

[David Caldwell]

[-- Attachment #1: Type: text/plain, Size: 466 bytes --]

On 9/12/14 10:44 AM, Stephen J. Turnbull wrote:
> 
> OTOH, rebase merely moves *trees*; those (and the blobs that they
> point to in turn) are not rewritten.

No, those are rewritten, too. Rebase just cherry-picks a bunch of
commits, and cherry-picking can certainly change blobs. The first
sentence of the cherry-pick man page:

> Given one or more existing commits, apply the change each one
> introduces, recording a new commit for each.

-David



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4219 bytes --]

Re: Everyone, please stop making my life more difficult

[Harald Hanche-Olsen]

["Stephen J. Turnbull" <stephen@xemacs.org> (2014-09-12 17:44:06 UTC)]

> OTOH, rebase merely moves *trees*; those (and the blobs that they
> point to in turn) are not rewritten.

Not true, other than in the sense that trees and blobs in the repo are
never rewritten, because they are named by the their own hash.

But the entire point of a rebase is that you incorporate changes that
were made to the main branch since the histories first diverged, in
order to create (the illusion of) a linear history; so the new commits
must contain references to changed files.

– Harald

Re: Everyone, please stop making my life more difficult

[Stephen J. Turnbull]

David Caldwell writes:

 > > Given one or more existing commits, apply the change each one
 > > introduces, recording a new commit for each.

Ah, right.  I was thinking of filter-branch in its
prune-commits-and-rejoin-branches applications.

Re: Everyone, please stop making my life more difficult

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: David Kastrup <dak@gnu.org>
>> Date: Fri, 12 Sep 2014 17:53:34 +0200
>> 
>> >> git show --date=raw
>> >
>> > This shows the author's timestamp, not the committer's one, right?
>> 
>> It shows what you tell it to show elsewhere.  For example, with
>> 
>> git show --pretty=fuller --date=raw
>> 
>> it shows both.  With --date=raw --pretty='format:...' it shows the
>> fields specified in the format.
>
> When I use --format='%ci', it doesn't seem to be affected by
> --date=raw, which is consistent with the documentation as I understand
> it.

So why would you use %ci then?

           ·   %cd: committer date

           ·   %cD: committer date, RFC2822 style

           ·   %cr: committer date, relative

           ·   %ct: committer date, UNIX timestamp

           ·   %ci: committer date, ISO 8601 format

It seems obvious that all commit date specifiers apart from %cd use a
fixed format.  And %ct will output the timestamp format.

-- 
David Kastrup

Re: Everyone, please stop making my life more difficult

[Joshua Judson Rosen]

Sam Steingold <sds@gnu.org> writes:
>
> I thought that rebase _moves_ commits, not _creates_ them anew.

I have found it helpful to think of it like this, so maybe you will too:

    * A commit in git or bzr is a node in a linked list; the node's cdr
      is the commit's parent-commit, so cdr'ing through the list takes
      you back through history toward the branch origin.

    * You can't "just move" a node in a linked list without changing any
      of its content.

    * The reasons that you'd want to create a new node when "moving" a
      commit, rather than just modifying it in place, are largely the
      same as the reasons that you might want to re-cons nodes onto the
      head of a list rather than just setcdr'ing the existing cons cells
      in place.


> Just like "mv" does not change ctime of a file,
> rebase must not change the commit time.

If you mv a subdirectory to a different parent directory,
the subdirectory does need to be written to have its
".." link updated, which should update its ctime....

(and even regular files actually may have their ctime updated,
 for reasons that I can't seem to find a way of fitting into
 the VC analogy)

-- 
"'tis an ill wind that blows no minds."

Re: Everyone, please stop making my life more difficult

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Fri, 12 Sep 2014 21:28:44 +0200
> 
>            ·   %cd: committer date
> 
>            ·   %cD: committer date, RFC2822 style
> 
>            ·   %cr: committer date, relative
> 
>            ·   %ct: committer date, UNIX timestamp
> 
>            ·   %ci: committer date, ISO 8601 format
> 
> It seems obvious that all commit date specifiers apart from %cd use a
> fixed format.  And %ct will output the timestamp format.

I was confused by the (apparently wrong) documentation which claims
that only %ad is affected by --date.

resolving ambiguity in action stamps (was: Everyone, please stop making my life more difficult)

[Joshua Judson Rosen]

"Eric S. Raymond" <esr@thyrsus.com> writes:
>
> Andreas Schwab <schwab@linux-m68k.org>:
> > >> That's pretty easy if you are rebasing.
> > >
> > > Explain?
> > 
> > Rebasing takes only a fraction of a second.
>
> And actually changes the timestamps, which I would not have expected -
> as someone else observed, I would have expected that to move the commits
> wuthout changing the metadata. 
>
> That is unfortunate.  I'd mumble that someone could have pointed this
> out six months ago, but it wouldn't have helped because there's no
> better alternative.
>
> Looking across VCS metadata, the only things you can always get are
> committer name and committer data. Sometimes author name and author
> are available, but it is not even guaranteed that the distinction is
> portable.  It wasn't in darcs last I checked.

Perhaps I should have pointed this out six months ago, but bzr presents
the same issue as git does re: commit timestamps vs. authorship timestamps.

The authorship timestamp is what's displayed by "bzr log" in the
"timestamp:" field; the commit timestamp is the decimal string in the
middle of the revision-id.

I gather that what you've been using thus far in the conversion from bzr
is the *authorship* timestamps, not the commit-timestamps.

Sorry--I mostly just wasn't paying close enough attention six months ago
to point out all of the angles of this issue...; though I *did* raise
this "ambiguity of reference by timestamp" issue at that time--and thought
that you'd given an ACK with a statement you were going to fix it by using
"#{sequence-number}" suffixes to distingiush between multiple distinct
items in the same "{ISO-date}!{email-address}" hashbucket:

    https://lists.gnu.org/archive/html/emacs-devel/2014-01/msg01117.html

    https://lists.gnu.org/archive/html/emacs-devel/2014-01/msg01113.html

(that first link is your response; the second is my message--which
 included an example of why the "same timestamps, different commit-
 objects" situation might exist in a repository)

-- 
"'tis an ill wind that blows no minds."

Re: Everyone, please stop making my life more difficult

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 1339 bytes --]

() Stefan Monnier <monnier@iro.umontreal.ca>
() Fri, 12 Sep 2014 13:06:19 -0400

   In text such as ChangeLog and commit messages (i.e. for
   human consumption), please use something human can
   understand, typically author + date.  The fact that it
   can be ambiguous is not a serious concern, AFAIK, and
   human should/will be able to figure out what was meant.

I am a human, and find using YYYY-MM-DD, "TITLE", e.g.:

 2014-06-20, "Diagnose failures due to colons in directory
 names (Bug#17278)"

to be relatively easy to manage.  (The comma and quotes
make writing such refs in paragraphs more "natural".)

I was initially offended by lack of WHO, but no longer.
There is no real win to pointing the finger at a single
person, when often the mistake is a shared misconception.

The date is the commit date, as that is what is recorded
in the ChangeLog files, and the primary venue for such
references.

I made this suggestion before during previous gyrations,
however i didn't mention my humanity, which i suppose is
the only excuse for this noise.  Enough!  Over and out!

-- 
Thien-Thi Nguyen
   GPG key: 4C807502
   (if you're human and you know it)
      read my lisp: (responsep (questions 'technical)
                               (not (via 'mailing-list)))
                     => nil

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: resolving ambiguity in action stamps (was: Everyone, please stop making my life more difficult)

[Eric S. Raymond]

Joshua Judson Rosen <rozzin@geekspace.com>:
> Sorry--I mostly just wasn't paying close enough attention six months ago
> to point out all of the angles of this issue...; though I *did* raise
> this "ambiguity of reference by timestamp" issue at that time--and thought
> that you'd given an ACK with a statement you were going to fix it by using
> "#{sequence-number}" suffixes to distingiush between multiple distinct
> items in the same "{ISO-date}!{email-address}" hashbucket:
> 
>     https://lists.gnu.org/archive/html/emacs-devel/2014-01/msg01117.html
> 
>     https://lists.gnu.org/archive/html/emacs-devel/2014-01/msg01113.html
> 
> (that first link is your response; the second is my message--which
>  included an example of why the "same timestamps, different commit-
>  objects" situation might exist in a repository)

I remeber that.  I thought I was going to fix it that way too.

Then I found a showstopper. Possibly it has a solution I haven't found yet.

The problem is this: suppose you have such a suffix, and several nodes
with the same date on a DAG.  What's your rule for mapping suffix 
to individual node?

I couldn't invent one.  I believe this is mathematically equivalent to 
total-ordering the DAG. Which you can't do.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: resolving ambiguity in action stamps

[Joshua Judson Rosen]

"Eric S. Raymond" <esr@thyrsus.com> writes:
>
> Joshua Judson Rosen <rozzin@geekspace.com>:
> > Sorry--I mostly just wasn't paying close enough attention six months ago
> > to point out all of the angles of this issue...; though I *did* raise
> > this "ambiguity of reference by timestamp" issue at that time--and thought
> > that you'd given an ACK with a statement you were going to fix it by using
> > "#{sequence-number}" suffixes to distingiush between multiple distinct
> > items in the same "{ISO-date}!{email-address}" hashbucket:
> > 
> >     https://lists.gnu.org/archive/html/emacs-devel/2014-01/msg01117.html
> > 
> >     https://lists.gnu.org/archive/html/emacs-devel/2014-01/msg01113.html
> > 
> > (that first link is your response; the second is my message--which
> >  included an example of why the "same timestamps, different commit-
> >  objects" situation might exist in a repository)
>
> I remeber that.  I thought I was going to fix it that way too.
>
> Then I found a showstopper. Possibly it has a solution I haven't found yet.
>
> The problem is this: suppose you have such a suffix, and several nodes
> with the same date on a DAG.  What's your rule for mapping suffix 
> to individual node?

Hrm. I'd figured you were just going to assign sequence-numbers
according to the order in which you encountered the commits in the
git-fast-import stream, or reverse of the order in the git-fast-import
stream, or something like that. It wasn't obvious to me how that would
actually be usable in the _general_ case, but I guess I assumed you'd
found it to be sufficient in practice, and/or that you had something up
your sleeve that you hadn't explicitly mentioned (and wasn't really
keenly intent on pressing the issue at the time).

> I couldn't invent one.  I believe this is mathematically equivalent to 
> total-ordering the DAG. Which you can't do.

In the specific case that Andreas put forward (*commit* times in a
single series incrementing more rapidly than their timestamp-precision
could account for), I'd sort-of expect that to be workable with pretty
good usability--the since commits along a single branch (as in a rebase)
would seem to already have intrinsic sequencing with respect to each other.

What I put forward in my example was the sort of case where I was
skeptical of `*authorship* timestamp_bucket + seqno' being usable, but I
also noted the possibility of that situation not existing in the emacs
repository (if you don't actually encounter the general case, does it
matter whether your algorithm handles it?). Have you actually found that
sort of collision in the emacs DAG?

Actually, I guess the nature (and relevance) of that last question
changes if you're using *commit* timestamps rather and not *authorship*
timestamps--like, maybe you only have to worry about the "rebase"
situation that Andreas described, and _that_ can handled trivially?

-- 
"'tis an ill wind that blows no minds."

Re: resolving ambiguity in action stamps (was: Everyone, please stop making my life more difficult)

[Stephen J. Turnbull]

Eric S. Raymond writes:

 > I couldn't invent one.  I believe this is mathematically equivalent
 > to total-ordering the DAG. Which you can't do.

You must have other conditions in mind for your allowable total
orders, since any finite partial order can be extended to a total
order (topological sort), usually in several ways.  Not even the Emacs
repo has an uncountable number of revisions (although working with bzr
sometimes makes me feel like it does).

Re: resolving ambiguity in action stamps (was: Everyone, please stop making my life more difficult)

[Eric S. Raymond]

Stephen J. Turnbull <stephen@xemacs.org>:
> Eric S. Raymond writes:
> 
>  > I couldn't invent one.  I believe this is mathematically equivalent
>  > to total-ordering the DAG. Which you can't do.
> 
> You must have other conditions in mind for your allowable total
> orders, since any finite partial order can be extended to a total
> order (topological sort), usually in several ways.  Not even the Emacs
> repo has an uncountable number of revisions (although working with bzr
> sometimes makes me feel like it does).

Sorry, you're right. I should have said equivalent to the existence of a 
*unique* total ordering.  The multiplicity of possible extended orders is the
problem, not the solution.

Look at it this way.  For a suffix-numbering scheme to be useful, a
browsing tool or anything else chasing references needs to converge on
*the same ordering that you generated* using only the DAG topology -
because it's not going to know, in particular, what order you
encountered the commits in.

/me puts on his "I used to be a mathematician..." hat...

The problem admits of a unique solution if every clique of commits
has the property that the minimal subgraph required to connect it
is totally ordered.  Think of the dates as node colors.  Then, in
this graph:

A <-- B <-- C <-- B <-- D

we can number uniquely as 

A <-- B1 <-- C <-- B2 <-- D

based on the parent-of ordering.  Then there's this case:

A <-- B1 <-- C <-- B2 <-- D
              \
               +--- E <-- F

We can still uniquely order B1 and B2 because the minimal subgraph that
joins them is totally ordered.  The problem case is this:

A <-- B <-- C <-- B <-- D
              \
               +--- B <-- E <- F

In this the minimal subgraph joining the clique is not totally ordered,
so we don't know how to uniquely order the Bs on the branches.

To solve this problem we need a unique order relation on the branch heads.
If branches had immutable names that would imply one, but we can't assume that.

Possibly there's a solution here using a constraint particular to VCSes that
I haven't taken into account, and someone cleverer than me will notice.
That would be nice.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: resolving ambiguity in action stamps (was: Everyone, please stop making my life more difficult)

[Stephen J. Turnbull]

Eric S. Raymond writes:

 > The problem case is this:
 > 
 > A <-- B <-- C <-- B <-- D
 >               \
 >                +--- B <-- E <- F
 > 
 > In this the minimal subgraph joining the clique is not totally ordered,
 > so we don't know how to uniquely order the Bs on the branches.
 > 
 > To solve this problem we need a unique order relation on the branch heads.

The branch heads *have* canonical names: the SHA1 of the commit.

Re: resolving ambiguity in action stamps

[Harald Hanche-Olsen]

["Stephen J. Turnbull" <stephen@xemacs.org> (2014-09-13 06:57:15 UTC)]

> The branch heads *have* canonical names: the SHA1 of the commit.

Eek. Isn't part of the point to not rely on internals of the version
control system? Even if the sources stay on git from now to eternity,
git itself might abandon SHA1 at one point. It is, after all, not
cryptographically secure. Also, branch heads move to new commits all
the time. And branches merge. Not at all sure I understand what you
are proposing.

– Harald

Re: resolving ambiguity in action stamps

[Andreas Schwab]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Eric S. Raymond writes:
>
>  > The problem case is this:
>  > 
>  > A <-- B <-- C <-- B <-- D
>  >               \
>  >                +--- B <-- E <- F
>  > 
>  > In this the minimal subgraph joining the clique is not totally ordered,
>  > so we don't know how to uniquely order the Bs on the branches.
>  > 
>  > To solve this problem we need a unique order relation on the branch heads.
>
> The branch heads *have* canonical names: the SHA1 of the commit.

The hash of the head doesn't define a unique order on the branches.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: resolving ambiguity in action stamps

[Eric S. Raymond]

Harald Hanche-Olsen <hanche@math.ntnu.no>:
> ["Stephen J. Turnbull" <stephen@xemacs.org> (2014-09-13 06:57:15 UTC)]
> 
> > The branch heads *have* canonical names: the SHA1 of the commit.
> 
> Eek. Isn't part of the point to not rely on internals of the version
> control system? Even if the sources stay on git from now to eternity,
> git itself might abandon SHA1 at one point. It is, after all, not
> cryptographically secure. Also, branch heads move to new commits all
> the time. And branches merge. Not at all sure I understand what you
> are proposing.

Both of Harald's objections are, alas, sound.  In particular, the 
branch total ordering is not useful if any commit can arbitrarily
permute it.

The hash of the branch's *root* node would be better.  But that still 
leaves the two of the three problems unsolved.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: resolving ambiguity in action stamps

[Yuri Khan]

On Sat, Sep 13, 2014 at 5:50 PM, Eric S. Raymond <esr@thyrsus.com> wrote:

> Both of Harald's objections are, alas, sound.  In particular, the
> branch total ordering is not useful if any commit can arbitrarily
> permute it.
>
> The hash of the branch's *root* node would be better.  But that still
> leaves the two of the three problems unsolved.

One of them being “How do you define the root node of a branch?”.

* You cannot mean the commit that is most distant one reachable from
the branch tip, because that root (the “Big Bang” of the repository)
is often common to all branches.
* Do you mean the most distant commit that is reachable from this
branch and unreachable from other branch tips? Then this is
invalidated by creating a new branch tip. If two branch tips point to
the same commit, then, under this definition, they have no root at
all.

No, if we want a mathematically sound and (mostly)
implementation-agnostic way to refer to commits, then it has to:

* be invariant under the operations that do not modify the graph —
branch tip and tag creation, deletion, renaming, resetting,
pushing/fetching;
* be invariant under the operations that are commonly performed on the
graph during normal repository use (creating new commits, pushing and
fetching);
* (optional but highly desired) be well-behaved under less common
operations (cherry-picking, rebasing, force-push).

An author + authoring timestamp combination is in my opinion the
closest thing in Git. It leaves ambiguity when a commit is
cherry-picked or rebased, but:

* This ambiguity may be resolved by tracing reachability from the
referring commit. It should be a rare situation when more than one
instance of the same rebased commit are reachable from a single
descendant/referrer.
** Under that heuristic, rebasing the referrer over to the branch
containing a different instance of the referree will “fix up” the
reference to point to that instance. I actually see this as a good
thing, e.g. when both commits are rebased in a single rebase
operation.
** However, on certain occasions, one may want to refer to a commit on
a different branch, e.g. when cherry-picking, add a reference to the
original commit. I do not have a solution here, apart from “don’t do
that”.

One could suggest ordering the commits having the same
author+timestamp by their commit timestamp (or in fact any other
property), and appending the numeric position according to that order
as a disambiguator. This will break if a reference is made without
knowledge of the combined global set of instances of the commit
referred to — uniting different subsets during fetch or push will
change the positions and invalidate references.

I suggest that, if it is desirable to be able to unambiguously refer
to commits in a rebaseful graph, the 4-tuple <author, authoring
timestamp, committer, commit timestamp> will provide a good
approximation.

In practice, I believe <author, authoring timestamp, optional part of
commit summary> will be precise enough for human software
archaeologists.

Re: resolving ambiguity in action stamps

[Stephen J. Turnbull]

Eric S. Raymond writes:

 > Both of Harald's objections are, alas, sound.

Only because your requirements are unsound.  It's not possible to
specify a revision compactly, let alone uniquely, in a "preorder of
patches" VCS like Darcs.  (Unlike the example below, it's not always
going to be the case that you can refer to a particular patch as a
summary of the situation.)  So broad portability across VCSes is out.
And it's not really necessary: at any given time there's only going to
be one canonical VCS for a project, so references can (and IMO should)
use that VCS's identifier.

Constructing a reference that is reasonably accurate and meaningful to
humans is also very difficult (at least).  Even restricting yourself
to a single VCS like git, there may be insufficient metadata to remove
ambiguity.  On the other hand, although for academics "Raymond [1997]"
is a form that makes a lot of sense, in referencing commits in a VCS
it's nowhere near as useful.  Human memory for commit content has a
halflife of about 2 weeks -- and that's assuming you were both
following emacs-devel and emacs-commits at the time.  Then you go and
look for it -- guess what, grepping the ChangeLog doesn't work if (as
in many projects, and I understand that many hope Emacs will
eventually be one of them) you don't have ChangeLogs.

I really don't see what's wrong with

2014-09-15  Stefan Monnier  <monnier@iro.umontreal.ca>

* JOKES (dumb_joke_internal):  Revert Eric Raymond's commit of
  2014-04-01 [deadbeefcafefeeddefaceddeafadd0123456789].
  It was a funny-once, Man.

Or whatever formatting you want, as long as the SHA1 ref is easily
parsable out of the text.

Regards,

Re: resolving ambiguity in action stamps

[Eric S. Raymond]

Stephen J. Turnbull <turnbull@sk.tsukuba.ac.jp>:
> I really don't see what's wrong with
> 
> 2014-09-15  Stefan Monnier  <monnier@iro.umontreal.ca>
> 
> * JOKES (dumb_joke_internal):  Revert Eric Raymond's commit of
>   2014-04-01 [deadbeefcafefeeddefaceddeafadd0123456789].
>   It was a funny-once, Man.
> 
> Or whatever formatting you want, as long as the SHA1 ref is easily
> parsable out of the text.

At least two things are wrong with it.

One is a genuinely funny gotcha.  You can't get to git hashes without
going through something semantically like my version stamps on the
way!

Think hard about what you'd need to do to know what the right SNA1
hash is for my hypothetical funny-once commit at the time you generate
the final version of this changelog comment about the
reversion. You'll get it, and when you do you might actually laugh.

The second problem is that it's not future-proof. Someday we might
have to change VCSes again; git is the *fifth*, after RCS CVS Arch bzr.
It would be unwise to assume that nobody will ever have a better idea.

At that time it would be a Really Good Thing if as few of our commit
refs as possible are opaque magic cookies - and in order to translate
them to whatever new commit-ref format we'll *still* have to go
through a semantic equivalent of revision stamps!

Thus, it seems best to me to just land on a VCS-independent and
human-readable version-stamp format and stay there, treating VCS-specific
commit-refs as a practice flaw to be avoided.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: resolving ambiguity in action stamps

[David Kastrup]

"Eric S. Raymond" <esr@thyrsus.com> writes:

> The second problem is that it's not future-proof. Someday we might
> have to change VCSes again; git is the *fifth*, after RCS CVS Arch
> bzr.

Being friendly to Arch mirrors does not imply that Emacs actually went
through Arch.

> It would be unwise to assume that nobody will ever have a better idea.
>
> At that time it would be a Really Good Thing if as few of our commit
> refs as possible are opaque magic cookies - and in order to translate
> them to whatever new commit-ref format we'll *still* have to go
> through a semantic equivalent of revision stamps!
>
> Thus, it seems best to me to just land on a VCS-independent and
> human-readable version-stamp format and stay there, treating
> VCS-specific commit-refs as a practice flaw to be avoided.

I disagree.  The whole point of the repo conversion is to make it more
convenient for the _current_ version control system to work with the
history and you are proposing to change to a format that is a nuisance
for _every_ version control system rather than to every version control
system _apart_ from the one being in active use.

When we change to something else but Git for the canonical VCS, we'll
want to convert all those references to whatever we are moving to in
order to make dealing with history reasonably nice.

-- 
David Kastrup

Re: resolving ambiguity in action stamps

[Stephen J. Turnbull]

Eric S. Raymond writes:

 > One is a genuinely funny gotcha.  You can't get to git hashes
 > without going through something semantically like my version stamps
 > on the way!

I disagree.  "Think hard" (your words!) about the workflow:

    Find a bug and make it work
    Then just "git blame" will find the jerk!

More seriously, it seems to me that in many cases the bug will be
localized to a particular commit using bisect.  In others, the *place*
of the bug is known, and git blame will indeed give a pretty good idea
of the commit whether you know *when* it was introduced or not.  (Of
course the bug patch may have been shadowed by a later change in the
same place, but it's not clear to me how else one would get a revision
id for an "old" bug found by place.)  If the bug is recent enough --
ie, the only commit in the pull that broke your Emacs -- it's trivial
to get the commit.  If it's a bit older, I suspect grepping the log
for file and function is more likely to catch the culprit than looking
for random authors and dates you think are incompetent and unlucky
respectively (or is it the other way around?)

So isn't it the reverse: an April Fool's joke is the only time
date!author is likely to find the commit faster?  (Of course if you
*have* the date as in the refs you advocate, git log --since/--until
will narrow it down.  But we're talking about how you figure out which
commit needs a ref to put in the log at this point.)

 > The second problem is that it's not future-proof. Someday we might
 > have to change VCSes again; git is the *fifth*, after RCS CVS Arch
 > bzr.  It would be unwise to assume that nobody will ever have a
 > better idea.

I don't.  But if the new one can't run

    $VCS filter-branch --commit-filter ...

as fast as git, I'd have serious doubts about the sanity of the
proponents.  Even on a 200,000 commit repo, that's just not going to
take a ton of time, and only needs to be done once.

 > At that time it would be a Really Good Thing if as few of our commit
 > refs as possible are opaque magic cookies

Actually, I disagree.  It would be a really good thing if they are
precise.  Do you really want to put anybody through the trouble of
translating randomized format cookies, which may point to any of
several commits, again?  Then revising their scripts every time a new
variant shows up?

A 40-hexit SHA1 is unlikely to be written in any variant, except for
case.  (Well, I suppose we have a few 133t hack0rz around, and
swapping 3s and Es would play havoc....)

The "magic cookie" mapping should be a by-product of the conversion
process (did I hear somebody say his git commits should be 1-1 with
bzr commits?), and then "filter-branch --commit-filter" is your
friend, if you have a database with the mapping lying around
somewhere.

 > Thus, it seems best to me to just land on a VCS-independent and
 > human-readable version-stamp format and stay there, treating
 > VCS-specific commit-refs as a practice flaw to be avoided.

Existence proof comes before characterization, please.

Re: resolving ambiguity in action stamps

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Eric S. Raymond writes:
>
>  > One is a genuinely funny gotcha.  You can't get to git hashes
>  > without going through something semantically like my version stamps
>  > on the way!
>
> I disagree.  "Think hard" (your words!) about the workflow:
>
>     Find a bug and make it work
>     Then just "git blame" will find the jerk!

After fixing a bug, "git blame" tends to point to the bug fixer.
Depending on who wrote that couplet for whom, this may actually be the
intended meaning.

-- 
David Kastrup

Re: resolving ambiguity in action stamps

[Eric S. Raymond]

Stephen J. Turnbull <stephen@xemacs.org>:
> Eric S. Raymond writes:
> 
>  > One is a genuinely funny gotcha.  You can't get to git hashes
>  > without going through something semantically like my version stamps
>  > on the way!
> 
> I disagree.

Well, maybe you have some case, but haring off into bug bisection 
techniques doesn't make it.  

It is still a technical fact that no git translation containing SHA1s
can be built without passing through a VCS-independent representation
of commit refs on the way.  This is because the SHA1-containing repo 
would have to be built one commit at a time in order for the correct target
hashes to *exist* to be patched into whatever refers to them.

That means the rewrite of the old commit refs has would need at some
point to pass through a mapping of old refs to something that identifies
a commit but cannot be a git hash *because the hash doesn't exist yet!*
That's the gotcha.

This is not limited to git: it would a general problem with hg or any 
other target VCS in which parent hashes are mixed into a commit's hash.

I have a design sketch in my head for a tool (a filter on fast-import
streams) to work around this problem, but it can't eliminate the need
for a VCS-independent commit-ref format, only hide the fact that one
was used during the move.

>  > The second problem is that it's not future-proof. Someday we might
>  > have to change VCSes again; git is the *fifth*, after RCS CVS Arch
>  > bzr.  It would be unwise to assume that nobody will ever have a
>  > better idea.
> 
> I don't.  But if the new one can't run
> 
>     $VCS filter-branch --commit-filter ...
> 
> as fast as git, I'd have serious doubts about the sanity of the
> proponents.  Even on a 200,000 commit repo, that's just not going to
> take a ton of time, and only needs to be done once.

Yeah, that'd be nice, if a functional equivalent of filter-branch could 
do the job at all by itself.  No chance of that: see above about hash mixing.

>  > At that time it would be a Really Good Thing if as few of our commit
>  > refs as possible are opaque magic cookies
> 
> Actually, I disagree.  It would be a really good thing if they are
> precise.  Do you really want to put anybody through the trouble of
> translating randomized format cookies, which may point to any of
> several commits, again?  Then revising their scripts every time a new
> variant shows up?

It has yet to be demonstrated that this is a problem in a real use case.
And, actually, I already checked this; the Emacs history doesn't have
any version-stamp collisions in actually referenced revisions. 

> Existence proof comes before characterization, please.

I don't understand that.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: resolving ambiguity in action stamps

[Stephen J. Turnbull]

Eric S. Raymond writes:

 > It is still a technical fact that no git translation containing SHA1s
 > can be built without passing through a VCS-independent representation
 > of commit refs on the way.

Fact!?  I would use the bzr revid, and insert the revid, SHA1 pair
after I commit each new revision in git on Pass 1.  What am I missing?
(Read your next paragraph and my reply first!)

 > Yeah, that'd be nice, if a functional equivalent of filter-branch
 > could do the job at all by itself.  No chance of that: see above
 > about hash mixing.

Of course my proposal needs a database that's writable, and must
update the bzr id to git id mapping every time filter-branch makes a
new commit.  It's not trivial, but it's not going to be hard.  Unless
you think there are Emacs developers smart enough to refer to bugs
that will occur in as-yet uncommitted revisions by revid?

Speaking of databases: since AFAIK you're basically done creating git
blobs and trees (ie, except for new commits to the public repo), I
assume you are using a pre-primed object db when you run your
conversion?  If not, you should get a 20% speed up or so.  You might
be able to get a lot more speed up if you could just work with bzr log
and git filter-branch.  (That's a pretty crazy idea and quite possibly
not at all worth the work even if possible.  But let me throw it out
there....)

 > > Actually, I disagree.  It would be a really good thing if they
 > > are precise.  Do you really want to put anybody through the
 > > trouble of translating randomized format cookies, which may point
 > > to any of several commits, again?  Then revising their scripts
 > > every time a new variant shows up?
 > 
 > It has yet to be demonstrated that this is a problem in a real use
 > case.  And, actually, I already checked this; the Emacs history
 > doesn't have any version-stamp collisions in actually referenced
 > revisions.

That's not what I'm talking about.  I'm talking about
2014/09/15!esr@thyrsus.com vs. 2014-09-15/esr@thyrsus.com vs.
9/15/2014!esr vs. ....  People *will* handwrite those references,
precisely because they're more or less human-readable.

 > > Existence proof comes before characterization, please.

Ie, I suppose you don't get any collisions in referenced revisions.
But we know that there could be.  Maybe "almost correct" is good
enough for you, but I think Emacs deserves better from its VCS.  Worse
is not better when best already exists.

Re: resolving ambiguity in action stamps

[Stephen J. Turnbull]

David Kastrup writes:

 > After fixing a bug, "git blame" tends to point to the bug fixer.

Not for me.  I would dig out the blamed revision before committing
because I am going to insert it in the commit log.  How do you propose
doing it?<wink />

Anyway, git blame supports a <rev> argument (not to mention that you
can get the same effect with git reset).

Re: resolving ambiguity in action stamps

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Eric S. Raymond writes:
>
>  > It is still a technical fact that no git translation containing SHA1s
>  > can be built without passing through a VCS-independent representation
>  > of commit refs on the way.
>
> Fact!?  I would use the bzr revid, and insert the revid, SHA1 pair
> after I commit each new revision in git on Pass 1.  What am I missing?
> (Read your next paragraph and my reply first!)
>
>  > Yeah, that'd be nice, if a functional equivalent of filter-branch
>  > could do the job at all by itself.  No chance of that: see above
>  > about hash mixing.
>
> Of course my proposal needs a database that's writable, and must
> update the bzr id to git id mapping every time filter-branch makes a
> new commit.  It's not trivial, but it's not going to be hard.  Unless
> you think there are Emacs developers smart enough to refer to bugs
> that will occur in as-yet uncommitted revisions by revid?

When revisions are numbered consecutively, that can actually happen.
When creating a commit in some other project referencing an issue
number, I am sometimes lazy enough to create the full commit message
before the issue even exists.  Depending on how many people are active
on some project, this may require doubling up.

But with Bzr revisions, referencing as-yet uncommitted revisions is
slightly more believable than with Git.

-- 
David Kastrup

Re: resolving ambiguity in action stamps

[Eric S. Raymond]

Stephen J. Turnbull <stephen@xemacs.org>:
> Eric S. Raymond writes:
> 
>  > It is still a technical fact that no git translation containing SHA1s
>  > can be built without passing through a VCS-independent representation
>  > of commit refs on the way.
> 
> Fact!?  I would use the bzr revid, and insert the revid, SHA1 pair
> after I commit each new revision in git on Pass 1.  What am I missing?

For one thing, variant forms of commit reference.  Somewhere in there
we'll need the equivalent of a canonicalization pass for the references.

If you go the database-of-pairs route, what you're actually doing is
temporarily creating a VCS-independendent reference ID that mimics a
bzr reference number.  A subtle point, I know - but in principle
there's no actual win in the database-of-pairs that you wouldn't also
get from unique inline reference cookies generated in an intermediate
pass.

In practise, the way my toolkit works, I basically have to have
something like a revision-stamp inline in intermediate versions (that
is, the database-of-pairs approach is out) even if it's massaged into
a SHA1 in the final version.  This is because my tools are an ecology
of import-stream processors built on the assumption that the stream
captures all relevant metadata.

Your instinct may be to come back that this approach is too limiting,
but there are very good reasons for it (beginning with the cross-VCS
portability of the stream files) and 22KLOC of algorithmically dense
tool code built around those reasons.  If you want a high-quality
conversion in reasonable time rather than an open-ended R&D project,
your odds of doing better are effectively nil.

> Speaking of databases: since AFAIK you're basically done creating git
> blobs and trees (ie, except for new commits to the public repo), I
> assume you are using a pre-primed object db when you run your
> conversion?  If not, you should get a 20% speed up or so.  You might
> be able to get a lot more speed up if you could just work with bzr log
> and git filter-branch.  (That's a pretty crazy idea and quite possibly
> not at all worth the work even if possible.  But let me throw it out
> there....)

It's not crazy, but it is too much work.  I'd effectively have to throw
away the rest of my tools.
 
>  > > Actually, I disagree.  It would be a really good thing if they
>  > > are precise.  Do you really want to put anybody through the
>  > > trouble of translating randomized format cookies, which may point
>  > > to any of several commits, again?  Then revising their scripts
>  > > every time a new variant shows up?
>  > 
>  > It has yet to be demonstrated that this is a problem in a real use
>  > case.  And, actually, I already checked this; the Emacs history
>  > doesn't have any version-stamp collisions in actually referenced
>  > revisions.
> 
> That's not what I'm talking about.  I'm talking about
> 2014/09/15!esr@thyrsus.com vs. 2014-09-15/esr@thyrsus.com vs.
> 9/15/2014!esr vs. ....  People *will* handwrite those references,
> precisely because they're more or less human-readable.

Engineering is tradeoffs. Readability (which is a good thing) 
comes with this price.

>  > > Existence proof comes before characterization, please.
> 
> Ie, I suppose you don't get any collisions in referenced revisions.
> But we know that there could be.  Maybe "almost correct" is good
> enough for you, but I think Emacs deserves better from its VCS.  Worse
> is not better when best already exists.

Engineering is tradeoffs.  "Best" by what metric? Readability and 
portability are not trivial features.

One significant disadvantage of building in SHA1s that I haven't mentioned
yet is that they make references brittle. Editing metadata invalidates 
all hashes downstream of it invalid.

Yes, this is a real problem which I have experienced before in big messy 
conversions like this one!  So, we put up a brand shiny new repo - and
a few days (or weeks, or months) later someone spots a conversion bug
that has to be fixed.  

It might be easy for you to say "oh, we just regenerate all the commit
references, then".  Actually doing that is a nasty, picky job even
with best-in-class tools like mine, especially on a repo this size.

I'm not sure anyone on this list but me properly groks the complexity scale 
of this conversion wgen they talk so casually about changing how
it's done.  To get some idea, fetch 

https://gitorious.org/emacs-transition/emacs-transition/raw/ca127b4e1a70cd17f2979b330b3f9dcedaf5bbd8:emacs.lift

and skim all 1018 lines of it - which doesn't count 2.5 Klines of program-
generated stuff included.

When I said this was the biggest, nastiest conversion I've ever done, I
wan't kidding.  Nothing else has even come close.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: resolving ambiguity in action stamps

[Stephen J. Turnbull]

Eric S. Raymond writes:

 > Your instinct may be to come back that this approach is too limiting,

Not at all.  My instinct is to tell you that what is good for a
machine designed to do many conversions with very high quality is
irrelevant to humans who have trouble keeping their noses snot-free at
their most excellent.

 > It might be easy for you to say "oh, we just regenerate all the commit
 > references, then".  Actually doing that is a nasty, picky job even
 > with best-in-class tools like mine, especially on a repo this size.

OK.  Go ahead, do it your way.  I don't really care -- I'm highly
unlikely to contribute code directly to Emacs anyway.

Re: Everyone, please stop making my life more difficult

[Randal L. Schwartz]

>>>>> "Joshua" == Joshua Judson Rosen <rozzin@geekspace.com> writes:

Joshua> (and even regular files actually may have their ctime updated,
Joshua>  for reasons that I can't seem to find a way of fitting into
Joshua>  the VC analogy)

ctime is the measure for "should this file be backed up again".  If a
file is moved to a different place, or given a different name, it *must*
be backed up on an incremental backup, or else the data would be missing
(or in the wrong place) if all the incrementals are restored.

That was the original purpose, and still useful.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix consulting, Technical writing, Comedy, etc. etc.
Still trying to think of something clever for the fourth line of this .sig