The VC to-do list

The VC to-do list

[Eric S. Raymond]

Dan Nicoleascu wrote a bunch of items into the to-do list of vc.el.  I
then appended some.  Here are my comments; I was originally going to
just mail Dan these privately but there are issues here on which
others might want to have design input.

(1) ;; - vc-dir-kill-dir-status-process should not be specific to dir-status,
;;   it should work for other async commands as well (pull/push/...).

Agreed.  Even the async diff code is in this category

Dan, the near term VC-specific fix for this one should probably be
yours, if you can find time to do it; I don't understand the ewoc
interface well enough yet, and there are other things I should do to
VC that are more urgent than learning ewoc.  Like item (2).

There's an entire layer of the VC front-end code that is really
independent of version control -- all the logic dealing with selection
sets, vc-dir, foreground/background command execution, and
parent-child buffer relationships.  I'm beginning to think this wants
to be factored out into its own module, sort of a generic
filesets-to-slave-commands dispatcher.  Such a thing would also be useful
for building IDE-like capabilities, for example.  It would call ewoc
to manage its selection-set buffer.

Your item 1 is really a way of pointing out that there ought to be a 
well-behaved generic command interrupt at this dispatcher level.

Hm.  I might just break vc.el in half this weekend along these lines.
But, Dan, if you see a fix for item (1) in the present organization,
don't let that stop you, because the entire fix should move over
cleanly.  It would all be in the dispatcher layer.

(2) ;; - vc-update/vc-merge should deal with VC systems that don't
;;   update/merge on a file basis, but on a whole repository basis.

There's actually a more basic design problem with the mode underlying
this, which is how the user *specifies* whole-repo operations.  I was
going to write a lengthy analysis here, but I think I'm going to ship
it to emacs-devel instead as a separate post.  Watch for a post on the
"null-fileset problem".  This one is mine to solve.  It's a bitch.

Actually, this UI design problem lives in the hypothetical dispatcher
layer I was just thinking about out loud.

(3) ;; - deal with push/pull operations.

Yes.  Not too hard once we solve the null-fileset problem, actually; 
I know what the keybindings ought to be, C-x v > and C-x v <, and the
rest is fairly straight-line coding.  I'll do it.

(4) ;; - "snapshots" should be renamed to "branches", and thoroughly reworked.

It's messier than that.  The real problem is that the present snapshot
implementations conflate two different concepts -- tags and branches.
When I originally wrote snapshot facility, it was a kluge to cope with
the apsence of repo-wide tags in SCCS.  Later, other backend aithors
co-opted those hooks to handle branching.  

This was an error but an understandable one; VCSes themselves have not
always been clear about the difference (yes, Subversion, I'm looking
at *you*).  These need to be pulled apart in VC.  I'll do it.

(5) ;; - when a file is in `conflict' state, turn on smerge-mode.

I know nothing about smerge.  This one is probably for you, Dan.

(6) ;; - figure out what to do with conflicts that are not caused by the
;;   file contents, but by metadata or other causes.

Example, please?

(7) ;; - add a generic mechanism for remembering the current branch names,
;;   display the branch name in the mode-line. Replace
;;   vc-cvs-sticky-tag with that.

Quite right.  I'll take this one.

(8) ;; - vc-diff should be able to show the diff for all files in a
;;   changeset, especially for VC systems that have per repository
;;   version numbers.  log-view should take advantage of this.

I thought we were already doing this, at least insofar as I understand
the specification here.  What behavior is missing?

(9) ;; - the *VC-log* buffer needs font-locking.

I know nothing about font-locking.  This one is probably for you, Dan.

(10) ;; - make it easier to write logs.  Maybe C-x 4 a should add to the log
;;   buffer, if one is present, instead of adding to the ChangeLog.

No.  The real problem here is that the combination of Changelogs and
a fileset-oriented VCS's revision history is intrinsically duplicative 
and bogus, and Changelogs should be shot through the head as soon as 
we migrate off CVS.

That's a policy issue we can't solve with code -- or, to put it a
different way, any code solution we do will be wrong where it doesn't
match a user's policy expectations.  So I'm not going to put any
energy into trying.

(11) ;; - add a mechanism for ignoring files.

I'm not clear why.  Under what circumstances do we need this when
the VCS doesn't do it for us?

(12) ;; - When vc-next-action calls vc-checkin it could pre-fill the
;;   *VC-log* buffer with some obvious items: the list of files that
;;   were added, the list of files that were removed.  If the diff is
;;   available, maybe it could even call something like
;;   `diff-add-change-log-entries-other-window' to create a detailed
;;   skeleton for the log...

Dan, this sounds reasonable, but also like a thing for you to do because
you know what your goals here are.

(13) ;; - Set `vc-dir-insert-directories' to t and check what operations
;;   and backends do not support directory arguments and fix them.

There be dragons here.  I think we have to clean up the front-end
problems with defining filesets and the meaning of the null
fileset (in particular, when it means 'current directory')
before tackling this will really make sense.

(14) ;; - a way to do repository wide log (instead of just per
;;   file/fileset) is needed.  Doing it per directory might be enough...

See the notes on the dispatcher and the null-fileset problem above.
This really has nothing to do with version-control per se, it would be
a UI issue with the front-end design for *anything* that has (a)
per-file, (b) per-directory, and (c) global operations.  How can the
user specify these contexts in an intuitive and consistent way?

(15) ;; - most vc-dir backends need more work.  They might need to
;;   provide custom headers, use the `extra' field and deal with all
;;   possible VC states.
;;
;; - add function that calls vc-dir to `find-directory-functions'.
;;
;; - vc-dir needs mouse bindings.
;;
;; - vc-dir needs more key bindings for VC actions.
;;
;; - vc-dir toolbar needs more icons.

None of these sound difficult.  It's basically a matter of defining
what's missing.

(16) ;; - vc-dir-next-line should not print an "end of buffer" message when
;;   invoked with the cursor on the last file.
;;
;; - add commands to move to the prev/next directory in vc-dir.

Relatively low-level coding issues.  Whichever one of us gets to 'em first.

(17) ;; - document vc-dir in the manual.

I'm willing to do that when the feature set stabilizes.

(18) ;; - vc-diff, vc-annotate, etc. need to deal better with unregistered
;;   files. Now that unregistered and ignored files are shown in
;;   vc-dired/vc-dir, it is possible that these commands are called
;;   for unregistered/ignored files.

Agreed.  This one shouldn't be too bad, I don't think it matters which
one of us takes it.

(19) ;; - do not default to RCS anymore when the current directory is not
;;   controlled by any VCS and the user does C-x v v

Likewise.

(20) ;; - vc-create-snapshot and vc-retrieve-snapshot should update the
;;   buffers that might be visiting the affected files.

I think this reduces to implementing vc-directory-resynch correctly for the 
vc-dir case.  And I think that's yours, Dan.

(21) ;; - Using multiple backends needs work.  Given a CVS directory with some
;;   files checked into git (but not all), using C-x v l to get a log file
;;   from a file only present in git, and then typing RET on some log entry,
;;   vc will bombs out because it wants to see the file being in CVS.
;;   Those logs should likely use a local variable to hardware the VC they
;;   are supposed to work with.

I've already explained on-list why I don't intend to put any energy
into this until the rest of the todo list is clear.  I (a) don't think
it's important, and (b) do fear the code bloat it seems likely to bring.

(22) ;; - backends that care about vc-stay-local should try to take it into
;;   account for vc-dir.  Is this likely to be useful???

I don't really understand stay-local or the hacks around it.  I wish
somebody else would take this one.

(23) ;; - vc-dir listing needs a footer generated when it's done to make it
;; obvious that it has finished.

This is totally a ewoc interface issue.  Yours, Dan, I think.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

When all government ...in little as in great things... shall be drawn to
Washington as the center of all power; it will render powerless the checks
provided of one government on another, and will become as venal and oppressive
as the government from which we separated."	-- Thomas Jefferson, 1821

Re: The VC to-do list

[Stefan Monnier]

> (6) ;; - figure out what to do with conflicts that are not caused by the
> ;;   file contents, but by metadata or other causes.

> Example, please?

File A gets renamed to B in one branch and to C in another and you merge
the two branches.  Or you locally add file FOO and then pull a change
that also adds a new file FOO, ...

> (8) ;; - vc-diff should be able to show the diff for all files in a
> ;;   changeset, especially for VC systems that have per repository
> ;;   version numbers.  log-view should take advantage of this.

> I thought we were already doing this, at least insofar as I understand
> the specification here.  What behavior is missing?

Do a log on foo.el, then for one of the changes, try to get the full
diff: that won't work, it'll only give you the foo.el part of that
changeset, not the changes made to other files.

> (10) ;; - make it easier to write logs.  Maybe C-x 4 a should add to the log
> ;;   buffer, if one is present, instead of adding to the ChangeLog.

> No.  The real problem here is that the combination of Changelogs and
> a fileset-oriented VCS's revision history is intrinsically duplicative 
> and bogus, and Changelogs should be shot through the head as soon as 
> we migrate off CVS.

I don't understand this "No" since you follow it by 4 lines of text that
seem 100% in agreement with point (10).  Looks like just mentioning
"ChangeLog" hits a nerve, even when the mention is going in the
direction you want.

> (14) ;; - a way to do repository wide log (instead of just per
> ;;   file/fileset) is needed.  Doing it per directory might be enough...

> See the notes on the dispatcher and the null-fileset problem above.
> This really has nothing to do with version-control per se, it would be
> a UI issue with the front-end design for *anything* that has (a)
> per-file, (b) per-directory, and (c) global operations.  How can the
> user specify these contexts in an intuitive and consistent way?

In PCL-CVS, I used different user commands with different key-bindings
for global operations.  Note that I only provided global operations for
"update" and "refresh status", so maybe it's not that relevant.

> I've already explained on-list why I don't intend to put any energy
> into this until the rest of the todo list is clear.  I (a) don't think
> it's important, and (b) do fear the code bloat it seems likely to bring.

I disagree.  I think it may require a lot of changes, but no code bloat
at all.  It's just a matter of being careful of where we decide which
backend to use.  Currently we make this decision over-and-over again at
many different places.  And if one of those places makes a different
decision, we have a bug.  So dealing with multiple backends is mostly
a matter of deciding which backend to use once and forall and then
properly preserving this info, which will eliminate the bugs.

Note that we've already seen such bugs for single-backend situations
because one of the files selected happens to be managed by no backend,
so we get bugs like "can't load vc-nil".  This is not some hypothetical
problem.  The fact that it will correctly handle multi-backend
situations is just the result of doing things right.

> (22) ;; - backends that care about vc-stay-local should try to take it into
> ;;   account for vc-dir.  Is this likely to be useful???

> I don't really understand stay-local or the hacks around it.  I wish
> somebody else would take this one.

I think stay-local should ultimately go.  It will be replaced by
a distinction between "status" and "pull --dry-run".

Another important thing: merge all the status-like backend operations.
We should remove dir-status, state, dir-state, and dir-status-files, and
replace them with just `status' which takes a fileset and a continuation
(like dir-status) and returns a buffer in which the process(es) are run
(or nil if it worked synchronously).  Hopefully we can define the old
4 operations in term of this one.


        Stefan

Re: The VC to-do list

[Dan Nicolaescu]

"Eric S. Raymond" <esr@snark.thyrsus.com> writes:

  > (1) ;; - vc-dir-kill-dir-status-process should not be specific to dir-status,
  > ;;   it should work for other async commands as well (pull/push/...).
  > 
  > Agreed.  Even the async diff code is in this category
  > 
  > Dan, the near term VC-specific fix for this one should probably be
  > yours, if you can find time to do it; I don't understand the ewoc
  > interface well enough yet, and there are other things I should do to
  > VC that are more urgent than learning ewoc.  Like item (2).

This is actually Stefan's addition.  It shouldn't be too hard to fix:
all the asynchronous commands need a buffer to run in, just find a way
to keep track of that buffer and kill it+the associated process when
necessary.

  > (11) ;; - add a mechanism for ignoring files.
  > 
  > I'm not clear why.  Under what circumstances do we need this when
  > the VCS doesn't do it for us?

This is about adding a way to tell the VCS to ignore certain files.

  > (23) ;; - vc-dir listing needs a footer generated when it's done to make it
  > ;; obvious that it has finished.
  > 
  > This is totally a ewoc interface issue.  Yours, Dan, I think.

I'm not convinced it's useful, the mode-line displays [Waiting...] when
something is going on.

Re: The VC to-do list

[Eric S. Raymond]

Stefan Monnier <monnier@iro.umontreal.ca>:
> > (6) ;; - figure out what to do with conflicts that are not caused by the
> > ;;   file contents, but by metadata or other causes.
> 
> > Example, please?
> 
> File A gets renamed to B in one branch and to C in another and you merge
> the two branches.  Or you locally add file FOO and then pull a change
> that also adds a new file FOO, ...

Noted, added as a paradigm example to that to-do item.

> > (8) ;; - vc-diff should be able to show the diff for all files in a
> > ;;   changeset, especially for VC systems that have per repository
> > ;;   version numbers.  log-view should take advantage of this.
> 
> > I thought we were already doing this, at least insofar as I understand
> > the specification here.  What behavior is missing?
> 
> Do a log on foo.el, then for one of the changes, try to get the full
> diff: that won't work, it'll only give you the foo.el part of that
> changeset, not the changes made to other files.

It's not clear to me that this is wrong behavior, given the context
in which the diff was requested.  But I'm willing to be convinced.
What's the UI/design-level argument?

> > (10) ;; - make it easier to write logs.  Maybe C-x 4 a should add to the log
> > ;;   buffer, if one is present, instead of adding to the ChangeLog.
> 
> > No.  The real problem here is that the combination of Changelogs and
> > a fileset-oriented VCS's revision history is intrinsically duplicative 
> > and bogus, and Changelogs should be shot through the head as soon as 
> > we migrate off CVS.
> 
> I don't understand this "No" since you follow it by 4 lines of text that
> seem 100% in agreement with point (10).  Looks like just mentioning
> "ChangeLog" hits a nerve, even when the mention is going in the
> direction you want.

Hm.  We must be parsing the term "log buffer" differently.  Perhaps Dan
will unpack his proposal a bit for us.

It is, however, true that Changelogs annoy me extremely.  SPOT violations
of all kinds make me pretty cranky, actually.

> > (14) ;; - a way to do repository wide log (instead of just per
> > ;;   file/fileset) is needed.  Doing it per directory might be enough...
> 
> > See the notes on the dispatcher and the null-fileset problem above.
> > This really has nothing to do with version-control per se, it would be
> > a UI issue with the front-end design for *anything* that has (a)
> > per-file, (b) per-directory, and (c) global operations.  How can the
> > user specify these contexts in an intuitive and consistent way?
> 
> In PCL-CVS, I used different user commands with different key-bindings
> for global operations.  Note that I only provided global operations for
> "update" and "refresh status", so maybe it's not that relevant.

Yes, the design problem is rather more difficult in the VC context.  I'm
going to write an analysis and post it sometime in the near future.  I'd
do it now, but I think the process of separating the dispatcher layer out 
of vc.el may give me insights so I'll probably do that first.
 
> I disagree.  I think it may require a lot of changes, but no code bloat
> at all.  It's just a matter of being careful of where we decide which
> backend to use.  Currently we make this decision over-and-over again at
> many different places.  And if one of those places makes a different
> decision, we have a bug.  So dealing with multiple backends is mostly
> a matter of deciding which backend to use once and forall and then
> properly preserving this info, which will eliminate the bugs.
> 
> Note that we've already seen such bugs for single-backend situations
> because one of the files selected happens to be managed by no backend,
> so we get bugs like "can't load vc-nil".  This is not some hypothetical
> problem.  The fact that it will correctly handle multi-backend
> situations is just the result of doing things right.

Persuade me by doing, please -- you or David Kastrup, I'm not picky.
I'm not saying that to be hostile, I've just got lots of other things
to do to VC that I consider higher priority.  And less than six weeks to do
them in, if we take our freeze schedule seriously.

> > (22) ;; - backends that care about vc-stay-local should try to take it into
> > ;;   account for vc-dir.  Is this likely to be useful???
> 
> > I don't really understand stay-local or the hacks around it.  I wish
> > somebody else would take this one.
> 
> I think stay-local should ultimately go.  It will be replaced by
> a distinction between "status" and "pull --dry-run".

I think I like that direction.  I'm not certain I understand all the
issues around it, though.

> Another important thing: merge all the status-like backend operations.
> We should remove dir-status, state, dir-state, and dir-status-files, and
> replace them with just `status' which takes a fileset and a continuation
> (like dir-status) and returns a buffer in which the process(es) are run
> (or nil if it worked synchronously).  Hopefully we can define the old
> 4 operations in term of this one.

Agreed.  I've been complaining to Dan that the backend API has gotten
warty and excessively complicated (mainly my fault, of course).  This seems a
good place to take a whack at simplifying it.  Adding your suggestion
as a to-do item...
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: The VC to-do list

[Eric S. Raymond]

Dan Nicolaescu <dann@ics.uci.edu>:
> "Eric S. Raymond" <esr@snark.thyrsus.com> writes:
> 
>> (1) ;; - vc-dir-kill-dir-status-process should not be specific to dir-status,
>> ;;   it should work for other async commands as well (pull/push/...).
>> 
>> Agreed.  Even the async diff code is in this category
>> 
>> Dan, the near term VC-specific fix for this one should probably be
>> yours, if you can find time to do it; I don't understand the ewoc
>> interface well enough yet, and there are other things I should do to
>> VC that are more urgent than learning ewoc.  Like item (2).
> 
> This is actually Stefan's addition.  It shouldn't be too hard to fix:
> all the asynchronous commands need a buffer to run in, just find a way
> to keep track of that buffer and kill it+the associated process when
> necessary.

Agreed.  The dispatcher layer I'm thinking about splitting out would need
this as a generic facility anyway.
 
>   > (11) ;; - add a mechanism for ignoring files.
>   > 
>   > I'm not clear why.  Under what circumstances do we need this when
>   > the VCS doesn't do it for us?
> 
> This is about adding a way to tell the VCS to ignore certain files.

Right; am rewording the to-do appropriately.
 
>> (23) ;; - vc-dir listing needs a footer generated when it's done to make it
>> ;; obvious that it has finished.
>> 
>> This is totally a ewoc interface issue.  Yours, Dan, I think.
> 
> I'm not convinced it's useful, the mode-line displays [Waiting...] when
> something is going on.

Really?  I didn't notice that...which tells its own story.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: The VC to-do list

[Stefan Monnier]

>> Do a log on foo.el, then for one of the changes, try to get the full
>> diff: that won't work, it'll only give you the foo.el part of that
>> changeset, not the changes made to other files.

> It's not clear to me that this is wrong behavior, given the context
> in which the diff was requested.  But I'm willing to be convinced.
> What's the UI/design-level argument?

I didn't mean to say that the above is a bug.  But that the other diff
can also be useful and that there is no way currently for VC to give you
that.  I.e. we need both to add a way for the user to request this info,
and a way for the backend to provide it.

>> > (10) ;; - make it easier to write logs.  Maybe C-x 4 a should add to the log
>> > ;;   buffer, if one is present, instead of adding to the ChangeLog.
>> 
>> > No.  The real problem here is that the combination of Changelogs and
>> > a fileset-oriented VCS's revision history is intrinsically duplicative 
>> > and bogus, and Changelogs should be shot through the head as soon as 
>> > we migrate off CVS.
>> 
>> I don't understand this "No" since you follow it by 4 lines of text that
>> seem 100% in agreement with point (10).  Looks like just mentioning
>> "ChangeLog" hits a nerve, even when the mention is going in the
>> direction you want.

> Hm.  We must be parsing the term "log buffer" differently.  Perhaps Dan
> will unpack his proposal a bit for us.

I probably wrote this TODO item.  The log buffer is *VC-Log*.
I.e. let C-x v 4 add the log-comment directly to the *VC-Log* buffer.

> Persuade me by doing, please -- you or David Kastrup, I'm not picky.

A first step was taken by having vc-deduce-fileset return both a list of
files and a backend.  We should then kill the `vc-call' macro
(introduced by yours truly).

> I'm not saying that to be hostile, I've just got lots of other things
> to do to VC that I consider higher priority.

Fine by me.

>> > (22) ;; - backends that care about vc-stay-local should try to take it into
>> > ;;   account for vc-dir.  Is this likely to be useful???
>> 
>> > I don't really understand stay-local or the hacks around it.  I wish
>> > somebody else would take this one.
>> 
>> I think stay-local should ultimately go.  It will be replaced by
>> a distinction between "status" and "pull --dry-run".

> I think I like that direction.  I'm not certain I understand all the
> issues around it, though.

Stay-local was introduced by yours truly basically to let vc-cvs work
better when the repository is far away.  The way it affects the behavior
is that diff/log/... tend to be async rather than sync (a fairly minor
issue so it doesn't really matter whether the user can configure it as
long as the "default" behavior works well) and more importantly, to use
state-heuristic (which interprets CVS/Entries) rather than running "cvs
status".  In DVCS, "foo status" corresponds roughyl to what
state-heurstic does (e.g. it'll tell you what you've changed but not
what other people have done), and "cvs status" corresponds to
"pull --dry-run" (except that "pull --dry-run" might only tell you about
what other people have done, whereas "cvs status" will combine it with
info about what you've done).


        Stefan

Re: The VC to-do list

[Eric S. Raymond]

Stefan Monnier <monnier@iro.umontreal.ca>:
> >> Do a log on foo.el, then for one of the changes, try to get the full
> >> diff: that won't work, it'll only give you the foo.el part of that
> >> changeset, not the changes made to other files.
> 
> > It's not clear to me that this is wrong behavior, given the context
> > in which the diff was requested.  But I'm willing to be convinced.
> > What's the UI/design-level argument?
> 
> I didn't mean to say that the above is a bug.  But that the other diff
> can also be useful and that there is no way currently for VC to give you
> that.  I.e. we need both to add a way for the user to request this info,
> and a way for the backend to provide it.

Noted.

> A first step was taken by having vc-deduce-fileset return both a list of
> files and a backend.  We should then kill the `vc-call' macro
> (introduced by yours truly).

Watch this space.  I'm in the process of prying the UI machinery loose
from the backend-aware stuff, as I was musing about doing two days
ago.  At some point in the not too distant future I'm going to
refactor the hell out of vc-deduce-fileset.  

The functional goal will be to clean up the UI selection logic so that
the user can always easily select either (a) the current directory,
(b) an explicit fileset, or (c) the null (repo-wide) context.

But the way I'm likeliest to actually do this (by retaining a
reference to a backend-capabilities object) will probably blow away
vc-call and a lot of the other mutually redundant backend dispatches
presently littered through the code.

Yes, I've found defstructs.  I knew about the CL version (though I'd
never used them), but when did they make it into Emacs?  For the last
30 hours or so, my Lisp chops and my OO chops have been fusing in
strange and wonderful ways.  It's interesting to actually watch my Lisp
style mutating towards Python...but then I write very Lispy Python.

(Yes, 30 hours of almost continuous hacking with short breaks for
meals; this is also my first experiment with modafinil aka Provigil.
Yes, it's legal, and yes, my doctor approved and prescribed it, and
yes, it seems remarkably benign.)

> >> > I don't really understand stay-local or the hacks around it.  I wish
> >> > somebody else would take this one.
> >> 
> >> I think stay-local should ultimately go.  It will be replaced by
> >> a distinction between "status" and "pull --dry-run".
> 
> > I think I like that direction.  I'm not certain I understand all the
> > issues around it, though.
> 
> Stay-local was introduced by yours truly basically to let vc-cvs work
> better when the repository is far away.  The way it affects the behavior
> is that diff/log/... tend to be async rather than sync (a fairly minor
> issue so it doesn't really matter whether the user can configure it as
> long as the "default" behavior works well) and more importantly, to use
> state-heuristic (which interprets CVS/Entries) rather than running "cvs
> status".  In DVCS, "foo status" corresponds roughyl to what
> state-heurstic does (e.g. it'll tell you what you've changed but not
> what other people have done), and "cvs status" corresponds to
> "pull --dry-run" (except that "pull --dry-run" might only tell you about
> what other people have done, whereas "cvs status" will combine it with
> info about what you've done).

Keeping a copy of this for later inspection...
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>