I can't commit a simple change

I can't commit a simple change

[Richard Stallman]

I tried to commit a documentation change in simple.el and lisp/ChangeLog.
I got an error message saying there are conflicts.  (There are no conficts
in those files.)

bzr conflicts gave me this output.

======================================================================
Conflict: can't delete info because it is not empty.  Not deleting.
Conflict adding file lisp/dired.el.BASE.  Moved existing file to lisp/dired.el.BASE.moved.
Conflict adding file lisp/dired.el.OTHER.  Moved existing file to lisp/dired.el.OTHER.moved.
Conflict adding file lisp/dired.el.THIS.  Moved existing file to lisp/dired.el.THIS.moved.
======================================================================

I don't understand any of this.  What do I have to do?
And why is any of this relevant to the change I want to commit?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: I can't commit a simple change

[Eli Zaretskii]

> Date: Sat, 21 Dec 2013 16:19:30 -0500
> From: Richard Stallman <rms@gnu.org>
> 
> I tried to commit a documentation change in simple.el and lisp/ChangeLog.
> I got an error message saying there are conflicts.  (There are no conficts
> in those files.)
> 
> bzr conflicts gave me this output.
> 
> ======================================================================
> Conflict: can't delete info because it is not empty.  Not deleting.
> Conflict adding file lisp/dired.el.BASE.  Moved existing file to lisp/dired.el.BASE.moved.
> Conflict adding file lisp/dired.el.OTHER.  Moved existing file to lisp/dired.el.OTHER.moved.
> Conflict adding file lisp/dired.el.THIS.  Moved existing file to lisp/dired.el.THIS.moved.
> ======================================================================
> 
> I don't understand any of this.  What do I have to do?

Delete the directory info with all that's in there, and then try

  bzr resolve --all --take-other

If this succeeds, the next "bzr conflicts" command should produce no
output.

> And why is any of this relevant to the change I want to commit?

Because your local repository is not up to date, and bzr won't let you
commit from an outdated repository.

Re: I can't commit a simple change

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

    Delete the directory info with all that's in there, and then try

I moved it away.

But why was it a problem?  It is not new;
the files in it are dated May 1.

      bzr resolve --all --take-other

That got an error, that --take-other is not valid.

I moved away the dired.el.* files and did

      bzr resolve --all

which seems to have fixed it.  But I would still like to know
what went wrong with the info directory.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: I can't commit a simple change

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

>     Delete the directory info with all that's in there, and then try
>
> I moved it away.
>
> But why was it a problem?  It is not new;
> the files in it are dated May 1.

The directory has been deleted recently, but you still had (ignored)
files in it.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: I can't commit a simple change

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

    > But why was it a problem?  It is not new;
    > the files in it are dated May 1.

    The directory has been deleted recently, but you still had (ignored)
    files in it.

Why was it deleted?  Where do the info files go, now?
I always build in the source tree.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: I can't commit a simple change

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

> Why was it deleted?  Where do the info files go, now?

Nothing has changed wrt. the info files.  It's just that the directory
is no longer kept in the repository.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: I can't commit a simple change

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

    > Why was it deleted?  Where do the info files go, now?

    Nothing has changed wrt. the info files.  It's just that the directory
    is no longer kept in the repository.

That seems like a mistake.  It makes building slower.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: I can't commit a simple change

[Eli Zaretskii]

> Date: Mon, 23 Dec 2013 22:28:12 -0500
> From: Richard Stallman <rms@gnu.org>
> CC: eliz@gnu.org, emacs-devel@gnu.org
> 
>     Nothing has changed wrt. the info files.  It's just that the directory
>     is no longer kept in the repository.
> 
> That seems like a mistake.  It makes building slower.

Which part of the build causes the slowdown?

Re: I can't commit a simple change

[Paul Eggert]

Richard Stallman wrote:
>     Nothing has changed wrt. the info files.  It's just that the directory
>     is no longer kept in the repository.
> 
> That seems like a mistake.  It makes building slower.

It makes other things faster, notably the human process
of reviewing patches to the repository.

It might be helpful to have two repositories: the first
would omit autogenerated files, and the second would contain
all files that can be generated automatically and portably.
Human reviewers could look only at the former, and people
who want to build quickly from the repository could use
the latter.

Re: I can't commit a simple change

[Glenn Morris]

Richard Stallman wrote:

>     Nothing has changed wrt. the info files.  It's just that the directory
>     is no longer kept in the repository.
>
> That seems like a mistake.  It makes building slower.

No, it does not. You have misunderstood.

Re: I can't commit a simple change

[Glenn Morris]

Paul Eggert wrote:

> It might be helpful to have two repositories: the first would omit
> autogenerated files, and the second would contain all files that can
> be generated automatically and portably. Human reviewers could look
> only at the former, and people who want to build quickly from the
> repository could use the latter.

Sorry, IMO this is not a helpful idea.
(Does any other project do anything like this?)

Anyone who finds building the info files a chore should simply install
makeinfo 4.13 in /usr/local, and move on with their life, as I did.

There are loads and loads of ways to get prebuilt versions of the Emacs
trunk if you don't have the resources to build it yourself.

Eg http://hydra.nixos.org/build/7323550, "source distribution" link,
updated in theory on every commit.

Please let's not reintroduce complexity that we spent some time
removing, all for the sake of what is clearly a misunderstanding on
rms's part (the .info files have never been in the repo. All that was
there was info/dir.)

Re: I can't commit a simple change

[Glenn Morris]

Glenn Morris wrote:

> Richard Stallman wrote:
>
>>     Nothing has changed wrt. the info files.  It's just that the directory
>>     is no longer kept in the repository.
>>
>> That seems like a mistake.  It makes building slower.
>
> No, it does not. You have misunderstood.

Let me elaborate: the .info files were never in the repository.
All that was there was a (pointless) info/dir file.
That's auto-generated now when needed.
You had a conflict when you updated from bzr, because bzr wanted to
remove the info directory, but you had non-versioned files in there, so
it could not.
Deal with the conflict and you will find that literally nothing has
changed about building Emacs from bzr in any meaningful way.

Re: I can't commit a simple change

[Lars Ingebrigtsen]

Glenn Morris <rgm@gnu.org> writes:

> Deal with the conflict and you will find that literally nothing has
> changed about building Emacs from bzr in any meaningful way.

Omitting these directories and files from the VC is obviously a good
thing, but would it be possible to include some kind of tooling during
the build process that would just blow these apparent "conflicts" away?

It may be a non-trivial thing to do ("if you've 'bzr update'-ed past
revision 322522, then 'rm -r info'" as the first thing in the build
process), but if this were possible, all these recurring queries in this
area would go away.

And it would be really helpful for us less than regular committers.  It
feels like every time I'm committing something after not having done
anything for a few weeks, I have to spend five minutes rm-ing stuff and
hoping that I'm not ruining anything the next commit.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: I can't commit a simple change

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

    It might be helpful to have two repositories: the first
    would omit autogenerated files, and the second would contain
    all files that can be generated automatically and portably.
    Human reviewers could look only at the former, and people
    who want to build quickly from the repository could use
    the latter.

I would like this.  I would be glad to be able to fetch
the current .elc files along with the current .el files.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: I can't commit a simple change

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

    Let me elaborate: the .info files were never in the repository.
    All that was there was a (pointless) info/dir file.
    That's auto-generated now when needed.

Thanks.  Now I understand.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: I can't commit a simple change

[Stephen J. Turnbull]

Lars Ingebrigtsen writes:

 > It may be a non-trivial thing to do ("if you've 'bzr update'-ed past
 > revision 322522, then 'rm -r info'" as the first thing in the build
 > process), but if this were possible, all these recurring queries in this
 > area would go away.

The right incantation is "rm `bzr ls --unknown`; bzr revert" before
merging.  But if that actually makes any changes to your tree, it's
destroying data that *you* put there[1].  Neither Bazaar nor Make
should do that automatically.

 > And it would be really helpful for us less than regular committers.
 > It feels like every time I'm committing something after not having
 > done anything for a few weeks, I have to spend five minutes rm-ing
 > stuff and hoping that I'm not ruining anything the next commit.

You're always at risk of ruining something with the next commit.  If
you can't safely use the cleanup incantation above to minimize that
risk with minimum effort and time, it's because your workspace has
acquired "cruft" irrelevant to that commit.  Use either shelve or
pipeline to organize the cruft so that the VCS can track your work.


Footnotes: 
[1]  Unless somebody forgot to commit the rebuilt Info along with the
changed Texi, which is a practical reason why getting those files out
of the repo is the RightThang.

Re: I can't commit a simple change

[Lars Ingebrigtsen]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> The right incantation is "rm `bzr ls --unknown`; bzr revert" before
> merging.  But if that actually makes any changes to your tree, it's
> destroying data that *you* put there[1].  Neither Bazaar nor Make
> should do that automatically.

I've put no data in the info directory, and neither has Richard.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: I can't commit a simple change

[Stephen J. Turnbull]

Lars Ingebrigtsen writes:

 > > The right incantation is "rm `bzr ls --unknown`; bzr revert" before
 > > merging.  But if that actually makes any changes to your tree, it's
 > > destroying data that *you* put there[1].  Neither Bazaar nor Make
 > > should do that automatically.
 > 
 > I've put no data in the info directory, and neither has Richard.

In that case, this is a one-off problem and removing and ignoring the
info directory entirely was the right fix.

Re: I can't commit a simple change

[Lars Ingebrigtsen]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> In that case, this is a one-off problem and removing and ignoring the
> info directory entirely was the right fix.

Like I said, there have been plenty of these one-off problems.  One
might suspect that they aren't one-offs any more.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: I can't commit a simple change

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: eliz@gnu.org,  Andreas Schwab <schwab@linux-m68k.org>,  rms@gnu.org,  emacs-devel@gnu.org
> Date: Wed, 25 Dec 2013 16:28:02 +0100
> 
> "Stephen J. Turnbull" <stephen@xemacs.org> writes:
> 
> > In that case, this is a one-off problem and removing and ignoring the
> > info directory entirely was the right fix.
> 
> Like I said, there have been plenty of these one-off problems.  One
> might suspect that they aren't one-offs any more.

The problem here is that bzr doesn't know whether the files in a
directory it needs to remove are precious to you or not.  So it punts.
You see, you and Richard did not make any changes in that directory,
but I and some other J. R. Hacker might.

If you can suggest a solution that doesn't risk removing precious
files in a directory that is removed from the repository, please do.

Re: I can't commit a simple change

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> If you can suggest a solution that doesn't risk removing precious
> files in a directory that is removed from the repository, please do.

In this instance, the problem was some of the auto-generated info
files.  We do know what those are, so it would be possible to have to
build process remove them.

In other instances, it's been some of...  er...  the leim files?  And
before that, I think...  the lisp/international stuff?  I forget.
Anyway, in all these cases there surely is a known set of checked-in
files that became autogenerated.  Enumerating these should be a
solveable problem.

Whether it's worth it is another issue.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: I can't commit a simple change

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: stephen@xemacs.org,  schwab@linux-m68k.org,  rms@gnu.org,  emacs-devel@gnu.org
> Date: Wed, 25 Dec 2013 19:32:58 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > If you can suggest a solution that doesn't risk removing precious
> > files in a directory that is removed from the repository, please do.
> 
> In this instance, the problem was some of the auto-generated info
> files.  We do know what those are, so it would be possible to have to
> build process remove them.

Remove them how?  The build process is too late: the conflict is
reported when you "bzr update" which is _before_ the build.

Re: I can't commit a simple change

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Remove them how?  The build process is too late: the conflict is
> reported when you "bzr update" which is _before_ the build.

I've only encountered these problems when I'm checking stuff in.  Which
is always after running a build, because I try to at least see whether
stuff builds before I do so.  >"?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: I can't commit a simple change

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: stephen@xemacs.org,  schwab@linux-m68k.org,  rms@gnu.org,  emacs-devel@gnu.org
> Date: Wed, 25 Dec 2013 20:51:22 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Remove them how?  The build process is too late: the conflict is
> > reported when you "bzr update" which is _before_ the build.
> 
> I've only encountered these problems when I'm checking stuff in.

That's because you don't pay attention to what "bzr up" tells you:

  Conflict: can't delete info because it is not empty.  Not deleting.
  1 conflicts encountered.

People who do pay attention are confused right there and then, before
they even try to build.

Besides, even if we do postpone the removal to build time, how do you
suggest doing this in practice?  Adding this to Makefile.in is
problematic, since it will thereafter remove those files again and
again, after the user did the build, until the removal commands are
deleted from Makefile.in.  And how to know when to delete them?  I
don't see how this can be workable.

Re: I can't commit a simple change

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Besides, even if we do postpone the removal to build time, how do you
> suggest doing this in practice?  Adding this to Makefile.in is
> problematic, since it will thereafter remove those files again and
> again, after the user did the build, until the removal commands are
> deleted from Makefile.in.  And how to know when to delete them?  I
> don't see how this can be workable.

Like I said in the original message -- if the build system were able to
detect that you had updated past revision foo, then it might be possible
to do something with that knowledge.  Perhaps there might be an
associated clean-up script for foo.

And I don't know whether that's workable, but I'm not a build genius,
and there's plenty of them here on this list.  >"?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: I can't commit a simple change

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Wed, 25 Dec 2013 21:18:59 +0100
> Cc: stephen@xemacs.org, schwab@linux-m68k.org, rms@gnu.org, emacs-devel@gnu.org
> 
> And I don't know whether that's workable, but I'm not a build genius,
> and there's plenty of them here on this list.  >"?

Let's hope one of them will deliver.

Re: I can't commit a simple change

[Stefan Monnier]

>> In that case, this is a one-off problem and removing and ignoring the
>> info directory entirely was the right fix.
> Like I said, there have been plenty of these one-off problems.  One
> might suspect that they aren't one-offs any more.

AFAIK there's been various one-off problems over the last few years, but
it's pretty hard to know how we could automatically resolve these
issues.  There might be some opportunity, indeed, but someone would have
to take a deep look at those problems, and try to find some common theme
maybe.  Seen from where I stand, each one was different.


        Stefan

Re: I can't commit a simple change

[Stephen J. Turnbull]

Lars Ingebrigtsen writes:
 > "Stephen J. Turnbull" <stephen@xemacs.org> writes:
 > 
 > > In that case, this is a one-off problem and removing and ignoring the
 > > info directory entirely was the right fix.
 > 
 > Like I said, there have been plenty of these one-off problems.

For you.  However, I haven't noticed a lot of reports though (not even
from you and Richard), and I do take an interest in any VCS problems
so I'd be likely to notice.

 > One might suspect that they aren't one-offs any more.

As I wrote before, if you aren't making unrecorded changes to the tree
(that includes personal notes files, apply-later-maybe patches, and
things like that), then "rm `bzr ls -u`; bzr revert" is safe and DWYWs.
But as Eli points out, Make cannot know that that is safe and
shouldn't do it.

Now, what I know is that a lot of developers just don't want to learn
anything about VCS (or, in less virulent form, about a VCS different
from the one they prefer) or change their workflows so that it is a
benefit rather than a cost.  Nothing wrong with that, but what I
suspect is that that is the root conflict here.

It caused the people who did the original import of the CVS repo to
Bazaar to mirror practices that are mostly benign in "collection of
files" VCSes like CVS (ie, keeping build products in the repo) but are
inconvenient or even malignant in a "tree-oriented" VCS.  If the build
products had been removed from the repo at the very beginning, nobody
ever would have had a problem (the repo would already be in good state
when the first checkout was done).

To the extent that the problems you are reporting are due to that,
each one is one-off, each one is a real bug (there is a tendency to
forget to commit build products and for various reasons the default
Emacs build doesn't build everything, so the built Emacs gets out of
sync with the sources, and should be fixed (by removing them from the
VCS registry), not papered over with additional build complexity --
which as Eli points out, won't work anyway because there are still
conflicts lurking every time you try to do anything that changes the
repo.

I admit I wouldn't have predicted the especially pernicious effect of
"bzr remove"-ing the info directory (turning it into an unregistered
but precious object because it *contains* build products, and then
that conflicts with the VCS unregister -- this wouldn't happen with
git because directories are not first-class objects in git:
unregistering "info/dir" would also effectively unregister "info").

AIUI, turning info/dir into a generated file was a developer decision,
and maybe it was a bad idea (I'm not familiar enough with Emacs's
build process to express an opinion one way of the other).  Once that
decision was made, however, the file and the directory needed to be
"bzr remove"-ed in order to *prevent* future conflicts.  The "rm `bzr
ls -u`; bzr revert" incantation is what is needed to guarantee a clean
tree.  In this case you could restrict to  "rm `bzr ls -u info`; bzr
revert" IIUC the Bazaar docs.