23 branch - can't push - lock

23 branch - can't push - lock

[David Reitter]

Can't push to the 23 branch - there's a lock.  Any suggestions?

The usual bzr comment:   This is not the only weird problem I had with Bazaar, and I was just trying to push a change to make the 23 branch compile again.  It's been slow, I've seen unclear error messages, and now I can't push.  Oh man.  To say something positive at least, "bzr resolve" trumps git's manual "add" operations in terms of convenience.  


dr@elin:~/Projects/emacs/emacs-23 $ bzr push
Using saved push location: bzr://bzr.savannah.gnu.org/emacs/emacs-23
bzr: ERROR: Cannot lock LockDir(filtered-160255308:///emacs/emacs-23/.bzr/branch/lock): Permission denied: "/srv/bzr/emacs/emacs-23/.bzr/branch/lock/lufitpwgqo.tmp": [Errno 13] Permission denied: '/srv/bzr/emacs/emacs-23/.bzr/branch/lock/lufitpwgqo.tmp'


dr@elin:~/Projects/emacs/emacs-23 $ bzr --version
Bazaar (bzr) 2.3.1
  Python interpreter: /usr/bin/python 2.6.1
  Python standard library: /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6
  Platform: Darwin-10.7.0-i386-64bit
  bzrlib: /Library/Python/2.6/site-packages/bzrlib
  Bazaar configuration: /Users/dr/.bazaar
  Bazaar log file: /Users/dr/.bzr.log

Copyright 2005-2010 Canonical Ltd.
http://bazaar.canonical.com/

bzr comes with ABSOLUTELY NO WARRANTY.  bzr is free software, and
you may use, modify and redistribute it under the terms of the GNU
General Public License version 2 or later.

Bazaar is part of the GNU Project to produce a free operating system.

Re: 23 branch - can't push - lock

[David Reitter]

On Jun 16, 2011, at 11:49 AM, David Reitter wrote:

> Can't push to the 23 branch - there's a lock.  Any suggestions?

> Using saved push location: bzr://bzr.savannah.gnu.org/emacs/emacs-23
> bzr: ERROR: Cannot lock LockDir(filtered-160255308:///emacs/emacs-23/.bzr/branch/lock): Permission denied: "/srv/bzr/emacs/emacs-23/.bzr/branch/lock/lufitpwgqo.tmp": [Errno 13] Permission denied: '/srv/bzr/emacs/emacs-23/.bzr/branch/lock/lufitpwgqo.tmp'

Looks like it's a permissions error rather than one concerning just the lock file.

In case any of you low-time committers run into the same problem, use the bzr+ssh protocol:

bzr push bzr+ssh://USERNAME@bzr.savannah.gnu.org/emacs/emacs-23

That worked.

Re: 23 branch - can't push - lock

[Eli Zaretskii]

> From: David Reitter <david.reitter@gmail.com>
> Date: Thu, 16 Jun 2011 11:49:05 -0400
> 
> Can't push to the 23 branch - there's a lock.  Any suggestions?

Yes, the obvious: "bzr break-lock".

But someone already did, because it isn't locked.  Maybe it's a local
lock on your machine, in which case "bzr break-lock" will fix that.

> The usual bzr comment:   This is not the only weird problem I had with Bazaar, and I was just trying to push a change to make the 23 branch compile again.  It's been slow, I've seen unclear error messages, and now I can't push.  Oh man.

What is your network bandwidth?  It's quite fast for me (I have 3.5Mbps).

  To say something positive at least, "bzr resolve" trumps git's manual "add" operations in terms of convenience.  

> dr@elin:~/Projects/emacs/emacs-23 $ bzr push
> Using saved push location: bzr://bzr.savannah.gnu.org/emacs/emacs-23
> bzr: ERROR: Cannot lock LockDir(filtered-160255308:///emacs/emacs-23/.bzr/branch/lock): Permission denied: "/srv/bzr/emacs/emacs-23/.bzr/branch/lock/lufitpwgqo.tmp": [Errno 13] Permission denied: '/srv/bzr/emacs/emacs-23/.bzr/branch/lock/lufitpwgqo.tmp'

That _is_ a local lock on your machine.  Did you Ctrl-C in the middle
of a commit or something?

Re: 23 branch - can't push - lock

[David Reitter]

On Jun 16, 2011, at 1:27 PM, Eli Zaretskii wrote:
> Yes, the obvious: "bzr break-lock".

I see... thanks.
>> 
>> dr@elin:~/Projects/emacs/emacs-23 $ bzr push
>> Using saved push location: bzr://bzr.savannah.gnu.org/emacs/emacs-23
>> bzr: ERROR: Cannot lock LockDir(filtered-160255308:///emacs/emacs-23/.bzr/branch/lock): Permission denied: "/srv/bzr/emacs/emacs-23/.bzr/branch/lock/lufitpwgqo.tmp": [Errno 13] Permission denied: '/srv/bzr/emacs/emacs-23/.bzr/branch/lock/lufitpwgqo.tmp'
> 
> That _is_ a local lock on your machine.  Did you Ctrl-C in the middle
> of a commit or something?

How do I tell it's a local lock?  I don't have /srv/bzr.  I thought that was on the server.  No, I didn't do C-c.  I have since pushed to bzr+ssh, so perhaps the problem was one of authentication, and really just permissions on the server.



>> The usual bzr comment:   This is not the only weird problem I had with Bazaar, and I was just trying to push a change to make the 23 branch compile again.  It's been slow, I've seen unclear error messages, and now I can't push.  Oh man.
> 
> What is your network bandwidth?  It's quite fast for me (I have 3.5Mbps).

I think we've got 800Mbps lines and we're members of Internet2.  My local ethernet connection is 100Mbps.  I think it's fast enough.  Besides, it's local operations that are slow.  "bzr log" isn't as awfully slow as it once was, but it still interrupts the workflow.

dr@elin:~/Projects/emacs/emacs-23 $ time bzr log >/dev/null
real    0m33.780s 

First page only:  
real    0m3.594s  [faster when repeated,i.e., in cache]


dr@elin:~/em23 emacs23$ time git log >/dev/null
real    0m5.427s

First page only:
real    0m0.810s  [faster when cached]

The merge of a single revision (-c) also took a long time.

I've updated bzr to the latest version.

FWIW,

dr@elin:~/Projects/emacs/emacs-23 $ bzr info
Repository tree (format: 2a)
Location:
  shared repository: /Users/dr/Projects/emacs
  repository branch: .

Related branches:
    push branch: bzr://bzr.savannah.gnu.org/emacs/emacs-23
  parent branch: bzr://bzr.savannah.gnu.org/emacs/emacs-23
  submit branch: bzr://bzr.savannah.gnu.org/emacs/emacs-23

Re: 23 branch - can't push - lock

[Eli Zaretskii]

> From: David Reitter <david.reitter@gmail.com>
> Date: Thu, 16 Jun 2011 13:44:46 -0400
> Cc: emacs-devel@gnu.org
> 
> How do I tell it's a local lock?  I don't have /srv/bzr.

Then I was mistaken, and it's not local.  I assume that when you said
"there's a lock", you had evidence there indeed was a lock.

> I thought that was on the server.  No, I didn't do C-c.  I have since pushed to bzr+ssh, so perhaps the problem was one of authentication, and really just permissions on the server.

Maybe.  I always use bzr+ssh.

> dr@elin:~/Projects/emacs/emacs-23 $ time bzr log >/dev/null
> real    0m33.780s 

Why in the world would you need the log of all the 104K revisions?

Btw, I suggest to set up a bzr alias so that "log" is actually 
"log --line", it's much more concise and faster, too.

> First page only:  
> real    0m3.594s  [faster when repeated,i.e., in cache]

What kind of machine is that?  On my 6-year-old Windows box with a
single 3 GHz core, I get this:

  D:\gnu\bzr\emacs\trunk>timep bzr --no-plugins log -l40 > nul

  real    00h00m00.437s
  user    00h00m00.343s
  sys     00h00m00.078s

  D:\gnu\bzr\emacs\trunk>timep bzr log -l40 > nul

  real    00h00m00.812s
  user    00h00m00.500s
  sys     00h00m00.281s

And anyway, 3.5 sec is hardly significantly different from 0.8, for
producing something that a human needs to _read_.

> dr@elin:~/em23 emacs23$ time git log >/dev/null
> real    0m5.427s

Irrelevant.  But OTOH, this is _very_ relevant for my work on the
display engine:

  D:\gnu\bzr\emacs\trunk>timep bzr annotate src/xdisp.c > nul

  real    00h01m23.421s
  user    00h01m18.734s
  sys     00h00m03.046s

  eliz@fencepost:~/git/emacs$ time git annotate src/xdisp.c > /dev/null

  real    8m28.865s
  user    7m45.490s
  sys     0m6.090s

(The second timing is from fencepost.gnu.org, a multicore x86_64
server that is much faster than what I have here.)

> The merge of a single revision (-c) also took a long time.

How long is "long" in this case?  (You can look it up in your
~/.bzr.log file, it logs the time and network throughput/speed of each
command, something I've searched high and low in git and couldn't
find.)

Re: 23 branch - can't push - lock

[Glenn Morris]

David Reitter wrote:

> Looks like it's a permissions error rather than one concerning just
> the lock file.

Yes; you were trying to commit anonymously.

> In case any of you low-time committers run into the same problem, use
> the bzr+ssh protocol:
>
> bzr push bzr+ssh://USERNAME@bzr.savannah.gnu.org/emacs/emacs-23

This is written in the instructions on:

http://savannah.gnu.org/bzr/?group=emacs

Re: 23 branch - can't push - lock

[David Reitter]

On Jun 16, 2011, at 4:09 PM, Eli Zaretskii wrote:
>> First page only:  
>> real    0m3.594s  [faster when repeated,i.e., in cache]
> 
> What kind of machine is that?  On my 6-year-old Windows box with a
> single 3 GHz core, I get this:

Core i7, 2.6GHz, 4GB RAM. 

> And anyway, 3.5 sec is hardly significantly different from 0.8, for
> producing something that a human needs to _read_.

Actually, I disagree.  A modern interface should not make the user wait that long - I would estimate anything beyond 30ms is discernible, and anything beyond 1s may be seen as interrupting someone's workflow.
At some point, people (perhaps including you) did an awesome job making Emacs start up fast.   I have had many complaints from Aquamacs users because it didn't start up as fast (loading a few more libraries, etc) - and we're talking 3-4 seconds here.

Just getting the first pages of a log should happen instantly - this is an operation one does all the time.  I'd find the timing you get acceptable, but mine is just sluggish.
(And I agree, vc-annotate should also be reasonably fast, even though I'd accept a few seconds delay here).


>> The merge of a single revision (-c) also took a long time.
> 
> How long is "long" in this case?  (You can look it up in your
> ~/.bzr.log file, it logs the time and network throughput/speed of each
> command, something I've searched high and low in git and couldn't
> find.)

I started out with the wrong command, "bzr merge -c 104024", because I thought that the revision ID is unique (sorry, Git thinking).  It took 65 seconds to give me an error message!
And the wording of the error message wasn't even very user-level:

InvalidRevisionSpec: Requested revision: u'104024' does not exist in branch: BzrBranch7('file:///Users/dr/Projects/emacs/emacs-23/')

Then I did the right thing, and it took 112 seconds (see below).

> Thu 2011-06-16 11:07:41 -0400
> 0.070  bazaar version: 2.1.0
> 0.070  bzr arguments: [u'merge', u'-c', u'104024', u'../trunk']
> 0.084  looking for plugins in /Users/dr/.bazaar/plugins
> 0.138  looking for plugins in /Library/Python/2.6/site-packages/bzrlib/plugins
> 0.282  Returning RevisionSpec RevisionSpec_before for before:104024
> 0.283  encoding stdout as sys.stdout encoding 'us-ascii'
> 0.396  opening working tree '/Users/dr/Projects/emacs/emacs-23'
> [64363] 2011-06-16 11:09:25.819 INFO:  M  src/ChangeLog
> [64363] 2011-06-16 11:09:25.885 INFO:  M  src/nsmenu.m
> [64363] 2011-06-16 11:09:25.885 WARNING: Text conflict in src/ChangeLog
> [64363] 2011-06-16 11:09:25.885 WARNING: Text conflict in src/nsmenu.m
> [64363] 2011-06-16 11:09:31.749 INFO: 2 conflicts encountered.
> 112.112  Transferred: 0KiB (0.0K/s r:0K w:0K)
> 112.112  return code 1

I updated Bazaar after that to 2.3.1, and did the same merge a second time (this one may be much easier for Bzr now - I don't know).
It still took 20 seconds.    That's sluggish in my book.  
Is there a way to reset a branch to a previous commit, i.e., the equivalent of "git reset --hard"?  Then I could test this better.

With --no-plugins, the time is cut in half.  Better, but still sluggish.  I'll see if I can eliminate some plugins if they hurt performance so much.  Thanks for this hint.

Re: 23 branch - can't push - lock

[Eli Zaretskii]

> From: David Reitter <david.reitter@gmail.com>
> Date: Fri, 17 Jun 2011 09:57:43 -0400
> Cc: emacs-devel@gnu.org
> 
> On Jun 16, 2011, at 4:09 PM, Eli Zaretskii wrote:
> >> First page only:  
> >> real    0m3.594s  [faster when repeated,i.e., in cache]
> > 
> > What kind of machine is that?  On my 6-year-old Windows box with a
> > single 3 GHz core, I get this:
> 
> Core i7, 2.6GHz, 4GB RAM. 

Strange.  It should be faster than mine.

> > And anyway, 3.5 sec is hardly significantly different from 0.8, for
> > producing something that a human needs to _read_.
> 
> Actually, I disagree.

If you see 3.5 sec as a significant delay, I wonder what's your
opinion about GCC compiling Emacs sources.  And in case of "bzr log"
the results need to be read by you, which will certainly take more
than just 3 seconds.

> A modern interface should not make the user wait that long - I would estimate anything beyond 30ms is discernible

How do you mean "discernible"?  Most humans are generally unable to
react in less than 200-300ms, so 30ms is an order of magnitude off the
mark.

> and anything beyond 1s may be seen as interrupting someone's workflow.

Typing a command takes more than 1s, so I guess your workflow is being
interrupted all the time, and you are used to it anyway ;-)

> At some point, people (perhaps including you) did an awesome job making Emacs start up fast.

I only start my Emacs once in several weeks, so the startup (which is
quite long, actually, because my sessions visit a lot of files)
doesn't bother me more than the time it takes the entire machine to
come up.

> Just getting the first pages of a log should happen instantly

Well, it takes less than 1s here, which is instant enough for me.

> I'd find the timing you get acceptable, but mine is just sluggish.

You should investigate the reasons for that sluggishness, then.

> I started out with the wrong command, "bzr merge -c 104024", because I thought that the revision ID is unique (sorry, Git thinking).  It took 65 seconds to give me an error message!

How about submitting a bug report (https://bugs.launchpad.net/bzr)
about that?

> And the wording of the error message wasn't even very user-level:
> 
> InvalidRevisionSpec: Requested revision: u'104024' does not exist in branch: BzrBranch7('file:///Users/dr/Projects/emacs/emacs-23/')

Is that what was displayed on the screen, or is it from .bzr.log?  The
latter is not only for users, so I would tolerate some technicalities
there for the sake of technical information the developers will want
to know.

> I updated Bazaar after that to 2.3.1, and did the same merge a second time (this one may be much easier for Bzr now - I don't know).
> It still took 20 seconds.    That's sluggish in my book.  

Most of my merges take less than 5 sec, but some took 20 or 30s.  I
don't consider that to be sluggish, but if you don't mind to live on
the edge, try the latest beta of bzr 2.4, I understand that "merge"
got a significant speedup there.

> Is there a way to reset a branch to a previous commit, i.e., the equivalent of "git reset --hard"?

That's git talk, and I don't really know what it means.  If you want
to be able to re-apply the same merge, I think you want "bzr
uncommit".  (But don't try that with a bound branch!)  Alternatively,
make a new branch from the revision before the merge, and then merge
onto that.

Re: 23 branch - can't push - lock

[Stefan Monnier]

> Just getting the first pages of a log should happen instantly - this is an
> operation one does all the time.  I'd find the timing you get acceptable,
> but mine is just sluggish.

Can you move this discussion elsewhere?
AFAIK Emacs can't make this operation faster (tho maybe we could create
a new branch and merge from trunk to that new branch (so the current
revision would become revision 2 or so), and hope that Bzr becomes
snappier because there are fewer revisions).


        Stefan

UI / reaction time - was: Re: 23 branch - can't push - lock

[David Reitter]

On Jun 17, 2011, at 11:39 AM, Eli Zaretskii wrote:
> How do you mean "discernible"?  Most humans are generally unable to
> react in less than 200-300ms, so 30ms is an order of magnitude off the
> mark.

Reaction requires a chain of not only perception and related encoding, but also planning possibly moderated by execute control, possibly memory access, motor planning and actual execution.  That takes time.  In some cases, after some reactions have been routinized, you can sidestep much of that, and react more quickly.  That said, not reactions are relevant, but change detection and perceived duration.   Current cognitive modeling frameworks assume a 50ms "clock" for cognitive processing (simple, IF-THEN rules, for instance, in Anderson's ACT-R), and this is where I derived this figure.  You may be right in that more than that  is needed to estimate relative duration, let alone be bothered by a wait.   Some people can perceive duration in a visual stimulus in around 100ms (a study by Efron in 1970 found 120ms as minimum).  Change detection seems to be similarly quick when subjects attend to the location (Pashler 1988).

We may both be missing the point with this, though.  At around 1s, isn't it much more about what users "feel" is fast or a delay, as opposed to whether they can make use of the actual speed-up?

The figures you obtained (less than 1 sec) seem OK to me (subjectively), while the pauses I experienced (several seconds) seemed unpleasant.  They were at a larger scale than that.

Thanks for the hints regarding speeding up bzr and undoing a commit.  I'll leave that portion of the discussion, as it has little to do with Emacs, as Stefan pointed out.

Re: UI / reaction time - was: Re: 23 branch - can't push - lock

[chad]

User testing continues to suggest that human reactions work very roughly on decimal seconds -- .1 seconds is `instant', 1 second is `fast', and 10 seconds is `slow'.  There are obvious outliers, but this guideline survived from (at least) the early Apple UI tests through ~2001, when I stopped paying attention -- but I'll be surprised if it's changed much.

To the point, bazaar/git continues to be much faster than git/bazaar for some operations and much slower for others.  Neither is fast at all of the `important' operations for all developers, and the architecture of each suggests that these things are unlikely to change (although we can hope for improvements to both, over time).

Hope that helps,
*Chad

Re: 23 branch - can't push - lock

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > > Is there a way to reset a branch to a previous commit, i.e., the equivalent of "git reset --hard"?
 > 
 > That's git talk, and I don't really know what it means.

It means moving the tip pointer of the current branch to point at a
different revision (usually older, but not necessarily so), and
simultaneously checkout the workspace corresponding to that revision.

The answer is yes, Robert Collins told me how to do it about three
years ago, but in vanilla bzr there was no simple equivalent to

    git tag tmp
    git reset --hard somewhere-else

so the bzr equivalent was necessarily destructive unless you extract
the rev-id and save it away by hand.  (Of course this is scriptable.)
I found that sufficiently painful that I just avoid the need, and
don't recall that necessary arcana.

Depending on the use case there are various commands that have the
same effect that "git --reset hard" does, but none of them work across
the board.  Look up the docs for update, revert, and uncommit.

I would assume that colo-enabled versions of bazaar have something
similar to git reset, though, and if not it shouldn't be hard to
implement.