`bzr viz' slooow

`bzr viz' slooow

[Werner LEMBERG]

The most valuable tool of git is `gitk' to visualize changes together
with the corresponding tree.  The pendant is `bzr viz', using the
`gtk' plugin for bazaar.  However, it is very slow.  Calling this
command for the emacs trunk takes five minutes to start on my laptop
with a a dual-core 1Ghz CPU having 1GByte of memory.[*] I know, thou
shalt not compare the holy bzr promoted by God himself against the
devilish git, however, I did just that: Calling it on the current git
repository of the linux 2.6 kernel (with approx. 180000 commits,
almost twice as much commits as for emacs), it only takes less than a
minute to build (and display) the tree.

Sigh.


    Werner


[*] I know how to limit the viewed commits so that it becomes faster,
    but this is not the point of this rant.

Re: `bzr viz' slooow

[Eli Zaretskii]

> Date: Tue, 26 Jan 2010 18:05:54 +0100 (CET)
> From: Werner LEMBERG <wl@gnu.org>
> 
> 
> The most valuable tool of git is `gitk' to visualize changes together
> with the corresponding tree.  The pendant is `bzr viz', using the
> `gtk' plugin for bazaar.  However, it is very slow.  Calling this
> command for the emacs trunk takes five minutes to start on my laptop
> with a a dual-core 1Ghz CPU having 1GByte of memory.

What kind of branch is your trunk?  Is it according to the workflow in
the wiki page (i.e., a bound branch with local repository) or
something else?

Regardless, look in your .bzr.log file and tell us what part takes
most of the time in your frequent operations.  Each line in the log
has a time stamp, so it's easy to see the sinks.

Re: `bzr viz' slooow

[Óscar Fuentes]

There are several similar tools for bzr, with a varying degree of
feature-completeness. Perhaps the most simple is `bzr qlog', from the
qbzr plugin. Bazaar Explorer is, possibly, the most complete.

qlog takes 20 seconds to show the graph on a 2.4 GHz machine
(the number of cores is not very relevant, as the process is
single-threaded). In any case I wouldn't expect to match the performance
of git, though. BTW, "less than a minute" qualifies as an slow operation
for me, so git slow, bzr is sloow (note just one extra `o')

Werner LEMBERG <wl@gnu.org> writes:

[snip]

Re: `bzr viz' slooow

[Werner LEMBERG]

>> The most valuable tool of git is `gitk' to visualize changes
>> together with the corresponding tree.  The pendant is `bzr viz',
>> using the `gtk' plugin for bazaar.  However, it is very slow.
>> Calling this command for the emacs trunk takes five minutes to
>> start on my laptop with a a dual-core 1Ghz CPU having 1GByte of
>> memory.
>
> What kind of branch is your trunk?

I've no idea.  How can I find it out?  Below the `emacs' directory I
have a `.bzr' directory with about 275MByte and a `trunk' directory
with about 116MByte.  If I do a `bzr pull' I change into `trunk'.  The
same for `bzr viz'.

BTW, `bzr info' within `trunk' gives

  Repository tree (format: 2a)
  Location:
    shared repository: /home/wl/bzr/emacs
    repository branch: .

  Related branches:
    parent branch: sftp://wl@bzr.savannah.gnu.org/srv/bzr/emacs/trunk/

> Is it according to the workflow in the wiki page (i.e., a bound
> branch with local repository) or something else?

I checked it out about a month ago; at that time those infos weren't
available at all...

> Regardless, look in your .bzr.log file and tell us what part takes
> most of the time in your frequent operations.  Each line in the log
> has a time stamp, so it's easy to see the sinks.

Do you mean this?

  0.048  bzr arguments: [u'viz']
  0.066  looking for plugins in /home/wl/.bazaar/plugins
  0.066  looking for plugins in /usr/lib/python2.6/site-packages/bzrlib/plugins
  0.161  encoding stdout as sys.stdout encoding 'UTF-8'
  0.554  Didn't find search plugin
  491.534  return code 0

This doesn't give any sensible information IMHO since apparently the
subprocesses of `bzr viz' don't write any log data into this file,
which is a pity.


    Werner

Re: `bzr viz' slooow

[Werner LEMBERG]

> In any case I wouldn't expect to match the performance of git,
> though.

Well...

> BTW, "less than a minute" qualifies as an slow operation for
> me, so git slow,

BUT: gitk displays something useful (the most recent commits) after a
few seconds, and I can start to inspect the data; apparently a
background process collects the remaining commit entries.  It takes
about a minute if you press the `End' button to see the very first
commit in the whole repository.  On the other hand, `bzr viz' is
unusable for really five minutes.

> bzr is sloow (note just one extra `o')

In the light of my previous paragraph, I strongly disagree.  bzr is
sloooow (note the four `o's); it takes 50 times more time until I can
actually do something.


    Werner

Re: `bzr viz' slooow

[Dan Nicolaescu]

Werner LEMBERG <wl@gnu.org> writes:

  > The most valuable tool of git is `gitk' to visualize changes together
  > with the corresponding tree.  The pendant is `bzr viz', using the
  > `gtk' plugin for bazaar.  However, it is very slow.  Calling this

You can also use C-x v L

Re: `bzr viz' slooow

[Óscar Fuentes]

Werner LEMBERG <wl@gnu.org> writes:

[snip]

> BUT: gitk displays something useful (the most recent commits) after a
> few seconds, and I can start to inspect the data; apparently a
> background process collects the remaining commit entries.

AFAIK, there is no reason preventing qlog from doing that. So file a bug
report on qbzr explaining the problem and let's hope that they'll
implement the feature.

[snip]

Re: `bzr viz' slooow

[Eli Zaretskii]

> Date: Tue, 26 Jan 2010 23:07:24 +0100 (CET)
> Cc: emacs-devel@gnu.org
> From: Werner LEMBERG <wl@gnu.org>
> 
> >> The most valuable tool of git is `gitk' to visualize changes
> >> together with the corresponding tree.  The pendant is `bzr viz',
> >> using the `gtk' plugin for bazaar.  However, it is very slow.
> >> Calling this command for the emacs trunk takes five minutes to
> >> start on my laptop with a a dual-core 1Ghz CPU having 1GByte of
> >> memory.
> >
> > What kind of branch is your trunk?
> 
> I've no idea.  How can I find it out?

With "bzr info", if with nothing else, like you did.

> > Regardless, look in your .bzr.log file and tell us what part takes
> > most of the time in your frequent operations.  Each line in the log
> > has a time stamp, so it's easy to see the sinks.
> 
> Do you mean this?
> 
>   0.048  bzr arguments: [u'viz']
>   0.066  looking for plugins in /home/wl/.bazaar/plugins
>   0.066  looking for plugins in /usr/lib/python2.6/site-packages/bzrlib/plugins
>   0.161  encoding stdout as sys.stdout encoding 'UTF-8'
>   0.554  Didn't find search plugin
>   491.534  return code 0
> 
> This doesn't give any sensible information IMHO since apparently the
> subprocesses of `bzr viz' don't write any log data into this file,
> which is a pity.

Yes, a pity.  So my guess would be that "bzr viz" tries to produce the
full DAG of the branch history, which is expensive.  Does it have any
options equivalent to the -n option of "log", which limits the depth
of the graph?

Re: `bzr viz' slooow

[Ken Hori]

Isn't this the problem of the bzr-gtk plugin and how it is
implemented? I think it's more productive and appropriate to file a
bug against it. (btw, my machine took about 57 sec and I'm fine with
that.)

On Tue, Jan 26, 2010 at 9:05 AM, Werner LEMBERG <wl@gnu.org> wrote:
>
> The most valuable tool of git is `gitk' to visualize changes together
> with the corresponding tree.  The pendant is `bzr viz', using the
> `gtk' plugin for bazaar.  However, it is very slow.  Calling this
> command for the emacs trunk takes five minutes to start on my laptop
> with a a dual-core 1Ghz CPU having 1GByte of memory.[*] I know, thou
> shalt not compare the holy bzr promoted by God himself against the
> devilish git, however, I did just that: Calling it on the current git
> repository of the linux 2.6 kernel (with approx. 180000 commits,
> almost twice as much commits as for emacs), it only takes less than a
> minute to build (and display) the tree.
>
> Sigh.
>
>
>    Werner
>
>
> [*] I know how to limit the viewed commits so that it becomes faster,
>    but this is not the point of this rant.
>
>
>

Re: `bzr viz' slooow

[Werner LEMBERG]

> Isn't this the problem of the bzr-gtk plugin and how it is
> implemented?

Maybe.  However, it is quite generic and basically independent of the
version of bzr.  In other words, it uses calls to the bzr
infrastructure to get the data.

> I think it's more productive and appropriate to file a bug against
> it.

Will do, of course.

> (btw, my machine took about 57 sec and I'm fine with that.)

Either your machine is much faster than mine, or you have `light'
checkout.


    Werner

Re: `bzr viz' slooow

[Werner LEMBERG]

> So my guess would be that "bzr viz" tries to produce the full DAG of
> the branch history, which is expensive.

It seems so.

> Does it have any options equivalent to the -n option of "log", which
> limits the depth of the graph?

Not in version 0.97.


    Werner

Re: `bzr viz' slooow

[Werner LEMBERG]

>> BUT: gitk displays something useful (the most recent commits) after
>> a few seconds, and I can start to inspect the data; apparently a
>> background process collects the remaining commit entries.
> 
> AFAIK, there is no reason preventing qlog from doing that. So file a
> bug report on qbzr explaining the problem and let's hope that
> they'll implement the feature.

I'm not using qbzr.


    Werner

Re: `bzr viz' slooow

[Óscar Fuentes]

Werner LEMBERG <wl@gnu.org> writes:

>> Isn't this the problem of the bzr-gtk plugin and how it is
>> implemented?
>
> Maybe.  However, it is quite generic and basically independent of the
> version of bzr.  In other words, it uses calls to the bzr
> infrastructure to get the data.

And why do you think that it is bzr the main culprit? Haven't you seen
the timings about qlog, which does the same thing but much faster? Have
you forgotten about the possibility of displaying the graph at the same
time that the data is collected, the same way gitk does? Or is this
about blaming bzr and make it look bad?

>> I think it's more productive and appropriate to file a bug against
>> it.
>
> Will do, of course.

Great, because the biggest slowdown by far is in viz, not in bzr.

>> (btw, my machine took about 57 sec and I'm fine with that.)
>
> Either your machine is much faster than mine, or you have `light'
> checkout.

Actually, building the graph on a lightweight checkout of a remote
branch will require *much* more time.

Re: `bzr viz' slooow

[Eli Zaretskii]

> Date: Wed, 27 Jan 2010 07:14:23 +0100 (CET)
> Cc: emacs-devel@gnu.org
> From: Werner LEMBERG <wl@gnu.org>
> 
> 
> > So my guess would be that "bzr viz" tries to produce the full DAG of
> > the branch history, which is expensive.
> 
> It seems so.

There's the global option --lsprof-file=FILE that all bzr commands
accept.  Running a command with that option will produce the profile
data on the named FILE.  If nothing else, that file can be submitted
to the maintainers of the plugin to show them where it spends most of
the time.  But maybe look into it first (it's a text file) for hints
and ideas.

Re: `bzr viz' slooow

[Werner LEMBERG]

>> Maybe.  However, it is quite generic and basically independent of
>> the version of bzr.  In other words, it uses calls to the bzr
>> infrastructure to get the data.
> 
> And why do you think that it is bzr the main culprit?  Haven't you
> seen the timings about qlog, which does the same thing but much
> faster?

I'll try qlog.

> Have you forgotten about the possibility of displaying the graph at
> the same time that the data is collected, the same way gitk does?

I've suggested exactly that on the bzr-gtk mailing list meanwhile.

> Or is this about blaming bzr and make it look bad?

Not at all.  However, gtk and not qt is the primary graphical
environment recommended for GNU projects, right?  From a political
point of view, bzr-gtk should thus be the preferred tool...

>> Either your machine is much faster than mine, or you have `light'
>> checkout.
> 
> Actually, building the graph on a lightweight checkout of a remote
> branch will require *much* more time.

OK.


    Werner

Re: `bzr viz' slooow

[Werner LEMBERG]

> I'll try qlog.

Well, it is indeed much faster in displaying the log tree.  However,
it has different weaknesses: Clicking on a commit from a branch (I
mean the pink stuff like rev. 99322.1.8, however it is named) to see
the diff, I have to wait almost 30 seconds until it gets displayed.
Additionally, there doesn't seem to be a possibility to configure the
behaviour of the `diff' window (for example, I would like to always
start it in unidiff mode).


    Werner

Re: `bzr viz' slooow

[Werner LEMBERG]

> There's the global option --lsprof-file=FILE that all bzr commands
> accept.

Hmm.  `bzr viz' apparently doesn't, neither before nor after the
`viz'.  Actually, this option doesn't seem to exist at all in bzr
2.0.4...


    Werner

Re: `bzr viz' slooow

[Eli Zaretskii]

> Date: Wed, 27 Jan 2010 12:51:33 +0100 (CET)
> From: Werner LEMBERG <wl@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> 
> > I'll try qlog.
> 
> Well, it is indeed much faster in displaying the log tree.  However,
> it has different weaknesses: Clicking on a commit from a branch (I
> mean the pink stuff like rev. 99322.1.8, however it is named) to see
> the diff, I have to wait almost 30 seconds until it gets displayed.
> Additionally, there doesn't seem to be a possibility to configure the
> behaviour of the `diff' window (for example, I would like to always
> start it in unidiff mode).

Maybe Someone(TM) should simply add a version DAG display to Emacs's
VC.  (Maybe it even already exists, I don't know.)  Don't we have
enough infrastructure in tree-widget to make that possible?  Nothing
will ever be as customizable as Emacs.

Re: `bzr viz' slooow

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> Maybe Someone(TM) should simply add a version DAG display to Emacs's
> VC.  (Maybe it even already exists, I don't know.)  Don't we have
> enough infrastructure in tree-widget to make that possible?  Nothing
> will ever be as customizable as Emacs.

Some random thoughts on this:

The VC DAG is not a tree.

Some projects (i.e. Emacs) have a large number of nodes on their DAG (of
the order of hundreds of thousands.) A C++ extension like qlog, for
instance, uses 280 MB of RES memory on GNU/Linux x86_64 while displaying
the Emacs DAG.

It would be great to have that functionality on Emacs. It's VC-log on
steroids.

Re: `bzr viz' slooow

[Eli Zaretskii]

> From: Óscar_Fuentes <ofv@wanadoo.es>
> Date: Wed, 27 Jan 2010 13:27:45 +0100
> 
> The VC DAG is not a tree.

Nevertheless, I see no reason why tree-widget couldn't support some
reasonable display based on "bzr log".

> Some projects (i.e. Emacs) have a large number of nodes on their DAG (of
> the order of hundreds of thousands.) A C++ extension like qlog, for
> instance, uses 280 MB of RES memory on GNU/Linux x86_64 while displaying
> the Emacs DAG.

According to this:

   bzr log --line -n0 | wc

we had 106759 revisions 3 weeks ago (I don't have an up to date tree
where I type this).  To use up 280MB for that means 2KB per revision,
which sounds almost unbelievable.  Maybe I'm missing something
important.

> It would be great to have that functionality on Emacs. It's VC-log on
> steroids.

Sure.  Any volunteers?

Re: `bzr viz' slooow

[Eli Zaretskii]

> Date: Wed, 27 Jan 2010 13:01:02 +0100 (CET)
> Cc: emacs-devel@gnu.org
> From: Werner LEMBERG <wl@gnu.org>
> 
> 
> > There's the global option --lsprof-file=FILE that all bzr commands
> > accept.
> 
> Hmm.  `bzr viz' apparently doesn't, neither before nor after the
> `viz'.  Actually, this option doesn't seem to exist at all in bzr
> 2.0.4...

I find it hard to believe.  Maybe it's a bug in 2.0.4 that you should
report ASAP.  (I use 2.0.3, FWIW.)