* bzr + savannah.gnu.org slow again
@ 2011-02-26 6:21 Paul Eggert
2011-02-26 6:50 ` Paul Eggert
0 siblings, 1 reply; 9+ messages in thread
From: Paul Eggert @ 2011-02-26 6:21 UTC (permalink / raw)
To: Emacs Development; +Cc: Savannah-hackers-public
Savannah.gnu.org is very slow for bzr commits on the Emacs trunk,
with symptoms much like they were about 24 hours ago. Perhaps the
same thing is happening again?
My most recent attempt to commit is taking more than 20 minutes.
It's not done yet, so I don't know how long it will take exactly.
Here are the symptoms at the shell level:
$ bzr commit -m 'Merge: lib-src changes mostly to avoid GCC warnings'
Committing to: bzr+ssh://eggert@bzr.savannah.gnu.org/emacs/trunk/
modified lib-src/ChangeLog
modified lib-src/ebrowse.c
modified lib-src/emacsclient.c
modified lib-src/etags.c
modified lib-src/fakemail.c
modified lib-src/make-docfile.c
[ It's hanging here now. ]
Here is my ~/.bzr.log:
Fri 2011-02-25 21:53:48 -0800
0.073 bazaar version: 2.2.2
0.073 bzr arguments: [u'commit', u'-m', u'Merge: lib-src changes mostly to avoid GCC warnings']
0.096 looking for plugins in /u/cs/fac/eggert/.bazaar/plugins
0.096 looking for plugins in /usr/local/cs/Python-2.6.6/lib/python2.6/site-packages/bzrlib/plugins
0.117 encoding stdout as sys.stdout encoding 'UTF-8'
0.245 opening working tree '/w/fac.2/cs/eggert/src/gnu/emacs/trunk'
0.566 ssh implementation is OpenSSH
38.887 preparing to commit
[10929] 2011-02-25 21:54:36.575 INFO: Committing to: bzr+ssh://eggert@bzr.savannah.gnu.org/emacs/trunk/
47.711 Selecting files for commit with filter None
[10929] 2011-02-25 21:54:37.032 INFO: modified lib-src/ChangeLog
[10929] 2011-02-25 21:54:37.033 INFO: modified lib-src/ebrowse.c
[10929] 2011-02-25 21:54:37.033 INFO: modified lib-src/emacsclient.c
[10929] 2011-02-25 21:54:37.033 INFO: modified lib-src/etags.c
[10929] 2011-02-25 21:54:37.034 INFO: modified lib-src/fakemail.c
[10929] 2011-02-25 21:54:37.034 INFO: modified lib-src/make-docfile.c
48.761 Using fetch logic to copy between CHKInventoryRepository('file:///w/fac.2/cs/eggert/src/gnu/emacs/.bzr/repository/')(RepositoryFormat2a()) and RemoteRepository(bzr+ssh://eggert@bzr.savannah.gnu.org/emacs/.bzr/)(RemoteRepositoryFormat(_network_name='Bazaar repository format 2a (needs bzr 1.16 or later)\n'))
48.761 fetch up to rev {eggert@cs.ucla.edu-20110226055436-mfvjfrh702e68yqk}
142.288 stripping trailing bytes from groupcompress block 627289 => 258611
It is now Fri Feb 25 22:20:20 PST 2011, so it's about
27 minutes and counting.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: bzr + savannah.gnu.org slow again
2011-02-26 6:21 bzr + savannah.gnu.org slow again Paul Eggert
@ 2011-02-26 6:50 ` Paul Eggert
2011-02-26 7:54 ` Eli Zaretskii
0 siblings, 1 reply; 9+ messages in thread
From: Paul Eggert @ 2011-02-26 6:50 UTC (permalink / raw)
To: Emacs Development; +Cc: Savannah-hackers-public
Following up on my previous message, my commit to the Emacs trunk
took over 50 minutes (!).
Here's the finished .bzr.log entry:
Fri 2011-02-25 21:53:48 -0800
0.073 bazaar version: 2.2.2
0.073 bzr arguments: [u'commit', u'-m', u'Merge: lib-src changes mostly to avoid GCC warnings']
0.096 looking for plugins in /u/cs/fac/eggert/.bazaar/plugins
0.096 looking for plugins in /usr/local/cs/Python-2.6.6/lib/python2.6/site-packages/bzrlib/plugins
0.117 encoding stdout as sys.stdout encoding 'UTF-8'
0.245 opening working tree '/w/fac.2/cs/eggert/src/gnu/emacs/trunk'
0.566 ssh implementation is OpenSSH
38.887 preparing to commit
[10929] 2011-02-25 21:54:36.575 INFO: Committing to: bzr+ssh://eggert@bzr.savannah.gnu.org/emacs/trunk/
47.711 Selecting files for commit with filter None
[10929] 2011-02-25 21:54:37.032 INFO: modified lib-src/ChangeLog
[10929] 2011-02-25 21:54:37.033 INFO: modified lib-src/ebrowse.c
[10929] 2011-02-25 21:54:37.033 INFO: modified lib-src/emacsclient.c
[10929] 2011-02-25 21:54:37.033 INFO: modified lib-src/etags.c
[10929] 2011-02-25 21:54:37.034 INFO: modified lib-src/fakemail.c
[10929] 2011-02-25 21:54:37.034 INFO: modified lib-src/make-docfile.c
48.761 Using fetch logic to copy between CHKInventoryRepository('file:///w/fac.2/cs/eggert/src/gnu/emacs/.bzr/repository/')(RepositoryFormat2a()) and RemoteRepository(bzr+ssh://eggert@bzr.savannah.gnu.org/emacs/.bzr/)(RemoteRepositoryFormat(_network_name='Bazaar repository format 2a (needs bzr 1.16 or later)\n'))
48.761 fetch up to rev {eggert@cs.ucla.edu-20110226055436-mfvjfrh702e68yqk}
142.288 stripping trailing bytes from groupcompress block 627289 => 258611
[10929] 2011-02-25 22:43:50.604 INFO: Committed revision 103424.
3001.740 Committed revid eggert@cs.ucla.edu-20110226055436-mfvjfrh702e68yqk as revno 103424.
3002.897 return code 0
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: bzr + savannah.gnu.org slow again
2011-02-26 6:50 ` Paul Eggert
@ 2011-02-26 7:54 ` Eli Zaretskii
2011-02-26 8:13 ` [Savannah-hackers-public] " Jim Meyering
0 siblings, 1 reply; 9+ messages in thread
From: Eli Zaretskii @ 2011-02-26 7:54 UTC (permalink / raw)
To: Paul Eggert; +Cc: Savannah-hackers-public, emacs-devel
> Date: Fri, 25 Feb 2011 22:50:33 -0800
> From: Paul Eggert <eggert@cs.ucla.edu>
> Cc: Savannah-hackers-public@gnu.org
>
> Following up on my previous message, my commit to the Emacs trunk
> took over 50 minutes (!).
I get "only" 5.5 minutes here, about 2 hours after Paul's commit:
331.016 Committed revid eliz@gnu.org-20110226074438-z9zfono0mhgzr4ri as revno 103425.
331.547 Transferred: 244kB (0.7kB/s r:137kB w:107kB)
But something's definitely wrong again. "bzr up" again took 3.5
minutes, which is about 4-5 times longer than it usually takes for me.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Savannah-hackers-public] Re: bzr + savannah.gnu.org slow again
2011-02-26 7:54 ` Eli Zaretskii
@ 2011-02-26 8:13 ` Jim Meyering
2011-02-26 8:49 ` Eli Zaretskii
0 siblings, 1 reply; 9+ messages in thread
From: Jim Meyering @ 2011-02-26 8:13 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: Paul Eggert, Savannah-hackers-public, emacs-devel
Eli Zaretskii wrote:
>> Date: Fri, 25 Feb 2011 22:50:33 -0800
>> From: Paul Eggert <eggert@cs.ucla.edu>
>> Cc: Savannah-hackers-public@gnu.org
>>
>> Following up on my previous message, my commit to the Emacs trunk
>> took over 50 minutes (!).
>
> I get "only" 5.5 minutes here, about 2 hours after Paul's commit:
>
> 331.016 Committed revid eliz@gnu.org-20110226074438-z9zfono0mhgzr4ri
> as revno 103425.
> 331.547 Transferred: 244kB (0.7kB/s r:137kB w:107kB)
>
> But something's definitely wrong again. "bzr up" again took 3.5
> minutes, which is about 4-5 times longer than it usually takes for me.
Load average was 150 -- mostly due to cgit.cgi processes.
I killed them, again. It may be robots that need to be told to go away.
FYI, now we're using the latest cgit: 0.8.3.4.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Savannah-hackers-public] Re: bzr + savannah.gnu.org slow again
2011-02-26 8:13 ` [Savannah-hackers-public] " Jim Meyering
@ 2011-02-26 8:49 ` Eli Zaretskii
2011-02-26 9:06 ` Eli Zaretskii
0 siblings, 1 reply; 9+ messages in thread
From: Eli Zaretskii @ 2011-02-26 8:49 UTC (permalink / raw)
To: Jim Meyering; +Cc: eggert, Savannah-hackers-public, emacs-devel
> From: Jim Meyering <jim@meyering.net>
> Cc: Paul Eggert <eggert@cs.ucla.edu>, Savannah-hackers-public@gnu.org, emacs-devel@gnu.org
> Date: Sat, 26 Feb 2011 09:13:50 +0100
>
> Eli Zaretskii wrote:
> >> Date: Fri, 25 Feb 2011 22:50:33 -0800
> >> From: Paul Eggert <eggert@cs.ucla.edu>
> >> Cc: Savannah-hackers-public@gnu.org
> >>
> >> Following up on my previous message, my commit to the Emacs trunk
> >> took over 50 minutes (!).
> >
> > I get "only" 5.5 minutes here, about 2 hours after Paul's commit:
> >
> > 331.016 Committed revid eliz@gnu.org-20110226074438-z9zfono0mhgzr4ri
> > as revno 103425.
> > 331.547 Transferred: 244kB (0.7kB/s r:137kB w:107kB)
> >
> > But something's definitely wrong again. "bzr up" again took 3.5
> > minutes, which is about 4-5 times longer than it usually takes for me.
>
> Load average was 150 -- mostly due to cgit.cgi processes.
> I killed them, again. It may be robots that need to be told to go away.
Thanks. I still get very slow commits, half an hour after you did it:
41.953 fetch up to rev {eliz@gnu.org-20110226083656-wtcszqlzmytktl8w}
[ 1288] 2011-02-26 10:43:20.466 INFO: Committed revision 103426.
393.406 Committed revid eliz@gnu.org-20110226083656-wtcszqlzmytktl8w as revno 103426.
393.812 Transferred: 307kB (0.8kB/s r:137kB w:170kB)
That's 6.5 minutes, with most of it sitting and waiting for the commit
to complete on the server.
So either the problem is back, or it takes bzr.savannah.gnu.org some
time to recover.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Savannah-hackers-public] Re: bzr + savannah.gnu.org slow again
2011-02-26 8:49 ` Eli Zaretskii
@ 2011-02-26 9:06 ` Eli Zaretskii
2011-02-28 8:03 ` Glenn Morris
0 siblings, 1 reply; 9+ messages in thread
From: Eli Zaretskii @ 2011-02-26 9:06 UTC (permalink / raw)
To: jim, eggert, Savannah-hackers-public, emacs-devel
> Date: Sat, 26 Feb 2011 10:49:33 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: eggert@cs.ucla.edu, Savannah-hackers-public@gnu.org, emacs-devel@gnu.org
>
> Thanks. I still get very slow commits, half an hour after you did it:
>
> 41.953 fetch up to rev {eliz@gnu.org-20110226083656-wtcszqlzmytktl8w}
> [ 1288] 2011-02-26 10:43:20.466 INFO: Committed revision 103426.
> 393.406 Committed revid eliz@gnu.org-20110226083656-wtcszqlzmytktl8w as revno 103426.
> 393.812 Transferred: 307kB (0.8kB/s r:137kB w:170kB)
>
> That's 6.5 minutes, with most of it sitting and waiting for the commit
> to complete on the server.
Btw, git.savannah.gnu.org is also affected: it took me a whopping 5
minutes to do a "git pull" (from gnulib) that said it's pulling 1837
objects.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Savannah-hackers-public] Re: bzr + savannah.gnu.org slow again
2011-02-26 9:06 ` Eli Zaretskii
@ 2011-02-28 8:03 ` Glenn Morris
2011-02-28 13:02 ` Jim Meyering
0 siblings, 1 reply; 9+ messages in thread
From: Glenn Morris @ 2011-02-28 8:03 UTC (permalink / raw)
To: Savannah-hackers-public; +Cc: Eli Zaretskii, eggert, jim, emacs-devel
At 23.30 PST on Sunday 27 Feb, it is very slow again. ~ 35 mins for a ~ 5
line commit. It was alright mid-afternoon PST.
Can you not just disable the cgit service (it seems to still be
available) until there is a solution for this problem? Being able to
commit is more important.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Savannah-hackers-public] Re: bzr + savannah.gnu.org slow again
2011-02-28 8:03 ` Glenn Morris
@ 2011-02-28 13:02 ` Jim Meyering
2011-02-28 17:54 ` Glenn Morris
0 siblings, 1 reply; 9+ messages in thread
From: Jim Meyering @ 2011-02-28 13:02 UTC (permalink / raw)
To: Glenn Morris; +Cc: Eli Zaretskii, Savannah-hackers-public, emacs-devel, eggert
Glenn Morris wrote:
> At 23.30 PST on Sunday 27 Feb, it is very slow again. ~ 35 mins for a ~ 5
> line commit. It was alright mid-afternoon PST.
>
> Can you not just disable the cgit service (it seems to still be
> available) until there is a solution for this problem? Being able to
> commit is more important.
Hi Glenn,
I've diagnosed and fixed the problem. git.sv is now running a new binary.
The trigger was a buggy spider that was provoking an infloop in cgit.
Once there were 15-20 cgit.cgi process stuck in this infloop, the
system would become essentially unusable.
I've just fixed the cgit bug and reported it privately to upstream
and as http://bugzilla.redhat.com/680905 (private, for now).
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2011-02-28 17:54 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-02-26 6:21 bzr + savannah.gnu.org slow again Paul Eggert
2011-02-26 6:50 ` Paul Eggert
2011-02-26 7:54 ` Eli Zaretskii
2011-02-26 8:13 ` [Savannah-hackers-public] " Jim Meyering
2011-02-26 8:49 ` Eli Zaretskii
2011-02-26 9:06 ` Eli Zaretskii
2011-02-28 8:03 ` Glenn Morris
2011-02-28 13:02 ` Jim Meyering
2011-02-28 17:54 ` Glenn Morris
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.