From: Bob Proulx <bob@proulx.com>
To: help-gnu-emacs@gnu.org
Subject: Re: Mail posting in newsgroups in Gnus
Date: Sun, 15 May 2016 16:25:17 -0600 [thread overview]
Message-ID: <20160515160711053121293@bob.proulx.com> (raw)
In-Reply-To: <83shxl6k3q.fsf@gnu.org>
Eli Zaretskii wrote:
> Lars Magne Ingebrigtsen wrote:
> > The gnu.org SMTP servers have been timing out on incoming mail for some
> > people the last couple of days, whether you're posting through Gmane or
> > not. But some seem to make it through, for some reason...
> >
> > Or perhaps it's just email from Norway? I wonder whether this one make
> > it through...
>
> gnu.org machines have connectivity problems lately. I hope they will
> be resolved soon.
Since Friday the 6th there was a large move with various assorted
related and unrelated changes. Which has made it difficult to tell
what broke what. I am deep in the middle of *.savannah and can give
the most information about it.
The *.savannah.gnu.org machines moved to a different host server in
the same data center. But their file systems got repackaged and the
copy process had problems. 1) All of the system uids were mapped to
the host used to do the copy. All of those had to be renamed back to
their proper uids. Fixed that. 2) All of the ACLs were lost. This
prevented the web team from being able to access files. Fixed that.
3) Swap was lost. Causing OOM activity. Fixed that. 4) Additionally
a very new kernel from the host system was booted instead of the
system one. That caused a subset of networking problems. The managed
switch was reporting a continuous high rate of dropped packets.
Reverting back to the system kernel fixed the switch's reports of
dropped packets. 5) This happened late on a Friday without notice
meaning that we had to scramble around over the weekend to find and
fix the problems. 6) Long term there has been persistent problems
with stuck connections to git, cvs, bzr, hg daemons. Since git is by
far most popular we mostly hear about git problems but it happens with
the others too. The stuck daemons stack up consuming slots until all
of the xinetd slots are filled and it hits the limit at 100 processes
running. At that point no new connections can be made. This has been
made worse by the Friday network move. Something is now tickly this
problem agressively. We are manually monitoring this and mitigating
it by killing stale processes. We hope this to be fixed as soon as we
can migrate onto the new operating sytsem VMs promised to us any day
now for the last two years.
On the same day the entire Boston FSF network was changed to a
different network routing of which I know little. I don't have much
visibility into this change. traceroute shows a different route now.
This changed for every system. People are reporting many problems
across all of the system. There are reports of differences seen
between IPv4 and IPv6.
Except for the stuck process problem I think the Savannah systems are
running within nominal limits. It is "normal". For Savannah anyway.
(Not to say things don't need fixing. But we are waiting in the
pipeline for a new VM that we can migrate onto.) However all of the
random network problems people are reporting include fencepost, eggs
(mail relay), lists.gnu.org (mailing list), www.gnu.org, and many
others. Those don't have anything in common except for the networking
and all are suffering.
If you are suffering problems I encourage a trouble report being
made. Please include details. Say where from and where to. Say
whether it was IPv4 or IPv6. Time the problem occurred. Then if
later it works update and say that. Because one of the problems is
that this seems worse from Europe than from the US. I in Colorado
have a hard time triggering any problems. But I have been able to on
occassion. But people in Europe have been most of the problem
reports. Most of them have been using IPv6.
And that is all I know.
Bob
next prev parent reply other threads:[~2016-05-15 22:25 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-05-13 10:11 Mail posting in newsgroups N. Raghavendra
2016-05-13 10:48 ` Mail posting in newsgroups in Gnus N. Raghavendra
2016-05-14 0:55 ` Bob Proulx
2016-05-14 3:06 ` Jude DaShiell
2016-05-14 3:30 ` Emanuel Berg
2016-05-14 8:01 ` Eli Zaretskii
2016-05-14 18:23 ` Bob Proulx
2016-05-14 3:45 ` Emanuel Berg
2016-05-14 7:25 ` N. Raghavendra
2016-05-14 13:36 ` Emanuel Berg
2016-05-14 18:26 ` Bob Proulx
2016-05-15 22:33 ` Bob Proulx
2016-05-16 15:31 ` N. Raghavendra
[not found] ` <mailman.2793.1463187329.7477.help-gnu-emacs@gnu.org>
2016-05-14 9:38 ` Lars Magne Ingebrigtsen
2016-05-14 9:54 ` Eli Zaretskii
2016-05-15 22:25 ` Bob Proulx [this message]
2016-05-16 7:54 ` tomas
2016-05-18 3:07 ` Emanuel Berg
[not found] ` <mailman.2864.1463351125.7477.help-gnu-emacs@gnu.org>
2016-05-16 10:03 ` Lars Magne Ingebrigtsen
[not found] ` <mailman.2911.1463419198.7477.help-gnu-emacs@gnu.org>
2016-05-16 17:44 ` Lars Magne Ingebrigtsen
2016-05-16 18:43 ` Bob Proulx
[not found] ` <mailman.2951.1463424214.7477.help-gnu-emacs@gnu.org>
2016-05-16 19:30 ` Lars Magne Ingebrigtsen
2016-05-14 9:57 ` N. Raghavendra
2016-05-14 13:38 ` Emanuel Berg
2016-05-14 12:14 ` Emanuel Berg
2016-05-16 20:26 ` Mail posting in newsgroups Michael Heerdegen
2016-05-17 2:45 ` N. Raghavendra
-- strict thread matches above, loose matches on Subject: below --
2016-05-13 10:38 Mail posting in newsgroups in Gnus N. Raghavendra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160515160711053121293@bob.proulx.com \
--to=bob@proulx.com \
--cc=help-gnu-emacs@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.