Re: Idea: 'ethical hosting' [formerly mailman service (free for FOSS projects)]

[Pjotr Prins <pjotr.public12@thebird.nl>]

On Tue, Apr 18, 2017 at 07:52:58PM +0000, ng0 wrote:
> (I am not a lawyer, and I haven't read every bit and piece of the German Vereinsrecht
> so far)
> With a registered non-profit under the Vereinsrecht in Germany it would not
> conflict with the non-profit status as long as the work supports the mission
> and structures of Guix Europe.

Exactly. It would confuse things. Guix-Europe has its own agenda,
currently hosting the build farm.

There is also the administrative overhead and choices one has to make.
I think any company has to be focused on its offerings.

> I haven't read the charta of GE in a while and haven't compared the two countries.
> 
> > I believe in fairness, so we'd have to come up with a way of
> > distributing any income fairly and giving some back to Guix. But that
> > is a separate discussion with the people who want to do this. That
> > does probably not belong on the mailing list. It needs thought.
> 
> It's difficult. Maybe where you have your permanent residence it is easier,
> I've had some expedition into legal entities in Germany for the last few
> weeks with the result that I delayed it for now (TL;DR: get professional
> advice offered by the state). Still some useful outcome in the process,
> you can contact me off list if you are interested.

I have set up limited companies in the past. Not hard and you need
about 2-3K per year in administrative costs. One interesting option is
Estonia. When you become an e-citizen you can set up a limited company
remotely with a group. It is within the EU and all legal. Estonia only
taxes 20% on dividend - which you can avoid paying when you have
enough expenses. 

My idea is that if we have an interesting business case we can
register a company any time. Estonia, NL and UK are interesting
possibilities. Anyway, this is a bit beyond the scope of this ML.

This is an example of a mailman setup I just had to analyse: 

* Mailman

I installed htop, mc and git.

** Ad hoc management

I don't think any form of software deployment is in place (other than
yum) - let alone deterministic software deployment. 

** No git in /etc (fixed)

Especially with shared sys admin I suggest to run git in /etc. I have done
it for you (after installing git with yum)

: cd /etc
: git init
: chmod 0600 .git
: git add .
: git commit -a -m init

Anyone doing updates should check git and commit changes.

** Firewall

Firewall rules are basic. Only ssh gets checked by fail2ban:

: iptables -L
:   fail2ban-SSH  tcp  --  anywhere             anywhere             tcp dpt:ssh

fail2ban is actually a running daemon. The /etc/hosts.deny file keeps
getting edited (currently 7K lines which means lookups get
slower!). There are multiple fail2ban's running - maybe that is by
design. fail2ban does not modify the firewall rules.

Also running is denyhosts.py - is it part of fail2ban? - which takes
25.7% of RAM(!?).

: 11061 root      30  10  344m 151m 2348 S  0.0 25.7  19:50.20 denyhosts.py

: root     27642  0.0  0.8 429812  5348 ?        Sl   Apr15   0:21 /usr/bin/python2.7 /usr/bin/fail2ban-server -b -s /var/run/fail2ban/fail2ban.sock -p /var/run/fail2ban/fail2ban.pid -x

which can be replaced by the following firewall rules which do not put the load, nor the
complexity beyond the firewall:

:            tcp  --  anywhere             anywhere             tcp dpt:ssh flags:FIN,SYN,RST,ACK/SYN recent: SET name: ssh side: source
: DROP       tcp  --  anywhere             anywhere             tcp dpt:ssh flags:FIN,SYN,RST,ACK/SYN recent: CHECK seconds: 127 hit_count: 10 name: ssh side: source
: ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh

** CRON

Virus checker gets restarted every day (apparently it crashes)

: 0 7 * * * /sbin/service restart mimedefang

** ssh

No password login allowed, good.

Root can access ssh with remote commands (but not shell and using keys only).

** RAM

RAM is fully in use which means SWAP is used quite a bit.

The following python processes

: 11061 root      30  10  344m 151m 2348 S  0.0 25.7  19:50.20 denyhosts.py
: 21223 defang    20   0  272m 116m 4624 S  0.0 19.8   3:28.71 mimedefang.pl
:  1643 mailman   20   0  338m  61m 1964 S  0.0 10.3  98:22.97 python2.7
: 23823 defang    20   0  182m  55m 4536 S  0.0  9.5   0:17.10 mimedefang.pl

take up more than half the RAM. denyhosts is part of sshd screening
(see firewall). Mimedefang does mail filtering of virusses (see CRON).

** CPU

Running htop for a while is interesting. The single core instance is
maxed out regularly every time a mail comes in.

Mailman qrunner and mimedefang.pl are the main culprits.

: Example:
:
:  CPU[|||||||||||||||||||||||||||||||||||||||||||100.0%]     Tasks: 84, 44 thr; 10 running
:  Mem[||||||||||||||||||||||||||||||||||||||||525/589MB]     Load average: 6.49 3.69 2.00
:  Swp[|||||||||                              738/4095MB]     Uptime: 8 days, 11:40:46
:
:   PID USER      PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command
: 26009 defang     20   0 72160 18908  3792 R 14.0  3.1  0:02.87 /usr/bin/perl /usr/bin/mimedefang.pl -server
: 26008 defang     20   0 72688 19312  3672 R 14.0  3.2  0:03.08 /usr/bin/perl /usr/bin/mimedefang.pl -server
: 25970 defang     20   0  118M 42388  4540 R 13.0  7.0  0:08.01 /usr/bin/perl /usr/bin/mimedefang.pl -server
: 21223 defang     20   0  319M 84720  4928 R 11.0 14.0  6:47.68 /usr/bin/perl /usr/bin/mimedefang.pl -server
: 25905 defang     20   0  179M 74812  7552 R 10.0 12.4  0:24.85 /usr/bin/perl /usr/bin/mimedefang.pl -server
: 23823 defang     20   0  189M 64584  6288 R 10.0 10.7  0:52.37 /usr/bin/perl /usr/bin/mimedefang.pl -server
: (...)

some messages coming in together here ... In fact, my terminal stopped
responding for a while.

** Disk space

: Filesystem      Size  Used Avail Use% Mounted on
: /dev/xvda1       15G  7.7G  6.9G  53% /
: devtmpfs        274M   28K  274M   1% /dev
: tmpfs           295M     0  295M   0% /dev/shm
: /dev/xvdg1       25G   17G  6.8G  71% /mailman

Using 'du' is very very slow for some reason. Probably because
both RAM and CPU are maxing out.

The mailman
/mailman/var-lib-mailman/archives/private/mailteam/attachments/ folder
is by far the largest and contains all sidelined MIME
attachments. Looking in these you can see they often have fake mail
addresses. Which can be stopped when mail comes in.

Also this attachment folder can be emptied - no need to keep these
around for more than 3 months. It would shave of 7G of the 17G stored for
mailman now.

** mailman

Mailman 2.1.15 is installed in /usr/lib/mailman/ and the configuration
file sits in the source tree. Installation date is 2015. Mailman's
security record is pretty good, but at least one security advisory is
probably relevant: https://www.debian.org/security/2016/dsa-3668

** sendmail

Sendmail accepts non-existing domains at this point.

Sendmail is used as a relay by events.xxxx.

Spamassassin is switched off.

Rbl lookups are switched off.

No checks on MX records of incoming mail.

The current configuration of sendmail/mailmain does not plug in the
correct host name of the sending server. My mailserver bounces this:

  Apr 6 11:11:03 mail postfix/smtpd[6691]: NOQUEUE: reject: RCPT from
  []: 450 4.7.1
  <ip-10-187-26-41.ec2.internal>: Helo command rejected: Host not
  found; from=<mailman-bounces@ip.ec2.internal>
  to=<pjotr.public@thebird.nl> proto=ESMTP
  helo=<ip.ec2.internal>

I.e., only the internal ec2 name is sent.

** MX settings

mailman has no backup MX (good!)

Although DNS configuration is outside this server, the TXT record of
mailman does not contain an spf in the TXT record. See,
for example,
https://support.rackspace.com/how-to/create-an-spf-txt-record/.

If you do a search against my own mail server:

   dig -t TXT mail.thebird.nl

you should see something like

   "v=spf1 ip4:95.154.246.10 +mx -all"

These days, an increasing number of mail daemons check the
SPF. Especially Microsoft is fussy and it means that people may not be
receiving mail, or it goes into SPAM.