Re: Some points on public-inbox

[Eric Wong <e@80x24.org>]

Leah Neukirchen <leah@vuxu.org> wrote:
> Hi,
> 
> over the last few days I've set up a public-inbox 1.1.0pre1 instance,
> and noticed some things:

Hey Leah, thanks for giving it a try!  Sorry for the late reply,
been trying to avoid being at the computer too much for health
reasons.

> 1) Makefile.PL only works properly when run from a checkout, not a tarball.
> I replaced the beginning with
> 
> my @EXE_FILES = split("\n", `printf '%s\n' script/* 2>/dev/null`);
> my $PM_FILES = `find lib 2>/dev/null`;

Thanks, I'd probably add "-name '*.pm'" to find(1) to filter out
directories.  But I wonder if it's better to grep the MANIFEST
file...

> 2) public-inbox-mda returns with status 1 when it gets a mail it
> doesn't know where to deliver to.  I think status 67 would be more
> appropriate (EX_NOUSER).

Sure.  There's a bunch of places where we just die() and ignore
sysexits.h or similar.  Could use some help checking for that
and patches are welcome :>

> 3) IPv6 support needs the Socket6 module, this is not stated anywhere.

Oops, I thought this was standard :x  Care to send a patch to
INSTALL for that?

> 4) I think it would be useful if the thread overview displayed
> the name of the initial poster, could this be added as an option?

If anything, I'd rather list ALL the recent participants in a
thread (it wouldn't require extra lookups).

But my philosophy is not to give anybody more credit/weight than
anybody else; and I'd rather people follow links if the Subject
seems interesting, not because who started a particular topic
(especially when it comes to emails with [PATCH] subjects).

I also prefer to avoid having too many options to reduce
support/documentation costs, so if we do something like this,
it would be the default.  But also, it's more clutter.

> 5) Is there a way for the HTML view to list all served lists?

Not currently...  I'm not sure how the UI or configuration
should be or how to avoid clutter/scalability problems with many
inboxes.  NNTP has standardized commands and clients can decide
how to show them, at least.

> / results in 404.  How did you add links to meta/ and test/ on
> https://public-inbox.org/ ?

mkdir /srv/public-inbox/{meta,test} # static directory listing

Rack::Builder routes meta/ and test/ to varnish => public-inbox-httpd

All the HTTPS, static files and reverse proxying is handled
using yet-another-horribly-named-server written in Ruby and Rack :)

==> config.ru snippet <==
# random crap from yahns extras/
require "autoindex"
require "try_gzip_static"
require "yahns/proxy_pass"
autoindex = lambda do |path|
  Autoindex.new(TryGzipStatic.new(path), skip_dotfiles: true)
end

pi = Rack::Builder.new do
  run Yahns::ProxyPass.new('http://127.0.0.1:6081', # varnish
      response_headers: {
        'Age' => :ignore,
        'X-Varnish' => :ignore,
        'Via' => :ignore
    })
end.to_app

unsub = Rack::Builder.new do
  run Yahns::ProxyPass.new('unix:/run/unsubscribe-psgi.sock')
end.to_app
unsub_re = %r{\A/u/[^/]+/[^/]+\z}

map('http://public-inbox.org/') do
  pfx = 'public-inbox'
  static = autoindex["/srv/#{pfx}"]
  txt_html = %r{\.txt\.html\z} # oops :x
  proxy = Yahns::ProxyPass.new("http://127.0.0.1:2080/$host$fullpath",
			        proxy_buffering: false)
  cascade = Rack::Cascade.new([static, proxy])
  run(lambda do |env|
    return redirect(env, nil, nil) if env['rack.url_scheme'] == -'http'
    case path_info = env["PATH_INFO"]
    when %r{\A/test(?:/?.*)\z}
      redirect(env, "try.public-inbox.org", path_info)
    when %r{\A/(?:git|meta|public-inbox(?:\.git)?)(?:/|\z)}x,
	 '/HEAD', '/info/refs', '/git-upload-pack', '/description', '/cloneurl',
         %r{\A/objects/}
      pi.call(env)
    when unsub_re
      unsub.call(env)
    when txt_html
      redirect(env, 'public-inbox.org', path_info.sub(txt_html, '.html'))
    else
      cascade.call(env)
    end
  end)
end
==> end snippet <=

> 6) I have a user account that uses .forward to call public-inbox-mda,
> and use /etc/aliases to route the lists that are hosted primarily on
> the server to it.  What's the best approach to do this for mailing
> lists I only mirror? Subscribe with a "secret" second address to the
> list, and add this second adress to publicinbox.<name>.address?
> Or can public-inbox-mda also scan for List-Id etc and sort by it somehow?

I prefer to use public-inbox-watch for mirroring existing lists.

-mda is also a bit strict and opinionated (though I have plans to
make it less so, optionally), so it's mainly for non-mirrored
inboxes.

-watch is also safer and less likely to lose/bounce mail since
it hits a Maildir, first.  -watch will scan for List-Id (or any
other header, such as X-Mailing-List) and put it into the
correct inbox.  If space is a problem, a cronjob to remove
old files will help, but maybe it can unlink-on-import-commit
in the future.

I haven't thought much about mirroring with -mda, but I suppose
having a per-list subscriber address and extra
publicinbox.<name>.address entry works, too.

> Thank you very much for your work,

No problem :>