From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, WEIRD_PORT shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id D0AB31F403; Tue, 12 Jun 2018 10:09:15 +0000 (UTC) Date: Tue, 12 Jun 2018 10:09:15 +0000 From: Eric Wong To: Leah Neukirchen Cc: meta@public-inbox.org Subject: Re: Some points on public-inbox Message-ID: <20180612100915.shfo3ltn6aj55mrf@dcvr> References: <871sdfzy80.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <871sdfzy80.fsf@gmail.com> List-Id: Leah Neukirchen wrote: > Hi, > > over the last few days I've set up a public-inbox 1.1.0pre1 instance, > and noticed some things: Hey Leah, thanks for giving it a try! Sorry for the late reply, been trying to avoid being at the computer too much for health reasons. > 1) Makefile.PL only works properly when run from a checkout, not a tarball. > I replaced the beginning with > > my @EXE_FILES = split("\n", `printf '%s\n' script/* 2>/dev/null`); > my $PM_FILES = `find lib 2>/dev/null`; Thanks, I'd probably add "-name '*.pm'" to find(1) to filter out directories. But I wonder if it's better to grep the MANIFEST file... > 2) public-inbox-mda returns with status 1 when it gets a mail it > doesn't know where to deliver to. I think status 67 would be more > appropriate (EX_NOUSER). Sure. There's a bunch of places where we just die() and ignore sysexits.h or similar. Could use some help checking for that and patches are welcome :> > 3) IPv6 support needs the Socket6 module, this is not stated anywhere. Oops, I thought this was standard :x Care to send a patch to INSTALL for that? > 4) I think it would be useful if the thread overview displayed > the name of the initial poster, could this be added as an option? If anything, I'd rather list ALL the recent participants in a thread (it wouldn't require extra lookups). But my philosophy is not to give anybody more credit/weight than anybody else; and I'd rather people follow links if the Subject seems interesting, not because who started a particular topic (especially when it comes to emails with [PATCH] subjects). I also prefer to avoid having too many options to reduce support/documentation costs, so if we do something like this, it would be the default. But also, it's more clutter. > 5) Is there a way for the HTML view to list all served lists? Not currently... I'm not sure how the UI or configuration should be or how to avoid clutter/scalability problems with many inboxes. NNTP has standardized commands and clients can decide how to show them, at least. > / results in 404. How did you add links to meta/ and test/ on > https://public-inbox.org/ ? mkdir /srv/public-inbox/{meta,test} # static directory listing Rack::Builder routes meta/ and test/ to varnish => public-inbox-httpd All the HTTPS, static files and reverse proxying is handled using yet-another-horribly-named-server written in Ruby and Rack :) ==> config.ru snippet <== # random crap from yahns extras/ require "autoindex" require "try_gzip_static" require "yahns/proxy_pass" autoindex = lambda do |path| Autoindex.new(TryGzipStatic.new(path), skip_dotfiles: true) end pi = Rack::Builder.new do run Yahns::ProxyPass.new('http://127.0.0.1:6081', # varnish response_headers: { 'Age' => :ignore, 'X-Varnish' => :ignore, 'Via' => :ignore }) end.to_app unsub = Rack::Builder.new do run Yahns::ProxyPass.new('unix:/run/unsubscribe-psgi.sock') end.to_app unsub_re = %r{\A/u/[^/]+/[^/]+\z} map('http://public-inbox.org/') do pfx = 'public-inbox' static = autoindex["/srv/#{pfx}"] txt_html = %r{\.txt\.html\z} # oops :x proxy = Yahns::ProxyPass.new("http://127.0.0.1:2080/$host$fullpath", proxy_buffering: false) cascade = Rack::Cascade.new([static, proxy]) run(lambda do |env| return redirect(env, nil, nil) if env['rack.url_scheme'] == -'http' case path_info = env["PATH_INFO"] when %r{\A/test(?:/?.*)\z} redirect(env, "try.public-inbox.org", path_info) when %r{\A/(?:git|meta|public-inbox(?:\.git)?)(?:/|\z)}x, '/HEAD', '/info/refs', '/git-upload-pack', '/description', '/cloneurl', %r{\A/objects/} pi.call(env) when unsub_re unsub.call(env) when txt_html redirect(env, 'public-inbox.org', path_info.sub(txt_html, '.html')) else cascade.call(env) end end) end ==> end snippet <= > 6) I have a user account that uses .forward to call public-inbox-mda, > and use /etc/aliases to route the lists that are hosted primarily on > the server to it. What's the best approach to do this for mailing > lists I only mirror? Subscribe with a "secret" second address to the > list, and add this second adress to publicinbox..address? > Or can public-inbox-mda also scan for List-Id etc and sort by it somehow? I prefer to use public-inbox-watch for mirroring existing lists. -mda is also a bit strict and opinionated (though I have plans to make it less so, optionally), so it's mainly for non-mirrored inboxes. -watch is also safer and less likely to lose/bounce mail since it hits a Maildir, first. -watch will scan for List-Id (or any other header, such as X-Mailing-List) and put it into the correct inbox. If space is a problem, a cronjob to remove old files will help, but maybe it can unlink-on-import-commit in the future. I haven't thought much about mirroring with -mda, but I suppose having a per-list subscriber address and extra publicinbox..address entry works, too. > Thank you very much for your work, No problem :>