workflow problems and possible public-inbox solutions

workflow problems and possible public-inbox solutions

[Eric Wong]

It seems like there's a bunch of problems the workflows@vger
list is trying to solve.  Some can be solved independently
of others.

Here's a summary of them and where items in
https://public-inbox.org/TODO list can be possible solution
(I'm not claiming it can solve everything :P)


1. vger deliverability problems to major mail providers

(NNTP|public-inbox) => POP3 bridge, since all major providers
support importing mail from POP3.

 https://public-inbox.org/meta/20160411034104.GA7817@dcvr.yhbt.net/

Locally-run NNTP -> mail fetchers can also work, but requires
more effort to setup.


2. attestation of messages

Self-publishing "Sent Mail" folders via NNTP or git,
(similar to Certificate Transparency):
https://lore.kernel.org/workflows/20191010192852.wl622ijvyy6i6tiu@chatter.i7.local/

This would provide redundancy, too; and it's probably more
usable and learnable than GPG :P



3. patch series evolution tracking

Further optimize searching and linkification for patches and
blob OIDs.

public-inbox can already search based on pre/post-image OIDs
(dfpre:/dfpost:) and use that recreate blobs from a series of
patches.

Future: Arbitrary diffs can be generated off recreated blobs.
--interdiff output can be parsed and searched on.

Some proposed solutions to make --range-diff output more
search-friendly, too:

  https://public-inbox.org/git/b9fb52b8-8168-6bf0-9a72-1e6c44a281a5@oracle.com/
  ("email as a bona fide git transport")

  https://public-inbox.org/git/20191017121045.GA15364@dcvr/
  ("range-diff: show old/new blob OIDs in comments")



4. secret exchanges for embargoed issues

No idea, really; not my department :P

It would be good if all messages in the exchange can be
published and mirrored once the embargo is lifted, though.



5. bug tracking

public-inbox already learned to parse and apply patches; so
parsing a debbugs-like language could be done, too.  Maybe
public-inbox can be optimized to improve searching on
"Fixes: $COMMIT_OID" in trailers until we decide on other
parts of the grammar.


Maybe some other stuff, too (feel free to add)
Thanks for reading.

Re: workflow problems and possible public-inbox solutions

[Eric Wong]

Eric Wong <e@80x24.org> wrote:
> Maybe some other stuff, too (feel free to add)

Ease-of-use is another...  I was an early adopter of git
and a user of tla (GNU arch) before that, which probably
makes me too numb to deal with this matter :P

Perhaps web UIs which tells users about command-line tools would
be helpful.

Linking to the git-send-email(1) manpage is one example of
that in public-inbox (but IMHO not as important as informing
users about Message-IDs).

Pointers to inform casual readers which tools (e.g. git
request-pull, git format-patch) were used to generate certain
emails could be useful, too...  I remember being asked how I
generated my request pull emails to a low-key project several
years ago, at least...

Re: workflow problems and possible public-inbox solutions

[Konstantin Ryabitsev]

On Fri, Oct 18, 2019 at 03:25:16AM +0000, Eric Wong wrote:
>It seems like there's a bunch of problems the workflows@vger
>list is trying to solve.  Some can be solved independently
>of others.
>
>Here's a summary of them and where items in
>https://public-inbox.org/TODO list can be possible solution
>(I'm not claiming it can solve everything :P)

Well, then you'd have to rename the project as "emacs". :)

>1. vger deliverability problems to major mail providers
>
>(NNTP|public-inbox) => POP3 bridge, since all major providers
>support importing mail from POP3.
>
> https://public-inbox.org/meta/20160411034104.GA7817@dcvr.yhbt.net/
>
>Locally-run NNTP -> mail fetchers can also work, but requires
>more effort to setup.

There is also this:
https://git.kernel.org/pub/scm/linux/kernel/git/dborkman/l2md.git/

With a bit more tweaking (see TODO), it can be made into a very nice 
generic interface tool -- in fact, I hope to use it at some point for 
feeding patches into patchwork instead of piping directly from postfix.

>2. attestation of messages
>
>Self-publishing "Sent Mail" folders via NNTP or git,
>(similar to Certificate Transparency):
>https://lore.kernel.org/workflows/20191010192852.wl622ijvyy6i6tiu@chatter.i7.local/
>
>This would provide redundancy, too; and it's probably more
>usable and learnable than GPG :P

Well, it still uses GPG, but it hides away the horrible parts. The 
easiest way to accomplish auto-signing is to create a dedicated ECC 
subkey and store it (and just it) in a dedicated dir used by the 
self-publishing tool.

>4. secret exchanges for embargoed issues
>
>No idea, really; not my department :P
>
>It would be good if all messages in the exchange can be
>published and mirrored once the embargo is lifted, though.

There is ongoing separate effort to address this, using remail:
https://git.kernel.org/pub/scm/linux/kernel/git/tglx/remail.git/

I haven't started implementing it yet, but my hope is to provide builtin 
archives that can be released once the embargo is lifted.

-K

Re: workflow problems and possible public-inbox solutions

[Eric Wong]

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> On Fri, Oct 18, 2019 at 03:25:16AM +0000, Eric Wong wrote:
> > 1. vger deliverability problems to major mail providers
> > 
> > (NNTP|public-inbox) => POP3 bridge, since all major providers
> > support importing mail from POP3.
> > 
> > https://public-inbox.org/meta/20160411034104.GA7817@dcvr.yhbt.net/
> > 
> > Locally-run NNTP -> mail fetchers can also work, but requires
> > more effort to setup.
> 
> There is also this:
> https://git.kernel.org/pub/scm/linux/kernel/git/dborkman/l2md.git/
> 
> With a bit more tweaking (see TODO), it can be made into a very nice generic
> interface tool -- in fact, I hope to use it at some point for feeding
> patches into patchwork instead of piping directly from postfix.

Looks like that still needs local storage?  I'm planning a tool
which would work remotely while allowing local caching/memoization.

> > 2. attestation of messages
> > 
> > Self-publishing "Sent Mail" folders via NNTP or git,
> > (similar to Certificate Transparency):
> > https://lore.kernel.org/workflows/20191010192852.wl622ijvyy6i6tiu@chatter.i7.local/
> > 
> > This would provide redundancy, too; and it's probably more
> > usable and learnable than GPG :P
> 
> Well, it still uses GPG, but it hides away the horrible parts. The easiest
> way to accomplish auto-signing is to create a dedicated ECC subkey and store
> it (and just it) in a dedicated dir used by the self-publishing tool.

Self-hosting an .onion would get rid of the need for GPG.
That should be doable on most home Internet connections, but
requires constant uptime...  But more self-hosting and not
having to pay registrars is good :>

HTTPS still relies on trust authorities, and it's "good enough"
for a lot of people so probably OK... *shrug*

> > 4. secret exchanges for embargoed issues
> > 
> > No idea, really; not my department :P
> > 
> > It would be good if all messages in the exchange can be
> > published and mirrored once the embargo is lifted, though.
> 
> There is ongoing separate effort to address this, using remail:
> https://git.kernel.org/pub/scm/linux/kernel/git/tglx/remail.git/
> 
> I haven't started implementing it yet, but my hope is to provide builtin
> archives that can be released once the embargo is lifted.

Cool :>