Re: [RFC2 Patch 5/5] lib: iterator API for message properties

[Daniel Kahn Gillmor <dkg@fifthhorseman.net>]

On Tue 2016-05-31 21:52:06 -0400, Daniel Kahn Gillmor wrote:
> Thanks for working on this, David!  I think this is going to be really
> useful!

just thinking about this series this morning in a bigger-picture way, i
figure it's worth asking the hard questions now rather than later --
maybe the answers are obvious, and we just need to write them down.
Please accept these questions in the spirit of supportive inquiry :)
Here goes:

do we actually need this abstraction?  If we're aiming to build specific
new features (the two i'm thinking of are cryptographic-session-keys and
reference-adjustments), couldn't we implement those features explicitly
in xapian with their own special prefix, rather than treating them as a
generic "property"?  If we make a generic "property", that seems likely
to be exposed to the user, who can then manipulate them directly
externally from notmuch.

We already have a bit of an uncomfortable fit with tags and special
flags (encrypted, signed, attachment, etc), where some are expected to
be set and cleared automagically and some are expected to be manipulated
directly by the user.  Are we setting ourselves up for more of the same,
or is there a principled way that a user can know which properties it's
kosher for them to set and clear, and which ones they should leave
alone?

If we add new specific features, we could potentially augment the dump
format explicitly for them, without having the property abstraction.  We
already have some explicit features for each message (subject, from, to,
attachment, mimetype, thread id, etc), and most of them are derived from
the message itself, with the hope that it could be re-derived given just
the message body.  Is there a distinction between properties that can be
derived from the message body and properties that need to be
additionally derived from some other data?

   --dkg