PGP/MIME signature verification

PGP/MIME signature verification

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 2073 bytes --]

hey folks--

the signature-verification branch on my git repo [0] contains functional
PGP/MIME signature verification if you supply the --verify argument to

 notmuch show --format=json

It relies on gpg being in the path, and on the user having the signer's
key in their gnupg keyring.

The patchset depends on the MIME reorganization produced initially by
dme (and rebased against 0.5 in my mp3-on-0.5 branch).

the emacs frontend doesn't yet know how to deal with the resultant data,
so the signature verification doesn't reach the user yet.

The JSON this change produces is similar (but not identical) to my
earlier proposal on this list.

In particular, the parent MIME part (content-type: multipart/signed) is
what gets the sigstatus object, which is itself a list of signature
objects, like this:

"sigstatus": [
 {
  "created": 1290557507,
  "fingerprint": "0EE5BE979282D80B9F7540F1CCD2ED94D21739E9",
  "status": "good"
 },
 {
  "keyid": "DEADBEEFDECAFBAD",
  "status": "bad"
 }
]

(this example shows a mime part signed by two keys, the signature from
one checking out properly, and the signature from the other failing --
most multipart/signed MIME parts will only be signed by one party)

I'd appreciate any feedback people have on the change, and any
suggestions on how to integrate its results into the emacs interface.

Also, i'm curious what i can do to make these changes more acceptable
for the mainline.  should i rebase them against mainline master?  or
demonstrate a functional merge?  i'm new to how this community likes to
use git, so pointers to suggested approaches would be welcome.

I should also note that in the course of pursuing this, i found that
GMime's upstream author is very receptive to suggestions and feedback
about crypto.  If anyone else in the notmuch community wants to make
suggestions about what we might need for future extensions in this
direction, he's probably a good resource to help us get there.

Regards,

	--dkg

[0] git://lair.fifthhorseman.net/~dkg/notmuch


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

Re: PGP/MIME signature verification

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 488 bytes --]

On Sat, 27 Nov 2010 14:35:03 -0500, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:
> the signature-verification branch on my git repo [0] contains functional
> PGP/MIME signature verification if you supply the --verify argument to
> 
>  notmuch show --format=json

Daniel, this is really incredible.  I can't tell you how much I
appreciate all of your effort on this front.  This is really a very
fundamental improvement to this tool.  I can't wait to test it out!

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: PGP/MIME signature verification

[David Bremner]

On Sat, 27 Nov 2010 14:35:03 -0500, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:
> hey folks--
> 
> the signature-verification branch on my git repo [0] contains functional
> PGP/MIME signature verification if you supply the --verify argument to
> 
>  notmuch show --format=json
> 
> It relies on gpg being in the path, and on the user having the signer's
> key in their gnupg keyring.
> 
> The patchset depends on the MIME reorganization produced initially by
> dme (and rebased against 0.5 in my mp3-on-0.5 branch).

I had a look at dkg's patch and it looks reasonable to me. Except for
the small amount of self described "crufty boilplate" which I was too
lazy to understand.  I also compiled and ran it, it seemed to work OK on
the two test cases I had handy.  

I've started writing a testsuite for the Mail::GnuPG perl module;
perhaps the messages in
        
        http://pivot.cs.unb.ca/git/?p=mail-gnupg.git;a=tree;f=t/msg

can be helpful for testing notmuch as well. 

d

Re: PGP/MIME signature verification

[David Bremner]

On Sat, 27 Nov 2010 14:35:03 -0500, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:
> 
> The JSON this change produces is similar (but not identical) to my
> earlier proposal on this list.
> 
I forgot to mention, I also had a look at the produced json, and it
looks sensible enough. 

One thing I'm a bit confused about, several of the commits in branch
dkg/mp3-on-0.5 have dkg as author. Are these part of basic multipart
support? I'm afraid my last look was just at the one commit on top of
mp3-on-0.5.

d

Re: PGP/MIME signature verification

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 751 bytes --]

On 12/13/2010 05:02 PM, David Bremner wrote:
> I had a look at dkg's patch and it looks reasonable to me. Except for
> the small amount of self described "crufty boilplate" which I was too
> lazy to understand.  I also compiled and ran it, it seemed to work OK on
> the two test cases I had handy.  

the crufty boilerplate is just to make the GMime business know how to
not hassle the user for a password.  GMime upstream appears to be
deprecating that particular interface, so we should be able to remove
the boilerplate once gmime 2.6 is released (and notmuch decides to
abandon gmime 2.4).

This isn't about to happen tomorrow though, so i think the boilerplate
needs to stay in for now.

Thanks for the review, David!

	--dkg


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

Re: PGP/MIME signature verification

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 546 bytes --]

On 12/13/2010 05:10 PM, David Bremner wrote:

> One thing I'm a bit confused about, several of the commits in branch
> dkg/mp3-on-0.5 have dkg as author. Are these part of basic multipart
> support? I'm afraid my last look was just at the one commit on top of
> mp3-on-0.5.

i did make a handful of cleanup commits on top of the branch after i
rebased dme's mp3 branch.

They were focused on normalizing output, and making sure the test suite
completed cleanly *before* i started implementing the signature
verification.

	--dkg


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

Re: PGP/MIME signature verification

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 1438 bytes --]

On Sat, 27 Nov 2010 14:35:03 -0500, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:
> the signature-verification branch on my git repo [0] contains functional
> PGP/MIME signature verification if you supply the --verify argument to
> 
>  notmuch show --format=json
> 
> It relies on gpg being in the path, and on the user having the signer's
> key in their gnupg keyring.

I was able to merge dkg's signature-verification branch on top of
cworth's current master head (b3caef1f) with only a tiny
easily-resolvable conflict.  It seems to work exactly as advertised, and
I wholeheartedly approve and endorse this patch set for inclusion
upstream:

Approved-and-Tested-By: Jameson Rollins <jrollins@finestructure.net>

I have pushed a merged "signature-verification" branch to my repo, along
with a single commit to fix an errant newline in the json --verify
output:

git://finestructure.net/notmuch

Carl: can you give an indication of whether or not you intend to
ultimately accept this patch set?  If so, I would like to start work on
representing signature verification information in the emacs UI.  I want
to make sure that we're good with this effort first, though.

We should probably also start a conversation about how best to represent
the signature verification info.  I'll bring that up in a separate
thread, though.

dkg: again, great work on this!  Thanks!

Next up: decrypting!

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: PGP/MIME signature verification

[Sebastian Spaeth]

[-- Attachment #1: Type: text/plain, Size: 1062 bytes --]

On Mon, 20 Dec 2010 13:22:07 -0500, Jameson Rollins <jrollins@finestructure.net> wrote:
> I was able to merge dkg's signature-verification branch on top of
> cworth's current master head (b3caef1f) with only a tiny
> easily-resolvable conflict.  It seems to work exactly as advertised, and
> I wholeheartedly approve and endorse this patch set for inclusion
> upstream:

> I have pushed a merged "signature-verification" branch to my repo, along

So what am I doing wrong, I pulled your branch and compiled/installed,
but I don't see any signature verification in my emacs?

Unless all work is still in the notmuch binary, I see for your mail,
e.g. 
"sigstatus": [{"status": "error","keyid": "ED34CEABE27BAABC", "errors":
2}]
(perhaps due to me not having your key???)


On a side note:
notmuch show format=json --verify id:"87aak08fu8.fsf@servo.finestructure.net"

is ok but:
notmuch show --verify id:"87aak08fu8.fsf@servo.finestructure.net"

neither omplains nor shows anything in addition. Should it do one of the
two?

Sebastian

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: PGP/MIME signature verification

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 1326 bytes --]

On 12/21/2010 04:51 AM, Sebastian Spaeth wrote:
> So what am I doing wrong, I pulled your branch and compiled/installed,
> but I don't see any signature verification in my emacs?
> 
> Unless all work is still in the notmuch binary, 

yes, the branch as it stands just concerns itself with the notmuch binary.

> I see for your mail,
> e.g. 
> "sigstatus": [{"status": "error","keyid": "ED34CEABE27BAABC", "errors":
> 2}]
> (perhaps due to me not having your key???)

yup, that is why you get that error.

> On a side note:
> notmuch show format=json --verify id:"87aak08fu8.fsf@servo.finestructure.net"
> 
> is ok but:
> notmuch show --verify id:"87aak08fu8.fsf@servo.finestructure.net"
> 
> neither omplains nor shows anything in addition. Should it do one of the
> two?

the patch is specifically only for the json output.  If you'd like the
same features for the non-json output, that would be cool to see, but i
don't understand those formats well enough to know how to incude that
kind of info in them.

If this seems like i've only done one small corner of the task needed to
get signature verification: you're right!  I deliberately scoped the
initial task to be small-but-doable.  i want to make sure this seems
acceptable/reasonable before digging into the other pieces.

	--dkg


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

Re: PGP/MIME signature verification

[Sebastian Spaeth]

> yes, the branch as it stands just concerns itself with the notmuch binary.

OK, fair enough.

> > "sigstatus": [{"status": "error","keyid": "ED34CEABE27BAABC", "errors":
> > 2}]
> > (perhaps due to me not having your key???)
> 
> yup, that is why you get that error.

Is there a possibility to squeeze a nicer error message out of it? :-)

> > is ok but:
> > notmuch show --verify id:"87aak08fu8.fsf@servo.finestructure.net"
> > 
> > neither omplains nor shows anything in addition. Should it do one of the
> > two?
> 
> the patch is specifically only for the json output.  If you'd like the
> same features for the non-json output, that would be cool to see, but i
> don't understand those formats well enough to know how to incude that
> kind of info in them.

No problem, I was just wondering about it simply silently ignoring an
option (in plain output), but that is fine for now.
 
> If this seems like i've only done one small corner of the task needed to
> get signature verification: you're right!

NONONO :) Don't misunderestimate me. This is phantastic work!!!

You got the really hard part working, which is great. The rest is sugar
on top :). Much appreciated and another:

Tested-and-approved-and-+1for_merging: S. Spaeth <Sebastian@SSpaeth.de>

Sebastian

Re: PGP/MIME signature verification

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 2986 bytes --]

On 12/22/2010 09:38 AM, Sebastian Spaeth wrote:
>>> "sigstatus": [{"status": "error","keyid": "ED34CEABE27BAABC", "errors":
>>> 2}]
>>> (perhaps due to me not having your key???)
>>
>> yup, that is why you get that error.
> 
> Is there a possibility to squeeze a nicer error message out of it? :-)

that's a good question -- i'm not sure libgmime gives us much else to
work with (it is itself calling out to gpg, which in turn might not give
*it* much to work with), but i'll see if there's anything we can do
about that.

One potential concern with this kind of reporting is that (due to the
dirty details of the OpenPGP data signature format), differentiating
"you don't have this key in your keyring" from "you have this key, but
the signature is bad" is not fully possible.

For those who want the details:

 0) an OpenPGP signature contains the signature data itself, plus an
indicator of what key made the signature (the "issuer"):

  https://tools.ietf.org/html/rfc4880#section-5.2.3.5

 1) The indicator is the lower 64-bits of the OpenPGP fingerprint of the
signing key.

 2) OpenPGP fingerprints themselves are SHA-1 sums of (some fiddly
concatenation of) the creation time of the key and the public key
material itself:

  https://tools.ietf.org/html/rfc4880#section-12.2

However, the 64 bits of the key ID (while longer than the 32-bit pattern
commonly expressed like "0xDEADBEEF") isn't really a big-enough search
space to resist collision attacks by moderately-well-resourced adversaries.

That is, it's currently possible to have a computer farm powerful enough
to crank through the search space to create a valid OpenPGP key with any
chosen 64-bit key ID.

So if Alice has her legitimate key with key ID 0x0000DECAFBAD0000, and
Eve has a big computer, Eve can create a new OpenPGP key with the same
Key ID.  If Eve convinces Bob to load her own key into his keyring, then
what happens when Bob receives a signed message from Alice?

Bob's verifier will look at the message, see that it's signed by a key
with key ID 0x0000DECAFBAD0000, and then check the signature against his
copy of Eve's key.  But the signature doesn't verfiy!  This is actually
because Bob doesn't have Alice's key, of course, but his client has no
way of knowing that -- it just sees that it has a matching key that
doesn't verify.

So an attacker can convert a "you don't have this key" to a "you have
this key but the signature doesn't validate" without even tampering with
the message in transit (of course, an attacker who can tamper with the
message in transit can *also* make a signature fail to validate, just by
flipping a bit or two in either the message or the signature).

(note: don't despair!  the full 160-bit OpenPGP fingerprint space is not
currently under threat; only the 64-bit "key ID" has the above problem,
given our current understanding of cryptanalysis and the power of our
computing hardware).

	--dkg


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

Re: PGP/MIME signature verification

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 1253 bytes --]

On Mon, 20 Dec 2010 13:22:07 -0500, Jameson Rollins <jrollins@finestructure.net> wrote:
> On Sat, 27 Nov 2010 14:35:03 -0500, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:
> > the signature-verification branch on my git repo [0] contains functional
> > PGP/MIME signature verification if you supply the --verify argument to
> > 
> >  notmuch show --format=json
> > 
> > It relies on gpg being in the path, and on the user having the signer's
> > key in their gnupg keyring.
> 
> I was able to merge dkg's signature-verification branch on top of
> cworth's current master head (b3caef1f) with only a tiny
> easily-resolvable conflict.  It seems to work exactly as advertised, and
> I wholeheartedly approve and endorse this patch set for inclusion
> upstream:
> 
> Approved-and-Tested-By: Jameson Rollins <jrollins@finestructure.net>

Hi, all.  I have rebased the "signature-verification" branch against the
current (as of now) head of cworth's master (74cb76a6) and pushed it to
my repo:

git://finestructure.net/notmuch

Carl: please let me know if this is suitable for pulling into your
master branch, and definitely let me know if there's anything else I can
do to facilitate getting these patches included.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 1120 bytes --]

Hi, all.  I have pushed a new branch called "crypto" to my notmuch
repository [0].  This branch provides full support for PGP/MIME signed
and encrypted messages, including emacs UI support.  It has been applied
on top of cworth's current master (21e97c50).  It includes the
following:

* David Edmondson's improved multipart handling patch series (cherry-picked)
* Daniel Gillmor's PGP/MIME signature verification patch series (cherry-picked)
* my PGP/MIME decryption+verification patch series
* a test suite for signature verification and decryption
* emacs support for the above

I have signed-off on all cherry-picked commits, as I have reviewed and
tested them extensively (I have been using them in my daily notmuch use
for many months now).  The multipart and sig-verification patches were
cherry-picked to get around a couple of accidental spurious commits in
their originating branches that would have needed to be reverted.

Please test and provide feedback.  I would really like to see this
series merged into the mainline for the next release, if at all
possible.

jamie.

[0] git://finestructure.net/notmuch

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[micah anderson]

[-- Attachment #1: Type: text/plain, Size: 2912 bytes --]

On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins <jrollins@finestructure.net> wrote:
> Hi, all.  I have pushed a new branch called "crypto" to my notmuch
> repository [0].  This branch provides full support for PGP/MIME signed
> and encrypted messages, including emacs UI support.  It has been applied
> on top of cworth's current master (21e97c50).  It includes the
> following:
> 
> * David Edmondson's improved multipart handling patch series (cherry-picked)
> * Daniel Gillmor's PGP/MIME signature verification patch series (cherry-picked)
> * my PGP/MIME decryption+verification patch series
> * a test suite for signature verification and decryption
> * emacs support for the above

Don't forget that you also included man page changes!

> Please test and provide feedback.  I would really like to see this
> series merged into the mainline for the next release, if at all
> possible.

I've really really really wanted this functionality, so I pulled this
right away and have been testing it, its really slick! I like how the
emacs UI gives you good visual feedback for different signature states
(I had red for a signature from Sebastian Spaeth because I did not have
the key; orange for when I obtained that key; and green for Jameson and
dkg's mails because I have exchanged keys with them and have full
validity for them; and purple for a decryption error). The minor delay
in opening a thread with signatures is not bad, and is to be expected.

And messages that are PGP/MIME encrypted are decrypted automatically,
wow, this is amazing. I enthusiastically support merging this into
mainline for the next release.

I have a couple points of feedback that I do not think should hold up
merging this work:

1. I personally think notmuch-show-process-pgpmime should default to
true

2. in-line pgp messages don't have any processing done on them. getting
the mime-encoded processing work is a huge step and I'm happy that
works, in-line can (and IMHO, should) come later

3. i'm not sure expired/revoked keys are handled properly - tested on a
message that was encrypted by a key that was revoked and got "End of
file during parsing"

4. messages that I sent encrypted to someone are not also encrypted to
myself, which means that a thread which contains my replies isn't able
to decrypt my messages in that thread and results in a purple
'decryption error'. Perhaps this is an emacs UI tweak that needs to be
made to get messages also encrypted to my own key?

5. unknown keys are represented in a long format,
eg. '0x5585F58CC827A062' when most tools represent them just with their
shortened keyid (in this case this one would be: 0xC827A062), is there a
particular reason for this? I recognize some people's keyids in the
short form, but do not in the longform.

6. this is awesome, huge thanks to everyone who has worked on this!

micah

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 2401 bytes --]

On 02/02/2011 08:18 PM, Jameson Rollins wrote:
> Hi, all.  I have pushed a new branch called "crypto" to my notmuch
> repository [0].  This branch provides full support for PGP/MIME signed
> and encrypted messages, including emacs UI support.

I have tested this, and am now using it.  I'm very happy with it.  I
support its inclusion in the mainline.

Thanks for doing this, Jamie.  This is excellent!

 ----less important stuff follows----

I want to raise one (non-blocking) question about the decryption to see
if anyone has any suggestions:

If you do "notmuch show --format=json" on a PGP/MIME-encrypted plaintext
message, it emits the base message, which is structured like this:

1 └┬╴multipart/encrypted
2  ├╴application/pgp-encrypted attachment
3  └╴application/octet-stream inline [msg.asc]

with these patches, if you do "notmuch show --format=json --decrypt", it
emits this instead:

1 └┬╴multipart/encrypted
2  └╴text/plain inline

and it attaches an encstatus (and possibly sigstatus, if the message was
signed) to part 1.  I'll call this "method A".

There are other methods that could be used as well, and it's worth
making sure we've chosen one that we think is what we'll want in the
future.  here are two other proposals:

Method B:

1 └┬╴multipart/encrypted
2  ├╴application/pgp-encrypted attachment
3  └╴text/plain inline

That is, just replace part 3 (the actual encrypted body) with the
decrypted material.

This has the advantage that for single-part encrypted messages, the
structure and part numbers of the message remains the same as without
--decrypt.


Method C:

1 └╴text/plain inline

That is, replace the entire multipart/encrypted with the decrypted material.

This avoids having an explicitly-labeled "multipart/encrypted" wrapper
around cleartext (which might be considered odd).  It would mean
attaching the encstatus and sigstatus directly to the decrypted part,
though.



I don't actually see any of these methods as being significantly better
than the others -- i think they all have some inherent ugliness.  So i'm
fine with going with method A as Jamie chose it and has it working.  But
i wanted to see if anyone had strong arguments in favor of the other
methods (or if there are other --decrypt methods we could use, for that
matter)

	--dkg


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 1030 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 2869 bytes --]

Hi Micah--

just wanted to follow up on your points/questions:

On 02/03/2011 11:25 AM, micah anderson wrote:
> 1. I personally think notmuch-show-process-pgpmime should default to
> true

note that with it set to false, you can still M-RET (instead of RET) on
an item in the summary window to have it set for that particular view.

> 2. in-line pgp messages don't have any processing done on them. getting
> the mime-encoded processing work is a huge step and I'm happy that
> works, in-line can (and IMHO, should) come later

yeah, inline is a whole different thing, and much more difficult to
manage programmatically in the notmuch backend, i think.  I dealing with
inline signatures and encryption should be a job for the emacs (or vim
or whatever) frontend.

> 3. i'm not sure expired/revoked keys are handled properly - tested on a
> message that was encrypted by a key that was revoked and got "End of
> file during parsing"

when you say "encrypted by" do you mean "encrypted to"?  do you have
access to the corresponding secret key?

> 4. messages that I sent encrypted to someone are not also encrypted to
> myself, which means that a thread which contains my replies isn't able
> to decrypt my messages in that thread and results in a purple
> 'decryption error'. Perhaps this is an emacs UI tweak that needs to be
> made to get messages also encrypted to my own key?

this is an issue for the emacs message modes (or maybe for your gpg
configuration), not for notmuch.

You either want to fix this in your emacs config by putting your
fingerprint into mml2015-signers and setting mml2015-encrypt-to-self

Or you want to set gpg's default-recipient-self option  (and
default-recipient option if you hold more than one secret key and want
to be sure it chooses the right one)

> 5. unknown keys are represented in a long format,
> eg. '0x5585F58CC827A062' when most tools represent them just with their
> shortened keyid (in this case this one would be: 0xC827A062), is there a
> particular reason for this?

Short keyIDs are easily spoofable; believing anything based on a matched
short keyID is a mistake.  "unknown keys" themselves may or may not have
properly signed a message -- since we don't have the key handy, we don't
have a way of checking.

note that "unknown key" is different from "good signature from known key
but we do not know who controls the key"

> I recognize some people's keyids in the
> short form, but do not in the longform.

You can derive the short form from the long form by ignoring everything
but the last 8 hex characters.  But if you actually recognize the short
form, i'd expect you to have the relevant key in your keyring; is this
really a use case worth bothering with?

do you have a suggestion for how you think it should behave differently?

	--dkg


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 1030 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 2102 bytes --]

On Thu, 03 Feb 2011 14:52:01 -0500, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:
> On 02/03/2011 11:25 AM, micah anderson wrote:
> > 1. I personally think notmuch-show-process-pgpmime should default to
> > true
> 
> note that with it set to false, you can still M-RET (instead of RET) on
> an item in the summary window to have it set for that particular view.

This is also useful if you set notmuch-show-process-pgpmime and ever
come across a message that is causing crypto problems.  M-RET will
return you to the normal view.

> > 3. i'm not sure expired/revoked keys are handled properly - tested on a
> > message that was encrypted by a key that was revoked and got "End of
> > file during parsing"
> 
> when you say "encrypted by" do you mean "encrypted to"?  do you have
> access to the corresponding secret key?

I also seem to be noticing issues with revoked keys.  I'm looking in to
the issue.  If anyone else notices something similar, please do relay
your experience.

> > 4. messages that I sent encrypted to someone are not also encrypted to
> > myself, which means that a thread which contains my replies isn't able
> > to decrypt my messages in that thread and results in a purple
> > 'decryption error'. Perhaps this is an emacs UI tweak that needs to be
> > made to get messages also encrypted to my own key?
> 
> this is an issue for the emacs message modes (or maybe for your gpg
> configuration), not for notmuch.
> 
> You either want to fix this in your emacs config by putting your
> fingerprint into mml2015-signers and setting mml2015-encrypt-to-self
> 
> Or you want to set gpg's default-recipient-self option  (and
> default-recipient option if you hold more than one secret key and want
> to be sure it chooses the right one)

Actually, I think the gpg option we're looking for here is
"encrypt-to".  "default-recipient-self" sets the recipient only if none
other is specified.  I just set "encrypt-to <mykeyid>" in my gpg.conf
and it seems to do as expected (all encrypted messages are also
encrypted to myself).

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[Darren McGuicken]

[-- Attachment #1: Type: text/plain, Size: 612 bytes --]

On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins <jrollins@finestructure.net> wrote:
> Please test and provide feedback.  I would really like to see this
> series merged into the mainline for the next release, if at all
> possible.

Fan.  Tastic.  Thanks so much for this, incredible work!

Grabbed and testing now, the only slight niggle I see is that when I
reply to an encrypted mail I still get:

  On Thu, 03 Feb 2011 19:17:45 +0000, Someone <foo@bar> wrote:
  Non-text part: application/pgp-encrypted
  Non-text part: application/octet-stream

Can we pass the decrypted text to message mode on a reply?

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 1036 bytes --]

On Thu, 03 Feb 2011 20:42:38 +0000, Darren McGuicken <mailing-notmuch@fernseed.info> wrote:
> On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins <jrollins@finestructure.net> wrote:
> > Please test and provide feedback.  I would really like to see this
> > series merged into the mainline for the next release, if at all
> > possible.
> 
> Fan.  Tastic.  Thanks so much for this, incredible work!

Thanks!  I hope people find it useful.

> Grabbed and testing now, the only slight niggle I see is that when I
> reply to an encrypted mail I still get:
> 
>   On Thu, 03 Feb 2011 19:17:45 +0000, Someone <foo@bar> wrote:
>   Non-text part: application/pgp-encrypted
>   Non-text part: application/octet-stream
> 
> Can we pass the decrypted text to message mode on a reply?

Yeah, some folks pointed this out on #notmuch this morning.  The
decryption is only happening in notmuch-show, but we need to do it in
notmuch-reply as well for the decrypted text to show up in the reply.
I'm working on it now.

jamie.


[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

always encrypting messages to self [was: Re: new "crypto" branch providing full PGP/MIME support]

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 846 bytes --]

On 02/03/2011 03:34 PM, Jameson Rollins wrote:
> On Thu, 03 Feb 2011 14:52:01 -0500, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:
>> You either want to fix this in your emacs config by putting your
>> fingerprint into mml2015-signers and setting mml2015-encrypt-to-self
>>
>> Or you want to set gpg's default-recipient-self option  (and
>> default-recipient option if you hold more than one secret key and want
>> to be sure it chooses the right one)
> 
> Actually, I think the gpg option we're looking for here is
> "encrypt-to".  "default-recipient-self" sets the recipient only if none
> other is specified.  I just set "encrypt-to <mykeyid>" in my gpg.conf
> and it seems to do as expected (all encrypted messages are also
> encrypted to myself).

Yes, this is correct.  thanks for figuring that out, Jamie.

	--dkg


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 1030 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[Darren McGuicken]

[-- Attachment #1: Type: text/plain, Size: 622 bytes --]

On Thu, 03 Feb 2011 13:02:41 -0800, Jameson Rollins <jrollins@finestructure.net> wrote:
> > Can we pass the decrypted text to message mode on a reply?
> Yeah, some folks pointed this out on #notmuch this morning.  The
> decryption is only happening in notmuch-show, but we need to do it in
> notmuch-reply as well for the decrypted text to show up in the reply.
> I'm working on it now.

Great news, thanks!

Also: another confirmation for the revoked/expired key behaviour.  Emacs
UI just reports:

  json-read-string: Wrong type argument: characterp, :json-eof

Let me know if you need any samples of misbehaving mails.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[David Bremner]

[-- Attachment #1: Type: text/plain, Size: 717 bytes --]

On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins <jrollins@finestructure.net> wrote:
> Hi, all.  I have pushed a new branch called "crypto" to my notmuch
> repository [0].  This branch provides full support for PGP/MIME signed
> and encrypted messages, including emacs UI support.  It has been applied
> on top of cworth's current master (21e97c50).  It includes the
> following:
> 

For a signed and encrypted message, if the signing key can't be found,
one just gets:

 [ multipart/encrypted: decryption error ]

which is a bit mystifying.  At least gpg on the command line will
decrypt the message and print a warning about the signature.

All the best, and thanks for working on this, 

d


[-- Attachment #2: Type: application/pgp-signature, Size: 315 bytes --]

Re: always encrypting messages to self [was: Re: new "crypto" branch providing full PGP/MIME support]

[Sebastian Spaeth]

[-- Attachment #1: Type: text/plain, Size: 203 bytes --]


I just love seeing all those yellow signatures. It reminds me to meet
you all in real life to verify your keys :-).

Thanks for implementing this, the whole lot of you. It works really
well.

Sebastian

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[micah anderson]

[-- Attachment #1: Type: text/plain, Size: 3392 bytes --]

On Thu, 03 Feb 2011 14:52:01 -0500, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:

> On 02/03/2011 11:25 AM, micah anderson wrote:
> > 1. I personally think notmuch-show-process-pgpmime should default to
> > true
> 
> note that with it set to false, you can still M-RET (instead of RET) on
> an item in the summary window to have it set for that particular view.

Yes, that is true. However, I still think the default should be
on. Especially considering all three people (myself included) who I've
seen try this have been puzzled by why it wasn't working and had to be
told that they needed to turn it on.

> > 3. i'm not sure expired/revoked keys are handled properly - tested on a
> > message that was encrypted by a key that was revoked and got "End of
> > file during parsing"
> 
> when you say "encrypted by" do you mean "encrypted to"?  do you have
> access to the corresponding secret key?

If I open a message that was sent to me and was encrypted by the sender
using a key that they have since revoked, or has expired, I saw this.

> > 5. unknown keys are represented in a long format,
> > eg. '0x5585F58CC827A062' when most tools represent them just with their
> > shortened keyid (in this case this one would be: 0xC827A062), is there a
> > particular reason for this?
> 
> Short keyIDs are easily spoofable; believing anything based on a matched
> short keyID is a mistake. 

Most humans I know reference their key by the keyid, where keyid means
the 8 character representation of their key. When they need to ensure
that their key is not being spoofed they either rely on the tools to do
cryptographic verification, or inspect the fingerprint. I dont think
moving towards using a longer format of the keyid helps here. The key is
unknown, and to get it known requires going through a key signing
process, or fingerprint verification, which isn't aided by having a
longer form keyid. 

If I am presented with a short form keyid, and I go and try and receive
that key from the keyservers and then I am presented with two different
options, then I'm going to inspect the two and determine then which one
is the right one. I dont need to front-load that knowledge in the
interface.

> > I recognize some people's keyids in the
> > short form, but do not in the longform.
> 
> You can derive the short form from the long form by ignoring
> everything but the last 8 hex characters.

Yes, I know that, but my brain parser looks at '0x5585F58CC827A062' and
has to squint to try and ignore the last 8 characters. In fact, its
often quicker for me to use my cursor to go up there and count backwards
8 characters and then hit a space so I can visually separate it. Visual
interface parse failure with long form.

> But if you actually recognize the short form, i'd expect you to have
> the relevant key in your keyring;

Not always true. In fact the parse error I was discussing above was
about a key I used to use, but have removed from my keyring, and my
secret keyring. In the long form, I did *not* recognize it. If it were
simply 1CF2D62A I would have instantly recognized it as my old key that
I've revoked.

> is this really a use case worth bothering with?

No.

> do you have a suggestion for how you think it should behave
> differently?

I think my suggestion was already made: short form.

m

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[David Bremner]

[-- Attachment #1: Type: text/plain, Size: 705 bytes --]

On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins <jrollins@finestructure.net> wrote:
> 
> Please test and provide feedback.  I would really like to see this
> series merged into the mainline for the next release, if at all
> possible.

I reported some pretty weird experiences yesterday on IRC with replies
not working. It turns out they were caused by having old versions of
xapian installed in /usr/local (which I think might have caused some
mixup with headers and libraries being different versions).

The decryption is working ok for me now (aside from the issue I
mentioned already with unknown encrypted+signed).

d

tl;dr: yesterdays weirdness was not the fault of J. Rollins.

[-- Attachment #2: Type: application/pgp-signature, Size: 315 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 1028 bytes --]

On Fri, 04 Feb 2011 08:24:38 -0400, David Bremner <bremner@unb.ca> wrote:
> On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins <jrollins@finestructure.net> wrote:
> > Hi, all.  I have pushed a new branch called "crypto" to my notmuch
> > repository [0].  This branch provides full support for PGP/MIME signed
> > and encrypted messages, including emacs UI support.  It has been applied
> > on top of cworth's current master (21e97c50).  It includes the
> > following:
> > 
> 
> For a signed and encrypted message, if the signing key can't be found,
> one just gets:
> 
>  [ multipart/encrypted: decryption error ]
> 
> which is a bit mystifying.  At least gpg on the command line will
> decrypt the message and print a warning about the signature.

So I need to look at this closer, but I think this is a GMime problem.
That error message is generated when the GMime decryption code returns
NULL for the decrypted part.  I'll try to make a test case for this, and
file a bug with GMime upstream.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: always encrypting messages to self [was: Re: new "crypto" branch providing full PGP/MIME support]

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 432 bytes --]

On Fri, 04 Feb 2011 14:04:23 +0100, Sebastian Spaeth <Sebastian@SSpaeth.de> wrote:
> I just love seeing all those yellow signatures. It reminds me to meet
> you all in real life to verify your keys :-).
> 
> Thanks for implementing this, the whole lot of you. It works really
> well.

Yes!  I'm really glad you're finding it useful.  Hopefully you're seeing
some green fully-valid-userid sigs in there as well...

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 2839 bytes --]

On 02/04/2011 11:59 AM, micah anderson wrote:
> On Thu, 03 Feb 2011 14:52:01 -0500, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:
>> when you say "encrypted by" do you mean "encrypted to"?  do you have
>> access to the corresponding secret key?
> 
> If I open a message that was sent to me and was encrypted by the sender
> using a key that they have since revoked, or has expired, I saw this.

so you hold key A, the sender holds key B, they encrypt the message with
B, and then revoke key B, and then you try to read the message (without
holding key B)?

or are you saying you hold key A, sender holds key B, they encrypt to A,
*sign* the message with B, and then revoke B?  If it's this case, does
the same thing happen even if the message is not encrypted?

Sorry for the nit-picking pedantry here -- I'm trying to get a clear
workflow for a test case.

> Most humans I know reference their key by the keyid, where keyid means
> the 8 character representation of their key. When they need to ensure
> that their key is not being spoofed they either rely on the tools to do
> cryptographic verification, or inspect the fingerprint. I dont think
> moving towards using a longer format of the keyid helps here. The key is
> unknown, and to get it known requires going through a key signing
> process, or fingerprint verification, which isn't aided by having a
> longer form keyid. 
> 
> If I am presented with a short form keyid, and I go and try and receive
> that key from the keyservers and then I am presented with two different
> options, then I'm going to inspect the two and determine then which one
> is the right one. I dont need to front-load that knowledge in the
> interface.

and if you're presented with only one key, you will just accept that
one?  that puts you at the mercy of the network (or the keyserver
operator at least, and we can't all have the luxury of being or knowing
the keyserver operator directly).

If you're arguing "i would like to be able to recognize the key by
32-bit key ID" i don't think it's actually possible to do so because of
the spoofability; a tool that provides cryptographic assurances should
discourage attempts to make the user vulnerable to spoofing.

If you're saying we should remove the 64-bit keyID entirely and just say
"signature from unknown, non-validated key", i think that's a defensible
position, though it's not one i would take.

>> is this really a use case worth bothering with?
> 
> No.

:)

>> do you have a suggestion for how you think it should behave
>> differently?
> 
> I think my suggestion was already made: short form.

and as i said above, i think this is a mistake if we're trying to
provide tools that give cryptographic assurances.

thanks for the testing and the corner cases!

	--dkg


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 1030 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 1047 bytes --]

On Fri, 04 Feb 2011 12:09:48 +0000, Darren McGuicken <mailing-notmuch@fernseed.info> wrote:
> On Thu, 03 Feb 2011 13:02:41 -0800, Jameson Rollins <jrollins@finestructure.net> wrote:
> > > Can we pass the decrypted text to message mode on a reply?
> > Yeah, some folks pointed this out on #notmuch this morning.  The
> > decryption is only happening in notmuch-show, but we need to do it in
> > notmuch-reply as well for the decrypted text to show up in the reply.
> > I'm working on it now.
> 
> Great news, thanks!

Thanks for the feedback!

> Also: another confirmation for the revoked/expired key behaviour.  Emacs
> UI just reports:
> 
>   json-read-string: Wrong type argument: characterp, :json-eof
> 
> Let me know if you need any samples of misbehaving mails.

Yeah, I've found some emails that are doing this as well.  I'm looking
in to what we can do to mitigate the problem, which I think is
ultimately a problem in GMime.  The json output should not be breaking,
though, so that I will definitely fix.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 739 bytes --]

On Fri, 04 Feb 2011 13:12:27 -0400, David Bremner <bremner@unb.ca> wrote:
> I reported some pretty weird experiences yesterday on IRC with replies
> not working. It turns out they were caused by having old versions of
> xapian installed in /usr/local (which I think might have caused some
> mixup with headers and libraries being different versions).
> 
> The decryption is working ok for me now (aside from the issue I
> mentioned already with unknown encrypted+signed).
...
> tl;dr: yesterdays weirdness was not the fault of J. Rollins.

phew!  Although there remains the issue of replying to encrypted
messages, since notmuch-reply was not modified to decrypt messages
before replying.  I'm working on that now.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 247 bytes --]

Hey, folks.  I've pushed some improvements to the "crypto" branch that:

* add support for replying to encrypted messages
* don't show sigstatus button for unsigned encrypted messages

As always, please let me know encounter any problems.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: new "crypto" branch providing full PGP/MIME support

[Darren McGuicken]

[-- Attachment #1: Type: text/plain, Size: 776 bytes --]

On Fri, 04 Feb 2011 09:32:23 -0800, Jameson Rollins <jrollins@finestructure.net> wrote:
> Yeah, I've found some emails that are doing this as well.  I'm looking
> in to what we can do to mitigate the problem, which I think is
> ultimately a problem in GMime.  The json output should not be
> breaking, though, so that I will definitely fix.

I pulled your latest updates to the crypto branch (I can reply to
encrypted mails without resorting to Gnus!  W00t!) and have noticed that
I'm not seeing the json eof anymore, but if I grab the keys for the
various folks in this thread as a test (Micah seems to be signing with
an expired key?) then emacs just hangs when trying to produce the
notmuch-show buffer.

I can actually see the notmuch show process pegging the cpu in top.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

[Review] Re: new "crypto" branch providing full PGP/MIME support

[David Bremner]

On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins <jrollins@finestructure.net> wrote:
> Hi, all.  I have pushed a new branch called "crypto" to my notmuch
> repository [0].  This branch provides full support for PGP/MIME signed
> and encrypted messages, including emacs UI support.  It has been applied
> on top of cworth's current master (21e97c50).  It includes the
> following:
> 

Further to our discussion on IRC the other day about providing feedback
on patches, I have run these patches pretty much all of February with
no glitches.  I am running them on 3 different machines, although they
are all Debian AMD64 boxes.

David

P.S. I also started the (possibly goofy) idea of putting [Review] in the
subject to highlight the fact there is feedback on the patches.

Re: [Review] Re: new "crypto" branch providing full PGP/MIME support

[Darren McGuicken]

[-- Attachment #1: Type: text/plain, Size: 700 bytes --]

On Sat, 26 Feb 2011 20:45:15 -0400, David Bremner <bremner@unb.ca> wrote:
> Further to our discussion on IRC the other day about providing
> feedback on patches, I have run these patches pretty much all of
> February with no glitches.  I am running them on 3 different machines,
> although they are all Debian AMD64 boxes.

If feedback is needed here then likewise, I've been running the crypto
branch since it was made available.  The only strangeness I've seen was
that which was reported in id:"87sjw2h6xy.fsf@bookbinder.fernseed.info"
for expired keys.

Even with that glitch, I really can't see me going back to vanilla
notmuch until these patches are pulled.  They're just ridiculously
useful.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: [Review] Re: new "crypto" branch providing full PGP/MIME support

[Sebastian Spaeth]

[-- Attachment #1: Type: text/plain, Size: 432 bytes --]

On Sun, 27 Feb 2011 10:41:48 +0000, Darren McGuicken <mailing-notmuch@fernseed.info> wrote:
> If feedback is needed here then likewise, I've been running the crypto
> branch since it was made available.  The only strangeness I've seen was
> that which was reported in id:"87sjw2h6xy.fsf@bookbinder.fernseed.info"
> for expired keys.

I also run the crypto branch since it has been published and it is
working just  fine.

Sebastian

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: [Review] Re: new "crypto" branch providing full PGP/MIME support

[Ross Glover]

I too am now running the crypto branch and find it quite amazing.  The
one feature I would like added, though, is some face color or
auto-tagging in the search buffer for mail with encrypted mime parts.
It seems like this could be achieved with notmuch effort (by someone
notme) by adding similar functionality to that of attachments in
index.cc.

ross

-- 


Sent from an Emacs buffer.

Re: [Review] Re: new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 778 bytes --]

On Mon, 28 Feb 2011 08:52:45 -0500, Ross Glover <ross@ross.mayfirst.org> wrote:
> I too am now running the crypto branch and find it quite amazing.  The
> one feature I would like added, though, is some face color or
> auto-tagging in the search buffer for mail with encrypted mime parts.
> It seems like this could be achieved with notmuch effort (by someone
> notme) by adding similar functionality to that of attachments in
> index.cc.

Yes, this is a good idea, Ross, and one that I've actually been wanting
to implement.  I was thinking of auto-tagging messages with signed parts
with something like "signed", and encrypted messages with "encrypted".
Do people like those tags, or would they prefer to see something
different?  Or more specific, like "pgp-signed"?

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [Review] Re: new "crypto" branch providing full PGP/MIME support

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 2597 bytes --]

On 02/28/2011 01:25 PM, Jameson Rollins wrote:
> On Mon, 28 Feb 2011 08:52:45 -0500, Ross Glover <ross@ross.mayfirst.org> wrote:
>> I too am now running the crypto branch and find it quite amazing.  The
>> one feature I would like added, though, is some face color or
>> auto-tagging in the search buffer for mail with encrypted mime parts.
>> It seems like this could be achieved with notmuch effort (by someone
>> notme) by adding similar functionality to that of attachments in
>> index.cc.
> 
> Yes, this is a good idea, Ross, and one that I've actually been wanting
> to implement.  I was thinking of auto-tagging messages with signed parts
> with something like "signed", and encrypted messages with "encrypted".
> Do people like those tags, or would they prefer to see something
> different?  Or more specific, like "pgp-signed"?

i don't care much about the difference between PGP/MIME and S/MIME
message formats, so i prefer the term "signed" to "pgp-signed" and
"encrypted" to "pgp-encrypted".

  ----

But: what does the "signed" tag mean? i wouldn't want to necessarily
conflate these four ideas:

 0) "this message claims to be cryptographically-signed"

 1) "we have verified a cryptographic signature over this message"

 2) "we have verified a cryptographic signature over this message from a
known key (that is, we believe we know who the key belongs to)"

 3) "we have verified a cryptographic signature on this message from the
sender claimed in the From: line"

3 implies 2, 2 implies 1, and 1 implies 0, of course.  But which level
would a "signed" tag signify?

I'll also note that signed+encrypted messages would not get tagged with
"signed" unless the recipient has successfully decrypted them.  And
then, it's possible that some sub-parts of a message are signed, and
others are not.  Would the tags indicate the maximum "level" found? or
the minimum?  something else?

  ----

For that matter, what would an automatically-placed "encrypted" tag
mean?  i can think of a few different approaches:

 0) some part of this message is wrapped in an encrypted MIME block

 1) some part of this message is wrapped in an encrypted MIME block that
claims to be decryptable by a key you control

 2) some part of this message is wrapped in an encrypted MIME block and
you can actually decrypt it (have decrypted it in the past?).

2 in particular couldn't be auto-assigned without having access to the
user's secret key material in the first place, but maybe it could be
assigned after a decryption succeeds?


	--dkg


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 1030 bytes --]

Re: [Review] Re: new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 747 bytes --]

On Mon, 28 Feb 2011 13:59:54 -0500, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:
> But: what does the "signed" tag mean? i wouldn't want to necessarily
> conflate these four ideas:

These are good points, Daniel.  However, I had actually just been
thinking of something much simpler, along the lines of just tagging
"signed" any message with a "multipart/signed" part, and "encrypted" any
message with a "multipart/encrypted" part.

This simpler approach would certainly satisfy my needs, without having
to get into sorting out all the complicated details in the points you
brought up.

Does that sound like it would work for folks, or would they like to see
a more nuanced approach to handling tagging of signed/encrypted
messages?

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [Review] Re: new "crypto" branch providing full PGP/MIME support

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 893 bytes --]

On 02/28/2011 02:56 PM, Jameson Rollins wrote:
> On Mon, 28 Feb 2011 13:59:54 -0500, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:
>> But: what does the "signed" tag mean? i wouldn't want to necessarily
>> conflate these four ideas:
> 
> These are good points, Daniel.  However, I had actually just been
> thinking of something much simpler, along the lines of just tagging
> "signed" any message with a "multipart/signed" part, and "encrypted" any
> message with a "multipart/encrypted" part.

this is a fair answer to my questions, not an evasion -- you're
selecting level 0 in both tracks, which is not a bad thing (it's
certainly simpler to get right!)

The outstanding question in my mind is whether those tags could be
mistaken by a naïve user for meaning one of the other concepts.  Is
there a way to name the tags to minimize that kind of confusion?

	--dkg


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 1030 bytes --]

Re: [Review] Re: new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 941 bytes --]

On Mon, 28 Feb 2011 15:08:39 -0500, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote:
> The outstanding question in my mind is whether those tags could be
> mistaken by a naïve user for meaning one of the other concepts.  Is
> there a way to name the tags to minimize that kind of confusion?

I think that would be difficult without using a long and cumbersome tag
name ("signed-but-not-verified"??).  But I think it might be a bit of a
moot point, since I kind of think that any user that actually
understands what a signature is, and what signature verification means,
is sophisticated enough to understand that the mere presence of a
signature does not mean it's been verified.  I could be wrong, though.

If folks have suggestions for disambiguating tag names that don't
themselves create further confusion on some other front, then I'm
inclined to just go with the simplest and most straightforward tag name.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [Review] Re: new "crypto" branch providing full PGP/MIME support

[Rob Browning]

Jameson Rollins <jrollins@finestructure.net> writes:

> If folks have suggestions for disambiguating tag names that don't
> themselves create further confusion on some other front, then I'm
> inclined to just go with the simplest and most straightforward tag name.

Are persistent tags required here?  The original question at least,
seemed to just be asking for a visual indicator that a message has
encrypted or signed bits.  So I wondered if that might be accomplished
without actual tags.

Just curious.
-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

Re: [Review] Re: new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 1223 bytes --]

On Mon, 28 Feb 2011 21:16:13 -0600, Rob Browning <rlb@defaultvalue.org> wrote:
> Are persistent tags required here?  The original question at least,
> seemed to just be asking for a visual indicator that a message has
> encrypted or signed bits.  So I wondered if that might be accomplished
> without actual tags.

Hey, Rob.  It probably could, but given that we already have
infrastructure for modifying the face of lines in the search output
based on tags, it therefore seems like the easiest way to achieve the
indicator that Ross was interested in would also be via a tag.  Any
other method would probably require extra hacking of the search
function, and hacking of the emacs interface to parse it and act on it.

To me personally the issue was more about wanting to be able to easily
find signed or encrypted messages.  The easiest way to do that would be
with a tag also, since that's kind of what they're for (again I can
imagine some other sort of internal flag in the database, but that seems
like it would be a lot more work).

Given that it should be fairly easy to tag these messages during notmuch
new, and that tags can be easily leveraged by existing functions, tags
seem to me to be the way to go.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [Review] Re: new "crypto" branch providing full PGP/MIME support

[Simon Fondrie-Teitler]

[-- Attachment #1: Type: text/plain, Size: 404 bytes --]

On Mon, 28 Feb 2011 14:24:03 +0100, Sebastian Spaeth <Sebastian@SSpaeth.de> wrote:
> On Sun, 27 Feb 2011 10:41:48 +0000, Darren McGuicken <mailing-notmuch@fernseed.info> wrote:
> I also run the crypto branch since it has been published and it is
> working just  fine.

I have just started using the crypto branch, and after a few misteps on
my part it is working. It works great. 

     --Simon

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

signed/encrypted tagging in crypto branch [was: Re: [Review] Re: new "crypto" branch providing full PGP/MIME support]

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 1077 bytes --]

Hey, folks.  I just pushed a couple of patches to my "crypto" branch [0]
that add support for auto-tagging of multipart/signed and
multipart/encrypted messages with the "signed" and "encrypted" tags
respectively.  Only new messages are thus tagged, so a database rebuild
is required to auto-tag old messages.

To be clear, after the previous discussion, these tags indicate nothing
about the validity of signatures or the decryptability of encrypted
messages.  They only indicate that a message contains a signed or
encrypted part according to the self-declared mime type.

This certainly makes it easier to find crypto messages, and it should
allow people to highlight signed or encrypted messages in the search
output using the "notmuch-search-lines-faces" customization variable
[1].

jamie.

[0] git://finestructure.net/notmuch
[1] The notmuch-search-lines-faces variable needs to be rewritten to
allow for full face customization support, but it also needs to continue
to specify a tag/face mapping.  Any elisp experts out there have any
good suggestions how to fix this?

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: signed/encrypted tagging in crypto branch

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 1284 bytes --]

On Sat, 05 Mar 2011 00:26:46 -0800, Jameson Rollins <jrollins@finestructure.net> wrote:
> Hey, folks.  I just pushed a couple of patches to my "crypto" branch [0]
> that add support for auto-tagging of multipart/signed and
> multipart/encrypted messages with the "signed" and "encrypted" tags
> respectively.  Only new messages are thus tagged, so a database rebuild
> is required to auto-tag old messages.

So I realized last night, what now seems obvious, that restoring tags
after a notmuch new will override any initial auto tagging.  This means
that doing a database rebuild will *not* crypto tag all your old mail if
you then restore tags from a tag dump afterwords.

I'm not sure if there's anything that can be done about this.  I think
we either have to have a way to merge tags, or the signed and encrypted
indicators need to exist in a different field in the database.  Tags
allow more flexibility in the UIs, but maybe we could just tag based on
a the new database field somehow?

It's not such a big deal that we only get "signed" and "encrypted" from
here forward, but it would be nice to re-tag old messages this way.  I
can imagine that something like this will come up again in the future,
and it would be nice if we had a solution.  I'm open to suggestions.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: signed/encrypted tagging in crypto branch

[Florian Friesdorf]

[-- Attachment #1: Type: text/plain, Size: 1845 bytes --]


I am also using the crypto branch and am very satisfied with it - thank
you!

There are some issues, but I will further debug before bothering you.

On Sun, 06 Mar 2011 11:15:00 -0800, Jameson Rollins <jrollins@finestructure.net> wrote:
> On Sat, 05 Mar 2011 00:26:46 -0800, Jameson Rollins <jrollins@finestructure.net> wrote:
> > Hey, folks.  I just pushed a couple of patches to my "crypto" branch [0]
> > that add support for auto-tagging of multipart/signed and
> > multipart/encrypted messages with the "signed" and "encrypted" tags
> > respectively.  Only new messages are thus tagged, so a database rebuild
> > is required to auto-tag old messages.
> 
> So I realized last night, what now seems obvious, that restoring tags
> after a notmuch new will override any initial auto tagging.  This means
> that doing a database rebuild will *not* crypto tag all your old mail if
> you then restore tags from a tag dump afterwords.
> 
> I'm not sure if there's anything that can be done about this.  I think
> we either have to have a way to merge tags, or the signed and encrypted
> indicators need to exist in a different field in the database.  Tags
> allow more flexibility in the UIs, but maybe we could just tag based on
> a the new database field somehow?
> 
> It's not such a big deal that we only get "signed" and "encrypted" from
> here forward, but it would be nice to re-tag old messages this way.  I
> can imagine that something like this will come up again in the future,
> and it would be nice if we had a solution.  I'm open to suggestions.

Is there a solution for retagging or selective tag merging already?

-- 
Florian Friesdorf <flo@chaoflow.net>
  GPG FPR: 7A13 5EEE 1421 9FC2 108D  BAAF 38F8 99A3 0C45 F083
Jabber/XMPP: flo@chaoflow.net
IRC: chaoflow on freenode,ircnet,blafasel,OFTC

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: signed/encrypted tagging in crypto branch

[Pieter Praet]

On Sun, 06 Mar 2011 11:15:00 -0800, Jameson Rollins <jrollins@finestructure.net> wrote:
> On Sat, 05 Mar 2011 00:26:46 -0800, Jameson Rollins <jrollins@finestructure.net> wrote:
> > Hey, folks.  I just pushed a couple of patches to my "crypto" branch [0]
> > that add support for auto-tagging of multipart/signed and
> > multipart/encrypted messages with the "signed" and "encrypted" tags
> > respectively.  Only new messages are thus tagged, so a database rebuild
> > is required to auto-tag old messages.
> 
> So I realized last night, what now seems obvious, that restoring tags
> after a notmuch new will override any initial auto tagging.  This means
> that doing a database rebuild will *not* crypto tag all your old mail if
> you then restore tags from a tag dump afterwords.
> 
> I'm not sure if there's anything that can be done about this.  I think
> we either have to have a way to merge tags, or the signed and encrypted
> indicators need to exist in a different field in the database.  Tags
> allow more flexibility in the UIs, but maybe we could just tag based on
> a the new database field somehow?
> 
> It's not such a big deal that we only get "signed" and "encrypted" from
> here forward, but it would be nice to re-tag old messages this way.  I
> can imagine that something like this will come up again in the future,
> and it would be nice if we had a solution.  I'm open to suggestions.
> 
> jamie.
Non-text part: application/pgp-signature
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch


As long as we're talking solely about "signed" and "encrypted" (so no
verification-wise information whatsoever), I'd definitely vote for a
dedicated database field.

It's absolutely immutable metadata, embedded in the message content.

No point in using tags for that, though it's not mutually exclusive:
"notmuch tag +signed -- is:signed" (or whatever, knock yourself out)

If folders -which DO change, although rarely- got one, so should crypto.


...but that's just my (insufficiently) humble opinion.


Peace

-Pieter

Re: new "crypto" branch providing full PGP/MIME support

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 173 bytes --]

Quick!  While cworth is working I want to push this back to the top of
the queue!

crypto branch is still ready for merge!

git://finestructure.net/notmuch

Thanks!

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]