[RFE] Call `gpg --recv-keys` asynchronously

[RFE] Call `gpg --recv-keys` asynchronously

[Steven Allen]

[-- Attachment #1: Type: text/plain, Size: 347 bytes --]


Currently, notmuch fetches keys synchronously. This can be *very* slow
(I fetch keys over tor) and locks up emacs for the duration. Therefore,
this fetch really should be asynchronous.

Unfortunately, getting this to work well (i.e. not interleave output in
the `*notmuch-crypto-gpg-out*` buffer) requires a non-trivial amount of
work.

- Steven

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: [RFE] Call `gpg --recv-keys` asynchronously

[Sebastian Schwarz]

Allow me add to this RFE.

On 2017-02-28, Steven Allen wrote:
> Currently, notmuch fetches keys synchronously.  This can be
> *very* slow (I fetch keys over tor) and locks up emacs for the
> duration.  Therefore, this fetch really should be
> asynchronous.

Even with all keys already present signature verification takes
some time as well.  It would be nice if this was done
asynchronously as well.  This would greatly improve the speed of
opening threads with large numbers of signed messages.

Re: [RFE] Call `gpg --recv-keys` asynchronously

[David Bremner]

Sebastian Schwarz <seschwar@gmail.com> writes:

> Allow me add to this RFE.
>
> On 2017-02-28, Steven Allen wrote:
>> Currently, notmuch fetches keys synchronously.  This can be
>> *very* slow (I fetch keys over tor) and locks up emacs for the
>> duration.  Therefore, this fetch really should be
>> asynchronous.
>
> Even with all keys already present signature verification takes
> some time as well.  It would be nice if this was done
> asynchronously as well.  This would greatly improve the speed of
> opening threads with large numbers of signed messages.

Offhand, that sounds even harder.  With a noticable loss in security, we
could cache the verifications in the database so the cost is only paid
once.

d

Re: [RFE] Call `gpg --recv-keys` asynchronously

[David Bremner]

David Bremner <david@tethera.net> writes:

> Sebastian Schwarz <seschwar@gmail.com> writes:
>
>> Allow me add to this RFE.
>>
>> On 2017-02-28, Steven Allen wrote:
>>> Currently, notmuch fetches keys synchronously.  This can be
>>> *very* slow (I fetch keys over tor) and locks up emacs for the
>>> duration.  Therefore, this fetch really should be
>>> asynchronous.
>>
>> Even with all keys already present signature verification takes
>> some time as well.  It would be nice if this was done
>> asynchronously as well.  This would greatly improve the speed of
>> opening threads with large numbers of signed messages.
>
> Offhand, that sounds even harder.  With a noticable loss in security, we
> could cache the verifications in the database so the cost is only paid
> once.
>
> d

FWIW, these changes are pretty unrelated at an implimentation level. The
key fetching is done from emacs, while the verification is done in the
notmuch binary.

d

Re: [RFE] Call `gpg --recv-keys` asynchronously

[Sebastian Schwarz]

On 2017-02-28, David Bremner wrote:
> FWIW, these changes are pretty unrelated at an implimentation
> level.  The key fetching is done from emacs, while the
> verification is done in the notmuch binary.

I presumed the verification was done in Emacs as well.  But I
can see how asynchronous signature verification would basically
impossible without large scale refactoring, when it's done by
notmuch itself.

Re: [RFE] Call `gpg --recv-keys` asynchronously

[Jani Nikula]

On Tue, 28 Feb 2017, David Bremner <david@tethera.net> wrote:
> Sebastian Schwarz <seschwar@gmail.com> writes:
>> Even with all keys already present signature verification takes
>> some time as well.  It would be nice if this was done
>> asynchronously as well.  This would greatly improve the speed of
>> opening threads with large numbers of signed messages.
>
> Offhand, that sounds even harder.

Indeed. I think this could only be sensibly considered if we ever
decided to support decryption/verification in notmuch-emacs.

> With a noticable loss in security, we could cache the verifications in
> the database so the cost is only paid once.

From implementation POV one downside is that a read-only operation would
have to be turned into a read-write operation to store the cached result
in the database.

Assuming the database is as secure as the key store, I suppose the
reduction in security is related to key expiry and revocation. Those
bring in the problem of invalidating the cached verification. All of
this is something that has to be opt-in, judged by the user.

Even if the mechanisms might be fairly straightforward to implement, I'm
not really fond of the idea of adding somewhat complicated features not
enabled by default for all users. or features we couldn't even recommend
for people who can't make the judgement themselves.

BR,
Jani.

Re: [RFE] Call `gpg --recv-keys` asynchronously

[Daniel Kahn Gillmor]

On Tue 2017-02-28 12:35:05 -0800, Steven Allen wrote:
> Currently, notmuch fetches keys synchronously. This can be *very* slow
> (I fetch keys over tor) and locks up emacs for the duration. Therefore,
> this fetch really should be asynchronous.
>
> Unfortunately, getting this to work well (i.e. not interleave output in
> the `*notmuch-crypto-gpg-out*` buffer) requires a non-trivial amount of
> work.

i agree with you that this should be asynchronous, but my elisp-foo
isn't sufficient to know how to do this correctly.  But i think you've
identified a concrete and useful improvement, fwiw.

      --dkg

Re: [RFE] Call `gpg --recv-keys` asynchronously

[David Bremner]

Steven Allen <steven@stebalien.com> writes:

> Currently, notmuch fetches keys synchronously. This can be *very* slow
> (I fetch keys over tor) and locks up emacs for the duration. Therefore,
> this fetch really should be asynchronous.
>
> Unfortunately, getting this to work well (i.e. not interleave output in
> the `*notmuch-crypto-gpg-out*` buffer) requires a non-trivial amount of
> work.

Do I misunderstand, or was this fixed in a1d6e406f6f ?

Of course in the meantime the keyserver network basically went to
hell...

d

Re: [RFE] Call `gpg --recv-keys` asynchronously

[Steven Allen]

[-- Attachment #1.1: Type: text/plain, Size: 148 bytes --]

David Bremner <david@tethera.net> writes:
> Do I misunderstand, or was this fixed in a1d6e406f6f ?

Yes, this was fixed long ago. Thanks!

- Steven

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]