Re: libnettle/libhogweed WIP

[Eli Zaretskii <eliz@gnu.org>]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Mon, 15 May 2017 17:55:34 -0400
> 
> On Fri, 21 Apr 2017 09:14:02 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

It's hard to have a dialog with 4 weeks between responses ;-)

> EZ> I think both this and the other sub-thread arrived at a point where it
> EZ> is important to present a list of use cases we envision and would like
> EZ> to support, because these kinds of decisions are prone to errors if we
> EZ> don't hold the use cases in our minds.
> 
> EZ> So could you please present such a list, describing the source of the
> EZ> text to be encrypted/decrypted/hashed, the purpose of the operation
> EZ> (i.e. some higher-level context), and the destination where the
> EZ> encrypted/decrypted/hashed text will go?  The list doesn't have to be
> EZ> exhaustive, but it should include the most important use cases.
> 
> Right now, I am mirroring the GnuTLS API. That API is in wide use. So I
> think the use cases are a large subset of the general GnuTLS use cases;
> we're enabling the same things.

I don't think this could be the case.  We are talking about using
these APIs in Emacs Lisp programs, where objects being manipulated
aren't C strings, and where file I/O is relatively rare and mostly
implied.  The use cases I was asking about are use cases for Lisp
programs.

> 
> On Fri, 21 Apr 2017 09:21:14 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 
> 
> >> From: Ted Zlatanov <tzz@lifelogs.com>
> >> Date: Thu, 20 Apr 2017 17:54:32 -0400
> >> 
> >> The KEY is secret and ideally would come from a file and never be
> >> seen at the Lisp level. But tests and other use cases may need it from a
> >> buffer (more secure but still accessible to Lisp) or a string (visible
> >> to all as a function parameter).
> 
> EZ> For testing, we could always write the key to a file before using it.
> EZ> What other use cases would need the key from other sources?
> 
> I think if the key is not on disk, we shouldn't force it to disk just to
> fit a always-a-file usage model.

My point was that the test suite cannot be a use case that drives our
design, because we can tweak any test to fit into any usage pattern we
want.  Valid and interesting use cases should come from outside the
test suite, they should come from real-life uses of these features by
Lisp programs.

> >> Getting the INPUT from a file enables large files (not in the first
> >> version probably) and other interesting use cases.
> 
> EZ> What other cases?  Large files is only theoretically useful, since
> EZ> generally Emacs cannot do useful things on files larger than
> EZ> most-positive-fixnum, and on 64-bit machines that is far enough to not
> EZ> care.
> 
> I think anything over 1 MB is pretty big.

Not for Emacs.  1MB is actually pretty small, and shouldn't even
bother us.  A system that cannot spare 1MB is already in trouble.

> There also pipes (pretty easy to fit the file model)

How do you fit a pipe into the (file "FOO") model?

> and network streams (probably a separate spec).

Well, this part of the discussion was specifically about the
(file "FOO") specs, so additional specs are a separate discussion.

OTOH, all of these are readable into a buffer, so the buffer model
could easily cover them all, no?

> EZ> I think we need to weigh flexibility against the complexity, and find
> EZ> the optimal balance.  So making the interfaces too complicated for use
> EZ> cases that will happen only very rarely, if at all, should be avoided.
> 
> I agree in general, BUT these are not end-user APIs. They are for
> application developers. They have to be flexible so we don't have to
> bolt-on these things later.

They have to be flexible enough, but not more flexible.  We are
talking about where to draw the line.

> EZ> The data will always leave traces, because doing the above involves
> EZ> reallocation of memory, so you are likely to leave traces in the page
> EZ> file and in memory.  But I don't think you can avoid that, whatever
> EZ> you do: as long as data needs to be read into memory to process it, it
> EZ> will always leave traces.
> 
> Would you agree the tight loop that overwrites the read block will leave
> fewer traces and offer fewer exposure opportunities?

I don't know, I'm not a specialist on how these usage patterns affect
VM and swap.  Maybe you are right.

> So how about a compromise for now: I can leave the `(file "foo")'
> capability out. I'll adjust the docs to remove mentions of it. Then,
> after the main patch is done, I can propose a followup patch to
> implement `(file "foo")' and we can decide if it's good or bad.
> 
> That will get the GnuTLS API integration working, and we can have a
> separate discussion about `(file "foo")' later. Lars, Eli, would that be
> acceptable?

I think this should be fine, thanks.