Re: libnettle/libhogweed WIP

[Ted Zlatanov <tzz@lifelogs.com>]

On Sat, 15 Apr 2017 12:32:32 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Fri, 14 Apr 2017 16:48:39 -0400
>> 
TZ> * TODO from Eli: avoid allocating a scratch buffer and then copying its
TZ> data (inside make_unibyte_string) into a newly-allocated string.
TZ> Instead, use make_uninit_string.
>> 
>> I've done this as much as possible. For AEAD output, I'm not sure how to
>> set the length of an already-allocated string; I didn't want to modify
>> `s.size' directly. I didn't see a function or macro to do it. This is
>> needed for gnutls_symmetric_aead().

EZ> I'm not sure I understand what you say here.  In particular, I see no
EZ> s.size in gnutls_symmetric_aead.  What did I miss?

EZ> I do see some issues in gnutls_symmetric_aead with how you create Lisp
EZ> strings.  You first correctly call make_uninit_string, which gives you
EZ> a unibyte string with no contents.  Then you populate that string's
EZ> data by calling gnutls_aead_cipher_encrypt resp. decrypt functions.
EZ> But then you call make_unibyte_string with the resulting data, which
EZ> is redundant: you already have the unibyte string with the correct
EZ> data in the 'storage' variable.  So you should just return 'storage',
EZ> like you do in, e.g., gnutls_symmetric.

These two comments are related: for example, the decryption with
CAMELLIA-256-GCM produces less bytes of output that the input. But I
don't want to try to anticipate that byte count--it complicates the code
needlessly. So instead I want to cut the Lisp string `storage' to
`storage_length' bytes after gnutls_aead_cipher_{encrypt,decrypt}()
modifies `storage_length'. I can't find a macro or function to do it, so
I used make_unibyte_string() for now and am asking how to do it better.

EZ> I see your methods are still strings, whereas I suggested making them
EZ> symbols.  Any reasons why you didn't?

Forgot :) Done now.

EZ> A minor nit: in doc strings, please always leave 2 spaces between
EZ> sentences, not 1.

That was the auto reformat. Fixed, thanks.

EZ> Only data structures defined via DEFVAR are accessible to Lisp, so
EZ> keeping the data in C and providing accessors for Lisp programs will
EZ> achieve the result, I think.  The accessor could wipe the data after N
EZ> accesses.

OK, I'll work on that later.

>> 2) Could there be a built-in C way to let C core functions take strings,
>> but callers can invoke them with '(buffer-string) to tell *the core
>> function* to call that form. In other words, I want the eval to be done
>> at the C level, so that looking at the call tree won't reveal the actual
>> string that was passed to the function. I think that would simplify my
>> code and other C code too. I can probably fake it with eval()? WDYT?

EZ> Why not simply pass nil as the input, with the meaning that it stands
EZ> for the current buffer's text?  Or, better yet, pass START and END to
EZ> allow a substring of current buffer's text.  We do that in a lot of
EZ> places (for different reasons, of course).

EZ> IOW, I see no reason to involve the Lisp interpreter for this job.  Am
EZ> I missing something?

We're assuming that there's only three ways to pass data to the
function, and they can all be expressed in the parameters instead of
code: buffer-string, buffer-substring, and direct string. I think there
may be other use cases, but maybe I'm wrong? Streaming data, event
handlers, and coding system adjustments maybe?

This is not standardized for core C functions AFAIK; I don't want to
rewrite the first 150 lines of secure_hash() and extend them when I need
to support more ways to pass data. So I think this should be provided by
the core somehow, in a way that can be reused. I thought of the
following possibilities:

a) Could the core C functions use the `interactive' spec? That may be the
best way, but I don't know of any core C functions that use it.

b) Another way is to write a special core C function to interpret these
special parameters and give back text. I could start writing this
function with the first 150 lines of secure_hash() and then try to
standardize the parameters.

c) my earlier idea, to eval a form in the core C function, but that's
slow and awkward. It *could* be a little better for performance, if the
C function doesn't call the form when it's not needed. And it's very
flexible.

Ted