bug#42998: 27.1; gnutls-symmetric-encrypt slow when providing cipher as symbol

bug#42998: 27.1; gnutls-symmetric-encrypt slow when providing cipher as symbol

[Vasilij Schneidermann]

[-- Attachment #1: Type: text/plain, Size: 4396 bytes --]

I've recently debugged bad performance for a piece of code showing
abysmal performance in its encryption loop.  Compare the following
benchmark results:

(benchmark 10000
  '(gnutls-symmetric-encrypt 'AES-128-CBC (make-string 16 0)
                                          (make-string 16 0)
                                          (make-string 16 0)))
;; => "Elapsed time: 1.808647s (1.708427s in 124 GCs)"
(benchmark 10000
  '(gnutls-symmetric-encrypt 4 (make-string 16 0)
                               (make-string 16 0)
                               (make-string 16 0)))
;; => "Elapsed time: 0.103690s (0.089148s in 6 GCs)"

This seems to be the fault of how the cipher value is mapped to the
cipher, if it's a symbol, the C code performs the equivalent of `(assoc
cipher (gnutls-ciphers))`.  Generating the cipher list over and over
again seems like a waste of processing power.  I'd consider either to
document this behavior or making the mapping more efficient, for example
by caching the cipher list instead of recomputing it over and over
again.  The latter would match the behavior as seen in other
cryptographic library wrappers, for example the OpenSSL one in Ruby.


In GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.22)
 of 2020-08-13 built on juergen
Windowing system distributor 'The X.Org Foundation', version 11.0.12008000
System Description: Arch Linux

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-wide-int
 --with-modules 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt'
 CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY
ACL GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON
PDUMPER LCMS2 GMP

Important settings:
  value of $LC_MESSAGES: 
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 45005 6273)
 (symbols 48 6004 1)
 (strings 32 15483 2032)
 (string-bytes 1 504714)
 (vectors 16 10090)
 (vector-slots 8 130067 10258)
 (floats 8 19 27)
 (intervals 56 196 0)
 (buffers 1000 11))

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

bug#42998: 27.1; gnutls-symmetric-encrypt slow when providing cipher as symbol

[Lars Ingebrigtsen]

Vasilij Schneidermann <mail@vasilij.de> writes:

> I'd consider either to
> document this behavior or making the mapping more efficient, for example
> by caching the cipher list instead of recomputing it over and over
> again.

The following fixes the problem, but I'm not quite sure whether it's
correct...  Is initialising a Lisp_Object to NULL something that's
allowed over all platforms?  I can't initialise it to the more natural
Qnil, since that's not a constant, C-wise.

diff --git a/src/gnutls.c b/src/gnutls.c
index 416fb15470..75bc7d970f 100644
--- a/src/gnutls.c
+++ b/src/gnutls.c
@@ -2303,6 +2303,8 @@ gnutls_symmetric (bool encrypting, Lisp_Object cipher,
                   Lisp_Object key, Lisp_Object iv,
                   Lisp_Object input, Lisp_Object aead_auth)
 {
+  static Lisp_Object cipher_cache = NULL;
+
   if (BUFFERP (key) || STRINGP (key))
     key = list1 (key);
 
@@ -2329,7 +2331,12 @@ gnutls_symmetric (bool encrypting, Lisp_Object cipher,
 
   if (SYMBOLP (cipher))
     {
-      info = Fassq (cipher, Fgnutls_ciphers ());
+      if (! cipher_cache)
+	{
+	  cipher_cache = Fgnutls_ciphers ();
+	  staticpro (&cipher_cache);
+	}
+      info = Fassq (cipher, cipher_cache);
       if (!CONSP (info))
 	xsignal2 (Qerror,
 		  build_string ("GnuTLS cipher is invalid or not found"),


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#42998: 27.1; gnutls-symmetric-encrypt slow when providing cipher as symbol

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Mon, 24 Aug 2020 15:01:01 +0200
> Cc: 42998@debbugs.gnu.org
> 
> Vasilij Schneidermann <mail@vasilij.de> writes:
> 
> > I'd consider either to
> > document this behavior or making the mapping more efficient, for example
> > by caching the cipher list instead of recomputing it over and over
> > again.
> 
> The following fixes the problem, but I'm not quite sure whether it's
> correct...  Is initialising a Lisp_Object to NULL something that's
> allowed over all platforms?

Since Lisp_Object is not really a pointer, at least not in all builds,
this might work, but is fragile, and thus not recommended.

> I can't initialise it to the more natural Qnil, since that's not a
> constant, C-wise.

Such initializations shall be done in the syms_of_* functions, in this
case syms_of_gnutls -- those run at build time, and are recorded in
the dumped memory.  yes, that means the static variable will not be
private to the function, but I don't think it's a problem.  I think
it'd also allow you to staticpro the variable only once.

Thanks.

bug#42998: 27.1; gnutls-symmetric-encrypt slow when providing cipher as symbol

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Such initializations shall be done in the syms_of_* functions, in this
> case syms_of_gnutls -- those run at build time, and are recorded in
> the dumped memory.  yes, that means the static variable will not be
> private to the function, but I don't think it's a problem.

OK, I'll do that instead.

> I think it'd also allow you to staticpro the variable only once.

Don't we need to staticpro the variable only once in any case, or am I
misunderstanding how staticpro works?  I assumed that it took the
address of the variable, and then it'll protect whatever that variable
points to, no matter whether we change the value or not?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#42998: 27.1; gnutls-symmetric-encrypt slow when providing cipher as symbol

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 42998@debbugs.gnu.org,  mail@vasilij.de
> Date: Mon, 24 Aug 2020 16:22:33 +0200
> 
> > I think it'd also allow you to staticpro the variable only once.
> 
> Don't we need to staticpro the variable only once in any case, or am I
> misunderstanding how staticpro works?

Yes.  I meant that one time should be in syms_of_gnutls.

bug#42998: 27.1; gnutls-symmetric-encrypt slow when providing cipher as symbol

[Lars Ingebrigtsen]

OK, I've redone this as a file-local variable, but I'm not totally sure
that I'm putting this inside the correct #ifdefs.  :-/

gnutls.c is a mess if #ifdefs -- does Emacs have a command that lists
all the #ifdefs that are in effect at the current point?

diff --git a/src/gnutls.c b/src/gnutls.c
index 416fb15470..0010553a9d 100644
--- a/src/gnutls.c
+++ b/src/gnutls.c
@@ -2298,6 +2298,8 @@ gnutls_symmetric_aead (bool encrypting, gnutls_cipher_algorithm_t gca,
 # endif
 }
 
+static Lisp_Object cipher_cache;
+
 static Lisp_Object
 gnutls_symmetric (bool encrypting, Lisp_Object cipher,
                   Lisp_Object key, Lisp_Object iv,
@@ -2329,7 +2331,9 @@ gnutls_symmetric (bool encrypting, Lisp_Object cipher,
 
   if (SYMBOLP (cipher))
     {
-      info = Fassq (cipher, Fgnutls_ciphers ());
+      if (NILP (cipher_cache))
+	cipher_cache = Fgnutls_ciphers ();
+      info = Fassq (cipher, cipher_cache);
       if (!CONSP (info))
 	xsignal2 (Qerror,
 		  build_string ("GnuTLS cipher is invalid or not found"),
@@ -2914,6 +2918,9 @@ syms_of_gnutls (void)
   defsubr (&Sgnutls_hash_digest);
   defsubr (&Sgnutls_symmetric_encrypt);
   defsubr (&Sgnutls_symmetric_decrypt);
+
+  cipher_cache = Qnil;
+  staticpro (&cipher_cache);
 #endif
 
   DEFVAR_INT ("gnutls-log-level", global_gnutls_log_level,


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#42998: 27.1; gnutls-symmetric-encrypt slow when providing cipher as symbol

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: mail@vasilij.de,  42998@debbugs.gnu.org
> Date: Mon, 24 Aug 2020 16:28:21 +0200
> 
> gnutls.c is a mess if #ifdefs -- does Emacs have a command that lists
> all the #ifdefs that are in effect at the current point?

I suggest "C-u 1 C-x $".

bug#42998: 27.1; gnutls-symmetric-encrypt slow when providing cipher as symbol

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> gnutls.c is a mess if #ifdefs -- does Emacs have a command that lists
>> all the #ifdefs that are in effect at the current point?
>
> I suggest "C-u 1 C-x $".

Thanks, but that didn't really help that much in gnutls.c, apparently.
But I worked my way up in the ifdef/endif mess and I think I put the
init in the right section.

Pushed to Emacs 28.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no