[bug#41363] knot-resolver: Enable reloading of policy files (add lua-cqueues)

[bug#41363] knot-resolver: Enable reloading of policy files (add lua-cqueues)

[Simon South]

This patch series enables the automatic reloading of response-policy
zone (RPZ) files by Knot Resolver. Specifically these patches

- Add package definitions for the cqueues Lua extension module and the
  luaossl module on which it relies, and

- Add lua5.1-cqueues as an input to knot-resolver.

With these changes applied, Knot Resolver can be configured with lines
like

    modules = { 'policy' }
    policy.add(policy.rpz(policy.DENY, '/etc/dns/blacklist.txt', true))

and it will automatically reload RPZ rules from /etc/dns/blacklist.txt
whenever that file changes. This makes it easy to use Knot Resolver to
block unwanted sites using a list of domains downloaded periodically
from the Internet.

I've tested these changes on x86-64 and aarch64. On x86-64 everything
works as expected.

On aarch64, the packages build and install fine but Knot Resolver fails
to load the configuration above with

    policy.lua:430: [poli] lua-cqueues required to watch and reload RPZ file

This is due to a known issue with LuaJIT on aarch64 (see e.g.
https://github.com/LuaJIT/LuaJIT/pull/230):

    $ ./pre-inst-env guix environment knot-resolver --ad-hoc knot-resolver
    $ $(head -n 3 `which kresd` | tail -n 2)  # set LUA_PATH, LUA_CPATH
    $ luajit -e 'require("cqueues")'
    luajit: bad light userdata pointer
    stack traceback:
            [C]: at 0xffffa556a960
            [C]: in function 'require'
            ...
    $

Otherwise (i.e. after changing "true" to "false" in the configuration
above) Knot Resolver continues to work as it did before, so I expect
existing users will not be affected.

I'll work on diagnosing the upstream bug but thought I'd submit these
patches in the meantime.

--
Simon South
simon@simonsouth.net

[bug#41363] [PATCH 1/3] gnu: Add lua-ossl.

[Simon South]

* gnu/packages/lua.scm (make-lua-ossl): New function.
(lua-ossl, lua5.1-ossl, lua5.2-ossl): New variables.
---
 gnu/packages/lua.scm | 63 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 63 insertions(+)

diff --git a/gnu/packages/lua.scm b/gnu/packages/lua.scm
index 181ce76559..147ed8d9f7 100644
--- a/gnu/packages/lua.scm
+++ b/gnu/packages/lua.scm
@@ -11,6 +11,7 @@
 ;;; Copyright © 2018, 2019 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2018 Fis Trivial <ybbs.daans@hotmail.com>
 ;;; Copyright © 2020 Nicolas Goaziou <mail@nicolasgoaziou.fr>
+;;; Copyright © 2020 Simon South <simon@simonsouth.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -38,6 +39,7 @@
   #:use-module (guix build-system trivial)
   #:use-module (gnu packages)
   #:use-module (gnu packages readline)
+  #:use-module (gnu packages m4)
   #:use-module (gnu packages tls)
   #:use-module (gnu packages xml)
   #:use-module (gnu packages glib)
@@ -292,6 +294,67 @@ directory structure and file attributes.")
 (define-public lua5.2-filesystem
   (make-lua-filesystem "lua5.2-filesystem" lua-5.2))
 
+(define (make-lua-ossl name lua)
+  (package
+    (name name)
+    (version "20170903")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append "https://25thandclement.com/~william/"
+                                  "projects/releases/luaossl-" version ".tgz"))
+              (sha256
+               (base32
+                "10392bvd0lzyibipblgiss09zlqh3a5zgqg1b9lgbybpqb9cv2k3"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:make-flags
+       (let ((out (assoc-ref %outputs "out"))
+             (lua-api-version ,(version-major+minor (package-version lua))))
+         (list "CC=gcc"
+               "CFLAGS='-D HAVE_SYS_SYSCTL_H=0'" ; sys/sysctl.h is deprecated
+               (string-append "DESTDIR=" out)
+               (string-append "LUA_APIS=" lua-api-version)
+               "prefix="))
+       #:phases
+       (modify-phases %standard-phases
+         (delete 'configure)
+         (delete 'check)
+         (add-after 'install 'check
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let ((out (assoc-ref outputs "out"))
+                   (lua-version ,(version-major+minor (package-version lua))))
+               (setenv "LUA_CPATH"
+                       (string-append out "/lib/lua/" lua-version "/?.so;;"))
+               (setenv "LUA_PATH"
+                       (string-append out "/share/lua/" lua-version "/?.lua;;"))
+               (with-directory-excursion "regress"
+                 (for-each (lambda (f)
+                             (invoke "lua" f))
+                           (find-files "." "^[0-9].*\\.lua$"))))
+             #t)))))
+    (native-inputs
+     `(("m4", m4)))
+    (inputs
+     `(("lua" ,lua)
+       ("openssl" ,openssl)))
+    (home-page "https://25thandclement.com/~william/projects/luaossl.html")
+    (synopsis "OpenSSL bindings for Lua")
+    (description "The luaossl extension module for Lua provides comprehensive,
+low-level bindings to the OpenSSL library, including support for certificate and
+key management, key generation, signature verification, and deep bindings to the
+distinguished name, alternative name, and X.509v3 extension interfaces.  It also
+binds OpenSSL's bignum, message digest, HMAC, cipher, and CSPRNG interfaces.")
+    (license license:expat)))
+
+(define-public lua-ossl
+  (make-lua-ossl "lua-ossl" lua))
+
+(define-public lua5.1-ossl
+  (make-lua-ossl "lua5.1-ossl" lua-5.1))
+
+(define-public lua5.2-ossl
+  (make-lua-ossl "lua5.2-ossl" lua-5.2))
+
 (define (make-lua-sec name lua)
   (package
     (name name)
-- 
2.26.2

[bug#41363] [PATCH 2/3] gnu: Add lua-cqueues.

[Simon South]

* gnu/packages/lua.scm (make-lua-cqueues): New function.
(lua-cqueues, lua5.1-cqueues, lua5.2-cqueues): New variables.
---
 gnu/packages/lua.scm | 106 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 106 insertions(+)

diff --git a/gnu/packages/lua.scm b/gnu/packages/lua.scm
index 147ed8d9f7..958b317d62 100644
--- a/gnu/packages/lua.scm
+++ b/gnu/packages/lua.scm
@@ -403,6 +403,112 @@ secure session between the peers.")
 (define-public lua5.2-sec
   (make-lua-sec "lua5.2-sec" lua-5.2))
 
+(define (make-lua-cqueues name lua lua-ossl)
+  (package
+    (name name)
+    (version "20171014")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append "https://25thandclement.com/~william/"
+                                  "projects/releases/cqueues-" version ".tgz"))
+              (sha256
+               (base32
+                "1dabhpn6r0hlln8vx9hxm34pfcm46qzgpb2apmziwg5z51fi4ksb"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:modules ((guix build gnu-build-system)
+                  (guix build utils)
+                  (ice-9 string-fun))
+       #:make-flags
+       (let ((out (assoc-ref %outputs "out"))
+             (lua-api-version ,(version-major+minor (package-version lua))))
+         (list "CC=gcc"
+               (string-append "LUA_APIS=" lua-api-version)))
+       #:phases
+       (modify-phases %standard-phases
+         (delete 'configure)
+         (delete 'check)
+         (replace 'install
+           (lambda* (#:key make-flags outputs #:allow-other-keys)
+             (let ((out (assoc-ref outputs "out")))
+               (apply invoke "make" "install"
+                      (append make-flags
+                              (list (string-append "DESTDIR=" out)
+                                    "prefix="))))))
+         (add-after 'install 'check
+           (lambda* (#:key inputs outputs make-flags #:allow-other-keys)
+             (let*
+                 ((lua-version ,(version-major+minor (package-version lua)))
+                  (env-suffix (if (equal? lua-version "5.1")
+                                  ""
+                                  (string-append
+                                   "_"
+                                   (string-replace-substring lua-version "." "_"))))
+
+                  (lua-ossl (assoc-ref inputs "lua-ossl"))
+                  (out (assoc-ref outputs "out"))
+
+                  (lua-cpath (lambda (p)
+                               (string-append p "/lib/lua/" lua-version "/?.so")))
+                  (lua-path (lambda (p)
+                              (string-append p "/share/lua/" lua-version "/?.lua"))))
+               ;; The test suite sets Lua-version-specific search-path variables
+               ;; when available so we must do the same, as these take
+               ;; precedence over the generic "LUA_CPATH" and "LUA_PATH"
+               (setenv (string-append "LUA_CPATH" env-suffix)
+                       (string-append
+                        (string-join (map lua-cpath (list out lua-ossl)) ";")
+                        ";;"))
+               (setenv (string-append "LUA_PATH" env-suffix)
+                       (string-append
+                        (string-join (map lua-path (list out lua-ossl)) ";")
+                        ";;"))
+
+               ;; Skip regression tests we expect to fail
+               (with-directory-excursion "regress"
+                 (for-each (lambda (f)
+                             (rename-file f (string-append f ".skip")))
+                           (append
+                            ;; Regression tests that require network
+                            ;; connectivity
+                            '("22-client-dtls.lua"
+                              "30-starttls-completion.lua"
+                              "62-noname.lua"
+                              "153-dns-resolvers.lua")
+
+                            ;; Regression tests that require LuaJIT
+                            '("44-resolvers-gc.lua"
+                              "51-join-defunct-thread.lua")
+
+                            ;; Regression tests that require Lua 5.3
+                            (if (not (equal? lua-version "5.3"))
+                                '("152-thread-integer-passing.lua")
+                                '()))))
+
+               (apply invoke "make" "check" make-flags)))))))
+    (native-inputs
+     `(("m4" ,m4)))
+    (inputs
+     `(("lua" ,lua)
+       ("openssl" ,openssl)))
+    (propagated-inputs
+     `(("lua-ossl" ,lua-ossl)))
+    (home-page "https://25thandclement.com/~william/projects/cqueues.html")
+    (synopsis "Event loop for Lua using continuation queues")
+    (description "The cqueues extension module for Lua implements an event loop
+that operates through the yielding and resumption of coroutines.  It is designed
+to be non-intrusive, composable, and embeddable within existing applications.")
+    (license license:expat)))
+
+(define-public lua-cqueues
+  (make-lua-cqueues "lua-cqueues" lua lua-ossl))
+
+(define-public lua5.1-cqueues
+  (make-lua-cqueues "lua5.1-cqueues" lua-5.1 lua5.1-ossl))
+
+(define-public lua5.2-cqueues
+  (make-lua-cqueues "lua5.2-cqueues" lua-5.2 lua5.2-ossl))
+
 (define-public lua-penlight
   (package
     (name "lua-penlight")
-- 
2.26.2

[bug#41363] [PATCH 3/3] gnu: knot-resolver: Enable automatic reloading of policy files.

[Simon South]

* gnu/packages/dns.scm (knot-resolver)[inputs]: Add lua5.1-cqueues.
---
 gnu/packages/dns.scm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/dns.scm b/gnu/packages/dns.scm
index 469ef7605d..fdf9ed9dea 100644
--- a/gnu/packages/dns.scm
+++ b/gnu/packages/dns.scm
@@ -723,8 +723,9 @@ synthesis, and on-the-fly re-configuration.")
        ("libuv" ,libuv)
        ("lmdb" ,lmdb)
        ("luajit" ,luajit)
-       ;; TODO: Add optional lua modules: basexx, cqueues and psl.
+       ;; TODO: Add optional lua modules: basexx and psl.
        ("lua-bitop" ,lua5.1-bitop)
+       ("lua-cqueues" ,lua5.1-cqueues)
        ("lua-filesystem" ,lua5.1-filesystem)
        ("lua-sec" ,lua5.1-sec)
        ("lua-socket" ,lua5.1-socket)))
-- 
2.26.2

[bug#41363] [PATCH 1/3] gnu: Add lua-ossl.

[Simon South]

[-- Attachment #1: Type: text/plain, Size: 131 bytes --]

luaossl doesn't actually require M4 to build, unlike cqueues.

Here's a replacement patch that omits M4 from the package inputs.



[-- Attachment #2: [PATCH 1/3] gnu: Add lua-ossl. --]
[-- Type: text/x-patch, Size: 3984 bytes --]

From 610918a771b84a081af24940ae94d35b1af7511e Mon Sep 17 00:00:00 2001
From: Simon South <simon@simonsouth.net>
Date: Fri, 15 May 2020 11:18:44 -0400
Subject: [PATCH 1/3] gnu: Add lua-ossl.
To: 41363@debbugs.gnu.org

* gnu/packages/lua.scm (make-lua-ossl): New function.
(lua-ossl, lua5.1-ossl, lua5.2-ossl): New variables.
---
 gnu/packages/lua.scm | 61 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 61 insertions(+)

diff --git a/gnu/packages/lua.scm b/gnu/packages/lua.scm
index 181ce76559..defb7b68e6 100644
--- a/gnu/packages/lua.scm
+++ b/gnu/packages/lua.scm
@@ -11,6 +11,7 @@
 ;;; Copyright © 2018, 2019 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2018 Fis Trivial <ybbs.daans@hotmail.com>
 ;;; Copyright © 2020 Nicolas Goaziou <mail@nicolasgoaziou.fr>
+;;; Copyright © 2020 Simon South <simon@simonsouth.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -38,6 +39,7 @@
   #:use-module (guix build-system trivial)
   #:use-module (gnu packages)
   #:use-module (gnu packages readline)
+  #:use-module (gnu packages m4)
   #:use-module (gnu packages tls)
   #:use-module (gnu packages xml)
   #:use-module (gnu packages glib)
@@ -292,6 +294,65 @@ directory structure and file attributes.")
 (define-public lua5.2-filesystem
   (make-lua-filesystem "lua5.2-filesystem" lua-5.2))
 
+(define (make-lua-ossl name lua)
+  (package
+    (name name)
+    (version "20170903")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append "https://25thandclement.com/~william/"
+                                  "projects/releases/luaossl-" version ".tgz"))
+              (sha256
+               (base32
+                "10392bvd0lzyibipblgiss09zlqh3a5zgqg1b9lgbybpqb9cv2k3"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:make-flags
+       (let ((out (assoc-ref %outputs "out"))
+             (lua-api-version ,(version-major+minor (package-version lua))))
+         (list "CC=gcc"
+               "CFLAGS='-D HAVE_SYS_SYSCTL_H=0'" ; sys/sysctl.h is deprecated
+               (string-append "DESTDIR=" out)
+               (string-append "LUA_APIS=" lua-api-version)
+               "prefix="))
+       #:phases
+       (modify-phases %standard-phases
+         (delete 'configure)
+         (delete 'check)
+         (add-after 'install 'check
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let ((out (assoc-ref outputs "out"))
+                   (lua-version ,(version-major+minor (package-version lua))))
+               (setenv "LUA_CPATH"
+                       (string-append out "/lib/lua/" lua-version "/?.so;;"))
+               (setenv "LUA_PATH"
+                       (string-append out "/share/lua/" lua-version "/?.lua;;"))
+               (with-directory-excursion "regress"
+                 (for-each (lambda (f)
+                             (invoke "lua" f))
+                           (find-files "." "^[0-9].*\\.lua$"))))
+             #t)))))
+    (inputs
+     `(("lua" ,lua)
+       ("openssl" ,openssl)))
+    (home-page "https://25thandclement.com/~william/projects/luaossl.html")
+    (synopsis "OpenSSL bindings for Lua")
+    (description "The luaossl extension module for Lua provides comprehensive,
+low-level bindings to the OpenSSL library, including support for certificate and
+key management, key generation, signature verification, and deep bindings to the
+distinguished name, alternative name, and X.509v3 extension interfaces.  It also
+binds OpenSSL's bignum, message digest, HMAC, cipher, and CSPRNG interfaces.")
+    (license license:expat)))
+
+(define-public lua-ossl
+  (make-lua-ossl "lua-ossl" lua))
+
+(define-public lua5.1-ossl
+  (make-lua-ossl "lua5.1-ossl" lua-5.1))
+
+(define-public lua5.2-ossl
+  (make-lua-ossl "lua5.2-ossl" lua-5.2))
+
 (define (make-lua-sec name lua)
   (package
     (name name)
-- 
2.26.2

[bug#41363] knot-resolver: Enable reloading of policy files (add lua-cqueues)

[Simon South]

Simon South <simon@simonsouth.net> writes:
> This is due to a known issue with LuaJIT on aarch64...

Just realized the URIs I used for cqueues and luaossl are out-of-date
and this issue has been addressed in newer releases
(https://github.com/wahern/cqueues/pull/225).

I'll send updated patches shortly.

-- 
Simon South
simon@simonsouth.net

[bug#41363] knot-resolver: Enable reloading of policy files (add lua-cqueues)

[Simon South]

[-- Attachment #1: Type: text/plain, Size: 1599 bytes --]

I'm posting updated patches here as a work-in-progress, in case anyone
else is interested. The patches are not yet ready to be applied.

The package definitions are pretty much complete (I believe), however
the regression tests for luajit-cqueues are failing for me on both
aarch64 and x86_64.

On aarch64, the test for issue #71 (only) fails with

    71-empty-cqueue: .......
    71-empty-cqueue: testing issue 71A
    71-empty-cqueue: 71A OK
    71-empty-cqueue: testing 71B
    71-empty-cqueue: setting alert on inner loop
    71-empty-cqueue: stepping inner loop
    71-empty-cqueue: polling inner loop
    71-empty-cqueue: stepping inner loop
    71-empty-cqueue: timeout before inner loop test completed

This test, along with the rest of the suite, passes fine if the stock
Lua 5.1 interpreter is used instead, which suggests a possible
regression in LuaJIT. (Building LuaJIT from the latest revision in its
source repository leads to the same error.)

On x86_64, that test passes but a different one fails:

    141-segfault-on-accept: OK
    PANIC: unprotected error in call to Lua API (attempt to call a thread value)

I've written to the authors of cqueues regarding the first issue but
have not received a response. I may try following up with LuaJIT's
author as a next step; a bit of analysis might reveal why these tests
are failing but my interest here is in getting a DNS server up and
running, not in learning Lua or diving into the internals of a compiler.

Perhaps someone more familiar with these libraries, language and tools
could help?

--
Simon South
ssouth@simonsouth.net


[-- Attachment #2: 0001-gnu-Add-luajit-ossl.patch --]
[-- Type: text/x-patch, Size: 5213 bytes --]

From d648ce07cd4828b4f2a848464a31efd76f00cb7c Mon Sep 17 00:00:00 2001
From: Simon South <simon@simonsouth.net>
Date: Fri, 15 May 2020 11:18:44 -0400
Subject: [PATCH 1/3] gnu: Add luajit-ossl.

* gnu/packages/lua.scm (make-lua-ossl): New function.
(luajit-ossl): New variable.
---
 gnu/packages/lua.scm | 82 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 82 insertions(+)

diff --git a/gnu/packages/lua.scm b/gnu/packages/lua.scm
index 181ce76559..ab86e24bb9 100644
--- a/gnu/packages/lua.scm
+++ b/gnu/packages/lua.scm
@@ -11,6 +11,7 @@
 ;;; Copyright © 2018, 2019 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2018 Fis Trivial <ybbs.daans@hotmail.com>
 ;;; Copyright © 2020 Nicolas Goaziou <mail@nicolasgoaziou.fr>
+;;; Copyright © 2020 Simon South <simon@simonsouth.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -38,6 +39,7 @@
   #:use-module (guix build-system trivial)
   #:use-module (gnu packages)
   #:use-module (gnu packages readline)
+  #:use-module (gnu packages m4)
   #:use-module (gnu packages tls)
   #:use-module (gnu packages xml)
   #:use-module (gnu packages glib)
@@ -292,6 +294,86 @@ directory structure and file attributes.")
 (define-public lua5.2-filesystem
   (make-lua-filesystem "lua5.2-filesystem" lua-5.2))
 
+(define (make-lua-ossl name lua)
+  (package
+    (name name)
+    (version "20190731")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/wahern/luaossl.git")
+                    (commit (string-append "rel-" version))))
+              (sha256
+               (base32
+                "03xmhy90qrby8pbwqdwy2xa7bk5jbpfzzrmpj83klzw7zkyf2k96"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:modules ((guix build gnu-build-system)
+                  (guix build utils)
+                  (ice-9 popen)
+                  (ice-9 rdelim))
+       #:make-flags
+       (let ((out (assoc-ref %outputs "out"))
+             (lua-api-version ,(if (eq? lua luajit)
+                                   "5.1"
+                                   (version-major+minor
+                                    (package-version lua)))))
+         (list "CC=gcc"
+               "CFLAGS='-D HAVE_SYS_SYSCTL_H=0'" ; sys/sysctl.h is deprecated
+               (string-append "DESTDIR=" out)
+               (string-append "LUA_APIS=" lua-api-version)
+               "prefix="))
+       #:phases
+       (modify-phases %standard-phases
+         (delete 'configure)
+         (delete 'check)
+         (add-after 'install 'check
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((lua-api-version ,(if (eq? lua luajit)
+                                          "5.1"
+                                          (version-major+minor
+                                           (package-version lua))))
+                    (lua-cpath (lambda (p)
+                                 (string-append p "/lib/lua/" lua-api-version
+                                                "/?.so")))
+                    (lua-path (lambda (p)
+                                (string-append p "/share/lua/" lua-api-version
+                                               "/?.lua")))
+                    (lua-interp (begin
+                                  (setenv "CC" "gcc")
+                                  (read-line
+                                   (open-pipe* OPEN_READ "mk/luapath" "lua"))))
+                    (out (assoc-ref outputs "out")))
+               (setenv "LUA_CPATH" (string-append (lua-cpath out) ";;"))
+               (setenv "LUA_PATH" (string-append (lua-path out) ";;"))
+
+               (with-directory-excursion "regress"
+                 (for-each (lambda (f)
+                             (invoke lua-interp f))
+                           (filter
+                            ;; Omit a test that requires lua-cqueues, as that
+                            ;; requires this package also
+                            (lambda (f)
+                              (not (equal? f "./148-custom-extensions.lua")))
+                            (find-files "." "^[0-9].*\\.lua$")))))
+             #t)))))
+    (inputs
+     `(("lua" ,lua)
+       ;; TODO: Support interpreters other than LuaJIT using luaffifb or a
+       ;; similar FFI-compatibility module
+       ("openssl" ,openssl)))
+    (home-page "https://25thandclement.com/~william/projects/luaossl.html")
+    (synopsis "OpenSSL bindings for Lua")
+    (description "The luaossl extension module for Lua provides comprehensive,
+low-level bindings to the OpenSSL library, including support for certificate and
+key management, key generation, signature verification, and deep bindings to the
+distinguished name, alternative name, and X.509v3 extension interfaces.  It also
+binds OpenSSL's bignum, message digest, HMAC, cipher, and CSPRNG interfaces.")
+    (license license:expat)))
+
+(define-public luajit-ossl
+  (make-lua-ossl "luajit-ossl" luajit))
+
 (define (make-lua-sec name lua)
   (package
     (name name)
-- 
2.26.2


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-gnu-Add-lua-cqueues.patch --]
[-- Type: text/x-patch, Size: 5778 bytes --]

From 0cf2c3b798071048d70c39d9f6697e8137e24251 Mon Sep 17 00:00:00 2001
From: Simon South <simon@simonsouth.net>
Date: Fri, 15 May 2020 18:18:51 -0400
Subject: [PATCH 2/3] gnu: Add lua-cqueues.

* gnu/packages/lua.scm (make-lua-cqueues): New function.
(luajit-cqueues): New variable.
---
 gnu/packages/lua.scm | 106 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 106 insertions(+)

diff --git a/gnu/packages/lua.scm b/gnu/packages/lua.scm
index ab86e24bb9..dd14f7dd43 100644
--- a/gnu/packages/lua.scm
+++ b/gnu/packages/lua.scm
@@ -422,6 +422,112 @@ secure session between the peers.")
 (define-public lua5.2-sec
   (make-lua-sec "lua5.2-sec" lua-5.2))
 
+(define (make-lua-cqueues name lua lua-ossl)
+  (package
+    (name name)
+    (version "20190813")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/wahern/cqueues.git")
+                    (commit (string-append "rel-" version))))
+              (sha256
+               (base32
+                "0yin39cldhc1l6rr33nj2z8la6rvv0c70g47rkvjb7s0wy760ggz"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:modules ((guix build gnu-build-system)
+                  (guix build utils)
+                  (ice-9 string-fun))
+       #:make-flags
+       (let ((lua-api-version ,(if (eq? lua luajit)
+                                   "5.1"
+                                   (version-major+minor (package-version lua)))))
+         (list "CC=gcc"
+               (string-append "LUA_APIS=" lua-api-version)))
+       #:phases
+       (modify-phases %standard-phases
+         (delete 'configure)
+         (delete 'check)
+         (replace 'install
+           (lambda* (#:key make-flags outputs #:allow-other-keys)
+             (let ((out (assoc-ref outputs "out")))
+               (apply invoke "make" "install"
+                      (append make-flags
+                              (list (string-append "DESTDIR=" out)
+                                    "prefix="))))))
+         (add-after 'install 'check
+           (lambda* (#:key inputs make-flags outputs #:allow-other-keys)
+             (let* ((lua-api-version ,(if (eq? lua luajit)
+                                          "5.1"
+                                          (version-major+minor
+                                           (package-version lua))))
+                    (lua-cpath (lambda (p)
+                                 (string-append p "/lib/lua/" lua-api-version
+                                                "/?.so")))
+                    (lua-path (lambda (p)
+                                (string-append p "/share/lua/" lua-api-version
+                                               "/?.lua")))
+                    (env-suffix (if (equal? lua-api-version "5.1")
+                                    ""
+                                    (string-append
+                                     "_"
+                                     (string-replace-substring lua-api-version
+                                                               "." "_"))))
+                    (lua-ossl (assoc-ref inputs "lua-ossl"))
+                    (out (assoc-ref outputs "out")))
+               ;; The test suite sets Lua-version-specific search-path variables
+               ;; when available so we must do the same, as these take
+               ;; precedence over the generic "LUA_CPATH" and "LUA_PATH"
+               (setenv (string-append "LUA_CPATH" env-suffix)
+                       (string-append
+                        (string-join (map lua-cpath (list out lua-ossl)) ";")
+                        ";;"))
+               (setenv (string-append "LUA_PATH" env-suffix)
+                       (string-append
+                        (string-join (map lua-path (list out lua-ossl)) ";")
+                        ";;"))
+
+               ;; Skip regression tests we expect to fail
+               (with-directory-excursion "regress"
+                 (for-each (lambda (f)
+                             (rename-file f (string-append f ".skip")))
+                           (append
+                            ;; Tests that require network connectivity
+                            '("22-client-dtls.lua"
+                              "30-starttls-completion.lua"
+                              "62-noname.lua"
+                              "153-dns-resolvers.lua")
+
+                            ;; Tests that require LuaJIT
+                            (if ,(not (eq? lua luajit))
+                                '("44-resolvers-gc.lua"
+                                  "51-join-defunct-thread.lua")
+                                '())
+
+                            ;; Tests that require Lua 5.3
+                            (if (not (equal? lua-api-version "5.3"))
+                                '("152-thread-integer-passing.lua")
+                                '()))))
+
+               (apply invoke "make" "check" make-flags)))))))
+    (native-inputs
+     `(("m4" ,m4)))
+    (inputs
+     `(("lua" ,lua)
+       ("openssl" ,openssl)))
+    (propagated-inputs
+     `(("lua-ossl" ,lua-ossl)))
+    (home-page "https://25thandclement.com/~william/projects/cqueues.html")
+    (synopsis "Event loop for Lua using continuation queues")
+    (description "The cqueues extension module for Lua implements an event loop
+that operates through the yielding and resumption of coroutines.  It is designed
+to be non-intrusive, composable, and embeddable within existing applications.")
+    (license license:expat)))
+
+(define-public luajit-cqueues
+  (make-lua-cqueues "luajit-cqueues" luajit luajit-ossl))
+
 (define-public lua-penlight
   (package
     (name "lua-penlight")
-- 
2.26.2


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0003-gnu-knot-resolver-Enable-automatic-reloading-of-poli.patch --]
[-- Type: text/x-patch, Size: 1009 bytes --]

From c8b4d696c96a3dc81e84aa05a24220cbadf90809 Mon Sep 17 00:00:00 2001
From: Simon South <simon@simonsouth.net>
Date: Sat, 16 May 2020 14:35:27 -0400
Subject: [PATCH 3/3] gnu: knot-resolver: Enable automatic reloading of policy
 files.

* gnu/packages/dns.scm (knot-resolver)[inputs]: Add lua5.1-cqueues.
---
 gnu/packages/dns.scm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/dns.scm b/gnu/packages/dns.scm
index 7ff0501ab2..dea24cf7c9 100644
--- a/gnu/packages/dns.scm
+++ b/gnu/packages/dns.scm
@@ -723,8 +723,9 @@ synthesis, and on-the-fly re-configuration.")
        ("libuv" ,libuv)
        ("lmdb" ,lmdb)
        ("luajit" ,luajit)
-       ;; TODO: Add optional lua modules: basexx, cqueues and psl.
+       ;; TODO: Add optional lua modules: basexx and psl.
        ("lua-bitop" ,lua5.1-bitop)
+       ("lua-cqueues" ,lua5.1-cqueues)
        ("lua-filesystem" ,lua5.1-filesystem)
        ("lua-sec" ,lua5.1-sec)
        ("lua-socket" ,lua5.1-socket)))
-- 
2.26.2

[bug#41363] knot-resolver: Enable reloading of policy files (add lua-cqueues)

[Ludovic Courtès]

Hi,

Julien, could you take a look at this patch series?  I figured you
probably know Knot better than I do.

  https://issues.guix.gnu.org/41363

Thanks in advance,
Ludo’.

Simon South <simon@simonsouth.net> skribis:

> This patch series enables the automatic reloading of response-policy
> zone (RPZ) files by Knot Resolver. Specifically these patches
>
> - Add package definitions for the cqueues Lua extension module and the
>   luaossl module on which it relies, and
>
> - Add lua5.1-cqueues as an input to knot-resolver.
>
> With these changes applied, Knot Resolver can be configured with lines
> like
>
>     modules = { 'policy' }
>     policy.add(policy.rpz(policy.DENY, '/etc/dns/blacklist.txt', true))
>
> and it will automatically reload RPZ rules from /etc/dns/blacklist.txt
> whenever that file changes. This makes it easy to use Knot Resolver to
> block unwanted sites using a list of domains downloaded periodically
> from the Internet.
>
> I've tested these changes on x86-64 and aarch64. On x86-64 everything
> works as expected.
>
> On aarch64, the packages build and install fine but Knot Resolver fails
> to load the configuration above with
>
>     policy.lua:430: [poli] lua-cqueues required to watch and reload RPZ file
>
> This is due to a known issue with LuaJIT on aarch64 (see e.g.
> https://github.com/LuaJIT/LuaJIT/pull/230):
>
>     $ ./pre-inst-env guix environment knot-resolver --ad-hoc knot-resolver
>     $ $(head -n 3 `which kresd` | tail -n 2)  # set LUA_PATH, LUA_CPATH
>     $ luajit -e 'require("cqueues")'
>     luajit: bad light userdata pointer
>     stack traceback:
>             [C]: at 0xffffa556a960
>             [C]: in function 'require'
>             ...
>     $
>
> Otherwise (i.e. after changing "true" to "false" in the configuration
> above) Knot Resolver continues to work as it did before, so I expect
> existing users will not be affected.
>
> I'll work on diagnosing the upstream bug but thought I'd submit these
> patches in the meantime.
>
> --
> Simon South
> simon@simonsouth.net

bug#41363: knot-resolver: Enable reloading of policy files (add lua-cqueues)

[Ludovic Courtès]

Hi Simon,

Simon South <simon@simonsouth.net> skribis:

> This patch series enables the automatic reloading of response-policy
> zone (RPZ) files by Knot Resolver. Specifically these patches
>
> - Add package definitions for the cqueues Lua extension module and the
>   luaossl module on which it relies, and
>
> - Add lua5.1-cqueues as an input to knot-resolver.
>
> With these changes applied, Knot Resolver can be configured with lines
> like
>
>     modules = { 'policy' }
>     policy.add(policy.rpz(policy.DENY, '/etc/dns/blacklist.txt', true))
>
> and it will automatically reload RPZ rules from /etc/dns/blacklist.txt
> whenever that file changes. This makes it easy to use Knot Resolver to
> block unwanted sites using a list of domains downloaded periodically
> from the Internet.
>
> I've tested these changes on x86-64 and aarch64. On x86-64 everything
> works as expected.

I went ahead and applied this patch series (builds fine on x86_64).

> On aarch64, the packages build and install fine but Knot Resolver fails
> to load the configuration above with
>
>     policy.lua:430: [poli] lua-cqueues required to watch and reload RPZ file
>
> This is due to a known issue with LuaJIT on aarch64 (see e.g.
> https://github.com/LuaJIT/LuaJIT/pull/230):
>
>     $ ./pre-inst-env guix environment knot-resolver --ad-hoc knot-resolver
>     $ $(head -n 3 `which kresd` | tail -n 2)  # set LUA_PATH, LUA_CPATH
>     $ luajit -e 'require("cqueues")'
>     luajit: bad light userdata pointer
>     stack traceback:
>             [C]: at 0xffffa556a960
>             [C]: in function 'require'
>             ...
>     $
>
> Otherwise (i.e. after changing "true" to "false" in the configuration
> above) Knot Resolver continues to work as it did before, so I expect
> existing users will not be affected.
>
> I'll work on diagnosing the upstream bug but thought I'd submit these
> patches in the meantime.

Should we disable the Lua dependency on AArch64?

Thank you, and apologies for the delay!

Ludo’.

[bug#41363] knot-resolver: Enable reloading of policy files (add lua-cqueues)

[Simon South]

Ludovic Courtès <ludo@gnu.org> writes:
> Should we disable the Lua dependency on AArch64?

I'd leave it as-is (despite the confusing error message), since you can
work around the issue on AArch64 with a rebuilt kernel configured to use
39-bit virtual addresses rather than the default 48.

Plus I'm most of the way towards updated patches that use more recent
versions of cqueues and luaossl, which avoids the problem
altogether. This is still on my to-do list though not a priority at the
moment.

-- 
Simon South
simon@simonsouth.net

[bug#41363] knot-resolver: Enable reloading of policy files (add lua-cqueues)

[Ludovic Courtès]

Simon South <simon@simonsouth.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>> Should we disable the Lua dependency on AArch64?
>
> I'd leave it as-is (despite the confusing error message), since you can
> work around the issue on AArch64 with a rebuilt kernel configured to use
> 39-bit virtual addresses rather than the default 48.
>
> Plus I'm most of the way towards updated patches that use more recent
> versions of cqueues and luaossl, which avoids the problem
> altogether. This is still on my to-do list though not a priority at the
> moment.

OK, sounds good!

Thanks,
Ludo’.