From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id KBqpLVD4rGL7lgAAbAwnHQ (envelope-from ) for ; Fri, 17 Jun 2022 23:55:28 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id 2J6cLVD4rGKN9AAA9RJhRA (envelope-from ) for ; Fri, 17 Jun 2022 23:55:28 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 82F853A828 for ; Fri, 17 Jun 2022 23:55:27 +0200 (CEST) Received: from localhost ([::1]:46602 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o2Jw6-0006dQ-8J for larch@yhetil.org; Fri, 17 Jun 2022 17:55:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45740) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2Jvi-0006dI-DL for guix-patches@gnu.org; Fri, 17 Jun 2022 17:55:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:53233) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1o2Jvh-0005zj-Uq for guix-patches@gnu.org; Fri, 17 Jun 2022 17:55:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1o2Jvh-0002HT-NU for guix-patches@gnu.org; Fri, 17 Jun 2022 17:55:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#56046] [PATCH] gnu: services: opensmtpd-records-task-list.org: Some notes about how I thought about building this service. And some additional task lists, as well as the WIP documentation. References: <20220617214618.12377-1-jbranso@dismail.de> In-Reply-To: <20220617214618.12377-1-jbranso@dismail.de> Resent-From: Joshua Branson Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 17 Jun 2022 21:55:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 56046 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 56046@debbugs.gnu.org Cc: Joshua Branson Received: via spool by 56046-submit@debbugs.gnu.org id=B56046.16555028728727 (code B ref 56046); Fri, 17 Jun 2022 21:55:01 +0000 Received: (at 56046) by debbugs.gnu.org; 17 Jun 2022 21:54:32 +0000 Received: from localhost ([127.0.0.1]:47131 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o2JvE-0002Gh-CT for submit@debbugs.gnu.org; Fri, 17 Jun 2022 17:54:32 -0400 Received: from mx1.dismail.de ([78.46.223.134]:21709) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o2JvC-0002GT-Gv for 56046@debbugs.gnu.org; Fri, 17 Jun 2022 17:54:31 -0400 Received: from mx1.dismail.de (localhost [127.0.0.1]) by mx1.dismail.de (OpenSMTPD) with ESMTP id 6e36d91a for <56046@debbugs.gnu.org>; Fri, 17 Jun 2022 23:54:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=dismail.de; h=from:to:cc :subject:date:message-id:mime-version:content-type :content-transfer-encoding; s=20190914; bh=q2Ce8q6uAxJu4Oen6Bhg4 +NbiTB2Phd+AjQxSfHsI3I=; b=DA2lvnFTF/3ht7A4bZpJqFMFSitWzwzlI3B81 iPlTnzmYO1gl0dlTpbcAhtM2SJL1AwI/Zcwzms6tBP3Q58azxDh8oJ0082t/1fcR QXT7ORoAFKCOERnj6Wkp/5ib5Z7oTY8qEh69sf4FQN8ywAew57wGq7RxeRieALrf MFx0ZFM+ew3lk59FyT/RlRLKbueRnGFNsDHLu25Wg7rw3WIIng1fZY9dcBSOcVA4 ddm9R+C6j2RVeJW7AmVfs0H2fF4i+JL4j5A/Pb6eKKnZa9erDT7bRz/76SoFLYLD MZKe1kYsprR+CXm3HWt4S8rY5n4t48ZHQz26LFtyDqmO1D/dw== Received: from smtp2.dismail.de ( [10.240.26.12]) by mx1.dismail.de (OpenSMTPD) with ESMTP id 779a86ac for <56046@debbugs.gnu.org>; Fri, 17 Jun 2022 23:54:22 +0200 (CEST) Received: from smtp2.dismail.de (localhost [127.0.0.1]) by smtp2.dismail.de (OpenSMTPD) with ESMTP id caf993c3 for <56046@debbugs.gnu.org>; Fri, 17 Jun 2022 23:54:22 +0200 (CEST) Received: by dismail.de (OpenSMTPD) with ESMTPSA id c1362d04 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO); Fri, 17 Jun 2022 23:54:18 +0200 (CEST) Date: Fri, 17 Jun 2022 17:54:07 -0400 Message-Id: <20220617215407.21290-1-jbranso@dismail.de> X-Mailer: git-send-email 2.36.1 MIME-Version: 1.0 Content-Type: text/plain; charset=y Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" Reply-to: Joshua Branson X-ACL-Warn: , Joshua Branson via Guix-patches From: Joshua Branson via Guix-patches via X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1655502928; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=Ke3Vgz9hSmTdPo/bn9cs7k6uy2L3by9Y50/Nihh3c+0=; b=LQyjYcWGzN9b1bRHzcf8XmkrRKOYaAkOlj9aruFQ5OIcZxs0mYH5Ak6S+N4IZv7aX5QCym 2uQDiWj/nL89fa2AD2zs/7bFNZ7SBti1YgHND/W3hPvqsGN3JJS7YY57E11V/VwtK76jhq wpqX2Ty2N2vyudfQ/a0V3Tyvw89VafPtZVluGSXeLSHoQok5cWA628mLp2qKuM6Dqupgrb HN3fqGTqxdNsfVkCrwNsdaDcQGbTH8FZY70DKMq61vJe6d8IOv3+meeCm3GiiXHMSE3sbn D8tn8Yrf6zdoymBIg1ompd16rWIjqr8eq7JhBg0D0q4zjb+vv2XaaLKfBE3lpg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1655502928; a=rsa-sha256; cv=none; b=PzIflDfarrmwQgBJCrUaJHaCO9yMsSthn+Mxo1fhBFas4YYUd/9iPEwvJ0LVBEJQl2fGsn e/NIWgV0wdfcrEtWPGMWJRY7dWWUQR9NCXBn5NUTjvU3+lxTvQsTlGU/Jdk+NhY89A+Qb9 9nKAaRqR3U4VK9Q4HaXYzoHDzElsKSUnXiaj01/GMw/IIArAvREGzsS/IRGNospDuWfZ7F wt1BJsPpaDPMBlnpyzkLD5jVDlasRXlmeTG7SlP4BHhYK+5T5AgZkHsvXFfM3DwPYIYZaS pau/Q8b2majpPSgSGaPnxggMn21oOfpBwD7jxDc4y19gwifWxvgNEnHtOEgARQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=dismail.de header.s=20190914 header.b=DA2lvnFT; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.59 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=dismail.de header.s=20190914 header.b=DA2lvnFT; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 82F853A828 X-Spam-Score: -4.59 X-Migadu-Scanner: scn0.migadu.com X-TUID: RJxOnB1MpgdQ --- gnu/services/opensmtpd-records-task-list.org | 5122 ++++++++++++++++++ 1 file changed, 5122 insertions(+) create mode 100644 gnu/services/opensmtpd-records-task-list.org diff --git a/gnu/services/opensmtpd-records-task-list.org b/gnu/services/opensmtpd-records-task-list.org new file mode 100644 index 0000000000..c138aab8fe --- /dev/null +++ b/gnu/services/opensmtpd-records-task-list.org @@ -0,0 +1,5122 @@ +#+title: Opensmtpd Records Task List +#+AUTHOR: Joshua Branson + + +(service (@ (gnu services mail) opensmtpd-service-type) + ((@ (gnu services mail) opensmtpd-configuration) + (config-file …))) + +* tasks +** PROJ I have decent data structures. now let's get some good code. [0/6] +*** why are good data structures important? +**** nckx's advice: use a simple 1-1 mapping +"...as I think Guix services ought to faithfully wrap the native +syntax whenever possible (implement alternative simple APIs on top of +that — fine)." + +-nckx from irc on #guix + +**** To follow nckx's advice, one might create the == like this: +#+BEGIN_SRC scheme + (service opensmtpd-service + (opensmtpd-configuration + (includes ...) + (tables ...) + (pkis ...) + (filters ...) + (listen-on ...) + (actions ...) + (matches ...))) +#+END_SRC + +Defining the service this way, makes it VERY easy from a development point of +view. But it makes it possible for users to create simple mistakes when +defining the service. + +For example, it is possible to define an nginx service that will successfully +reconfigure the system. BUT after reboot nginx refuses to start. Why? Who knows. +Guix won't tell you. Neither will the Shepherd. To fix this, the user has to go +digging into the nginx logs, and he might not know where to find those. If +possible, when the user specificies a == that has +obvious errors, then the guix services should make reconfigure fail and print a +helpful error message. + +**** BUT it would be better if the service uses better datastructures. + +I should follow nckx's advice, and Linus' advice: good programmers use good +datastructures. If you have good datastructures, then your code will almost +write itself. + +It might make the service a little harder to develop, but end-users will find +the service easier to use. This would eliminate common errors like misspellings +and give appropriate error messages. Practically it would ensure each +== has a corresponding ==, +creating a table name and then misspelling the table name later, and defining +a table but never using it, etc. + +**** Example configuration + +#+BEGIN_SRC scheme +(service opensmtpd-service-type + (let ([interface "lo"] + [creds-table (opensmtpd-table-configuration + (name "creds") + (data + (list + (cons "joshua" + "$6$Ec4m8FgKjT2F/03Y$k66ABdse9TzCX6qaALB3WBL9GC1rmAWJmaoSjFMpbhzat7DOpFqpnOwpbZ34wwsQYIK8RQlqwM1I/v6vsRq86."))))] + [receive-action (opensmtpd-action-local-delivery-configuration + (name "receive") + (method (opensmtpd-maildir-configuration + (pathname "/home/%{rcpt.user}/Maildir") + (junk #t))) + (virtual (opensmtpd-table-configuration + (name "virtual") + (data (list "josh" "jbranso@dismail.de")))))] + [filter-dkimsign (opensmtpd-filter-configuration + (name "dkimsign") + (exec #t) + (proc (string-append "/path/to/dkimsign -d gnucode.me -s 2021-09-22 -c relaxed/relaxed -k " + "/path/to/dkimsign-key user nobody group nobody")))] + [smtp.gnucode.me (opensmtpd-pki-configuration + (domain "smtp.gnucode.me") + (cert "opensmtpd.scm") + (key "opensmtpd.scm"))]) + (opensmtpd-configuration + (mta-max-deferred 50) + (queue + (opensmtpd-queue-configuration + (compression #t))) + (smtp + (opensmtpd-smtp-configuration + (max-message-size "10M"))) + (srs + (opensmtpd-srs-configuration + (ttl-delay "5d"))) + (listen-ons + (list + (opensmtpd-listen-on-configuration + (interface interface) + (port 25) + (secure-connection "tls") + (filters (list (opensmtpd-filter-phase-configuration + (name "noFRDNS") + (phase "commit") + (options (list (opensmtpd-option-configuration + (option "fcrdns") + (not #t)))) + (decision "disconnect") + (message "433 No FCRDNS")))) + (pki smtp.gnucode.me)) + ;; this lets local users logged into the system via ssh send email + (opensmtpd-listen-on-configuration + (interface interface) + (port 465) + (secure-connection "smtps") + (pki smtp.gnucode.me) + (auth creds-table) + (filters (list filter-dkimsign))) + (opensmtpd-listen-on-configuration + (interface interface) + (port 587) + (secure-connection "tls-require") + (pki smtp.gnucode.me) + (auth creds-table) + (filters (list filter-dkimsign))))) + (matches (list + (opensmtpd-match-configuration + (action (opensmtpd-action-relay-configuration + (name "relay"))) + (options (list (opensmtpd-option-configuration + (option "for any")) + (opensmtpd-option-configuration + (option "from any")) + (opensmtpd-option-configuration + (option "auth"))))) + (opensmtpd-match-configuration + (action receive-action) + (options (list (opensmtpd-option-configuration + (option "from any")) + (opensmtpd-option-configuration + (option "for domain") + (data (opensmtpd-table-configuration + (name "domain-table") + (data (list "gnucode.me" "gnu-hurd.com")))))))) + (opensmtpd-match-configuration + (action receive-action) + (options (list (opensmtpd-option-configuration + (option "for local")))))))))) +#+END_SRC + +:OldConfigurationSyntax: +#+BEGIN_SRC scheme + (service opensmtpd-service-type + (opensmtpd-configuration + (pkis (list + (opensmtpd-pki-configuration + ...))) + (tables (list + (opensmtpd-table-configuration + ...) + (opensmtpd-table-configuration + ...))) + (listen-ons + (list + (opensmtpd-listen-on-configuration + ...) + (opensmtpd-listen-on-configuration + ...))) + (actions + (list + (opensmtpd-action + ...) + (opensmtpd-action + ...))) + (matches (list + (opensmtpd-match-configuration + ...) + (opensmtpd-match-configuration + ...))) + (filter-chains + (list + (opensmtpd-filter-chain + (name "dropDumbEmails") + (filter-names (list "nofcrdnsDisconnect" + "nordnsDisconnect"))))) + (filter-phases + (list (opensmtpd-filter-phase-configuration + ...) + (opensmtpd-filter-phase-configuration + ...))))) +#+END_SRC + +Here you have to define the =pki=s twice! You define it once in the =pkis= +section, and then you reference it later. This could potentially cause a +mispelling error. That would be silly to debug as an end-user. + +:END: + +*** PROJ tweek the code for == & == records [4/7] +**** Why I chose the current datastructures of == & == + +According to the man page I have a four kinds of filters: + +#+BEGIN_EXAMPLE +1. filter chain-name chain {filter-name [, ...]} + Register a chain of filters chain-name, consisting of the filters listed from filter-name. + Filters part of a filter chain are executed in order of declaration for each phase that + they are registered for. A filter chain may be used in place of a filter for any direc‐ + tive but filter chains themselves. +2. filter filter-name phase phase-name match conditions decision + Register a filter filter-name. A decision about what to do with the mail is taken at + phase phase-name when matching conditions. Phases, matching conditions, and decisions are + described in MAIL FILTERING, below. +3. filter filter-name proc proc-name + Register "proc" filter filter-name backed by the proc-name process. +4. filter filter-name proc-exec command + Register and execute "proc" filter filter-name from command. If command starts with a + slash it is executed with an absolute path, else it will be run from + “/gnu/store/2d13sdz76ldq8zgwv4wif0zx7hkr3mh2-opensmtpd-6.8.0p2/libexec/opensmtpd”. +#+END_EXAMPLE + +=chain-name= could be easily represented as a list of filters. in the +opensmtpd-configuration-filter fieldname: + +#+BEGIN_SRC scheme +(opensmtpd-configuration + (listen-on + (filter + (list (opensmtpd-filter-configuration) + (opensmtpd-filter-configuration) + (opensmtpd-filter-configuration))))) +#+END_SRC + +For example, this is probably easier + +#+BEGIN_SRC scheme + (opensmtpd-configuration + (actions (list + (opensmtpd-action + (name "relay") + (method (opensmtpd-relay-configuration + (domain (opensmtpd-table-configuration + ;;(name "domains") ;; with some smart coding, the name would NOT be needed. + (data (list + "gnucode.me" + "gnu-hurd.com")))))))))) +#+END_SRC + +than the alternative: + +#+BEGIN_SRC scheme + (opensmtpd-configuration + (tables (list + (opensmtpd-table-configuration + (name "domains") + (data (list + "gnucode.me" + "gnu-hurd.com"))))) + (actions (list + (opensmtpd-action + (name "relay") + (method (opensmtpd-relay-configuration + (domain "domains"))))))) +#+END_SRC + +**** some example code for each of the 3 types of filters + +1. filter phase +#+BEGIN_SRC scheme +(opensmtpd-filter-phase-configuration + (name "phase") + (phase "connect") + (options + (list + (opensmtpd-option-configuration + (option "src") + (not #t) + (regex #t) + (table (opensmtpd-table-configuration (name "src-option-table") + (data (list "cat" "hot"))))))) + (decision "reject") + (message "We do not want spam here!")) +#+END_SRC + +#+RESULTS: + +2. filter proc +this is a filter-proc +#+BEGIN_SRC scheme +(opensmtpd-filter + (name "proc") + (proc "dkimsign")) +#+END_SRC + +3. filter proc-exec +#+BEGIN_SRC scheme +(opensmtpd-filter + (name "proc") + (exec #t) + (proc "dkimsign")) +#+END_SRC + +***** Why am I doing the data structure like the above? + +filter-proc and proc-exec as defined in man smtpd.conf can both use the same + record. That works just fine. + +But filter-phase is a different beast. I do NOT want someone to accidentally +define something like the following which is BAD data: + +#+BEGIN_SRC scheme +(opensmtpd-filter + (name "proc") + (exec #t) + (proc "dkimsign")) + +#+END_SRC +**** NO Is it advantageous/desireable to merge == & + +When a user creates a filter, he is either going to create a +=~ or an ~= NOT both. If +we define separate records, then it is impossible for a user to accidentally +define a filter record using fieldnames from both filter types. eg: + +#+BEGIN_SRC scheme +(opensmtpd-filter-configuration + (name "filter") + (exec #t) + (proc "dkimsign") + (phase "connect")) ;; this phase should NOT be there. this is a +#+END_SRC + +If == & == are separte then the above +would correctly result in an error message for free. +**** TODO make fieldname 'proc' accept a list of strings and/or a s + +Suppose you want to do dkimsigning in smtpd.conf. Here is how you might +register the official opensmtpd dkimsign filter: + +#+BEGIN_EXAMPLE +filter "dkimsign" proc-exec "filter-dkimsign -d -s \ + -k /gnu/store/2d13sdz76ldq8zgwv4wif0zx7hkr3mh2-opensmtpd-6.8.0p2/etc/dkim/private.key" user _dkimsign group _dkimsign +#+END_EXAMPLE + +For example my hacky code to do dkimsigning looks like: + +#+BEGIN_SRC scheme +filter \"dkimsign\" \ + proc-exec \"" path-to-filter-dkimsign " -d gnucode.me -s 2021-09-22 -c relaxed/relaxed -k " etc-dkimsign-key-file + " \" \ + user nobody group nogroup + +#+END_SRC + +Here is some example code of how we could create an +== that registers a dkimsign filter. The code +below probably will NOT work. + +#+BEGIN_SRC scheme +(let ((etc-dkimsign-key-file "filename.key") + (path-to-dkimsign-key "/etc/opensmtpd/"))) +(opensmtpd-filter-configuration + (name "dkimsign") + (proc (list + (file-append opensmtpd-filter-dkimsign "/libexec/opensmtpd/filter-dkimsign") + " -d gnucode.me -s 2021-09-22 -c relaxed/relaxed -k " + ~#(let ([UID-nobody (passwd:uid (getpw "nobody"))] + [GID-root (group:gid (getgr "root"))] + [GID-nogroup (group:gid (getgr "nogroup"))]) + ;; #o550 user root can read/open the directory + ;; and the group "root" can read/open the directory. + ;; change these two lines to (mkdir-p) ? + (unless (file-exists? "/etc/opensmtpd") + (mkdir "/etc/opensmtpd" #o550)) + + ;; root can read/write/execute on directory dkimsign + ;; group "root" can read and execute + (unless (file-exists? "/etc/opensmtpd/dkimsign") + (mkdir "/etc/opensmtpd/dkimsign" #o750)) + + (copy-file path-to-dkimsign-key etc-dkimsign-key-file) + ;; make the etc-dkimsign-key-file to owned by nobody and group nogroup. + (chown "/etc/opensmtpd" UID-nobody GID-root) + (chown "/etc/opensmtpd/dkimsign" UID-nobody GID-root) + (chown etc-dkimsign-key-file UID-nobody GID-nogroup) + "/etc/opensmtpd/dkimsign/2021-09-22-rsa1024-gnucode.me.key") + "user nobody group nogroup")) + (exec #)) +#+END_SRC + +Here is the full for how I currently run opensmtpd: + +#+BEGIN_SRC sh :dir ~/prog/gnu/guix/guix-config/linode-guix-system-configuration/ :results raw +cat opensmtpd.scm +#+END_SRC + +#+RESULTS: +#+BEGIN_SRC scheme +(define-module (opensmtpd) + #:use-module (guix gexp) + #:use-module (guix records) + #:use-module (gnu packages mail) ;;for finding location of filter-dkimsign + #:export ( + %smtpd.conf + )) + + +;; to create credentials for now, I need to do the following: +;; find /gnu/store -name '*encrypt*' | grep opensmtpd +;; /gnu/store/blah/opensmtpd/encrypt +(define creds + (plain-file "creds" + ;; this is my joshua's password for server. This can be found on dobby's /home/joshua/.authinfo/ + "joshua $6$Ec4m8FgKjT2F/03Y$k66ABdse9TzCX6qaALB3WBL9GC1rmAWJmaoSjFMpbhzat7DOpFqpnOwpbZ34wwsQYIK8RQlqwM1I/v6vsRq86.")) + +(define vdoms + (plain-file + "vdoms" + "gnucode.me +gnu-hurd.com")) + +(define vusers + (plain-file + "vusers" + "joshua@gnucode.me joshua +jbranso@gnucode.me joshua +postmaster@gnucode.me joshua")) + +(define path-to-filter-dkimsign + (file-append opensmtpd-filter-dkimsign "/libexec/opensmtpd/filter-dkimsign")) + +(define path-to-dkimsign-key (string-append (getcwd) "/email-dkim-ssh-keys/2021-09-22-rsa1024-gnucode.me.key")) +(define etc-dkimsign-key-file "/etc/opensmtpd/dkimsign/2021-09-22-rsa1024-gnucode.me.key") + +;; FIXME: This should become a derivation. Currently it just runs when I evaluate +;; %smtpd.conf. For example it should look like this? +;; (define build-exp +;; #~(begin +;; (mkdir #$output) +;; (chdir #$output) +;; (symlink (string-append #$coreutils "/bin/ls") +;; "list-files"))) + +;; I will need to extend the opensmtpd service, to create a directory +;; in etc. This line needs to be added to etc-service. +;; (service-extension etc-service-type opensmtpd-etc-service) +;; I'll then need to create a opensmtpd-etc-service procedure. ganeti has +;; a good example. + +;; It should also use the /etc service, which is a service for creating +;; directories and files in /etc ? +(define (create-etc-dkimsign-key-file) + #~(let ([UID-nobody (passwd:uid (getpw "nobody"))] + [GID-root (group:gid (getgr "root"))] + [GID-nogroup (group:gid (getgr "nogroup"))]) + ;; #o550 user root can read/open the directory + ;; and the group "root" can read/open the directory. + ;; change these two lines to (mkdir-p) ? + (unless (file-exists? "/etc/opensmtpd") + (mkdir "/etc/opensmtpd" #o550)) + + ;; root can read/write/execute on directory dkimsign + ;; group "root" can read and execute + (unless (file-exists? "/etc/opensmtpd/dkimsign") + (mkdir "/etc/opensmtpd/dkimsign" #o750)) + + (copy-file path-to-dkimsign-key etc-dkimsign-key-file) + ;; ;; ;; make the etc-dkimsign-key-file to owned by nobody and group nogroup. + (chown "/etc/opensmtpd" UID-nobody GID-root) + (chown "/etc/opensmtpd/dkimsign" UID-nobody GID-root) + (chown etc-dkimsign-key-file UID-nobody GID-nogroup) + etc-dkimsign-key-file)) + +(define %smtpd.conf + (mixed-text-file "smtpd.conf" + " +# This is the smtpd server system-wide configuration file. +# See smtpd.conf(5) for more information. +# borrowed from the archlinux guix +# https://wiki.archlinux.org/index.php/OpenSMTPD#Simple_OpenSMTPD/mbox_configuration + +# My TLS certificate and key +table aliases file:/etc/aliases +pki smtp.gnucode.me cert \"/etc/letsencrypt/live/gnucode.me/fullchain.pem\" +pki smtp.gnucode.me key \"/etc/letsencrypt/live/gnucode.me/privkey.pem\" + +# for now I am NOT using the virtual credentials +# table creds { joshua = $6$Ec4m8FgKjT2F/03Y$k66ABdse9TzCX6qaALB3WBL9GC1rmAWJmaoSjFMpbhzat7DOpFqpnOwpbZ34wwsQYIK8RQlqwM1I/v6vsRq86. } +table creds \"" creds "\" +table vdoms \"" vdoms "\" +# table vdoms { gnucode.me, gnu-hurd.com } +# table vusers { joshua@gnucode.me = joshua, jbranso@gnucode.me = joshua, postmaster@gnucode.me = joshua } +table vusers \"" vusers "\" + +# this totally works! run this as user nobody! +# info about dkimsign ...ing +# https://openports.pl/path/mail/opensmtpd-filters/dkimsign +# sudo -u nobody /gnu/store/g17vdv4l03bacn7qbdpb5v8l8vgdxcld-opensmtpd-filter-dkimsign-0.5/libexec/opensmtpd/filter-dkimsign -d gnucode.me -s 2020 -c relaxed/relaxed -k etc-dkimsign-key-file /home/joshua/linode-guix-system-configuration/email-dkim-ssh-keys/20201004-gnucode.me.key user nobody group nogroup + +filter \"dkimsign\" \ + proc-exec \"" path-to-filter-dkimsign " -d gnucode.me -s 2021-09-22 -c relaxed/relaxed -k " etc-dkimsign-key-file ;;(create-etc-dkimsign-key-file) + " \" \ + user nobody group nogroup + +# port 25 is used only for receiving from external servers, and they may start a +# TLS session if the want. +listen on eth0 port 25 tls pki smtp.gnucode.me + +# For sending messages from outside of this server, you need to authenticate and use +# TLS +listen on eth0 port 465 smtps pki smtp.gnucode.me auth filter \"dkimsign\" +listen on eth0 port 587 tls-require pki smtp.gnucode.me auth filter \"dkimsign\" + +# users logged-in/ssh-ed into the system can send email +listen on lo port 25 tls pki smtp.gnucode.me + +# receive email action +action \"receive\" maildir \"/home/%{rcpt.user}/Maildir\" junk virtual +# action send the email to the world +action \"send\" relay + +# We accept to send email from any mail from authenticated users +match for any from any auth action \"send\" + +#finally we receive any incoming email +# maybe the next \"from any\" should be changed to \"for rdns\". +match from any for domain action \"receive\" +match for local action \"receive\"")) +(define-module (opensmtpd) + #:use-module (guix gexp) + #:use-module (guix records) + #:use-module (gnu packages mail) ;;for finding location of filter-dkimsign + #:export ( + %smtpd.conf + )) + + +;; to create credentials for now, I need to do the following: +;; find /gnu/store -name '*encrypt*' | grep opensmtpd +;; /gnu/store/blah/opensmtpd/encrypt +(define creds + (plain-file "creds" + ;; this is my joshua's password for server. This can be found on dobby's /home/joshua/.authinfo/ + "joshua $6$Ec4m8FgKjT2F/03Y$k66ABdse9TzCX6qaALB3WBL9GC1rmAWJmaoSjFMpbhzat7DOpFqpnOwpbZ34wwsQYIK8RQlqwM1I/v6vsRq86.")) + +(define vdoms + (plain-file + "vdoms" + "gnucode.me +gnu-hurd.com")) + +(define vusers + (plain-file + "vusers" + "joshua@gnucode.me joshua +jbranso@gnucode.me joshua +postmaster@gnucode.me joshua")) + +(define path-to-filter-dkimsign + (file-append opensmtpd-filter-dkimsign "/libexec/opensmtpd/filter-dkimsign")) + +(define path-to-dkimsign-key (string-append (getcwd) "/email-dkim-ssh-keys/2021-09-22-rsa1024-gnucode.me.key")) +(define etc-dkimsign-key-file "/etc/opensmtpd/dkimsign/2021-09-22-rsa1024-gnucode.me.key") + +;; FIXME: This should become a derivation. Currently it just runs when I evaluate +;; %smtpd.conf. For example it should look like this? +;; (define build-exp +;; #~(begin +;; (mkdir #$output) +;; (chdir #$output) +;; (symlink (string-append #$coreutils "/bin/ls") +;; "list-files"))) + +;; I will need to extend the opensmtpd service, to create a directory +;; in etc. This line needs to be added to etc-service. +;; (service-extension etc-service-type opensmtpd-etc-service) +;; I'll then need to create a opensmtpd-etc-service procedure. ganeti has +;; a good example. + +;; It should also use the /etc service, which is a service for creating +;; directories and files in /etc ? +(define (create-etc-dkimsign-key-file) + #~(let ([UID-nobody (passwd:uid (getpw "nobody"))] + [GID-root (group:gid (getgr "root"))] + [GID-nogroup (group:gid (getgr "nogroup"))]) + ;; #o550 user root can read/open the directory + ;; and the group "root" can read/open the directory. + ;; change these two lines to (mkdir-p) ? + (unless (file-exists? "/etc/opensmtpd") + (mkdir "/etc/opensmtpd" #o550)) + + ;; root can read/write/execute on directory dkimsign + ;; group "root" can read and execute + (unless (file-exists? "/etc/opensmtpd/dkimsign") + (mkdir "/etc/opensmtpd/dkimsign" #o750)) + + (copy-file path-to-dkimsign-key etc-dkimsign-key-file) + ;; ;; ;; make the etc-dkimsign-key-file to owned by nobody and group nogroup. + (chown "/etc/opensmtpd" UID-nobody GID-root) + (chown "/etc/opensmtpd/dkimsign" UID-nobody GID-root) + (chown etc-dkimsign-key-file UID-nobody GID-nogroup) + etc-dkimsign-key-file)) + +(define %smtpd.conf + (mixed-text-file "smtpd.conf" + " +# This is the smtpd server system-wide configuration file. +# See smtpd.conf(5) for more information. +# borrowed from the archlinux guix +# https://wiki.archlinux.org/index.php/OpenSMTPD#Simple_OpenSMTPD/mbox_configuration + +# My TLS certificate and key +table aliases file:/etc/aliases +pki smtp.gnucode.me cert \"/etc/letsencrypt/live/gnucode.me/fullchain.pem\" +pki smtp.gnucode.me key \"/etc/letsencrypt/live/gnucode.me/privkey.pem\" + +# for now I am NOT using the virtual credentials +# table creds { joshua = $6$Ec4m8FgKjT2F/03Y$k66ABdse9TzCX6qaALB3WBL9GC1rmAWJmaoSjFMpbhzat7DOpFqpnOwpbZ34wwsQYIK8RQlqwM1I/v6vsRq86. } +table creds \"" creds "\" +table vdoms \"" vdoms "\" +# table vdoms { gnucode.me, gnu-hurd.com } +# table vusers { joshua@gnucode.me = joshua, jbranso@gnucode.me = joshua, postmaster@gnucode.me = joshua } +table vusers \"" vusers "\" + +# this totally works! run this as user nobody! +# info about dkimsign ...ing +# https://openports.pl/path/mail/opensmtpd-filters/dkimsign +# sudo -u nobody /gnu/store/g17vdv4l03bacn7qbdpb5v8l8vgdxcld-opensmtpd-filter-dkimsign-0.5/libexec/opensmtpd/filter-dkimsign -d gnucode.me -s 2020 -c relaxed/relaxed -k etc-dkimsign-key-file /home/joshua/linode-guix-system-configuration/email-dkim-ssh-keys/20201004-gnucode.me.key user nobody group nogroup + +filter \"dkimsign\" \ + proc-exec \"" path-to-filter-dkimsign " -d gnucode.me -s 2021-09-22 -c relaxed/relaxed -k " etc-dkimsign-key-file ;;(create-etc-dkimsign-key-file) + " \" \ + user nobody group nogroup + +# port 25 is used only for receiving from external servers, and they may start a +# TLS session if the want. +listen on eth0 port 25 tls pki smtp.gnucode.me + +# For sending messages from outside of this server, you need to authenticate and use +# TLS +listen on eth0 port 465 smtps pki smtp.gnucode.me auth filter \"dkimsign\" +listen on eth0 port 587 tls-require pki smtp.gnucode.me auth filter \"dkimsign\" + +# users logged-in/ssh-ed into the system can send email +listen on lo port 25 tls pki smtp.gnucode.me + +# receive email action +action \"receive\" maildir \"/home/%{rcpt.user}/Maildir\" junk virtual +# action send the email to the world +action \"send\" relay + +# We accept to send email from any mail from authenticated users +match for any from any auth action \"send\" + +#finally we receive any incoming email +# maybe the next \"from any\" should be changed to \"for rdns\". +match from any for domain action \"receive\" +match for local action \"receive\"")) + +#+END_SRC +**** TODO what does rewrite needs value mean? Should it be a number? this is for == +from the documentation + +rewrite value the command parameter is rewritten with value +**** DONE sanitize the fieldname 'filters'. + +I can probably reuse existing code from the sanitize procedure found in +== fieldname 'filters'. +**** DONE write a get-opensmtpd-filters procedure + +This procedure takes all the values of fieldname 'filters' +and fieldname 'filters'. It returns a list of +, , and filter-chains, which is a list +of and . An example of what this +might return is: + +#+BEGIN_SRC scheme +(list (list (opensmtpd-filter)) + (list (opensmtpd-filter-phase-configuration)) + (list (opensmtpd-filter) ; this list is a filter-chain + (opensmtpd-filter-phase-configuration)) + (list (opensmtpd-filter-phase-configuration) ; this list is also a filter chain + (opensmtpd-filter) + (opensmtpd-filter))) +#+END_SRC + +These are some example bits of code that I can test my resulting code on. + +All unique filters of and . 4 of +them. One listen-on has no filters. +#+BEGIN_SRC scheme +(let ([interface "lo"] + [filter-dkimsign (opensmtpd-filter + (name "dkimsign") + (exec #t) + (proc (string-append "/path/to/dkimsign -d gnucode.me -s 2021-09-22 -c relaxed/relaxed -k " + "/path/to/dkimsign-key user nobody group nobody")))]) + (opensmtpd-configuration + (listen-on-socket + (opensmtpd-listen-on-socket-configuration-configuration + (filters (list (opensmtpd-filter + (name "rspamd") + (proc "rspamd")))))) + (listen-ons + (list + (opensmtpd-listen-on-configuration + (interface interface) + (filters (list (opensmtpd-filter-phase-configuration + (name "noFRDNS") + (phase "commit") + (options (list (opensmtpd-option-configuration + (option "fcrdns") + (not #t)))) + (decision "disconnect") + (message "No FCRDNS"))))) + ;; this lets local users logged into the system via ssh send email + (opensmtpd-listen-on + (interface interface) + (port 27) + (filters (list filter-dkimsign))) + (opensmtpd-listen-on-configuration + (port 29) + (interface interface)) + (opensmtpd-listen-on-configuration + (interface interface) + (filters (list (opensmtpd-filter + (name "rspamd") + (proc "rspamd") + (exec #t))))))))) +#+END_SRC + +4 unique filters. One of the filters is a filter chain. +#+BEGIN_SRC scheme +(let ([interface "lo"] + [filter-dkimsign (opensmtpd-filter + (name "dkimsign") + (exec #t) + (proc (string-append "/path/to/dkimsign -d gnucode.me -s 2021-09-22 -c relaxed/relaxed -k " + "/path/to/dkimsign-key user nobody group nobody")))]) + (opensmtpd-configuration + (listen-on-socket + (opensmtpd-listen-on-socket-configuration-configuration + (filters (list (opensmtpd-filter + (name "spamassassain") + (proc "spamassassain")) + (opensmtpd-filter-phase-configuration + (name "rdns") + (phase "data") + (options (list + (opensmtpd-option-configuration + (option "rdns") + (not #t)))) + (decision "reject") + (message "No RDNS")) + (opensmtpd-filter + (name "block") + (proc "block")) + (opensmtpd-filter-phase-configuration + (name "auth") + (phase "commit") + (options (list + (opensmtpd-option-configuration + (option "auth") + (regex #t) + (not #t) + (table (opensmtpd-table-configuration + (name "auth-table") + (data (list ".*@gmail.com" + ".*@dismail.de"))))))) + (decision "junk")))))) + (listen-ons + (list + (opensmtpd-listen-on-configuration + (interface interface) + (filters (list (opensmtpd-filter-phase-configuration + (name "noFRDNS") + (phase "commit") + (options (list (opensmtpd-option-configuration + (option "fcrdns") + (not #t)))) + (decision "disconnect") + (message "No FCRDNS"))))) + ;; this lets local users logged into the system via ssh send email + (opensmtpd-listen-on-configuration + (interface interface) + (port 27) + (filters (list filter-dkimsign))) + (opensmtpd-listen-on-configuration + (port 29) + (interface interface)) + (opensmtpd-listen-on-configuration + (interface interface) + (filters (list (opensmtpd-filter + (name "rspamd") + (proc "rspamd") + (exec #t))))))))) +#+END_SRC + +No filters at all. +#+BEGIN_SRC scheme +(get-opensmtpd-filters (opensmtpd-configuration)) ; no filters at all. +#+END_SRC + +This one prints rspamd twice! The get-opensmtpd-filters procedure returns a +duplicate filter. While get-opensmtpd-filters does return a duplicate filter. +#+BEGIN_SRC scheme +(opensmtpd-configuration + (listen-on-socket + (opensmtpd-listen-on-configuration-socket-configuration + (filters + (list (opensmtpd-filter + (name "rando") + (proc "rando")))))) + (listen-ons + (list + (opensmtpd-listen-on-configuration + (port 25) + (filters (list (opensmtpd-filter-phase-configuration + (name "noFRDNS") + (phase "commit") + (options (list (opensmtpd-option-configuration + (option "fcrdns") + (not #t)))) + (decision "disconnect") + (message "No FCRDNS")) + (opensmtpd-filter + (name "rspamd") + (proc "rspamd"))))) + (opensmtpd-listen-on-configuration + (port 465) + (filters (list + (opensmtpd-filter + (name "rspamd") + (proc "rspamd")) + (opensmtpd-filter + (name "block") + (proc "block"))))) + (opensmtpd-listen-on-configuration + (port 587)) + (opensmtpd-listen-on-configuration + (port 999) + (filters (list + (opensmtpd-filter + (name "bogofilter") + (proc "bogofilter")))))))) +#+END_SRC + +#+BEGIN_SRC scheme +(opensmtpd-configuration + (listen-ons + (list + (opensmtpd-listen-on-configuration + (filters (list (opensmtpd-filter + (name "bogofilter") + (proc "bogofilter"))))) + (opensmtpd-listen-on-configuration + (filters (list (opensmtpd-filter + (name "noFRDNS") + (proc "noFRDNS")) + (opensmtpd-filter + (name "rspamd") + (proc "rspamd")))))))) +#+END_SRC +**** PROJ by which method should I turn the filters in == into strings? [1/3] + +in (opensmtpd-configuration->mixed-text-file ) do either + +1. one line of code. faster, but violates the convention set by the other lines + of code around it. + #+BEGIN_SRC scheme + ;; write out all the filters + (opensmtpd-filters->string (get-opensmtpd-filters record)) + #+END_SRC + + This task in done: [[*(opensmtpd-filters->string (get-opensmtpd-filters record))][(opensmtpd-filters->string (get-opensmtpd-filters record))]] +2. two lines of code. slower, but follows the convention set by the other lines + of code around it. + #+BEGIN_SRC scheme + (opensmtpd-configuration-fieldname->string record get-opensmtpd-filters-and-filter-phases opensmtpd-filter-and-filter-phase->string) + (opensmtpd-configuration-fieldname->string record get-opensmtpd-filter-chains opensmtpd-filter-chain->string) + #+END_SRC + +***** DONE (opensmtpd-filters->string (get-opensmtpd-filters record)) + +Have one procedure that prints out all filters. + + #+BEGIN_SRC scheme +;; write out all the filters +(opensmtpd-filters->string (get-opensmtpd-filters record)) + #+END_SRC + +***** PROJ 4 procedures: get-filter-and-filter-phases, filter-and-filter-phases->string, get-filter-chains, filter-chains->string + +The bonus with this method is that I can add these two lines in +opensmtpd-configuration->mixed-text-file and keep a consistent coding framework: + +#+BEGIN_SRC scheme +;; write out all the filters and filter-phases +(opensmtpd-configuration-fieldname->string record get-opensmtpd-filters-and-filter-phases opensmtpd-filter-and-filter-phase->string) +;; write out all the filter chains +(opensmtpd-configuration-fieldname->string record get-opensmtpd-filter-chains opensmtpd-filter-chain->string) +#+END_SRC +**** DONE fix the sanitize procedure for == fieldnames 'phase-name', 'decision', etc. [5/5] +***** DONE sanitize == so that fieldname 'decision' option "reject" and "disconnect" requires a 'message'. +:LOGBOOK: +CLOCK: [2022-04-01 Fri 22:45]--[2022-04-02 Sat 04:13] => 5:28 +:END: + +This message must be RFC compliant. The message must start with 4xx or 5xx +status code. + +#+BEGIN_SRC scheme +(opensmtpd-listen-on-configuration + (filters (list + (opensmtpd-filter-phase-configuration + (name "junk") + (phase "connect") + (decision "junk") + (options + (list + (opensmtpd-option-configuration + (option "rdns"))))) + (opensmtpd-filter-phase-configuration + (name "src") + (phase "connect") + (options + (list + (opensmtpd-option-configuration + (option "src") + (data (opensmtpd-table-configuration + (name "src-table") + (data (list "cat" "hat"))))))) + (decision "reject"))))) +#+END_SRC + +#+RESULTS: + +#+BEGIN_EXAMPLE + fieldname: 'decision' options "disconnect" and "reject" require fieldname 'message' +to have a string. +ice-9/boot-9.scm:1685:16: In procedure raise-exception: +Throw to key `bad!' with args `(#< name: "src" phase: "connect" options: (#< option: "src" not: #f regex: #f table: #< name: "src-table" file-db: #f values: ("cat" "hat") type: #:5894:22 (x)>>>) decision: "reject" message: #f value: #f>)'. + +Entering a new prompt. Type `,bt' for a backtrace or `,q' to continue. +#+END_EXAMPLE + +#+BEGIN_SRC scheme +(opensmtpd-listen-on-configuration + (filters (list + (opensmtpd-filter-phase-configuration + (name "junk") + (phase "connect") + (decision "junk") + (options + (list + (opensmtpd-option-configuration + (option "rdns"))))) + (opensmtpd-filter-phase-configuration + (name "src") + (phase "connect") + (options + (list + (opensmtpd-option-configuration + (option "src") + (data (opensmtpd-table-configuration + (name "src-table") + (data (list "cat" "hat"))))))) + (decision "reject") + (message "322 Hello"))))) + fieldname: 'decision' options +"disconnect" and "reject" require fieldname 'message' to have a string. +The 'message' string must be RFC commpliant, which means that the string +must begin with a 4xx or 5xx status code. +ice-9/boot-9.scm:1685:16: In procedure raise-exception: +Throw to key `bad!' with args `(#< name: "src" phase: "connect" options: (#< option: "src" not: #f regex: #f data: #< name: "src-table" file-db: #f data: ("cat" "hat") type: #:1153:21 (x)>>>) decision: "reject" message: "322 Hello" value: #f>)'. + +Entering a new prompt. Type `,bt' for a backtrace or `,q' to continue. +#+END_SRC +***** DONE sanitize == so that fieldname 'decision' option "rewrite" requires a 'value'. +:LOGBOOK: +CLOCK: [2022-04-01 Fri 22:45]--[2022-04-02 Sat 04:13] => 5:28 +:END: + +#+BEGIN_SRC scheme +(opensmtpd-listen-on-configuration + (filters + (list + (opensmtpd-filter-phase-configuration + (name "noFRDNS") + (phase "commit") + (options (list (opensmtpd-option-configuration + (option "fcrdns") + (not #t)))) + (value #f) + (decision "rewrite")) ;; there needs to be a value here. rewrite requires a value! + ))) +$12 = #< interface: "lo" family: #f auth: #f auth-optional: #f filters: (#< name: "noFRDNS" phase: "commit" options: (#< option: "fcrdns" not: #t regex: #f data: #f>) decision: "rewrite" message: #f value: 343>) hostname: #f hostnames: #f mask-src: #f disable-dsn: #f pki: #f port: #f proxy-v2: #f received-auth: #f secure-connection: #f tag: #f> +scheme@(opensmtpd-records) [10]> (opensmtpd-listen-on-configuration + (filters + (list + (opensmtpd-filter-phase-configuration + (name "noFRDNS") + (phase "commit") + (options (list (opensmtpd-option-configuration + (option "fcrdns") + (not #t)))) + (decision "rewrite")) ;; there needs to be a value here. rewrite requires a value! + ))) + fieldname: 'decision' option +"rewrite" requires fieldname 'value' +to have a number. +ice-9/boot-9.scm:1685:16: In procedure raise-exception: +Throw to key `bad!' with args `(#< name: "noFRDNS" phase: "commit" options: (#< option: "fcrdns" not: #t regex: #f data: #f>) decision: "rewrite" message: #f value: #f>)'. + +Entering a new prompt. Type `,bt' for a backtrace or `,q' to continue. +#+END_SRC + +***** DONE sanitize == so that fieldname 'decision' option "junk" and "bypass" have no message or value + +#+BEGIN_SRC scheme +(opensmtpd-listen-on-configuration + (filters + (list + (opensmtpd-filter-phase-configuration + (name "noFRDNS") + (phase "commit") + (options (list (opensmtpd-option-configuration + (option "fcrdns") + ))) + (decision "junk") + (message "This is not a good email."))))) ; there should NOT be a message here! junk has no message. +#+END_SRC +***** DONE sanitize the options too. rdns requires a table for instance: + +#+BEGIN_EXAMPLE +At each phase, various conditions may be matched. The fcrdns, rdns, and src data are +available in all phases, but other data must have been already submitted before they are +available. + + fcrdns forward-confirmed reverse DNS is valid + rdns session has a reverse DNS + rdns session has a reverse DNS in table + src
source address is in table + helo
helo name is in table + auth session is authenticated + auth
session username is in table + mail-from
sender address is in table + rcpt-to
recipient address is in table +#+END_EXAMPLE +***** DONE sanitize make sure that 'junking happens before phase 'committed'. +#+BEGIN_EXAMPLE + Descisions can be taken at any phase, though junking can only happen before a message is committed. +#+END_EXAMPLE + +#+BEGIN_SRC scheme +(opensmtpd-listen-on-configuration + (filters + (list + (opensmtpd-filter-phase-configuration + (name "junk-after-commit") + (phase "commit") + (options (list (opensmtpd-option-configuration + (option "fcrdns") + ))) + (decision "junk"))))) +#+END_SRC +*** PROJ make fieldnames that need a table accept a value of table [3/4] +**** DONE opensmtpd-action-local-delivery-configuration [3/3] +***** DONE alias == [2/2] +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 03:53] +:END: +****** DONE change the sanitize portion of the fieldname alias in the + +The below code does work in a REPL. +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +(opensmtpd-action-local-delivery-configuration + (alias + (opensmtpd-table-configuration + (name "My-table") + (data (list "gnu-hurd.com" "gnucode.me"))))) +#+END_SRC + +#+RESULTS: +****** DONE change relevant portions in opensmtpd-action-local-delivery-configuration->string +This bit of code works in the repl too! +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +((@@ (opensmtpd-records) opensmtpd-action-local-delivery-configuration->string) + (opensmtpd-action-local-delivery-configuration + (alias + (opensmtpd-table-configuration + (name "My-table") + (data (list "gnu-hurd.com" "gnucode.me")))))) +#+END_SRC + +***** DONE userbase == [2/2] +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 03:53] +:END: + +****** DONE change the sanitize portion of the fieldname userbase in the + +The below code does work in a REPL. +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) +(opensmtpd-action-local-delivery-configuration + (userbase + (opensmtpd-table-configuration + (name "this") + (data (list "job" "done"))))) +#+END_SRC + +#+RESULTS: +****** DONE change relevant portions in opensmtpd-action-local-delivery-configuration->string +This bit of code works in the repl too! +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +((@@ (opensmtpd-records) opensmtpd-action-local-delivery-configuration->string) + (opensmtpd-action-local-delivery-configuration + (userbase + (opensmtpd-table-configuration + (name "this") + (data (list "job" "done")))))) +#+END_SRC + +***** DONE virtual [2/2] +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 03:53] +:END: +****** DONE change the sanitize portion of the fieldname virtual in the + +The below code does work in a REPL. +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) +(opensmtpd-action-local-delivery-configuration + (virtual + (opensmtpd-table-configuration + (name "this") + (data (list "job" "done"))))) +#+END_SRC + +#+RESULTS: +****** DONE change relevant portions in opensmtpd-action-local-delivery-configuration->string +This bit of code works in the repl too! +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +((@@ (opensmtpd-records) opensmtpd-action-local-delivery-configuration->string) + (opensmtpd-action-local-delivery-configuration + (virtual + (opensmtpd-table-configuration + (name "this") + (data (list "job" "done")))))) + +#+END_SRC + +**** DONE opensmtpd-relay-configuration [4/4] +***** DONE helo-src +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:00] +:END: +***** DONE domain == [2/2] +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:00] +:END: + +****** DONE change the sanitize portion of the fieldname domain in the + +The below code does work in a REPL. +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +(opensmtpd-action-relay-configuration + (domain + (opensmtpd-table-configuration + (name "this") + (data (list "gnucode.me" "gnu-hurd.com"))))) + +#+END_SRC + +#+RESULTS: +****** DONE change relevant portions in opensmtpd-action-relay-configuration->string +This bit of code works in the repl too! +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +((@@ (opensmtpd-records) opensmtpd-action-relay-configuration->string) + (opensmtpd-action-relay-configuration + (domain + (opensmtpd-table-configuration + (name "this") + (data (list "gnucode.me" "gnu-hurd.com")))))) + + +#+END_SRC + +***** DONE auth == [2/2] +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:00] +:END: +****** DONE change the sanitize portion of the fieldname == in the + +The below code does work in a REPL. +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +(opensmtpd-action-relay-configuration + (auth + (opensmtpd-table-configuration + (name "this") + (data (list "gnucode.me" "gnu-hurd.com"))))) + +#+END_SRC + +#+RESULTS: +****** DONE change relevant portions in opensmtpd-action-relay-configuration->string +This bit of code works in the repl too! +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +((@@ (opensmtpd-records) opensmtpd-action-relay-configuration->string) + (opensmtpd-action-relay-configuration + (auth + (opensmtpd-table-configuration + (name "this") + (data (list "gnucode.me" "gnu-hurd.com")))))) + + +#+END_SRC + +***** DONE src srcaddress | +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:01] +:END: +****** DONE change the sanitize portion of the fieldname == in the + +The below code does work in a REPL. +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +(opensmtpd-action-relay-configuration + (src + (opensmtpd-table-configuration + (name "this") + (data (list "gnucode.me" "gnu-hurd.com"))))) + +#+END_SRC + +#+RESULTS: +****** DONE change relevant portions in opensmtpd-action-relay-configuration->string +This bit of code works in the repl too! +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +((@@ (opensmtpd-records) opensmtpd-action-relay-configuration->string) + (opensmtpd-action-relay-configuration + (src + (opensmtpd-table-configuration + (name "this") + (data (list "gnucode.me" "gnu-hurd.com")))))) + + +#+END_SRC + +Use the string scraddress or list table for the source IP address. +**** DONE opensmtpd-listen-on-configuration [3/3] +***** DONE auth == [3/3] +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:01] +:END: +****** DONE change the sanitize portion of the fieldname 'auth' in the + +The below code does work in a REPL. +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +(opensmtpd-listen-on-configuration + (auth + (opensmtpd-table-configuration + (name "My-table") + (data '(("joshua" . "$some$Long$EncrytpedPassword")))))) +#+END_SRC + +#+RESULTS: +****** DONE change relevant portions in opensmtpd-listen-on-configuration->string +This bit of code works in the repl too! +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +((@@ (opensmtpd-records) opensmtpd-listen-on-configuration->string) + (opensmtpd-listen-on-configuration + (auth + (opensmtpd-table-configuration + (name "credentials") + (data '(("joshua" . "$someLongEncrytpedPassword"))))))) +#+END_SRC + +****** DONE sanitize the == so that it can only be an opensmtpd-table-configuration, whose fieldname values are an assoc-list + +#+BEGIN_SRC scheme +(opensmtpd-listen-on-configuration (auth (opensmtpd-table-configuration (name "the") (data (list "the" "cat"))))) +#+END_SRC + +#+RESULTS: == fieldname: 'auth' is of type boolean, or an == record whose fieldname 'values' are an assoc-list. + +***** DONE auth-optional == [2/2] +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:01] +:END: +****** DONE change the sanitize portion of the fieldname 'auth-optional' in the + +The below code does work in a REPL. +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +(opensmtpd-listen-on-configuration + (auth + (opensmtpd-table-configuration + (name "My-table") + (data '(("joshua" . "$some$Long$EncrytpedPassword")))))) +#+END_SRC + +#+RESULTS: + +AND the below code will correctly result in an error! +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +(opensmtpd-listen-on-configuration + (auth + (opensmtpd-table-configuration + (name "My-table") + (data '("joshua" "$some$Long$EncrytpedPassword"))))) +#+END_SRC + +#+RESULTS: +: == fieldname: 'auth' is of type boolean, or an == record whose fieldname 'values' are an assoc-list +: (eg: (opensmtpd-table-configuration (name "table") (data '("joshua" . "$encrypted$password")))). + +****** DONE change relevant portions in opensmtpd-listen-on-configuration->string +This bit of code works in the repl too! +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +((@@ (opensmtpd-records) opensmtpd-listen-on-configuration->string) + (opensmtpd-listen-on-configuration + (auth + (opensmtpd-table-configuration + (name "credentials") + (data '(("joshua" . "$someLongEncrytpedPassword"))))))) +#+END_SRC + +***** DONE hostnames == [2/2] +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:01] +:END: +****** DONE change the sanitize portion of the fieldname 'hostnames' in the + +The below code does work in a REPL. +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +(opensmtpd-listen-on-configuration + (hostnames + (opensmtpd-table-configuration + (name "My-table") + (data '(("joshua" . "$some$Long$EncrytpedPassword")))))) +#+END_SRC + +#+RESULTS: + +AND the below code will correctly result in an error! +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +(opensmtpd-listen-on-configuration + (hostnames + (opensmtpd-table-configuration + (name "My-table") + (data '("joshua" "$some$Long$EncrytpedPassword"))))) +#+END_SRC + +#+RESULTS: +: == fieldname: 'hostname' is of type boolean, or an == record whose fieldname 'values' are an assoc-list +: (eg: (opensmtpd-table-configuration (name "table") (data '("joshua" . "$encrypted$password")))). + +****** DONE change relevant portions in opensmtpd-listen-on-configuration->string +This bit of code works in the repl too! +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +((@@ (opensmtpd-records) opensmtpd-listen-on-configuration->string) + (opensmtpd-listen-on-configuration + (hostnames + (opensmtpd-table-configuration + (name "credentials") + (data '(("joshua" . "$someLongEncrytpedPassword"))))))) +#+END_SRC + +**** TODO opensmtpd-match [20/24] +******* NO list approach +Guix probably won't like the list approach. +#+BEGIN_SRC scheme + (openmstpd-match + (for + (list 'not "for domain regex" + (opensmtpd-table-configuration + (data (list "gnucode.me" "gnu-hurd.com")))))) +#+END_SRC + +#+BEGIN_SRC scheme + (openmstpd-match + (for + (list "! for domain" + (opensmtpd-table-configuration + (data (list "gnucode.me" "gnu-hurd.com")))))) +#+END_SRC +******* opensmtpd-options-configuration approach I like this one quite a bit. + +This method is a little bit more verbose. Well I guess it's a lot +more verbose. But it's easier for me to properly parse what the user wants. + +I would sanitize the options in the opensmtpd-match-configuration-for, +openmsmtpd-match-from, opensmtpd-match-configuration-auth, opensmtpd-match-configuration-helo, +opensmtpd-match-configuration-mail-from, opensmtpd-match-configuration-rcpt-to fieldnames. +********* for +#+BEGIN_SRC scheme + (openmstpd-match + (for + (opensmtpd-options-configuration + (not #t) + (method "domain regex") ;; valid options for "for" are "domain" or "domain regex" + (opensmtpd-table-configuration + (data (list "gnucode.me" "gnu-hurd.com")))))) +#+END_SRC + +Do I want a regex fieldname? Probably not. It makes it more verbose... + +#+BEGIN_SRC scheme + (openmstpd-match + (for + (opensmtpd-options-configuration + (not #t) + (regex #t) + (method "domain") ;; valid options for "for" are "domain" + (opensmtpd-table-configuration + (data (list "gnucode.me" "gnu-hurd.com")))))) +#+END_SRC + +********* from +#+BEGIN_SRC scheme + (openmstpd-match + (from + (opensmtpd-options-configuration + (not #t) + (method "rdns regex") ;;valid options for from are "auth" "auth regex", "mail-from" "mail-from regex", + ;; "rdns", "rdns regex", "src", "src regex" + (opensmtpd-table-configuration + (data (list "gnucode.me" "gnu-hurd.com")))))) +#+END_SRC + +Do I want a regex fieldname? + +#+BEGIN_SRC scheme + (openmstpd-match + (from + (opensmtpd-options-configuration + (not #t) + (regex #t) + (method "rdns") ;;valid options for from are "auth", "mail-from", "rdns", "src" + (opensmtpd-table-configuration + (data (list "gnucode.me" "gnu-hurd.com")))))) +#+END_SRC + +********* auth +#+BEGIN_SRC scheme + (openmstpd-match + (auth + (opensmtpd-options-configuration + (not #t) + (method "auth regex") + (opensmtpd-table-configuration + (data (list "gnucode.me" "gnu-hurd.com")))))) +#+END_SRC + +Do I want a regex fieldname? + +#+BEGIN_SRC scheme + (openmstpd-match + (auth + (opensmtpd-options-configuration + (not #t) + (regex #t) + (method "auth") ;; valid options for auth are "auth" or this method can be left blank. + (opensmtpd-table-configuration + (data (list "gnucode.me" "gnu-hurd.com")))))) +#+END_SRC + +********* mail-from +#+BEGIN_SRC scheme + (openmstpd-match + (mail-from + (opensmtpd-options-configuration + (not #t) + (method "mail from") + (opensmtpd-table-configuration + (data (list "gnucode.me" "gnu-hurd.com")))))) +#+END_SRC +****** I tweak opensmtpd-match record and add a opensmtpd-options + +#+BEGIN_SRC scheme + (opensmtpd-match-configuration + (name "action-name") + (options + (list + (opensmtpd-options-configuration + (method "for domain regex"))) + )) +#+END_SRC + +****** PROJ many of these options are not completely sanitized. + +For example: "for domain" requires a domain | BUT this record, which +does not have a domain gives no errors: + +#+BEGIN_SRC scheme +(opensmtpd-match-configuration + (for (opensmtpd-option-configuration + (option "for domain"))) + (action (opensmtpd-action-local-delivery-configuration + (name "local") ))) +#+END_SRC + +And there are a ton of other examples of this. +****** DONE for domain +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:17] +:END: + +The datastructures work: + +#+BEGIN_SRC scheme +(opensmtpd-match-configuration + (name "local") + (for (opensmtpd-option-configuration + (option "for domain") + (value (opensmtpd-table-configuration + (name "this") + (data (list "helo" "hello")))))) + (action (opensmtpd-action-local-delivery-configuration))) +#+END_SRC + +#Results +: $4 = #< name: "local" action: #< method: "mbox" alias: #f ttl: #f user: #f userbase: #f virtual: #f wrapper: #f> for: #< option: "for domain" not: #f regex: #f value: #< name: "this" file-db: #f values: ("helo" "hello") type: #:148:97 (x)>>> from: #f auth: #f helo: #f mail-from: #f rcpt-to: #f tag: #f tls: #f> + +****** DONE for domain regexp +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:19] +:END: + +the datastructure works +#+BEGIN_SRC scheme +(opensmtpd-match-configuration + (name "local") + (for (opensmtpd-option-configuration + (regex #t) + (option "for domain") + (value (opensmtpd-table-configuration + (name "this") + (data (list "helo" "hello")))))) + (action (opensmtpd-action-local-delivery-configuration))) +#+END_SRC + +#Results +: $4 = #< name: "local" action: #< method: "mbox" alias: #f ttl: #f user: #f userbase: #f virtual: #f wrapper: #f> for: #< option: "for domain" not: #f regex: #f value: #< name: "this" file-db: #f values: ("helo" "hello") type: #:148:97 (x)>>> from: #f auth: #f helo: #f mail-from: #f rcpt-to: #f tag: #f tls: #f> + +****** DONE make opensmtpd-match-configuration->string work print for rcpt the appropriate match lines if some values now accept + +Seems to work: + +#+BEGIN_SRC scheme +(opensmtpd-match-configuration->string (opensmtpd-match-configuration + (name "local") + (for (opensmtpd-option-configuration + (regex #t) + (option "for domain") + (value (opensmtpd-table-configuration + (name "this") + (data (list "helo" "hello")))))) + (action (opensmtpd-action-local-delivery-configuration)))) +$6 = "match for domain regex == action \"local\" \n" +#+END_SRC + +also seems to work + +#+BEGIN_SRC scheme +(opensmtpd-match-configuration->string (opensmtpd-match-configuration + (name "local") + (for (opensmtpd-option-configuration + (option "for domain") + (value (opensmtpd-table-configuration + (name "this") + (data (list "helo" "hello")))))) + (action (opensmtpd-action-local-delivery-configuration)))) +$7 = "match for domain == action \"local\" \n" +#+END_SRC +****** DONE for rcpt +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:19] +:END: +****** DONE for rcpt regexp +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:19] +:END: +****** DONE from auth user | +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:19] +:END: + +#+BEGIN_SRC scheme +(opensmtpd-option-configuration + (regex #t) + (option "from auth") + (value (opensmtpd-table-configuration + (name "this") + (data (list "helo" "hello"))))) +$8 = #< option: "from auth" not: #f regex: #t value: #< name: "this" file-db: #f values: ("helo" "hello") type: #:224:14 (x)>>> + +#+END_SRC +****** DONE from auth regex user | +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:20] +:END: +****** DONE make sure opensmtpd-option-configuration->string works for from auth if they use + +#+BEGIN_SRC scheme +(opensmtpd-option-configuration->string + (opensmtpd-option-configuration + (regex #t) + (option "from auth") + (value (opensmtpd-table-configuration + (name "this") + (data (list "helo" "hello")))))) +$10 = "from auth regex == " +#+END_SRC +****** DONE from mail-from sender | +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:20] +:END: +****** DONE from mail-from regexp +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:20] +:END: +****** DONE from rdns +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:20] +:END: +****** DONE from rdns regex +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:20] +:END: +****** DONE from src
+:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:21] +:END: +****** DONE from src regex
+:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:21] +:END: +****** TODO do some more sanitizing of these last couple of options +There may be some way to specify invalid data. For example: + +tls does not support regex, not, or value fields. The below code should be an error. +#+BEGIN_SRC scheme + (display (opensmtpd-match-configuration->string (opensmtpd-match-configuration + (tls (opensmtpd-option-configuration + (option "tls") ;; this should be auth!!! NOT "helo" + (regex #t) + (not #t) + (value (opensmtpd-table-configuration (name "mytable") + (data (list "cat" "kitten")))))) + (from (opensmtpd-option-configuration (option "from rdns") + (value (opensmtpd-table-configuration (name "table") + (data (list "cat" "hat")))))) + (action (opensmtpd-action-local-delivery-configuration + (name "matches")))))) +match from rdns
! tls regex action "matches" +#+END_SRC + +****** DONE auth +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:21] +:END: + +Well I need to fix this bug. Basically, I can make an + with a table of values for 'option' "auth". And +I can print that table with (opensmtpd-option-configuration->string)...That works +fine... + +But if I put that same record into an == ...for some reason that +'auth' table is not being printed. + +#+BEGIN_SRC scheme +(opensmtpd-option-configuration + (option "auth") + (value (opensmtpd-table-configuration (name "mytable") + (data (list "cat" "hat"))))) +$20 = #< option: "auth" not: #f regex: #f value: #< name: "mytable" file-db: #f values: ("cat" "hat") type: #:860:40 (x)>>> +scheme@(opensmtpd-records) [4]> (opensmtpd-option-configuration->string $20) +$21 = "auth == " +scheme@(opensmtpd-records) [4]> (opensmtpd-match (name "matches") + (auth (opensmtpd-option-configuration + (option "auth") + (value (opensmtpd-table-configuration (name "mytable") + (data (list "cat" "kitten")))))) + (from (opensmtpd-option-configuration (option "from rdns") + (value (opensmtpd-table-configuration (name "table") + (data (list "cat" "hat")))))) + (action (opensmtpd-action-local-delivery-configuration))) +$22 = #< name: "matches" action: #< method: "mbox" alias: #f ttl: #f user: #f userbase: #f virtual: #f wrapper: #f> for: #f from: #< option: "from rdns" not: #f regex: #f value: #< name: "table" file-db: #f values: ("cat" "hat") type: #:876:89 (x)>>> auth: #< option: "auth" not: #f regex: #f value: #< name: "mytable" file-db: #f values: ("cat" "kitten") type: #:873:63 (x)>>> helo: #f mail-from: #f rcpt-to: #f tag: #f tls: #f> +scheme@(opensmtpd-records) [5]> (opensmtpd-match-configuration->string $22) +$23 = "match from rdns =
= auth action \"matches\" \n" ;; THERE IS SUPPOSED TO BE a "auth " here +scheme@(opensmtpd-records) [5]> +#+END_SRC +****** TODO [!] auth +****** TODO auth regex +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:21] +:END: + +This does NOT show the regex for the auth option. or the table why? +#+BEGIN_SRC scheme +(opensmtpd-match-configuration + (auth (opensmtpd-option-configuration + (option "auth") ;; this should be auth!!! NOT "helo" + (regex #t) + (value (opensmtpd-table-configuration (name "mytable") + (data (list "cat" "kitten")))))) + (from (opensmtpd-option-configuration (option "from rdns") + (value (opensmtpd-table-configuration (name "table") + (data (list "cat" "hat")))))) + (action (opensmtpd-action-local-delivery-configuration + (name "matches")))) +$9 = #< action: #< name: "matches" method: "mbox" alias: #f ttl: #f user: #f userbase: #f virtual: #f wrapper: #f> for: #f from: #< option: "from rdns" not: #f regex: #f value: #< name: "table" file-db: #f values: ("cat" "hat") type: #:144:52 (x)>>> auth: #< option: "auth" not: #f regex: #t value: #< name: "mytable" file-db: #f values: ("cat" "kitten") type: #:141:15 (x)>>> helo: #f mail-from: #f rcpt-to: #f tag: #f tls: #f> +scheme@(opensmtpd-records) [4]> (display (opensmtpd-match-configuration->string $9)) +match from rdns
auth action "matches" ;; there should be a regex in there. +#+END_SRC +****** DONE helo +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:21] +:END: +#+BEGIN_SRC scheme +(opensmtpd-match-configuration + (helo (opensmtpd-option-configuration + (option "helo") ;; this should be auth!!! NOT "helo" + (regex #t) + (value (opensmtpd-table-configuration (name "mytable") + (data (list "cat" "kitten")))))) + (from (opensmtpd-option-configuration (option "from rdns") + (value (opensmtpd-table-configuration (name "table") + (data (list "cat" "hat")))))) + (action (opensmtpd-action-local-delivery-configuration + (name "matches")))) +$10 = #< action: #< name: "matches" method: "mbox" alias: #f ttl: #f user: #f userbase: #f virtual: #f wrapper: #f> for: #f from: #< option: "from rdns" not: #f regex: #f value: #< name: "table" file-db: #f values: ("cat" "hat") type: #:252:52 (x)>>> auth: #f helo: #< option: "helo" not: #f regex: #t value: #< name: "mytable" file-db: #f values: ("cat" "kitten") type: #:249:15 (x)>>> mail-from: #f rcpt-to: #f tag: #f tls: #f> +scheme@(opensmtpd-records) [5]> (opensmtpd-match-configuration->string $10) +$11 = "match from rdns
helo regex action \"matches\" \n" +#+END_SRC +****** DONE mail-from +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:22] +:END: +#+BEGIN_SRC scheme + (display (opensmtpd-match-configuration->string (opensmtpd-match-configuration + (mail-from (opensmtpd-option-configuration + (option "mail-from") ;; this should be auth!!! NOT "helo" + (regex #t) + (not #t) + (value (opensmtpd-table-configuration (name "mytable") + (data (list "cat" "kitten")))))) + (from (opensmtpd-option-configuration (option "from rdns") + (value (opensmtpd-table-configuration (name "table") + (data (list "cat" "hat")))))) + (action (opensmtpd-action-local-delivery-configuration + (name "matches")))))) +match from rdns
! mail-from regex action "matches" +#+END_SRC +****** DONE mail-from regex +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:22] +:END: +#+BEGIN_SRC scheme + (display (opensmtpd-match-configuration->string (opensmtpd-match-configuration + (mail-from (opensmtpd-option-configuration + (option "mail-from") ;; this should be auth!!! NOT "helo" + (regex #t) + (not #t) + (value (opensmtpd-table-configuration (name "mytable") + (data (list "cat" "kitten")))))) + (from (opensmtpd-option-configuration (option "from rdns") + (value (opensmtpd-table-configuration (name "table") + (data (list "cat" "hat")))))) + (action (opensmtpd-action-local-delivery-configuration + (name "matches")))))) +match from rdns
! mail-from regex action "matches" +#+END_SRC +****** DONE rcpt-to +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:22] +:END: +****** DONE rcpt-to regex +:LOGBOOK: +- State "TODO" from "TODO" [2021-11-02 Tue 04:23] +:END: + +*** PROJ sanitize the == records in == & == +**** PROJ testing the sanitize-list-of-options-for-match-configuration-assoc precodure [5/5] +***** DONE make sure each option is unique (no duplicate "for"s). + +#+BEGIN_SRC scheme +(opensmtpd-configuration + (matches (list + (opensmtpd-match-configuration + (options (list + (opensmtpd-option-configuration + (option "for any")) + (opensmtpd-option-configuration + (option "for local")))) + (action + (opensmtpd-action-relay-configuration + (name "relay"))))))) +#+END_SRC +***** DONE make sure there is no duplicate from's +#+BEGIN_SRC scheme +(opensmtpd-configuration + (matches (list + (opensmtpd-match-configuration + (options (list + (opensmtpd-option-configuration + (option "from any")) + (opensmtpd-option-configuration + (option "from auth")))) + (action + (opensmtpd-action-relay-configuration + (name "relay"))))))) +#+END_SRC +***** DONE for any data and regex must be false + +#+BEGIN_SRC scheme +(opensmtpd-configuration + (matches (list + (opensmtpd-match-configuration + (options (list + (opensmtpd-option-configuration + (option "for any") + (regex #t)))) + (action + (opensmtpd-action-relay-configuration + (name "relay"))))))) +#+END_SRC +***** DONE 'rcpt-to' must have data + +#+BEGIN_SRC scheme +(opensmtpd-configuration + (matches (list + (opensmtpd-match-configuration + (options (list + (opensmtpd-option-configuration + (option "rcpt-to")))) + (action + (opensmtpd-action-relay-configuration + (name "relay"))))))) +#+END_SRC +***** DONE 'tls' cannot have a 'data' or 'regex' + +#+BEGIN_SRC scheme +(opensmtpd-configuration + (matches (list + (opensmtpd-match-configuration + (options (list + (opensmtpd-option-configuration + (option "tls") + (data "hello") + ))) + (action + (opensmtpd-action-relay-configuration + (name "relay"))))))) +#+END_SRC +*** TODO sanitize fieldname 'matches' so that no two unique actions have the same name +I definitely should sanitize 'matches' a bit more. For example, you could have two different +actions, one for local delivery and one for remote, with the same name. I +should make sure that all unique actions have unique names. + +Here is an example of two actions that have the same name, but different ttl values: + +#+BEGIN_SRC scheme +(opensmtpd-configuration + (matches + (list (opensmtpd-match-configuration + (action + (opensmtpd-action-local-delivery-configuration + (name "my-local-delivery") + (ttl "50m")))) ; 50 minutes + (opensmtpd-match-configuration + (action + (opensmtpd-action-local-delivery-configuration + (name "my-local-delivery") + (ttl "50h"))))))) ; 50 hours +#+END_SRC + +*** PROJ follow the style guide and style up my project [0/4] +https://mumble.net/~campbell/scheme/style.txt + +:SchemeStyleGuide: +#+BEGIN_SRC org +Riastradh's Lisp Style Rules -*- outline -*- + + Copyright (C) 2007--2011 Taylor R. Campbell + + CC BY-NC-SA 3.0 + + This work is licensed under a Creative Commons + Attribution-NonCommercial-ShareAlike 3.0 Unported License: + . + +This is a guide to Lisp style, written by Taylor R. Campbell, to +describe the standard rules of Lisp style as well as a set of more +stringent rules for his own style. This guide should be useful for +Lisp in general, but there are [or will be in the final draft] parts +that are focussed on or specific to Scheme or Common Lisp. + +This guide is written primarily as a collection of rules, with +rationale for each rule. (If a rule is missing rationale, please +inform the author!) Although a casual reader might go through and read +the rules without the rationale, perhaps reasoning that reading of the +rationale would require more time than is available, such a reader +would derive little value from this guide. In order to apply the rules +meaningfully, their spirit must be understood; the letter of the rules +serves only to hint at the spirit. The rationale is just as important +as the rules. + +There are many references in this document to `Emacs', `GNU Emacs', +`Edwin', and so on. In this document, `Emacs' means any of a class of +editors related in design to a common ancestor, the EMACS editor macros +written for TECO on ITS on the PDP-10 in the middle of the nineteen +seventies. All such editors -- or `all Emacsen', since `Emacsen' is +the plural of `Emacs' -- have many traits in common, such as a very +consistent set of key bindings, extensibility in Lisp, and so on. `GNU +Emacs' means the member of the class of editors collectively known as +Emacsen that was written for the GNU Project in the middle of the +nineteen eighties, and which is today probably the most popular Emacs. +`Edwin' is MIT Scheme's Emacs, which is bundled as part of MIT Scheme, +and not available separately. There are other Emacsen as well, such as +Hemlock and Climacs, but as the author of this document has little +experience with Emacsen other than GNU Emacs and Edwin, there is little +mention of other Emacsen. + +This guide is a work in progress. To be written: + +- Indentation rules for various special operators. +- Philosophical rambling concerning naming. +- Rules for breaking lines. +- Many more examples. +- A more cohesive explanation of the author's principles for composing + programs, and their implications. +- Rules for writing portable code. +- Some thoughts concerning extensions to the lexical syntax. +- Rules for writing or avoiding macros. +- Some unfinished rationale. +- More on documentation. +- The `Dependencies' subsection of the `General Layout' section should + be put in a different section, the rest of which has yet to be + written, on organization of programs, module systems, and portable + code. + +Feedback is welcome; address any feedback by email to the host +mumble.net's user `campbell', or by IRC to Riastradh in the #scheme +channel on Freenode (irc.freenode.net). Feedback includes reports of +typos, questions, requests for clarification, and responses to the +rationale, except in the case of round brackets versus square +brackets, the argument surrounding which is supremely uninteresting +and now not merely a dead horse but a rotting carcass buzzing with +flies and being picked apart by vultures. + +As this document has grown, the line between standard Lisp rules and +the author's own style has been blurred. The author is considering +merging of the partition, but has not yet decided on this with +certainty. Opinions on the subject are welcome -- is the partition +still useful to keep the author's biases and idiosyncrasies out of the +standard rules, or has the partition with its arbitrary nature only +caused disorganization of the whole document? + +Unfortunately, this document is entirely unscientific. It is at best a +superstition or philosophy, but one that the author of this document +has found to have improved his programs. Furthermore, the author is +somewhat skeptical of claims of scientific analyses of these matters: +analyzing human behaviour, especially confined to the set of excellent +programmers who often have strong opinions about their methods for +building programs, is a very tricky task. + +,* Standard Rules + +These are the standard rules for formatting Lisp code; they are +repeated here for completeness, although they are surely described +elsewhere. These are the rules implemented in Emacs Lisp modes, and +auxiliary utilities such as Paredit. + +The rationale given here is merely the author's own speculation of the +origin of these rules, and should be taken as nothing more than it. +The reader shall, irrespective of the author's rationale, accept the +rules as sent by the reader's favourite deity, or Cthulhu if no such +deity strikes adequate fear into the heart of the reader. + +,** Parentheses + +,*** Terminology + +This guide avoids the term /parenthesis/ except in the general use of +/parentheses/ or /parenthesized/, because the word's generally accepted +definition, outside of the programming language, is a statement whose +meaning is peripheral to the sentence in which it occurs, and *not* the +typographical symbols used to delimit such statements. + +The balanced pair of typographical symbols that mark parentheses in +English text are /round brackets/, i.e. ( and ). There are several +other balanced pairs of typographical symbols, such as /square +brackets/ (commonly called simply `brackets' in programming circles), +i.e. [ and ]; /curly braces/ (sometimes called simply `braces'), i.e. { +and }; /angle brackets/ (sometimes `brokets' (for `broken brackets')), +i.e. < and >. + +In any balanced pair of typographical symbols, the symbol that begins +the region delimited by the symbols is called the /opening bracket/ or +the /left bracket/, such as ( or [ or { or <. The symbol that ends +that region is called the /right bracket/ or the /closing bracket/, +such as > or } or ] or ). + +,*** Spacing + +If any text precedes an opening bracket or follows a closing bracket, +separate that text from that bracket with a space. Conversely, leave +no space after an opening bracket and before following text, or after +preceding text and before a closing bracket. + + Unacceptable: + + (foo(bar baz)quux) + (foo ( bar baz ) quux) + + Acceptable: + + (foo (bar baz) quux) + + Rationale: This is the same spacing found in standard typography of + European text. It is more aesthetically pleasing. + +,*** Line Separation + +Absolutely do *not* place closing brackets on their own lines. + + Unacceptable: + + (define (factorial x) + (if (< x 2) + 1 + (* x (factorial (- x 1 + ) + ) + ) + ) + ) + + Acceptable: + + (define (factorial x) + (if (< x 2) + 1 + (* x (factorial (- x 1))))) + + Rationale: The parentheses grow lonely if their closing brackets are + all kept separated and segregated. + +,**** Exceptions to the Above Rule Concerning Line Separation + +Do not heed this section unless you know what you are doing. Its title +does *not* make the unacceptable example above acceptable. + +When commenting out fragments of expressions with line comments, it may +be necessary to break a line before a sequence of closing brackets: + + (define (foo bar) + (list (frob bar) + (zork bar) + ;; (zap bar) + )) + +This is acceptable, but there are other alternatives. In Common Lisp, +one can use the read-time optional syntax, `#+' or `#-', with a +feature optional that is guaranteed to be false or true -- `#+(OR)' +or `#-(AND)' --; for example, + + (define (foo bar) + (list (frob bar) + (zork bar) + ,#+(or) (zap bar))). + +Read-time optionals are expression-oriented, not line-oriented, so +the closing brackets need not be placed on the following line. Some +Scheme implementations, and SRFI 62, also support expression comments +with `#;', which are operationally equivalent to the above read-time +optionals for Common Lisp: + + (define (foo bar) + (list (frob bar) + (zork bar) + #; + (zap bar))) + +The expression is placed on another line in order to avoid confusing +editors that do not recognize S-expression comments; see the section +titled `Comments' below for more details. However, the `#;' notation +is not standard -- it appears in neither the IEEE 1178 document nor in +the R5RS --, so line comments are preferable for portable Scheme code, +even if they require breaking a line before a sequence of closing +brackets. + +Finally, it is acceptable to break a line immediately after an opening +bracket and immediately before a closing bracket for very long lists, +especially in files under version control. This eases the maintenance +of the lists and clarifies version diffs. Example: + + (define colour-names ;Add more colour names to this list! + '( + blue + cerulean + green + magenta + purple + red + scarlet + turquoise + )) + +,*** Parenthetical Philosophy + +The actual bracket characters are simply lexical tokens to which little +significance should be assigned. Lisp programmers do not examine the +brackets individually, or, Azathoth forbid, count brackets; instead +they view the higher-level structures expressed in the program, +especially as presented by the indentation. Lisp is not about writing +a sequence of serial instructions; it is about building complex +structures by summing parts. The composition of complex structures +from parts is the focus of Lisp programs, and it should be readily +apparent from the Lisp code. Placing brackets haphazardly about the +presentation is jarring to a Lisp programmer, who otherwise would not +even have seen them for the most part. + +,** Indentation and Alignment + +The operator of any form, i.e. the first subform following the opening +round bracket, determines the rules for indenting or aligning the +remaining forms. Many names in this position indicate special +alignment or indentation rules; these are special operators, macros, or +procedures that have certain parameter structures. + +If the first subform is a non-special name, however, then if the second +subform is on the same line, align the starting column of all following +subforms with that of the second subform. If the second subform is on +the following line, align its starting column with that of the first +subform, and do the same for all remaining subforms. + +In general, Emacs will indent Lisp code correctly. Run `C-M-q' +(indent-sexp) on any code to ensure that it is indented correctly, and +configure Emacs so that any non-standard forms are indented +appropriately. + + Unacceptable: + + (+ (sqrt -1) + (* x y) + (+ p q)) + + (+ + (sqrt -1) + (* x y) + (+ p q)) + + Acceptable: + + (+ (sqrt -1) + (* x y) + (+ p q)) + + (+ + (sqrt -1) + (* x y) + (+ p q)) + + Rationale: The columnar alignment allows the reader to follow the + operands of any operation straightforwardly, simply by scanning + downward or upward to match a common column. Indentation dictates + structure; confusing indentation is a burden on the reader who wishes + to derive structure without matching parentheses manually. + +,*** Non-Symbol Indentation and Alignment + +The above rules are not exhaustive; some cases may arise with strange +data in operator positions. + +,**** Lists + +Unfortunately, style varies here from person to person and from editor +to editor. Here are some examples of possible ways to indent lists +whose operators are lists: + + Questionable: + + ((car x) ;Requires hand indentation. + (cdr x) + foo) + + ((car x) (cdr x) ;GNU Emacs + foo) + + Preferable: + + ((car x) ;Any Emacs + (cdr x) + foo) + + ((car x) (cdr x) ;Edwin + foo) + + Rationale: The operands should be aligned, as if it were any other + procedure call with a name in the operator position; anything other + than this is confusing because it gives some operands greater visual + distinction, allowing others to hide from the viewer's sight. For + example, the questionable indentation + + ((car x) (cdr x) + foo) + + can make it hard to see that FOO and (CDR X) are both operands here + at the same level. However, GNU Emacs will generate that indentation + by default. (Edwin will not.) + +,**** Strings + +If the form in question is meant to be simply a list of literal data, +all of the subforms should be aligned to the same column, irrespective +of the first subform. + + Unacceptable: + + ("foo" "bar" "baz" "quux" "zot" + "mumble" "frotz" "gargle" "mumph") + + Questionable, but acceptable: + + (3 1 4 1 5 9 2 6 5 3 5 8 9 7 9 3 2 3 8 4 6 2 6 4 + 3 3 8 3 2 7 9 5 0 2 8 8 4 1 9 7 1 6 9 3 9 9 3) + + Acceptable: + + ("foo" "bar" "baz" "quux" "zot" + "mumble" "frotz" "gargle" "mumph") + + ("foo" + "bar" "baz" "quux" "zot" + "mumble" "frotz" "gargle" "mumph") + + Rationale: Seldom is the first subform distinguished for any reason, + if it is a literal; usually in this case it indicates pure data, not + code. Some editors and pretty-printers, however, will indent + unacceptably in the example given unless the second subform is on the + next line anyway, which is why the last way to write the fragment is + usually best. + +,** Names + +Naming is subtle and elusive. Bizarrely, it is simultaneously +insignificant, because an object is independent of and unaffected by +the many names by which we refer to it, and also of supreme +importance, because it is what programming -- and, indeed, almost +everything that we humans deal with -- is all about. A full +discussion of the concept of name lies far outside the scope of this +document, and could surely fill not even a book but a library. + +Symbolic names are written with English words separated by hyphens. +Scheme and Common Lisp both fold the case of names in programs; +consequently, camel case is frowned upon, and not merely because it is +ugly. Underscores are unacceptable separators except for names that +were derived directly from a foreign language without translation. + + Unacceptable: + + XMLHttpRequest + foreach + append_map + + Acceptable: + + xml-http-request + for-each + append-map + +,*** Funny Characters + +There are several different conventions in different Lisps for the use +of non-alphanumeric characters in names. + +,**** Scheme + +,***** Question Marks: Predicates + +Affix a question mark to the end of a name for a procedure whose +purpose is to ask a question of an object and to yield a boolean +answer. Such procedures are called `predicates'. Do not use a +question mark if the procedure may return any object other than a +boolean. + + Examples: pair? procedure? proper-list? + Non-examples: member assoc any every + +Pronounce the question mark as if it were the isolated letter `p'. For +example, to read the fragment (PAIR? OBJECT) aloud, say: `pair-pee +object.' + +,***** Exclamation Marks: Destructive Operations + +Affix an exclamation mark to the end of a name for a procedure (or +macro) whose primary purpose is to modify an object. Such procedures +are called `destructive'. + + Examples: set-car! append! + +Avoid using the exclamation mark willy nilly for just *any* procedure +whose operation involves any kind of mutation or side effect; instead, +use the exclamation mark to identify procedures that exist *solely* for +the purpose of destructive update (e.g., SET-CAR!), or to distinguish a +destructive, or potentially destructive (in the case of linear-update +operations such as APPEND!), variant of a procedure of which there also +exists a purely functional variant (e.g., APPEND). + +Pronounce the exclamation mark as `bang'. For example, to read the +fragment (APPEND! LIST TAIL) aloud, say: `append-bang list tail.' + +,***** Asterisks: Variants, Internal Routines, Mutable Globals + +Affix an asterisk to the end of a name to make a variation on a theme +of the original name. + + Example: let -> let* + +Prefer a meaningful name over an asterisk; the asterisk does not +explain what variation on the theme the name means. + +Affix an asterisk to the beginning of a name to make an internal +routine for that name. Again, prefer a meaningful name over an +asterisk. + +Affix asterisks to the beginning and end of a globally mutable +variable. This allows the reader of the program to recognize very +easily that it is badly written! + +,***** `WITH-' and `CALL-WITH-': Dynamic State and Cleanup + +Prefix `WITH-' to any procedure that establishes dynamic state and +calls a nullary procedure, which should be the last (required) +argument. The dynamic state should be established for the extent of +the nullary procedure, and should be returned to its original state +after that procedure returns. + + Examples: with-input-from-file with-output-to-file + + Exception: Some systems provide a procedure (WITH-CONTINUATION + ), which calls in the given + continuation, using that continuation's dynamic state. If + returns, it will return to , not to the continuation of + the call to WITH-CONTINUATION. This is acceptable. + +Prefix `CALL-WITH-' to any procedure that calls a procedure, which +should be its last argument, with some arguments, and is either somehow +dependent upon the dynamic state or continuation of the program, or +will perform some action to clean up data after the procedure argument +returns. Generally, `CALL-WITH-' procedures should return the values +that the procedure argument returns, after performing the cleaning +action. + + Examples: + + - CALL-WITH-INPUT-FILE and CALL-WITH-OUTPUT-FILE both accept a + pathname and a procedure as an argument, open that pathname (for + input or output, respectively), and call the procedure with one + argument, a port corresponding with the file named by the given + pathname. After the procedure returns, CALL-WITH-INPUT-FILE and + CALL-WITH-OUTPUT-FILE close the file that they opened, and return + whatever the procedure returned. + + - CALL-WITH-CURRENT-CONTINUATION is dependent on the continuation + with which it was called, and passes as an argument an escape + procedure corresponding with that continuation. + + - CALL-WITH-OUTPUT-STRING, a common but non-standard procedure + definable in terms of OPEN-OUTPUT-STRING and GET-OUTPUT-STRING from + SRFI 6 (Basic String Ports), calls its procedure argument with an + output port, and returns a string of all of the output written to + that port. Note that it does not return what the procedure + argument returns, which is an exception to the above rule. + +Generally, the distinction between these two classes of procedures is +that `CALL-WITH-...' procedures should not establish fresh dynamic +state and instead pass explicit arguments to their procedure arguments, +whereas `WITH-...' should do the opposite and establish dynamic state +while passing zero arguments to their procedure arguments. + +,** Comments + +Write heading comments with at least four semicolons; see also the +section below titled `Outline Headings'. + +Write top-level comments with three semicolons. + +Write comments on a particular fragment of code before that fragment +and aligned with it, using two semicolons. + +Write margin comments with one semicolon. + +The only comments in which omission of a space between the semicolon +and the text is acceptable are margin comments. + + Examples: + + ;;;; Frob Grovel + + ;;; This section of code has some important implications: + ;;; 1. Foo. + ;;; 2. Bar. + ;;; 3. Baz. + + (define (fnord zarquon) + ;; If zob, then veeblefitz. + (quux zot + mumble ;Zibblefrotz. + frotz)) + +,* Riastradh's Non-Standard Rules + +Three principles guide this style, roughly ordered according to +descending importance: + +1. The purpose of a program is to describe an idea, and not the way + that the idea must be realized; the intent of the program's meaning, + rather than peripheral details that are irrelevant to its intent, + should be the focus of the program, *irrespective* of whether a + human or a machine is reading it. [It would be nice to express this + principle more concisely.] + +2. The sum of the parts is easier to understand than the whole. + +3. Aesthetics matters. No one enjoys reading an ugly program. + +,** General Layout + +This section contains rules that the author has found generally helpful +in keeping his programs clean and presentable, though they are not +especially philosophically interesting. + +Contained in the rationale for some of the following rules are +references to historical limitations of terminals and printers, which +are now considered aging cruft of no further relevance to today's +computers. Such references are made only to explain specific measures +chosen for some of the rules, such as a limit of eighty columns per +line, or sixty-six lines per page. There is a real reason for each of +the rules, and this real reason is not intrinsically related to the +historical measures, which are mentioned only for the sake of +providing some arbitrary measure for the limit. + +,*** File Length + +If a file exceeds five hundred twelve lines, begin to consider +splitting it into multiple files. Do not write program files that +exceed one thousand twenty-four lines. Write a concise but +descriptive title at the top of each file, and include no content in +the file that is unrelated to its title. + + Rationale: Files that are any larger should generally be factored + into smaller parts. (One thousand twenty-four is a nicer number than + one thousand.) Identifying the purpose of the file helps to break it + into parts if necessary and to ensure that nothing unrelated is + included accidentally. + +,*** Top-Level Form Length + +Do not write top-level forms that exceed twenty-one lines, except for +top-level forms that serve only the purpose of listing large sets of +data. If a procedure exceeds this length, split it apart and give +names to its parts. Avoid names formed simply by appending a number +to the original procedure's name; give meaningful names to the parts. + + Rationale: Top-level forms, especially procedure definitions, that + exceed this length usually combine too many concepts under one name. + Readers of the code are likely to more easily understand the code if + it is composed of separately named parts. Simply appending a number + to the original procedure's name can help only the letter of the + rule, not the spirit, however, even if the procedure was taken from a + standard algorithm description. Using comments to mark the code with + its corresponding place in the algorithm's description is acceptable, + but the algorithm should be split up in meaningful fragments anyway. + + Rationale for the number twenty-one: Twenty-one lines, at a maximum + of eighty columns per line, fits in a GNU Emacs instance running in a + 24x80 terminal. Although the terminal may have twenty-four lines, + three of the lines are occupied by GNU Emacs: one for the menu bar + (which the author of this guide never uses, but which occupies a line + nevertheless in a vanilla GNU Emacs installation), one for the mode + line, and one for the minibuffer's window. The writer of some code + may not be limited to such a terminal, but the author of this style + guide often finds it helpful to have at least four such terminals or + Emacs windows open simultaneously, spread across a twelve-inch laptop + screen, to view multiple code fragments. + +,*** Line Length + +Do not write lines that exceed eighty columns, or if possible +seventy-two. + + Rationale: Following multiple lines that span more columns is + difficult for humans, who must remember the line of focus and scan + right to left from the end of the previous line to the beginning of + the next line; the more columns there are, the harder this is to do. + Sticking to a fixed limit helps to improve readability. + + Rationale for the numbers eighty and seventy-two: It is true that we + have very wide screens these days, and we are no longer limited to + eighty-column terminals; however, we ought to exploit our wide + screens not by writing long lines, but by viewing multiple fragments + of code in parallel, something that the author of this guide does + very often. Seventy-two columns leave room for several nested layers + of quotation in email messages before the code reaches eighty + columns. Also, a fixed column limit yields nicer printed output, + especially in conjunction with pagination; see the section + `Pagination' below. + +,*** Blank Lines + +Separate each adjacent top-level form with a single blank line (i.e. +two line breaks). If two blank lines seem more appropriate, break the +page instead. Do not place blank lines in the middle of a procedure +body, except to separate internal definitions; if there is a blank +line for any other reason, split the top-level form up into multiple +ones. + + Rationale: More than one blank line is distracting and sloppy. If + the two concepts that are separated by multiple blank lines are + really so distinct that such a wide separator is warranted, then + they are probably better placed on separate pages anyway; see the + next section, `Pagination'. + +,*** Pagination + +Separate each file into pages of no more than sixty-six lines and no +fewer than forty lines with form feeds (ASCII #x0C, or ^L, written in +Emacs with `C-q C-l'), on either side of which is a single line break +(but not a blank line). + + Rationale: Keeping distinct concepts laid out on separate pages + helps to keep them straight. This is helpful not only for the + writer of the code, but also for the reader. It also allows readers + of the code to print it onto paper without fiddling with printer + settings to permit pages of more than sixty-six lines (which is the + default number for many printers), and pagination also makes the + code easier to navigate in Emacs, with the `C-x [' and `C-x ]' keys + (`backward-page' and `forward-page', respectively). To avoid + excessively small increments of page-by-page navigation, and to + avoid wasting paper, each page should generally exceed forty lines. + + `C-x l' in Emacs will report the number of lines in the page on which + the point lies; this is useful for finding where pagination is + necessary. + +,*** Outline Headings + +Use Emacs's Outline Mode to give titles to the pages, and if +appropriate a hierarchical structure. Set `outline-regexp' (or +`outline-pattern' in Edwin) to "\f\n;;;;+ ", so that each form feed +followed by an line break followed by at least four semicolons and a +space indicates an outline heading to Emacs. Use four semicolons for +the highest level of headings in the hierarchy, and one more for each +successively nested level of hierarchy. + + Rationale: Not only does this clarify the organization of the code, + but readers of the code can then navigate the code's structure with + Outline Mode commands such as `C-c C-f', `C-c C-b', `C-c C-u', and + `C-c C-d' (forward, backward, up, down, respectively, headings). + +,*** Dependencies + +When writing a file or module, minimize its dependencies. If there are +too many dependencies, consider breaking the module up into several +parts, and writing another module that is the sum of the parts and that +depends only on the parts, not their dependencies. + + Rationale: A fragment of a program with fewer dependencies is less + of a burden on the reader's cognition. The reader can more easily + understand the fragment in isolation; humans are very good at local + analyses, and terrible at global ones. + +,** Naming + +This section requires an elaborate philosophical discussion which the +author is too ill to have the energy to write at this moment. + +Compose concise but meaningful names. Do not cheat by abbreviating +words or using contractions. + + Rationale: Abbreviating words in names does not make them shorter; + it only makes them occupy less screen space. The reader still must + understand the whole long name. This does not mean, however, that + names should necessarily be long; they should be descriptive. Some + long names are more descriptive than some short names, but there are + also descriptive names that are not long and long names that are not + descriptive. Here is an example of a long name that is not + descriptive, from SchMUSE, a multi-user simulation environment + written in MIT Scheme: + + frisk-descriptor-recursive-subexpr-descender-for-frisk-descr-env + + Not only is it long (sixty-four characters) and completely + impenetrable, but halfway through its author decided to abbreviate + some words as well! + +Do not write single-letter variable names. Give local variables +meaningful names composed from complete English words. + + Rationale: It is tempting to reason that local variables are + invisible to other code, so it is OK to be messy with their names. + This is faulty reasoning: although the next person to come along and + use a library may not care about anything but the top-level + definitions that it exports, this is not the only audience of the + code. Someone will also want to read the code later on, and if it is + full of impenetrably terse variable names without meaning, that + someone will have a hard time reading the code. + +Give names to intermediate values where their expressions do not +adequately describe them. + + Rationale: An `expression' is a term that expresses some value. + Although a machine needs no higher meaning for this value, and + although it should be written to be sufficiently clear for a human to + understand what it means, the expression might mean something more + than just what it says where it is used. Consequently, it is helpful + for humans to see names given to expressions. + + Example: A hash table HASH-TABLE maps foos to bars; (HASH-TABLE/GET + HASH-TABLE FOO #F) expresses the datum that HASH-TABLE maps FOO to, + but that expression gives the reader no hint of any information + concerning that datum. (LET ((BAR (HASH-TABLE/GET HASH-TABLE FOO + #F))) ...) gives a helpful name for the reader to understand the + code without having to find the definition of HASH-TABLE. + + Index variables such as i and j, or variables such as A and D naming + the car and cdr of a pair, are acceptable only if they are completely + unambiguous in the scope. For example, + + (do ((i 0 (+ i 1))) + ((= i (vector-length vector))) + (frobnicate (vector-ref vector i))) + + is acceptable because the scope of i is very clearly limited to a + single vector. However, if more vectors are involved, using more + index variables such as j and k will obscure the program further. + +Avoid functional combinators, or, worse, the point-free (or +`point-less') style of code that is popular in the Haskell world. At +most, use function composition only where the composition of functions +is the crux of the idea being expressed, rather than simply a procedure +that happens to be a composition of two others. + + Rationale: Tempting as it may be to recognize patterns that can be + structured as combinations of functional combinators -- say, `compose + this procedure with the projection of the second argument of that + other one', or (COMPOSE FOO (PROJECT 2 BAR)) --, the reader of the + code must subsequently examine the elaborate structure that has been + built up to obscure the underlying purpose. The previous fragment + could have been written (LAMBDA (A B) (FOO (BAR B))), which is in + fact shorter, and which tells the reader directly what argument is + being passed on to what, and what argument is being ignored, without + forcing the reader to search for the definitions of FOO and BAR or + the call site of the final composition. The explicit fragment + contains substantially more information when intermediate values are + named, which is very helpful for understanding it and especially for + modifying it later on. + + The screen space that can be potentially saved by using functional + combinators is made up for by the cognitive effort on the part of the + reader. The reader should not be asked to search globally for usage + sites in order to understand a local fragment. Only if the structure + of the composition really is central to the point of the narrative + should it be written as such. For example, in a symbolic integrator + or differentiator, composition is an important concept, but in most + code the structure of the composition is completely irrelevant to the + real point of the code. + +If a parameter is ignored, give it a meaningful name nevertheless and +say that it is ignored; do not simply call it `ignored'. + +In Common Lisp, variables can be ignored with (DECLARE (IGNORE ...)). +Some Scheme systems have similar declarations, but the portable way to +ignore variables is just to write them in a command context, where +their values will be discarded, preferably with a comment indicating +this purpose: + + (define (foo x y z) + x z ;ignore + (frobnitz y)) + + Rationale: As with using functional combinators to hide names, + avoiding meaningful names for ignored parameters only obscures the + purpose of the program. It is helpful for a reader to understand + what parameters a procedure is independent of, or if someone wishes + to change the procedure later on, it is helpful to know what other + parameters are available. If the ignored parameters were named + meaninglessly, then these people would be forced to search for call + sites of the procedure in order to get a rough idea of what + parameters might be passed here. + +When naming top-level bindings, assume namespace partitions unless in a +context where they are certain to be absent. Do not write explicit +namespace prefixes, such as FOO:BAR for an operation BAR in a module +FOO, unless the names will be used in a context known not to have any +kind of namespace partitions. + + Rationale: Explicit namespace prefixes are ugly, and lengthen names + without adding much semantic content. Common Lisp has its package + system to separate the namespaces of symbols; most Schemes have + mechanisms to do so as well, even if the RnRS do not specify any. It + is better to write clear names which can be disambiguated if + necessary, rather than to write names that assume some kind of + disambiguation to be necessary to begin with. Furthermore, explicit + namespace prefixes are inadequate to cover name clashes anyway: + someone else might choose the same namespace prefix. Relegating this + issue to a module system removes it from the content of the program, + where it is uninteresting. + +,** Comments + +Write comments only where the code is incapable of explaining itself. +Prefer self-explanatory code over explanatory comments. Avoid +`literate programming' like the plague. + + Rationale: If the code is often incapable of explaining itself, then + perhaps it should be written in a more expressive language. This may + mean using a different programming language altogether, or, since we + are talking about Lisp, it may mean simply building a combinator + language or a macro language for the purpose. `Literate programming' + is the logical conclusion of languages incapable of explaining + themselves; it is a direct concession of the inexpressiveness of the + computer language implementing the program, to the extent that the + only way a human can understand the program is by having it rewritten + in a human language. + +Do not write interface documentation in the comments for the +implementation of the interface. Explain the interface at the top of +the file if it is a single-file library, or put that documentation in +another file altogether. (See the `Documentation' section below if the +interface documentation comments grow too large for a file.) + + Rationale: A reader who is interested only in the interface really + should not need to read through the implementation to pick out its + interface; by putting the interface documentation at the top, not + only is such a reader's task of identifying the interface made + easier, but the implementation code can be more liberally commented + without fear of distracting this reader. To a reader who is + interested in the implementation as well, the interface is still + useful in order to understand what concepts the implementation is + implementing. + + Example: + + In this example of a single-file library implementing the skip list + data structure, the first page explains the purpose and dependencies + of the file (which are useful for anyone who intends to use it, even + though dependencies are really implementation details), and the next + few pages explain the usage of skip lists as implemented in that + file. On the first page of implementation, `Skip List Structure', + there are some comments of interest only to a reader who wishes to + understand the implementation; the same goes for the rest of the + file, none of which must a reader read whose interest is only in the + usage of the library. + +Avoid block comments (i.e. #| ... |#). Use S-expression comments (`#;' +in Scheme, with the expression to comment on the next line; `#+(OR)' or +`#-(AND)' in Common Lisp) to comment out whole expressions. Use blocks +of line comments for text. + + Rationale: Editor support for block comments is weak, because it + requires keeping a detailed intermediate parse state of the whole + buffer, which most Emacsen do not do. At the very least, #|| ... ||# + is better, because most Emacsen will see vertical bars as symbol + delimiters, and lose trying to read a very, very long symbol, if they + try to parse #| ... |#, whereas they will just see two empty symbols + and otherwise innocuous text between them if they try to parse #|| + ... ||#. In any case, in Emacs, `M-x comment-region RET', or `M-;' + (comment-dwim), is trivial to type. + + The only standard comments in Scheme are line comments. There are + SRFIs for block comments and S-expression comments, but support for + them varies from system to system. Expression comments are not hard + for editors to deal with because it is safe not to deal with them at + all; however, in Scheme S-expression comments, which are written by + prefixing an expression with `#;', the expression to be commented + should be placed on the next line. This is because editors that do + not deal with them at all may see the semicolon as the start of a + line comment, which will throw them off. Expression comments in + Common Lisp, however, are always safe. + + In Common Lisp, the two read-time optionals that are guaranteed to + ignore any form following them are `#+(OR)' and `#-(AND)'. `#+NIL' + is sometimes used in their stead, but, while it may appear to be an + obviously false optional, it actually is not. The feature + expressions are read in the KEYWORD package, so NIL is read not as + CL:NIL, i.e. the boolean false value, but as :NIL, a keyword symbol + whose name happens to be `NIL'. Not only is it not read as the + boolean false value, but it has historically been used to indicate a + feature that might be enabled -- in JonL White's New Implementation + of Lisp! However, the New Implementation of Lisp is rather old these + days, and unlikely to matter much...until Alastair Bridgewater writes + Nyef's Implementation of Lisp. + +,** Documentation + +On-line references and documentation/manuals are both useful for +independent purposes, but there is a very fine distinction between +them. Do not generate documentation or manuals automatically from the +text of on-line references. + + Rationale: /On-line references/ are quick blurbs associated with + objects in a running Lisp image, such as documentation strings in + Common Lisp or Emacs Lisp. These assume that the reader is familiar + with the gist of the surrounding context, but unclear on details; + on-line references specify the details of individual objects. + + /Documentation/ and /manuals/ are fuller, organized, and cohesive + documents that explain the surrounding context to readers who are + unfamiliar with it. A reader should be able to pick a manual up and + begin reading it at some definite point, perusing it linearly to + acquire an understanding of the subject. Although manuals may be + dominated by reference sections, they should still have sections that + are linearly readable to acquaint the reader with context. + +,** Round and Square Brackets + +Some implementations of Scheme provide a non-standard extension of the +lexical syntax whereby balanced pairs of square brackets are +semantically indistinguishable from balanced pairs of round brackets. +Do not use this extension. + + Rationale: Because this is a non-standard extension, it creates + inherently non-portable code, of a nature much worse than using a + name in the program which is not defined by the R5RS. The reason + that we have distinct typographical symbols in the first place is to + express different meaning. The only distinction between round + brackets and square brackets is in convention, but the precise nature + of the convention is not specified by proponents of square brackets, + who suggest that they be used for `clauses', or for forms that are + parts of enclosing forms. This would lead to such constructions as + + (let [(x 5) (y 3)] ...) + + or + + (let ([x 5] [y 3]) ...) + + or + + (let [[x 5] [y 3]] ...), + + the first two of which the author of this guide has seen both of, and + the last of which does nothing to help to distinguish the parentheses + anyway. + + The reader of the code should not be forced to stumble over a + semantic identity because it is expressed by a syntactic distinction. + The reader's focus should not be directed toward the lexical tokens; + it should be directed toward the structure, but using square brackets + draws the reader's attention unnecessarily to the lexical tokens. + +,* Attribution + +#+END_SRC +:END: + +**** TODO I have to get change (let ([x 5] [y 3])) -> (let ((x 5) (y 3))) +**** TODO comments + +#+BEGIN_SRC scheme + ;;;; Frob Grovel + + ;;; This section of code has some important implications: + ;;; 1. Foo. + ;;; 2. Bar. + ;;; 3. Baz. + + (define (fnord zarquon) + ;; If zob, then veeblefitz. + (quux zot + mumble ;Zibblefrotz. + frotz)) + +#+END_SRC +**** TODO literal data +Strings + +If the form in question is meant to be simply a list of literal data, +all of the subforms should be aligned to the same column, irrespective +of the first subform. + + Unacceptable: + + ("foo" "bar" "baz" "quux" "zot" + "mumble" "frotz" "gargle" "mumph") + + Questionable, but acceptable: + + (3 1 4 1 5 9 2 6 5 3 5 8 9 7 9 3 2 3 8 4 6 2 6 4 + 3 3 8 3 2 7 9 5 0 2 8 8 4 1 9 7 1 6 9 3 9 9 3) + + Acceptable: + + ("foo" "bar" "baz" "quux" "zot" + "mumble" "frotz" "gargle" "mumph") + + ("foo" + "bar" "baz" "quux" "zot" + "mumble" "frotz" "gargle" "mumph") +**** TODO follow this syntax convention proc args + +:alignment: +The operator of any form, i.e. the first subform following the opening +round bracket, determines the rules for indenting or aligning the +remaining forms. Many names in this position indicate special +alignment or indentation rules; these are special operators, macros, or +procedures that have certain parameter structures. + +If the first subform is a non-special name, however, then if the second +subform is on the same line, align the starting column of all following +subforms with that of the second subform. If the second subform is on +the following line, align its starting column with that of the first +subform, and do the same for all remaining subforms. + +In general, Emacs will indent Lisp code correctly. Run `C-M-q' +(indent-sexp) on any code to ensure that it is indented correctly, and +configure Emacs so that any non-standard forms are indented +appropriately. + + Unacceptable: + + (+ (sqrt -1) + (* x y) + (+ p q)) + + (+ + (sqrt -1) + (* x y) + (+ p q)) + + Acceptable: + + (+ (sqrt -1) + (* x y) + (+ p q)) + + (+ + (sqrt -1) + (* x y) + (+ p q)) + + Rationale: The columnar alignment allows the reader to follow the + operands of any operation straightforwardly, simply by scanning + downward or upward to match a common column. Indentation dictates + structure; confusing indentation is a burden on the reader who wishes + to derive structure without matching parentheses manually. + +:END: + +If you have a procedure, then it's arguments should be on the same line. +#+BEGIN_SRC scheme +(proc args + (proc (proc + args) + (proc args) + (proc (proc + args) + (proc (proc + (proc (proc + args)))))) + + (proc (proc + args))) +#+END_SRC + +*** TODO write various tests for == + +I have many bits of code in opensmtpd.org.archive that should result in an +error. I should write some tests for this. +** NO should I modifiy some of the records to include a sanitize field? +Probably not. It would be cool if this function ran automatically upon record +initiation, but there's no to make it do that. +** TODO which sanitize function is better? Pick the better sanitize method and use that one. + +The sanitize function found in opensmtpd-listen-on-configuration-filters + +Or the sanitize function +sanitize-list-of-options-for-match-configuration ? + +sanitize-list-of-options-for-match-configuration is probably faster. But is it? +It is an iteratize loop that checks for all issues as it loops through the +options. There is a lot of repetitive code in this procedure. + +BUT opensmtpd-listen-on-configuration-filters certainly seems easier to follow. +** TODO remove opensmtpd-table-type fieldname and instead move that it its own procedure outside of the record + +ONly use one function instead of + + +;; this procedure takes in one argument. +;; if that argument is an whose fieldname 'values' is an assoc-list, then it returns +;; #t, #f if otherwise. +;; TODO should I remove these two functions? And instead use the (opensmtpd-table-configuration-type) procedure? +(define (table-whose-data-are-assoc-list? table) + (if (not (opensmtpd-table-configuration? table)) + #f + (assoc-list? (opensmtpd-table-configuration-data table)))) + +;; this procedure takes in one argument +;; if that argument is an whose fieldname 'values' is a list of strings, then it returns +;; #t, #f if otherwise. +(define (table-whose-data-are-a-list-of-strings? table) + (if (not (opensmtpd-table-configuration? table)) + #f + (list-of-strings? (opensmtpd-table-configuration-data table)))) + + And opensmtpd-table-type +** TODO OpenSMTPD Service documentation + +OpenSMTPD is an easy-to-use mail transfer agent (MTA). Its configuration file is +throughly documented in man 5 =smtpd.conf=. OpenSMTPD *listens* for incoming +mail and *matches* the mail to *actions*. The following records represent those +stages: ~~, +~=, =~, +~~, and +~~. + +Additionally, each ~~ and +~~ may use a list of +~~, and/or +~~ records to filter email/spam. Also +numerous records' fieldnames use ~~ to hold lists +or key value pairs of data. + +A simple example configuration is below: + +#+BEGIN_SRC scheme +(let ((smtp.gnu.org (opensmtpd-pki-configuration + (domain "smtp.gnu.org") + (cert "file.cert") + (key "file.key")))) + (service opensmtpd-service-type + (opensmtpd-configuration + (listen-ons (list + (opensmtpd-listen-on-configuration + (pki smtp.gnu.org)) + (opensmtpd-listen-on-configuration + (pki smtp.gnu.org) + (secure-connection "smtps")))) + (matches (list + (opensmtpd-match-configuration + (action + (opensmtpd-action-local-delivery-configuration + (name "local-delivery")))) + (opensmtpd-match-configuration + (action + (opensmtpd-action-relay-configuration + (name "relay"))))))))) +#+END_SRC + +- Scheme Variable: opensmtpd-service-type + + Service type for the OpenSMTPD ([[https://www.opensmtpd.org][https://www.opensmtpd.org]]) email server. The + value for this service type is a ~~ record. + +- Data Type: opensmtpd-configuration + + Data type representing the configuration of OpenSMTPD. + + - ~package~ (default: ~opensmtpd~) + + The OpenSMTPD package to use. + + - ~config-file~ (default: ~#f~) + + File-like object of the OpenSMTPD configuration file to use. By default it + listens on the loopback network interface, and allows for mail from users + and daemons on the local machine, as well as permitting email to remote + servers. Run ~man smtpd.conf~ for more information. + + - ~bounce~ (default: ~(list "4h")~) + + ~bounce~ is a list of strings, which send warning messages to the envelope + sender when temporary delivery failures cause a message to remain in the + queue for longer than string _delay_. Each string _delay_ parameter consists + of a string beginning with a positive decimal integer and a unit s, m, h, + or d. At most four delay parameters can be specified. + + - ~listen-ons~ (default: ~(list (opensmtpd-listen-on-configuration))~ ) + + ~listen-ons~ is a list of ~~ records. + This list details what interfaces and ports OpenSMTPD listens on as well as + other information. + + - ~listen-on-socket~ (default: ~(opensmtpd-listen-on-socket-configuration-configuration)~ ) + + Listens for incoming connections on the Unix domain socket. + + - ~includes~ (default: ~#f~) + + # TODO ~includes~ should support a list of string filenames or gexps. + ~includes~ is a list of string _filenames_. Each filename's contents is + additional configuration that is inserted into the top of the configuration + file. + + - ~matches~ default: + + #+BEGIN_SRC scheme + (list (opensmtpd-match-configuration + (action (opensmtpd-action-local-delivery-configuration + (name "local") + (method "mbox"))) + (for (opensmtpd-option-configuration + (option "for local")))) + (opensmtpd-match-configuration + (action (opensmtpd-action-relay-configuration + (name "outbound"))) + (from (opensmtpd-option-configuration + (option "from local"))) + (for (opensmtpd-option-configuration + (option "for any"))))) + #+END_SRC + + ~matches~ is a list of ~~ records, which + matches incoming mail and sends it to a correspending action. The match + records are evaluated sequentially, with the first match winning. If an + incoming mail does not match any match records, then it is rejected. + + # TODO when the code supports mda-wrappers, add in this documentation. + # - ~mda-wrappers~ + + - ~mta-max-deferred~ (default: ~100~) + + When delivery to a given host is suspended due to temporary failures, cache + at most _number_ envelopes for that host such that they can be delivered as + soon as another delivery succeeds to that host. The default is 100. + + - ~queue~ (default: ~#f~) + + ~queue~ expects an ~~ record. With it, one may + compress and encrypt queue-ed emails as well as set the default expiration + time for temporarily undeliverable messages. + + - ~smtp~ (default: ~#f~) + + ~smtp~ expects an ~~ record, which lets one + specifiy how large email may be along with other settings. + + - ~srs~ (default: ~#f~) + + ~srs~ expects an ~~ record, which lets one set + up SRS, the Sender Rewritting Scheme. + +- Data Type: opensmtpd-listen-on-configuration + + Data type representing the configuration of an + ~~. Listen on the fieldname ~interface~ for + incoming connections, using the same syntax as for ifconfig(8). The interface + parameter may also be an string interface group, an string IP address, or a + string domain name. Listening can optionally be restricted to a specific + address fieldname ~family~, which can be either "inet4" or "inet6". + + - ~interface~ (default: "lo") + + The string interface to listen for incoming connections. These interface can + usually be found by the command ~ip link~. + + - ~family~ (default: ~#f~) + + The string IP family to use. Valid strings are "inet4" or "inet6". + + - ~auth~ (default: ~#f~) + + Support SMTPAUTH: clients may only start SMTP transactions after successful + authentication. If ~auth~ is ~#t~, then users are authenticated against + their own normal login credentials. Alternatively ~auth~ may be an + ~~ whose users are authenticated against + their passwords. + + - ~auth-optional~ (default: ~#f~) + + Support SMTPAUTH optionally: clients need not authenticate, but may do so. + This allows the ~~ to both accept + incoming mail from untrusted senders and permit outgoing mail from + authenticated users (using ~~ fieldname + ~auth~). It can be used in situations where it is not possible to listen on + a separate port (usually the submission port, 587) for users to + authenticate. + + - ~filters~ (default: ~#f~) + + A list of one or many ~~ or + ~~ records. The filters are applied + sequentially. These records listen and filter on connections handled by this + listener. + + - ~hostname~ (default: ~#f~) + + Use string "hostname" in the greeting banner instead of the default server + name. + + - ~hostnames~ (default: ~#f~) + + Override the server name for specific addresses. Use a + ~~ containing a mapping of string IP + addresses to hostnames. If the address on which the connection arrives + appears in the mapping, the associated hostname is used. + + - ~mask-src~ (default: ~#f~) + + If ~#t~, then omit the from part when prepending “Received” headers. + + - ~disable-dsn~ (default: ~#f~) + + When ~#t~, then disable the DSN (Delivery Status Notification) extension. + + - ~pki~ (default: ~#f~) + + For secure connections, use an ~~ + to prove a mail server's identity. + + - ~port~ (default: ~#f~) + + Listen on the _integer_ port instead of the default port of 25. + + - ~proxy-v2~ (default: ~#f~) + + If ~#t~, then support the PROXYv2 protocol, rewriting appropriately source + address received from proxy. + + - ~received-auth~ (default: ~#f~) + + If ~#t~, then in “Received” headers, report whether the session was + authenticated and by which local user. + + - ~senders~ (default: ~#f~) + + Look up the authenticated user in the supplied + ~~ to find the email addresses that user is + allowed to submit mail as. + + - ~secure-connection~ (default: ~#f~) + + This is a string of one of these options: + + |----------------------+---------------------------------------------| + | "smtps" | Support SMTPS, by default on port 465. | + |----------------------+---------------------------------------------| + | "tls" | Support STARTTLS, by default on port 25. | + |----------------------+---------------------------------------------| + | "tls-require-verify" | Like tls, but force clients to establish | + | | a secure connection before being allowed to | + | | start an SMTP transaction. With the verify | + | | option, clients must also provide a valid | + | | certificate to establish an SMTP session. | + |----------------------+---------------------------------------------| + + - ~tag~ (default: ~#f~) + + Clients connecting to the listener are tagged with the given string tag. + +- Data Type: opensmtpd-listen-on-socket-configuration + + Data type representing the configuration of an + ~~. Listen for incoming SMTP + connections on the Unix domain socket =/var/run/smtpd.sock=. This is done by + default, even if the directive is absent. + + - ~filters~ (default: ~#f~) + + A list of one or many ~~ or + ~~ records. These filter incoming + connections handled by this listener. + + - ~mask-src~ (default: ~#f~) + + If ~#t~, then omit the from part when prepending “Received” headers. + + - ~tag~ (default: ~#f~) + + Clients connecting to the listener are tagged with the given string tag. + +- Data Type: opensmtpd-match-configuration + + This data type represents the configuration of an + ~~ record. + + If at least one mail envelope matches the options of one match record, receive + the incoming message, put a copy into each matching envelope, and atomically + save the envelopes to the mail spool for later processing by the respective + ~~ found in fieldname ~action~. + + - ~action~ (default: ~#f~) + + If mail matches this match configuration, then do this action. Valid values + include ~~ or + ~~. + + - ~options~ (default: ~#f~) ~~ + The fieldname 'option' is a list of unique + ~~ records. + + Each ~~ record's fieldname 'option' has some + mutually exclusive options: there can be one "for" and one "from" option. + + |---------------------------+--------------------------------| + | for | from | + |---------------------------+--------------------------------| + | use one of the following: | only use one of the following: | + |---------------------------+--------------------------------| + | "for any" | "from any" | + | "for local" | "from auth" | + | "for domain" | "from local" | + | "for rcpt-to" | "from mail-from" | + | | "from socket" | + | | "from src" | + |---------------------------+--------------------------------| + + The following matching options are supported and can all be negated via (not + #t). The options that support a table (anything surrounded with '<' and '>' + eg:
), also support specifying regex via (regex #t). + + - =for any= + + Specify that session may address any destination. + + - =for local= + + Specify that session may address any local domain. This is the default, + and may be omitted. + + - =for domain _domain_ | = + + Specify that session may address the string or list table _domain_. + + - =for rcpt-to _recipient_ | = + + Specify that session may address the string or list table _recipient_. + + - =from any= + + Specify that session may originate from any source. + + - =from auth= + + Specify that session may originate from any authenticated user, no matter + the source IP address. + + - =from auth _user_ | = + + Specify that session may originate from authenticated _user_ or user list + user, no matter the source IP address. + + - =from local= + + Specify that session may only originate from a local IP address, or from + the local enqueuer. This is the default, and may be omitted. + + - =from mail-from _sender_ | = + + Specify that session may originate from _sender_ or table _sender_, no + matter the source IP address. + + - =from rdns= + + Specify that session may only originate from an IP address that resolves + to a reverse DNS. + + - =from rdns _hostname_ | = + + Specify that session may only originate from an IP address that resolves + to a reverse DNS matching string or list string _hostname_. + + - =from socket= + + Specify that session may only originate from the local enqueuer. + + - =from src _address_ |
= + + Specify that session may only originate from string or list table address + which can be a specific _address_ or a subnet expressed in CIDR-notation. + + - =auth= + + Matches transactions which have been authenticated. + + - =auth _username_ | = + + Matches transactions which have been authenticated for user or user list + _username_. + + - =helo _helo-name_ | = + + Specify that session's HELO / EHLO should match the string or list table + _helo-name_. + + - =mail-from _sender_ | = + + Specify that transactions's MAIL FROM should match the string or list + table _sender_. + + - =rcpt-to _recipient_ | = + + Specify that transaction's RCPT TO should match the string or list table + _recipient_. + + - =tag tag= + Matches transactions tagged with the given _tag_. + + - =tls= + Specify that transaction should take place in a TLS channel. + + Here is a simple example: + #+BEGIN_SRC scheme + (opensmtpd-option-configuration + (not #t) + (regex #f) + (option "for domain") + (data (opensmtpd-table-configuration + (name "domain-table") + (data (list "gnu.org" "dismail.de"))))) + #+END_SRC + + The mail must NOT come from the domains =gnu.org= or =dismail.de=. + + - Data Type: opensmtpd-option-configuration + +- Data Type: opensmtpd-action-local-delivery-configuration + + This data type represents the configuration of an + ~~ record. + + - ~name~ (default: ~#f~) + + ~name~ is the string name of the relay action. + + - ~method~ (default: ~"mbox"~) + + The email delivery option. Valid options are: + + - ~"mbox"~ + + Deliver the message to the user's mbox with mail.local(8). + + - ~"expand-only"~ + + Only accept the message if a delivery method was specified in an aliases + or _.forward file_. + + - ~"forward-only"~ + + Only accept the message if the recipient results in a remote address after + the processing of aliases or forward file. + + - ~~ + + Deliver the message to an LMTP server at + ~~'s fieldname ~destination~. The location + may be expressed as string host:port or as a UNIX socket. Optionally, + ~~'s fieldname ~rcpt-to~ might be specified + to use the recipient email address (after expansion) instead of the local + user in the LMTP session as RCPT TO. + + - ~~ + + Deliver the message to the maildir in + ~~'s fieldname ~pathname~ if specified, + or by default to =~/Maildir=. + + The pathname may contain format specifiers that are expanded before use + (see the below section about Format Specifiers). + + If ~~'s record fieldname ~junk~ is ~#t~, + then message will be moved to the ‘Junk’ folder if it contains a positive + ‘X-Spam’ header. This folder will be created under fieldname ~pathname~ if + it does not yet exist. + + - ~~ + + Delegate the delivery to the ~~'s fieldname + ~command~ (type string) that receives the message on its standard input. + + The ~command~ may contain format specifiers that are expanded before use + (see Format Specifiers). + + - ~alias~ (default: ~#f~) + + Use the mapping table for aliases expansion. ~alias~ is an + ~~. + + - ~ttl~ (default: ~#f~) + + ~ttl~ is a string specify how long a message may remain in the queue. It's + format is =n{s|m|h|d}=. eg: "4m" is four minutes. + + - ~user~ (default: ~#f~ ) + + ~user~ is the string username for performing the delivery, to be looked up + with getpwnam(3). + + This is used for virtual hosting where a single username is in charge of + handling delivery for all virtual users. + + This option is not usable with the mbox delivery method. + + - ~userbase~ (default: ~#f~) + + ~userbase~ is an ~~ record for mapping user + lookups instead of the getpwnam(3) function. + + The fieldnames ~user~ and ~userbase~ are mutually exclusive. + + - ~virtual~ (default: ~#f~) + + ~virtual~ is an ~~ record is used for virtual + expansion. + # TODO man 5 smtpd.conf says "The aliasing table format is described in + # table(5)." What is virtual expansion? I do NOT know how to use ~virtual~ + # properly. What sort of do I need? does the + # below work? + # (opensmtpd-table (name "virtual") (data '(("joshua" . "jbranso@dismail.de")))) + + # TODO fix this ~wrapper documentation~. Should it accept an + # ? If so, then I need to write an + # - ~wrapper~ (default: ) + + # TODO double check that these options are all correct + +- Data Type: opensmtpd-action-relay-configuration + + This data type represents the configuration of an + ~~ record. + + - ~name~ (default: ~#f~) + + ~name~ is the string name of the relay action. + + - ~backup~ (default: ~#f~) + + When ~#t~, operate as a backup mail exchanger delivering messages to any + mail exchanger with higher priority. + + - ~backup-mx~ (default: ~#f~) + + Operate as a backup mail exchanger delivering messages to any mail exchanger + with higher priority than mail exchanger identified as string name. + + - ~helo~ (default: ~#f~) + + Advertise string heloname as the hostname to other mail exchangers during + the HELO phase. + + - ~helo-src~ (default: ~#f~ ) + + Use the mapping ~~ to look up a hostname + matching the source address, to advertise during the HELO phase. + + - ~domain~ (default: ~#f~) + + Do not perform MX lookups but look up destination domain in an + ~~ and use matching relay url as relay host. + + - ~host~ (default: ~#f~) + + Do not perform MX lookups but relay messages to the relay host described by + the string relay-url. The format for relay-url is + =[proto://[label@]]host[:port]=. The following protocols are available: + + |------------+----------------------------------------------------------------| + | smtp | Normal SMTP session with opportunistic STARTTLS (the default). | + | smtp+tls | Normal SMTP session with mandatory STARTTLS. | + | smtp+notls | Plain text SMTP session without TLS. | + | lmtp | LMTP session. port is required. | + | smtps | SMTP session with forced TLS on connection, default port is | + | | 465. | + |------------+----------------------------------------------------------------| + + Unless noted, port defaults to 25. + + The label corresponds to an entry in a credentials table, as documented in + =table(5)=. It is used with the ="smtp+tls"= and ="smtps"= protocols for + authentication. Server certificates for those protocols are verified by + default. + + - ~pki~ (default: ~#f~) + + For secure connections, use the certificate associated with + ~~ (declared in a pki directive) to prove the + client's identity to the remote mail server. + + - ~srs~ (default: ~#f~) + + If ~#t~, then when relaying a mail resulting from a forward, use the Sender + Rewriting Scheme to rewrite sender address. + + - ~tls~ (default: ~#f~) boolean or string "no-verify" + + When ~#t~, Require TLS to be used when relaying, using mandatory STARTTLS by + default. When used with a smarthost, the protocol must not be + ="smtp+notls://"=. When string ~"no-verify"~, then do not require a valid + certificate. + + - ~auth~ (default: ~#f~) ~~ + + Use the alist ~~ for connecting to relay-url + using credentials. This option is usable only with fieldname ~host~ option. + + - ~mail-from~ (default: ~#f~) string + + Use the string _mailaddress_ as MAIL FROM address within the SMTP transaction. + + - ~src~ (default: ~#f~) string | ~~ + + Use the string or ~~ sourceaddr for the + source IP address, which is useful on machines with multiple interfaces. If + the list contains more than one address, all of them are used in such a way + that traffic is routed as efficiently as possible. + +- Data Type: opensmtpd-filter-configuration + + This data type represents the configuration of an + ~~. This is the filter record one should use + if they want to use an external package to filter email eg: rspamd or + spamassassin. + + - ~name~ (default: ~#f~) + + The string name of the filter. + + - ~proc~ (default: ~#f~) + + # TODO let ~proc~ be a gexp + The string command or process name. If ~proc-exec~ is ~#t~, ~proc~ is + treated as a command to execute. Otherwise, it is a process name. + + - ~proc-exec~ (default: ~#f~) + +- Data Type: opensmtpd-filter-phase-configuration + + This data type represents the configuration of an + ~~. + + In a regular workflow, smtpd(8) may accept or reject a message based only on + the content of envelopes. Its decisions are about the handling of the message, + not about the handling of an active session. + + Filtering extends the decision making process by allowing smtpd(8) to stop at + each phase of an SMTP session, check that options are met, then decide if a + session is allowed to move forward. + + With filtering via an ~~ record, a + session may be interrupted at any phase before an envelope is complete. A + message may also be rejected after being submitted, regardless of whether the + envelope was accepted or not. + + - ~name~ (default: ~#f~) + + The string name of the filter phase. + + - ~phase-name~ (default: ~#f~) + + The string name of the phase. Valid values are: + + |-------------+-----------------------------------------------| + | "connect" | upon connection, before a banner is displayed | + |-------------+-----------------------------------------------| + | "helo" | after HELO command is submitted | + |-------------+-----------------------------------------------| + | "ehlo" | after EHLO command is submitted | + |-------------+-----------------------------------------------| + | "mail-from" | after MAIL FROM command is submitted | + |-------------+-----------------------------------------------| + | "rcpt-to" | after RCPT TO command is submitted | + |-------------+-----------------------------------------------| + | "data" | after DATA command is submitted | + |-------------+-----------------------------------------------| + | "commit" | after message is fully is submitted | + |-------------+-----------------------------------------------| + + - ~options~ (default ~#f~) + + A list of unique ~~ records. + + At each phase, various options, specified by a list of + ~~, may be checked. The + ~~'s fieldname 'option' values of: "fcrdns", + "rdns", and "src" data are available in all phases, but other data must have + been already submitted before they are available. Options with a =
= + next to them require the ~~'s fieldname + ~data~ to be an ~~. There are the available + options: + + |-------------------+----------------------------------------| + | fcrdns | forward-confirmed reverse DNS is valid | + |-------------------+----------------------------------------| + | rdns | session has a reverse DNS | + |-------------------+----------------------------------------| + | rdns
| session has a reverse DNS in table | + |-------------------+----------------------------------------| + | src
| source address is in table | + |-------------------+----------------------------------------| + | helo
| helo name is in table | + |-------------------+----------------------------------------| + | auth | session is authenticated | + |-------------------+----------------------------------------| + | auth
| session username is in table | + |-------------------+----------------------------------------| + | mail-from
| sender address is in table | + |-------------------+----------------------------------------| + | rcpt-to
| recipient address is in table | + |-------------------+----------------------------------------| + + These conditions may all be negated by setting + ~~'s fieldname ~not~ to ~#t~. + + Any conditions that require a table may indicate that tables include regexs + setting ~~'s fieldname ~regex~ to ~#t~. + + - ~decision~ + + A string decision to be taken. Some decisions require an ~message~ or + ~value~. Valid strings are: + + |----------------------+------------------------------------------------| + | "bypass" | the session or transaction bypasses filters | + |----------------------+------------------------------------------------| + | "disconnect" message | the session is disconnected with message | + |----------------------+------------------------------------------------| + | "junk" | the session or transaction is junked, i.e., an | + | | ‘X-Spam: yes’ header is added to any messages | + |----------------------+------------------------------------------------| + | "reject" message | the command is rejected with message | + |----------------------+------------------------------------------------| + | "rewrite" value | the command parameter is rewritten with value | + |----------------------+------------------------------------------------| + + Decisions that involve a message require that the message be RFC valid, + meaning that they should either start with a 4xx or 5xx status code. + Descisions can be taken at any phase, though junking can only happen before + a message is committed. + + - ~message~ (default ~#f~) + + A string message beginning with a 4xx or 5xx status code. + + - ~value~ (default: ~#f~) + + A number value. ~value~ and ~message~ are mutually exclusive. + +- Data Type: opensmtpd-option-configuration + + This data type represents the configuration of an + ~~, which is used by + ~~ and ~~ + to match various options for email. + + - ~conditition~ (default ~#f~) + + A string option to be taken. Some options require a string or an + ~~ via the fieldname data. When the option + record is used inside of an ~~, then + valid strings are: + + At each phase, various options may be matched. The fcrdns, rdns, and src + data are available in all phases, but other data must have been already + submitted before they are available. + + |---------------------+----------------------------------------| + | "fcrdns" | forward-confirmed reverse DNS is valid | + | "rdns" | session has a reverse DNS | + | "rdns"
| session has a reverse DNS in table | + | "src"
| source address is in table | + | "helo"
| helo name is in table | + | "auth" | session is authenticated | + | "auth"
| session username is in table | + | "mail-from"
| sender address is in table | + | "rcpt-to"
| recipient address is in table | + |---------------------+----------------------------------------| + + When ~~ is used inside of an + ~~, then valid strigs for fieldname ~option~ + are: "for", "for any", "for local", "for domain", "for rcpt-to", "from any" + "from auth", "from local", "from mail-from", "from rdns", "from socket", + "from src", "auth", "helo", "mail-from", "rcpt-to", "tag", or "tls". + + - ~data~ (default ~#f~) ~~ + + Some options require a table to be present. One would specify that table + here. + - ~regex~ (default: ~#f~) boolean + + Any options using a table may indicate that tables hold regex by + prefixing the table name with the keyword regex. + - ~not~ (default: ~#f~) boolean + + When ~#t~, this option record is negated. + +- Data Type: opensmtpd-table-configuration + + This data type represents the configuration of an + ~~. + + - ~name~ (default ~#f~) + + ~name~ is the name of the ~~ record. + + - ~data~ (default: ~#f~) + + ~data~ expects a list of strings or an alist, which is a list of + cons cells. eg: ~(data (list ("james" . "password")))~ OR + ~(data (list ("gnu.org" "fsf.org")))~. + +- Data Type: opensmtpd-pki-configuration + + This data type represents the configuration of an + ~~. + + - ~domain~ (default ~#f~) + + ~domain~ is the string name of the ~~ record. + + - ~cert~ (default: ~#f~) + + ~cert~ (default: ~#f~) + + ~cert~ is the string certificate filename to use for this pki. + + - ~key~ (default: ~#f~) + + ~key~ is the string certificate falename to use for this pki. + + - ~dhe~ (default: ~"none"~) + + Specify the DHE string parameter to use for DHE cipher suites with host + pkiname. Valid parameter values are "none", "legacy", or "auto". For "legacy", a + fixed key length of 1024 bits is used, whereas for "auto", the key length is + determined automatically. The default is "none", which disables DHE cipher + suites. + +- Data Type: opensmtpd-maildir-configuration + + - ~pathname~ (default: ~"~/Maildir"~) + + Deliver the message to the maildir if pathname if specified, or by default + to =~/Maildir=. + + The pathname may contain format specifiers that are expanded before use + (see FORMAT SPECIFIERS). + + - ~junk~ (default: ~#f~) + + If the junk argument is ~#t~, then the message will be moved to the =‘Junk’= + folder if it contains a positive =‘X-Spam’= header. This folder will be + created under pathname if it does not yet exist. + +- Data Type: opensmtpd-mda-configuration + # Do we need a dataypte for mda configuration? + # this could just be a gexp in the fieldname opensmtpd-configuration-mda + + - ~name~ + + The string name for this MDA command. + + - ~command~ + + Delegate the delivery to a command that receives the message on its standard + input. + + The command may contain format specifiers that are expanded before use (see + FORMAT SPECIFIERS). + +- Data Type: opensmtpd-queue-configuration + + - ~compression~ (default ~#f~) + + Store queue files in a compressed format. This may be useful to save disk + space. + - ~encryption~ (default ~#f~) + + Encrypt queue files with EVP_aes_256_gcm(3). If no key is specified, it is + read with getpass(3). If the string stdin or a single dash (‘-’) is given + instead of a key, the key is read from the standard input. + - ~ttl-delay~ (default ~#f~) + + Set the default expiration time for temporarily undeliverable messages, + given as a positive decimal integer followed by a unit s, m, h, or d. The + default is four days ("4d"). + +- Data Type: opensmtpd-smtp-configuration + + Data type representing an ~~ record. + + - ~ciphers~ (default: ~#f~) + + Set the control string for SSL_CTX_set_cipher_list(3). The default is + "HIGH:!aNULL:!MD5". + - ~limit-max-mails~ (default: ~100~) + + Limit the number of messages to count for each sessio + - ~limit-max-rcpt~ (default: ~1000~) + + Limit the number of recipients to count for each transaction. + - ~max-message-size~ (default: ~35M~) + + Reject messages larger than size, given as a positive number of bytes or as + a string to be parsed with scan_scaled(3). + - ~sub-addr-delim character~ (default: ~+~) + + When resolving the local part of a local email address, ignore the ASCII + character and all characters following it. This is helpful for email + filters. ="admin+bills@gnu.org"= is the same email address as + ="admin@gnu.org"=. BUT an email filter can filter emails addressed to first + email address into a 'Bills' email folder. + +- Data Type: opensmtpd-srs-configuration + + - ~key~ (default: ~#f~) + + Set the secret key to use for SRS, the Sender Rewriting Scheme. + + - ~backup-key~ (default: ~#f~) + + Set a backup secret key to use as a fallback for SRS. This can be used to + implement SRS key rotation. + - ~ttl-delay~ (default: ~"4d"~) + + Set the time-to-live delay for SRS envelopes. After this delay, a bounce + reply to the SRS address will be discarded to limit risks of forged + addresses. + +- Format Specifiers + + Some configuration records support expansion of their parameters at + runtime. Such records (for example + ~~, ~~) may use + format specifiers which are expanded before delivery or relaying. The + following formats are currently supported: + + |---------------------+-------------------------------------------------------| + | =%{sender}= | sender email address, may be empty string | + | =%{sender.user}= | user part of the sender email address, may be empty | + | =%{sender.domain}= | domain part of the sender email address, may be empty | + | =%{rcpt}= | recipient email address | + | =%{rcpt.user}= | user part of the recipient email address | + | =%{rcpt.domain}= | domain part of the recipient email address | + | =%{dest}= | recipient email address after expansion | + | =%{dest.user}= | user part after expansion | + | =%{dest.domain}= | domain part after expansion | + | =%{user.username}= | local user | + | =%{user.directory}= | home directory of the local user | + | =%{mbox.from}= | name used in mbox From separator lines | + | =%{mda}= | mda command, only available for mda wrappers | + |---------------------+-------------------------------------------------------| + + Expansion formats also support partial expansion using the optional bracket notations + with substring offset. For example, with recipient domain =“example.org”=: + + |------------------------+----------------------| + | =%{rcpt.domain[0]}= | expands to “e” | + | =%{rcpt.domain[1]}= | expands to “x” | + | =%{rcpt.domain[8:]}= | expands to “org” | + | =%{rcpt.domain[-3:]}= | expands to “org” | + | =%{rcpt.domain[0:6]}= | expands to “example” | + | =%{rcpt.domain[0:-4]}= | expands to “example” | + |------------------------+----------------------| + + In addition, modifiers may be applied to the token. For example, with recipient + =“User+Tag@Example.org”=: + + |--------------------------+-----------------------------------| + | =%{rcpt:lowercase}= | expands to “user+tag@example.org” | + | =%{rcpt:uppercase}= | expands to “USER+TAG@EXAMPLE.ORG” | + | =%{rcpt:strip}= | expands to “User@Example.org” | + | =%{rcpt:lowercasestrip}= | expands to “user@example.org” | + |--------------------------+-----------------------------------| + + For security concerns, expanded values are sanitized and potentially dangerous + characters are replaced with ‘:’. In situations where they are desirable, the + “raw” modifier may be applied. For example, with recipient + =“user+t?g@example.org”=: + + |---------------+-----------------------------------| + | =%{rcpt}= | expands to “user+t:g@example.org” | + | =%{rcpt:raw}= | expands to “user+t?g@example.org” | + |---------------+-----------------------------------| +*** some example ~~ that are probably out of date + + #+BEGIN_SRC scheme + +;;this works! (opensmtpd-configuration->mixed-text-file (opensmtpd-configuration (smtp (opensmtpd-smtp-configuration (limit-max-rcpt 10))))) + +;; (tables (list +;; (opensmtpd-table-configuration +;; (name "aliases") +;; (data +;; (list +;; (cons "webmaster" "root") +;; (cons "postmaster" "root") +;; (cons "abuse" "root")))) +;; +;; (opensmtpd-table-configuration +;; (name "vdoms") +;; (data (list "gnucode.me" +;; "gnu-hurd.com"))) +;; (opensmtpd-table-configuration +;; (name (opensmtpd-table-configuration +;; (name "virtual") +;; (data (list "root" "postmaster@gnu.org")))) +;; (data (list (cons "joshua@gnucode.me" "joshua") +;; (cons "jbranso@gnucode.me" "joshua") +;; (cons "postmaster@gnucode.me" "joshua")))))) + + ;; (filter-chains + ;; (list + ;; (opensmtpd-filter-chain + ;; (name "dropDumbEmails") + ;; (filter-names (list "nofcrdnsDisconnect" + ;; "nordnsDisconnect"))))) + ;; (filter-phases + ;; (list (opensmtpd-filter-phase-configuration + ;; (name "nofcrdnsDisconnect") + ;; (phase-name "connect") + ;; (options (list "!fcrdns")) + ;; (decision "disconnect") + ;; (message "You have not set up forward confirmed DNS.")) + ;; (opensmtpd-filter-phase-configuration + ;; (name "nordnsDisconnect") + ;; (phase-name "connect") + ;; (options (list "!rdns")) + ;; + ;; (decision "reject") + ;; (message "You have not set up reverse DNS.")))) + ;; +(define example-opensmtpd-config-smaller + (opensmtpd-configuration + (listen-ons + (list + ;; this forum help suggests that I listen on 0.0.0.0 and NOT eth0 + ;; https://serverfault.com/questions/726795/opensmtpd-wont-work-at-reboot + ;; this listens for email from the outside world + ;; this lets local users logged into the system via ssh send email + (opensmtpd-listen-on-configuration + (interface "wlp2s0") + (port 465)))) + (matches (list + (opensmtpd-match-configuration + (name "maildir") + (action (opensmtpd-action-local-delivery-configuration + (method (opensmtpd-maildir-configuration + (pathname "/home/%{rcpt.user}/Maildir") + (junk #t))) + (virtual (opensmtpd-table-configuration + (name "virtual") + (data (list "root" "james@gnu.org")))))) + (for (opensmtpd-option-configuration + (option "for local")))))))) + +(define example-opensmtpd-config-small + (let ([interface "wlp2s0"] + [creds (opensmtpd-table-configuration + (name "creds") + (data + (list + (cons "joshua" + "$6$Ec4m8FgKjT2F/03Y$k66ABdse9TzCX6qaALB3WBL9GC1rmAWJmaoSjFMpbhzat7DOpFqpnOwpbZ34wwsQYIK8RQlqwM1I/v6vsRq86."))))] + [receive-action (opensmtpd-action-local-delivery-configuration + (name "receive") + (method (opensmtpd-maildir-configuration + (pathname "/home/%{rcpt.user}/Maildir") + (junk #t))) + (virtual (opensmtpd-table-configuration + (name "virtual") + (data (list "root" "james@gnu.org")))))] + [smtp.gnucode.me (opensmtpd-pki-configuration + (domain "smtp.gnucode.me") + (cert "opensmtpd.scm") + (key "opensmtpd.scm"))]) + (opensmtpd-configuration + (listen-ons + (list + ;; this forum help suggests that I listen on 0.0.0.0 and NOT eth0 + ;; https://serverfault.com/questions/726795/opensmtpd-wont-work-at-reboot + ;; this listens for email from the outside world + (opensmtpd-listen-on-configuration + (interface interface) + (port 25) + (secure-connection "tls") + (pki smtp.gnucode.me)) + ;; this lets local users logged into the system via ssh send email + (opensmtpd-listen-on-configuration + (interface interface) + (port 465) + (secure-connection "smtps") + (pki smtp.gnucode.me) + (auth creds)))) + (matches (list + (opensmtpd-match-configuration + (action receive-action) + (for (opensmtpd-option-configuration + (option "for local"))))))))) + +(define example-opensmtpd-config + (let ([interface "lo"] + [creds (opensmtpd-table-configuration + (name "creds") + (data + (list + (cons "joshua" + "$6$Ec4m8FgKjT2F/03Y$k66ABdse9TzCX6qaALB3WBL9GC1rmAWJmaoSjFMpbhzat7DOpFqpnOwpbZ34wwsQYIK8RQlqwM1I/v6vsRq86."))))] + [receive-action (opensmtpd-action-local-delivery-configuration + (name "receive") + (method (opensmtpd-maildir-configuration + (pathname "/home/%{rcpt.user}/Maildir") + (junk #t))) + (virtual (opensmtpd-table-configuration + (name "virtual") + (data (list "josh" "jbranso@dismail.de")))))] + [smtp.gnucode.me (opensmtpd-pki-configuration + (domain "smtp.gnucode.me") + (cert "opensmtpd.scm") + (key "opensmtpd.scm"))]) + (opensmtpd-configuration + ;; (mta-max-deferred 50) + ;; (queue + ;; (opensmtpd-queue-configuration + ;; (compression #t))) + ;; (smtp + ;; (opensmtpd-smtp-configuration + ;; (max-message-size "10M"))) + ;; (srs + ;; (opensmtpd-srs-configuration + ;; (ttl-delay "5d"))) + (listen-ons + (list + ;; this forum help suggests that I listen on 0.0.0.0 and NOT eth0 + ;; https://serverfault.com/questions/726795/opensmtpd-wont-work-at-reboot + ;; this listens for email from the outside world + (opensmtpd-listen-on-configuration + (interface interface) + (port 25) + (secure-connection "tls") + (pki smtp.gnucode.me)) + ;; this lets local users logged into the system via ssh send email + (opensmtpd-listen-on-configuration + (interface "lo") + (port 25) + (secure-connection "tls") + (pki smtp.gnucode.me)) + (opensmtpd-listen-on-configuration + (interface interface) + (port 465) + (secure-connection "smtps") + (pki smtp.gnucode.me) + (auth creds) + ;;(filter ) + ) + (opensmtpd-listen-on-configuration + (interface interface) + (port 587) + (secure-connection "tls-require") + (pki smtp.gnucode.me) + (auth creds)))) + (matches (list + (opensmtpd-match-configuration + (action (opensmtpd-action-relay-configuration + (name "send"))) + (for (opensmtpd-option-configuration + (option "for any"))) + (from (opensmtpd-option-configuration + (option "from any"))) + (auth (opensmtpd-option-configuration + (option "auth")))) + (opensmtpd-match-configuration + (action receive-action) + (from (opensmtpd-option-configuration + (option "from any"))) + (for (opensmtpd-option-configuration + (option "for domain") + (value (list "gnucode.me" "gnu-hurd.com"))))) + (opensmtpd-match-configuration + (action receive-action) + (for (opensmtpd-option-configuration + (option "for local"))))))))) + #+END_SRC + +*** some example smtpd.conf configs +*** serving multiple domains with one pki + +source: https://www.reddit.com/r/openbsd/comments/n41wkz/how_to_host_different_domains_for_an_email_server/ +#+BEGIN_EXAMPLE +​pki mail.primary.domain cert​pki mail.primary.domain cert "/etc/ssl/mail.primary.domain.fullchain.pem" + +pki mail.primary.domain key "/etc/ssl/private/mail.primary.domain.key" + + +filter check_dyndns phase connect match rdns regex { '.*\.dyn\..*', '.*\.dsl\..*' } \ + +disconnect "550 no residential connections" + + +filter check_rdns phase connect match !rdns \ + +disconnect "550 no rDNS is so 80s" + + +filter check_fcrdns phase connect match !fcrdns \ + +disconnect "550 no FCrDNS is so 80s" + + +filter senderscore \ + +proc-exec "filter-senderscore -blockBelow 10 -junkBelow 70 -slowFactor 5000" + + +filter rspamd proc-exec "filter-rspamd" + + +table usermap file:/etc/mail/usermap + +table credentials file:/etc/mail/credentials + +table domains { primary.domain, second.domain } + + +listen on all tls pki mail.primary.domain \ + +filter { check_dyndns, check_rdns, check_fcrdns, senderscore, rspamd } + + +listen on egress port 465 smtps pki mail.primary.domain \ + +auth ~~ filter rspamd + + +action "inbound" lmtp "/var/dovecot/lmtp" rcpt-to virtual ~~ #maildir junk alias + +action "outbound" relay helo mail.primary.domain + + +match from any for domain ~~ action "inbound" + +match for local action "inbound" + + +match from any auth for any action "outbound" + +match for any action "outbound" "/etc/ssl/mail.primary.domain.fullchain.pem" + +pki mail.primary.domain key "/etc/ssl/private/mail.primary.domain.key" + + +filter check_dyndns phase connect match rdns regex { '.*\.dyn\..*', '.*\.dsl\..*' } \ + +disconnect "550 no residential connections" + + +filter check_rdns phase connect match !rdns \ + +disconnect "550 no rDNS is so 80s" + + +filter check_fcrdns phase connect match !fcrdns \ + +disconnect "550 no FCrDNS is so 80s" + + +filter senderscore \ + +proc-exec "filter-senderscore -blockBelow 10 -junkBelow 70 -slowFactor 5000" + + +filter rspamd proc-exec "filter-rspamd" + + +table usermap file:/etc/mail/usermap + +table credentials file:/etc/mail/credentials + +table domains { primary.domain, second.domain } + + +listen on all tls pki mail.primary.domain \ + +filter { check_dyndns, check_rdns, check_fcrdns, senderscore, rspamd } + + +listen on egress port 465 smtps pki mail.primary.domain \ + +auth ~~ filter rspamd + + +action "inbound" lmtp "/var/dovecot/lmtp" rcpt-to virtual ~~ #maildir junk alias + +action "outbound" relay helo mail.primary.domain + + +match from any for domain ~~ action "inbound" + +match for local action "inbound" + + +match from any auth for any action "outbound" + +match for any action "outbound" + +#+END_EXAMPLE +** PROJ nice things to have [0/9] +*** TODO Should I delete ~~ ? or fieldname 'opensmtpd-configuration-mda-wrapppers'? + +~~'s fieldname 'method' allows +for an mda configuration. BUT instead of an mda-configuration record, you could +just use a list of strings and/or gexps. + +#+BEGIN_EXAMPLE + mda wrapper name command + Associate command with the mail delivery agent wrapper named name. When a local + delivery specifies a wrapper, the command associated with the wrapper will be ex‐ + ecuted instead. The command may contain format specifiers (see FORMAT + SPECIFIERS). +#+END_EXAMPLE + +If I choose to NOT delete ~~, then should I delete +'opensmtpd-configuration-mda-wrapppers'? + +Also should I delete the opensmtpd-action-local-delivery-configuration-wrapper? + +*** TODO make the 'auth-optional' and 'auth' fieldnames for ~~ autoencrypt passwords. [0/0] +Guix makes it pretty hard to find the openbsd binary file that encrypts +passwords for you. If I can progmatically find this file, it would be nice to +autoencrypt the users's passwords for you. + +What does this mean practically? + +#+BEGIN_SRC scheme +(opensmtpd-table-configuration + (name "credentials") + (data '(("joshua@gnu.org" . "somePassword") + ("postmaster@gnu.org") . "anotherSillyPassword"))) +#+END_SRC + +Gets stored in /gnu/store/ in the =smtpd.conf= as something like: + +#+BEGIN_EXAMPLE +table credentials { joshua@gnu.org = $some$Long$EncrytpedPassword, \ + postmaster@gnu.org = $some$Long$Other$EncrytpedPassword } +#+END_EXAMPLE + +You would need to encourage users NOT to have passwords in a public git repo. +With guile-git, it might be possible to sanitize the config, to ensure that the +passwords are NOT stored in the git repo. + +Alternatively, we could put the following in the documentation: + +#+BEGIN_SRC scheme +(use-modules (passwords)) + +(opensmtpd-table-configuration + (name "credentials") + (data %passwords)) +#+END_SRC + +*** PROJ Why does (opensmtpd-configuration) take so long to initialize? [0/1] + +For example, try to initialize this bit of code. It takes almost 5 seconds. +#+BEGIN_SRC scheme +(let ([interface "lo"] + [creds-table (opensmtpd-table-configuration + (name "creds") + (data + (list + (cons "joshua" + "$6$Ec4m8FgKjT2F/03Y$k66ABdse9TzCX6qaALB3WBL9GC1rmAWJmaoSjFMpbhzat7DOpFqpnOwpbZ34wwsQYIK8RQlqwM1I/v6vsRq86."))))] + [receive-action (opensmtpd-action-local-delivery-configuration + (name "receive") + (method (opensmtpd-maildir-configuration + (pathname "/home/%{rcpt.user}/Maildir") + (junk #t))) + (virtual (opensmtpd-table-configuration + (name "virtual") + (data (list "josh" "jbranso@dismail.de")))))] + [filter-dkimsign (opensmtpd-filter + (name "dkimsign") + (exec #t) + (proc (string-append "/path/to/dkimsign -d gnucode.me -s 2021-09-22 -c relaxed/relaxed -k " + "/path/to/dkimsign-key user nobody group nobody")))] + [smtp.gnucode.me (opensmtpd-pki-configuration + (domain "smtp.gnucode.me") + (cert "opensmtpd.scm") + (key "opensmtpd.scm"))]) + (opensmtpd-configuration + (mta-max-deferred 50) + (queue + (opensmtpd-queue-configuration + (compression #t))) + (smtp + (opensmtpd-smtp-configuration + (max-message-size "10M"))) + (srs + (opensmtpd-srs-configuration + (ttl-delay "5d"))) + (listen-ons + (list + (opensmtpd-listen-on-configuration + (interface interface) + (port 25) + (secure-connection "tls") + (filters (list (opensmtpd-filter-phase-configuration + (name "noFRDNS") + (phase "commit") + (options (list (opensmtpd-option-configuration + (option "fcrdns") + (not #t)))) + (decision "disconnect") + (message "No FCRDNS")))) + (pki smtp.gnucode.me)) + ;; this lets local users logged into the system via ssh send email + (opensmtpd-listen-on-configuration + (interface interface) + (port 465) + (secure-connection "smtps") + (pki smtp.gnucode.me) + (auth creds-table) + (filters (list filter-dkimsign))) + (opensmtpd-listen-on-configuration + (interface interface) + (port 587) + (secure-connection "tls-require") + (pki smtp.gnucode.me) + (auth creds-table) + (filters (list filter-dkimsign))))) + (matches (list + (opensmtpd-match-configuration + (action (opensmtpd-action-relay-configuration + (name "relay"))) + (for (opensmtpd-option-configuration + (option "for any"))) + (from (opensmtpd-option-configuration + (option "from any"))) + (auth (opensmtpd-option-configuration + (option "auth")))) + (opensmtpd-match-configuration + (action receive-action) + (from (opensmtpd-option-configuration + (option "from any"))) + (for (opensmtpd-option-configuration + (option "for domain") + (value (opensmtpd-table-configuration + (name "domain-table") + (data (list "gnucode.me" "gnu-hurd.com"))))))) + (opensmtpd-match-configuration + (action receive-action) + (for (opensmtpd-option-configuration + (option "for local")))))))) +#+END_SRC +**** TODO one area to look for speed up improvements would be in the sanitize function of =(opensmtpd-listen-on-configuration-filters)=. +*** PROJ check the code base for places to use apply, map, fold, eval, or remove [3/4] + +**** DONE string-in-list would be a good place. maybe is-value-right-type + +#+BEGIN_SRC scheme +(define (string-in-list? string list) + (if (null? list) + #f + (if (and (string? (car list)) (string=? string (car list))) + #t + (string-in-list? string (cdr list))))) + +(define (string-in-list? string list) + (primitive-eval (cons 'or (map (lambda (var) (string=? string var)) list)))) + +#+END_SRC + +**** DONE contains-duplicate +#+BEGIN_SRC scheme +(define (contains-duplicate? list) + (if (null? list) + #f + (or + ;; + (let loop ([list (cdr list)] + [1st (car list)]) + (if (null? list) + #f + (if (equal? 1st (car list)) + (data #t 1st) + (loop (cdr list) 1st)))) + ;; + (contains-duplicate? (cdr list))))) + +(define (contains-duplicate? list) + (if (null? list) + #f + (or (primitive-eval (cons 'or ; check if (car list) is in (cdr list) + (map (lambda (var) (equal? var (car list))) + (cdr list)))) + ;; check if (cdr list) contains duplicate + (contains-duplicate? (cdr list))))) +#+END_SRC + +**** DONE using remove and flatten and map +#+BEGIN_SRC scheme +(define (get-opensmtpd-table-configurations value) + (delete-duplicates + (let loop ([list (flatten + (cond ((opensmtpd-table-configuration? value) + value) + ((record? value) + (let* ([ (record-type-descriptor value)] + [list-of-record-fieldnames (record-type-fields )]) + (map (lambda (fieldname) + (get-opensmtpd-table-configurations ((record-accessor fieldname) value))) + list-of-record-fieldnames))) + ((and (list? value) (not (null? list))) + (map (lambda (element-in-list) + (if (record? element-in-list) + (get-opensmtpd-table-configurations element-in-list) + #f)) + value))))]) + (if (null? list) + '() + (if (opensmtpd-table-configuration? (car list)) + (cons (car list) (loop (cdr list))) + (loop (cdr list))))))) + +(define (get-opensmtpd-table-configurations value) + (let loop ([list (flatten ;; turn (list '(1) '(2 '(3))) -> '(1 2 3) + (cond ((opensmtpd-table-configuration? value) + value) + ((record? value) + (let* ([ (record-type-descriptor value)] + [list-of-record-fieldnames (record-type-fields )]) + (map (lambda (fieldname) + (get-opensmtpd-table-configurations ((record-accessor fieldname) value))) + list-of-record-fieldnames))) + ((and (list? value) (not (null? list))) + (map (lambda (element-in-list) + (if (record? element-in-list) + (get-opensmtpd-table-configurations element-in-list) + #f)) + value))))]) + (delete-duplicates (partition opensmtpd-table-configuration? list)))) + +#+END_SRC +**** TODO using map, apply, and fold is certainly awesome, but is it less efficient? + +For example, list-of-type? using a named let is pretty efficient. It loops +through the list once. + +#+BEGIN_SRC scheme +(define (list-of-type? list proc?) + (if (and (list? list) + (not (null? list))) + (let loop ([list list]) + (if (null? list) + #t + (if (proc? (car list)) + (loop (cdr list)) + #f))) + #f)) +#+END_SRC + +BUT when I using map on this, it is slightly less efficient. It has to apply a +simple procedure to each element in the list. Then it has to return the list of +booleans. Then it has to build the primitive eval list, then it has to eval it. +#+BEGIN_SRC scheme +(define (list-of-type? list proc?) + (if (and (list? list) + (not (null? list))) + (primitive-eval (cons 'and + (map (lambda (var) + (if (proc? var) + #t + #f)) + list))) + #f)) +#+END_SRC + +*** PROJ improve [0/2] +**** TODO it would be nice if ~~ supported aliasing tables, as described in man 5 table + +#+BEGIN_EXAMPLE + Aliasing tables + Aliasing tables are mappings that associate a recipient to one or many destinations. They can be + used in two contexts: primary domain aliases and virtual domain mapping. + + action name method alias
+ action name method virtual
+ + In a primary domain context, the key is the user part of the recipient address, whilst the value + is one or many recipients as described in aliases(5): + + user1 otheruser + user2 otheruser1,otheruser2 + user3 otheruser@example.com + + In a virtual domain context, the key is either a user part, a full email address or a catch all, + following selection rules described in smtpd.conf(5), and the value is one or many recipients as + described in aliases(5): + + user1 otheruser + user2@example.org otheruser1,otheruser2 + @example.org otheruser@example.com + @ catchall@example.com + +#+END_EXAMPLE + +Currently opensmtpd-table-configuration, does not support mapping a user to 5 email addresses. +For example, if user 'dave' can email as 'postmaster@gnu.org', and +'other@gnu.org', and 5 other email addresses... does not +support this kind of mapping. To support it, I may be able to just embed a +table in smtpd.conf, or I may need to create an /etc/aliases table as man 5 +aliases describes. +**** TODO make an with file-db #t, auto convert the table into a berkley database via makemap + +See man 5 table +and man smtpd.conf +*** TODO writing out the pkis when there are no pkis gives the string "\n"...it might be better to give "" instead + +=(opensmtpd-configuration-fieldname->string example-opensmtpd-with-0-pkis opensmtpd-configuration-pkis opensmtpd-pki-configuration->string)= +*** PROJ Can I make some improvements to my/sanitize procedure? [0/5] +**** how does my/sanitize procedure work? + +~(my/sanitize var "record-name" "fieldname" '(string? boolean? number?)')~ + +It is essentially asking? are you any of the following: string?, boolean?, +number? If not, then error out with a helpful error message. + +**** How does my hard-coded sanitized procedure work? eg [[file:opensmtpd-records.scm::(filters opensmtpd-listen-on-configuration-filters][opensmtpd-listen-on-configuration-filters]] + +This hard coded sanitize is a little different than the my/sanitize procedure. I +designed the thunk ~my/sanitize~, such that each thunk (string?, false?, +boolean?) has a corresponding entry in the procedure [[file:opensmtpd-records.scm::define (list-of-procedures->string + procedures][~(list-of-procedures->string procedures)~]]. + +However, it would be nice to have the sanitize invocation in +opensmtpd-listen-on-configuration-filters use a my/sanitize invocation like so. + +#+BEGIN_SRC scheme +(my/sanitize var "opensmtpd-listen-on-configuration" "filters" + (list false? + '(list-has-duplicates-or-non-filters + "is a list in which each unique element is of type \n" + "or .") + '(some-filters-in-list-need-message? + " fieldname: 'decision' options " + "\"disconnect\" and \"reject\" require fieldname 'message'\n" + "to have a string.\n") + '(some-filters-in-list-need-value? + " fieldname: 'decision' option " + "\"rewrite\" requires fieldname 'value'\n" + "to have a string.\n"))) +#+END_SRC + +**** PROJ better error messages for my/sanitize calls that use a lambda instead of a defined function [0/4] + +THIS IS HARD TO DO... NOT DOING IT! I just chose to use a hard-coded error +message baked into the lambda. I tried making my/sanitize better...but I could +not get it to work. The hard-coded method just works: + +#+BEGIN_SRC scheme +(phase-name opensmtpd-filter-phase-configuration-phase-name ;; string + (default #f) + (sanitize (lambda (var) + (if (and (string? var) + (or (string=? "connect" var) + (string=? "helo" var) + (string=? "mail-from" var) + (string=? "rcpt-to" var) + (string=? "data" var) + (string=? "commit" var))) + var + (begin + (display (string-append " fieldname: 'phase-name' is of type " + "string. The string can be either 'connect'," + " 'helo', 'mail-from', 'rcpt-to', 'data', or 'commit.'\n ")) + (throw 'bad! var)))))) +#+END_SRC + +Why? ~~ fieldnames only accept certain strings. I want to +sanitize each fieldname to make sure that it's strings is one of those strings. +How would I do this? + +For example, ~~ fieldname 'decision' uses +a lambda to sanitize itself. This will result in an error message that is +descriptive enough to solve the problem. If I decide to do this, then I probably +should create a non-exported record. + +#+BEGIN_SRC scheme +(opensmtpd-filter-phase-configuration (name "cat") (phase-name "connect") (options "fcrdns") (decision "bypasse")) +#+END_SRC + +#+RESULTS: +: (opensmtpd-filter-phase-configuration (name "cat") (phase-name "connect") (options "fcrdns") (decision "bypasse")) +: fieldname 'bypasse' is of type. +: #:972:0 (var)> +: ice-9/boot-9.scm:1685:16: In procedure raise-exception: +: Throw to key `bad!' with args `(#:972:0 (var)>)'. + +A solution may be to modify the my/sanitize procedure to accept something like + +#+BEGIN_SRC scheme +(my/sanitize var "" "'fieldname'" (list ((lambda (var) ...) . "list of unique numbers or strings"))) +#+END_SRC + +I have some example code [[file:opensmtpd-records.scm::;; TODO add in some code that accepts a (cons . cell) that of (proc . "error message") here.][here.]] It probably won't work, but it is a rough sketch +of what could work. +#+BEGIN_SRC scheme +[(eq? (cons? (car procedures))) +(cdr (car procedures))] +#+END_SRC + +Entering a new prompt. Type `,bt' for a backtrace or `,q' to continue. + +***** TODO Now make all the other sanitize sections that use a lambda use this new functionality: + +eg: + +#+BEGIN_SRC scheme +(family opensmtpd-listen-on-configuration-family + (default #f) + (sanitize (lambda (var) + (cond + [(eq? #f var) ;; var == #f + var] + [(and (string? var) + (or (string=? "inet4" var) + (string=? "inet6" var))) + var] + [else + (begin + (display " fieldname 'family' must be string \"inet4\" or \"inet6\".\n") + (throw 'bad! var))])))) +#+END_SRC + +***** TODO perhaps I can create an unexported record [0/2] + +fieldnames: 'procedure', 'error message'. +***** TODO Perhaps I could try to make my/sanitize work more like the opensmtpd-listen-on-configuration-filters does. + +It looks like tiny errors first. And shows you those relevent errors. When you +fix those tiny errors it starts looking for harder errors. + +This is nice because when you get something wrong in the config, you get the +specific error message. + +The way my/sanitize currently works, if you get one thing wrong, then you get 4 +reasons for what you might have done wrong. +***** TODO the has lost of hard coded error checking. +It would be nice to hook this up to my/sanitize. Along with other bits of the code. +**** TODO rework my/sanitize to be like opensmtpd-listen-on-configuration-filters + +opensmtpd-listen-on-configuration-filters works like so: + +Is the variable (not (false? var)) + +Is the variable (need-some-messages?) + +Does the variable (need-some-value) ? + +else var. + +For example, this +#+BEGIN_SRC scheme +(define-record-type* + opensmtpd-table-configuration make-opensmtpd-table-configuration + opensmtpd-table-configuration? + this-record + (name opensmtpd-table-configuration-name ;; string + (default #f) + (sanitize (lambda (var) + (my/sanitize var "opensmtpd-table-configuration" "name" (list string?))))) + (file-db opensmtpd-table-configuration-file-db + (default #f) + (sanitize (lambda (var) + (my/sanitize var "opensmtpd-table-configuration" "file-db" + (list boolean?))))) +#+END_SRC + +would become: +#+BEGIN_SRC scheme +(define-record-type* + opensmtpd-table-configuration make-opensmtpd-table-configuration + opensmtpd-table-configuration? + this-record + (name opensmtpd-table-configuration-name ;; string + (default #f) + (sanitize (lambda (var) + (my/sanitize var "opensmtpd-table-configuration" "name" (list not-string?))))) + (file-db opensmtpd-table-configuration-file-db + (default #f) + (sanitize (lambda (var) + (my/sanitize var "opensmtpd-table-configuration" "file-db" + (list not-boolean?))))) +#+END_SRC + +This +#+BEGIN_SRC scheme +(secure-connection opensmtpd-listen-on-configuration-secure-connection + (default #f) + (sanitize (lambda (var) + (cond [(boolean? var) + var] + [(and (string? var) + (string-in-list? var + (list "smtps" "tls" + "tls-require" + "tls-require-verify"))) + var] + [else + (begin + (display (string-append " fieldname 'secure-connection' can be " + "one of the following strings: \n'smtps', 'tls', 'tls-require', " + "or 'tls-require-verify'.\n")) + (throw 'bad! var))])))) +#+END_SRC + +would become (finish this thought exercise.) +#+BEGIN_SRC scheme +(secure-connection opensmtpd-listen-on-configuration-secure-connection + (default #f) + (sanitize (lambda (var) + (list not-boolean + (sanitize-proc-configuration + string-in-list? + (list "smtps" "tls" + "tls-require" + "tls-require-verify")) + [else + (begin + (display (string-append " fieldname 'secure-connection' can be " + "one of the following strings: \n'smtps', 'tls', 'tls-require', " + "or 'tls-require-verify'.\n")) + (throw 'bad! var))])))) +#+END_SRC + +**** PROJ can we merge my/sanitize syntax into (guix records)? + +#+BEGIN_SRC scheme +(define-record-type* + opensmtpd-option-configuration make-opensmtpd-option-configuration + opensmtpd-option-configuration? + (documentation (list " uses to\n" + "tweak various options.")) + (sanitize (sanitize-configuration ; this sanitizes the whole record. + (list (lambda (value) + ... + )))) + (option opensmtpd-option-configuration-option + (default #f) + (sanitize (sanitize-configuration + (list string?)))) + (not opensmtpd-option-configuration-not + (default #f) + (sanitize (sanitize-configuration + (list boolean?)) )) + (regex opensmtpd-option-configuration-regex + (default #f) + (sanitize (sanitize-configuration '(boolean?)))) + (value opensmtpd-option-configuration-value + (default #f) + (sanitize (sanitize-configuration + ;; note that it is smart enough to realize that opensmtpd-table-configuration? is a record, + ;; so the error message it returns is something like " fieldname is of + ;; type ." + (list false? string? opensmtpd-table-configuration?))))) +#+END_SRC +**** TODO maybe sanitize the ~~ fieldname 'matches' better + +#+BEGIN_SRC scheme +(opensmtpd-configuration (matches (list + (opensmtpd-match-configuration + (action + (opensmtpd-action-relay-configuration + (name "relay")))) + 345 + (opensmtpd-match-configuration + (for (opensmtpd-option-configuration + (option "for local"))) + (action + (opensmtpd-action-relay-configuration + (name "relay"))))))) +#+END_SRC + +#+BEGIN_EXAMPLE + fieldname: 'matches' is of type a list of unique records. + +ice-9/boot-9.scm:1685:16: In procedure raise-exception: +Throw to key `bad!' with args `((#< action: #< name: "relay" backup: #f backup-mx: #f helo: #f domain: #f host: #f pki: #f srs: #f tls: #f protocols: #f ciphers: #f auth: #f mail-from: #f src: #f> for: #f from: #f auth: #f helo: #f mail-from: #f rcpt-to: #f tag: #f tls: #f> 345 #< action: #< name: "relay" backup: #f backup-mx: #f helo: #f domain: #f host: #f pki: #f srs: #f tls: #f protocols: #f ciphers: #f auth: #f mail-from: #f src: #f> for: #< option: "for local" not: #f regex: #f value: #f> from: #f auth: #f helo: #f mail-from: #f rcpt-to: #f tag: #f tls: #f>))'. + +Entering a new prompt. Type `,bt' for a backtrace or `,q' to continue. +scheme@(opensmtpd-records) [24]> ,bt +In current input: + 2358:0 2 (_) +In /home/joshua/prog/gnu/guix/guix-config/linode-guix-system-configuration/opensmtpd-records.scm: + 278:8 1 (my/sanitize (#< action: #< name: "relay" backup: #f backup-mx: #f helo: #f domain…> …) …) +In ice-9/boot-9.scm: + 1685:16 0 (raise-exception _ #:continuable? _) +#+END_EXAMPLE + +It is not obvious from the error message what is wrong. The error message +should say + +#+BEGIN_SRC org +~= fieldname 'matches' is a list of unique =~. One of the items in the list +is '345'. +Throw to key `bad! with args 345 +#+END_SRC + +Alternatively, we could define a guix specific record printer to make it easier +to see the problem. + +#+BEGIN_SRC scheme + fieldname: 'matches' is of type a list of unique records. + +ice-9/boot-9.scm:1685:16: In procedure raise-exception: +Throw to key `bad!' with args: +`(list + (opensmtpd-match-configuration + (action + (opensmtpd-action-relay-configuration + (name "relay")))) + 345 + (opensmtpd-match-configuration + (for (opensmtpd-option-configuration + (option "for local"))) + (action + (opensmtpd-action-relay-configuration + (name "relay")))))'. + +Entering a new prompt. Type `,bt' for a backtrace or `,q' to continue. +scheme@(opensmtpd-records) [24]> ,bt +In current input: + 2358:0 2 (_) +In /home/joshua/prog/gnu/guix/guix-config/linode-guix-system-configuration/opensmtpd-records.scm: + 278:8 1 (my/sanitize (#< action: #< name: "relay" backup: #f backup-mx: #f helo: #f domain…> …) …) +In ice-9/boot-9.scm: + 1685:16 0 (raise-exception _ #:continuable? _) + +#+END_SRC +**** TODO some of the error messages say "bad var #f". This is not very helpful. + +Where it is useful I should do a ~(throw 'bad! record)~ instead of +~(throw `bad! #f)~ +*** PROJ add support for ~= fieldname 'senders': syntax "senders =~ [masquerade]" [0/4] +:LOGBOOK: +- State "TODO" from [2021-11-02 Tue 04:08] +:END: +**** TODO add a record type + +fieldnames: 'table (accepts '), and 'masquerade' (accepts boolean). +**** TODO change the sanitize portion of the fieldname 'senders' in the + +The below code does work in a REPL. +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +(opensmtpd-listen-on-configuration + (auth + (opensmtpd-table-configuration + (name "My-table") + (data '(("joshua" . "$some$Long$EncrytpedPassword")))))) +#+END_SRC + +#+RESULTS: + +AND the below code will correctly result in an error! +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +(opensmtpd-listen-on-configuration + (auth + (opensmtpd-table-configuration + (name "My-table") + (data '("joshua" "$some$Long$EncrytpedPassword"))))) +#+END_SRC + +#+RESULTS: +: ~= fieldname: 'auth' is of type boolean, or an =~ record whose fieldname 'values' are an assoc-list +: (eg: (opensmtpd-table-configuration (name "table") (data '("joshua" . "$encrypted$password")))). + +**** TODO change relevant portions in opensmtpd-listen-on-configuration->string +This bit of code works in the repl too! +#+BEGIN_SRC scheme :dir ~/prog/gnu/guix-config/linode-guix-system-configuration/ +(add-to-load-path (dirname (current-filename))) +(use-modules (opensmtpd-records)) + +((@@ (opensmtpd-records) opensmtpd-listen-on-configuration->string) + (opensmtpd-listen-on-configuration + (auth + (opensmtpd-table-configuration + (name "credentials") + (data '(("joshua" . "$someLongEncrytpedPassword"))))))) +#+END_SRC + +**** TODO support the masquerade option + +Right now, senders just accepts an , but I am not allowing the +user to turn on or off the masquerade option. +*** TODO write out of examples of ~~ records that will fail to start or do not make sense. +Provide appropriate error messages. + +There is a trend of guix services that "work" but are not dummy proof. For +example, the XMPP service wants a cert in the format of "~ record and just by looking at the +record tell, if you have done something silly that will make the service refuse +to start or behave in a weird way, and provide you appropriate error messages so +you don't have to go syntax hunting. + +Examples: + +- could be a filter that is defined but never used, which won't +be possible once [[id:89603b3f-7580-4531-8aee-2c115c97adfe][remove opensmtpd-configuration-filters]] is done. + +- (listen-on (interface "doesNotExist")) + +- (smtp-configuration (smtp-max-message-size "10G")) Are you sure you + want emails that large? + +- (pki (domain "name") (key "notAKeyfile.txt") (cert + "notACertFile.txt") + +- (ca (file "NotACaFile.txt")) + +- (opensmtpd-filter-phase-configuration (name "filter") (phase "helo") (decision "bypass")) + There is no fieldname =options= here. This has to be sanitized by + ~~'s fieldname 'filters'. + + +#+BEGIN_SRC scheme +(opensmtpd-listen-on-configuration + (filters + (list + (opensmtpd-filter-phase-configuration + (name "noFRDNS") + (phase "commit") + (options (list (opensmtpd-option-configuration + (option "fcrdns") + ))) + (decision "junk"))))) +#+END_SRC +* Some notes on working on the service workflows and such + +disabling centaur-tabs-mode seems to help. and NOT working in the console helps too. + +I think that having the geiser repl running via + +M-x geiser +M-x geiser-load-file RET opensmtpd-records.scm +,m (opensmtpd-records) + +May be causing Emacs to move slowly after a while. + + +It may be better to instead do: + +cd prog/gnu/guix-config/linode-system-configuration; +guile -L . --listen=9999 + +And then in Emacs (as described here: https://www.nongnu.org/geiser/geiser_3.html) +connect to the external repl via +M-x geiser-connect -- 2.36.1