Stages of WWW development compared to Emacs Lisp development

[Jean Louis <bugs@gnu.support>]

As in the context of templating engines, WWW development is very
similar to Emacs Lisp development or any programming language.

One has to try it out, there are maybe some trials and errors, there
is debugging involved, preview or testing, editing, preview or
testing, until it becomes what author wants it to be.

One bad example that I can think of in using `read-from-string' and
`eval' in templates is when strings from visitor's input would be
processed with it.

For example, if visitor would be asked for email, and that email
address is then, instead of obtained from environment like POST,
processed by `read-from-string' -- at this moment I would not know why
it should be so, unless website wants to evaluate some programming
code. There are today so many websites evaluating programming code in
browser. Who knows how secure they are...

By principle, I do not keep the database online, at least privacy is
ensured. Attacker on a website could eventually send spam or use some
band width, but nothing more than that.

The comparison of web templating engines shows that many of them are
using "functions":
https://en.wikipedia.org/wiki/Comparison_of_web_template_engines

I see that Perl's Template Toolkit probably uses `eval':
https://github.com/abw/Template2/search?q=eval&type=

In this list for Python templating engines, two of them are mentioned
using `eval' I rather think there are more than two:
https://wiki.python.org/moin/Templating

Or this Tiny Template, that says not to use `eval' but it is
restricted in operation:
https://github.com/yuanchuan/tiny-template

And my intention is not to be restricted in operation.

I was doing for years preparation of variables, etc. so that I can
pass it to the templating engine like CL-EMB, but I think that was
more because of Common Lisp nature of packages, I would need to call
functions with their prefixes. Global variables were anyway present,
and I could anyway evaluate any Common Lisp code in CL-EMB.

From CL-EMB:

- `<% ... %>` is a scriptlet tag, and wraps Common Lisp code.
- `<%= ... %>` is an expression tag. Its content gets evaluated and fed as a
            parameter to `(FORMAT T "~A" ...)`.
- `<%# ... #%>` is a comment. Everything within will be removed/ignored.  Can't
            be nested!

I have just minimized it to the system that can use various
delimiters, including just parenthesis (would the remaining text not
use parenthesis with spaces):

( anything ) would be evaluated while (anything) would not. And I can
skip the CL-EMB's tag <%= something %> into ( something ) or ⟦ something ⟧
and I skip using complex tags like <% @var VARIABLE %> as ⟦ VARIABLE ⟧
will work or expand empty.

In CL-EMB there are some template tags, while any Lisp can be embedded anyway:

Currently supported: `@if`, `@else`, `@endif`, `@ifnotempty`, `@unless`, 
`@endunless`, `@var`, `@repeat`, `@endrepeat`, `@loop`, `@endloop`, `@include`,
`@includevar`, `@call`, `@with`, `@endwith`, `@set`, `@genloop`, `@endgenloop`,
`@insert`.

It is more complex templating engine, one can have separate chunks of
embedded templating tags which then work together with the mixed text
inside, great, but also complex. It mixes 2 languages, one is
templating language and one is Lisp. Instead, why not just use Lisp.

Process of preparing pages for evaluation is simple:

- edit page
- preview
- edit and preview until satisfied
- publish

In that sense whatever can go wrong with sex-pression templates or
Scribe style of preparation, it can also go wrong with `eval' based
templating engine.

Muse can embed any kind of Emacs Lisp and anyway does not have good
visualization of it. Org mode embeds any kind of programming.

I am using Org mode and expanding it from outside of Emacs by using
Common Lisp which invokes Emacs each time, and I guess that makes
things slower. If I manage the Website Revision System directly from
Emacs, it will be somewhat faster to expand Org blobs from database.

Following Common Lisp is used to convert stream of Org data to
Markdown markup:

$ cat file.org | pipe-org-to-markdown.lisp



#!/home/data1/protected/bin/lisp
(defparameter memdir "/dev/shm/")
(defparameter tmp-file "rcd-wrs-XXXXXX")
(defparameter tmp-org (format nil "~a~a.org" memdir tmp-file))
(defparameter tmp-md (format nil "~a~a.md" memdir tmp-file))
;;(delete-file tmp-org)
;;(delete-file tmp-md)
(load "/home/data1/protected/Programming/git/RCDBusiness/lib/lisp/streamtools.lisp")
;; (require "syscalls")

(defparameter org-autoloads.el
  (concatenate
   'string
   (namestring (car (directory "/home/data1/protected/.emacs.d/elpa/org-20??????/")))
   "org-autoloads.el"))

(defun main ()
  (let ((input '())
        (output '()))
    (with-open-file (out tmp-org :direction :output :external-format "utf-8" :if-exists :supersede)
      (loop for line = (read-line *standard-input* nil nil)
         while line do
           (write-line line out)))
    (shell (format nil "emacs -Q -l ~a \"~a\" --batch -f org-mode -f org-md-export-to-markdown --kill" org-autoloads.el tmp-org))
    (setf output
            (with-open-file (stream tmp-md :direction :input :external-format "utf-8")
              (loop for line = (read-line stream nil nil)
                 while line collect line)))
    (setf output (format nil "~{~a~^~%~}" output))
    (setf output (slurp-stream-io-command "markdown -F 0x4" output))
    (princ output))
  (exit))

;; ;; (saveinitmem "org2html" :quiet t :init-function 'main :verbose nil :norc t :documentation "Converts Org standard input into markdown" :executable t)

(main)


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/