How to tame compiler?

How to tame compiler?

[Jean Louis]

As I have recently implemented new function that use `eval' to expand
various variables and these variables are not visibly used in the
program, I would like to tame the compiler, as I get these warnings:

In rcd-send-email:
rcd-mailing.el:225:62: Warning: Unused lexical variable `unsubscribe-url'
rcd-mailing.el:231:21: Warning: Unused lexical variable `hello-name'
rcd-mailing.el:234:72: Warning: Unused lexical variable `unsubscribe-text'
rcd-mailing.el:252:11: Warning: Unused lexical variable `body'

What is happening here is that those variables are used but inside of
`eval' form which is expanded dynamically when program runs. Compiler
cannot see that.

Is there a way to avoid these warnings?




Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: How to tame compiler?

[Stefan Monnier]

> Is there a way to avoid these warnings?

Yes: don't abuse `eval` ;-)


        Stefan

Re: How to tame compiler?

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2021-04-22 17:47]:
> > Is there a way to avoid these warnings?
> 
> Yes: don't abuse `eval` ;-)

I know that legendary saying.

In practice there was never any problem. Variables related to page,
like title, description, some images, footer, header, some SQL,
imaginary real time reports, those things.

Of course I could expose this way whole list of passwords on WWW, but
it is never going to happen as it is one person who controls the
system.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: How to tame compiler?

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2021-04-22 17:47]:
> > Is there a way to avoid these warnings?
> 
> Yes: don't abuse `eval` ;-)

I found that this works:

(ignore "Avoid compiler warnings" to-name hello-name body unsubscribe-text unsubscribe-url)

Now I don't get compiler warnings for those variables.

But is it the right way?



-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: How to tame compiler?

[Jorge P. de Morais Neto]

Hi all!

Em [2021-04-22 qui 10:46:59-0400], Stefan Monnier escreveu:

>> Is there a way to avoid these warnings?
>
> Yes: don't abuse `eval` ;-)

Jean Louis, could you provide a little more detail on what are you using
~eval~ for?  Some tasks accomplished by eval can be done more safely by
other means.  For example, if you just want symbol indirection, you can
use ~symbol-value~ (there is also ~symbol-function~).  If you want to
apply a function object to a sequence of arguments, you can use ~apply~
or ~funcall~.

I know little about Elisp; more experienced hackers may know about other
mechanisms that avoid the need for ~eval~.

Regards

-- 
- <https://stallmansupport.org> "In Support of Richard Stallman"
- I am Brazilian.  I hope my English is correct and I welcome feedback.
- Free Software Supporter: <https://www.fsf.org/free-software-supporter>
- If an email of mine arrives at your spam box, please notify me.

rcd-template-eval - was Re: How to tame compiler?

[Jean Louis]

* Jorge P. de Morais Neto <jorge+list@disroot.org> [2021-04-30 17:29]:
> Hi all!
> 
> Em [2021-04-22 qui 10:46:59-0400], Stefan Monnier escreveu:
> 
> >> Is there a way to avoid these warnings?
> >
> > Yes: don't abuse `eval` ;-)
> 
> Jean Louis, could you provide a little more detail on what are you using
> ~eval~ for?  Some tasks accomplished by eval can be done more safely by
> other means.  For example, if you just want symbol indirection, you can
> use ~symbol-value~ (there is also ~symbol-function~).  If you want to
> apply a function object to a sequence of arguments, you can use ~apply~
> or ~funcall~.
> 
> I know little about Elisp; more experienced hackers may know about other
> mechanisms that avoid the need for ~eval~.

Gladly. I use it to expand some values in templates. That is very
handy and I use it already for many years. Normally it is for HTML and
WWW publishing. But I may use it for local reports on the fly. It is
very common beyond Emacs. It is similar to the nature of M4
pre-processing, which I have used back in past to create bunch of WWW
pages. It is similar to various server side templates with embedded
Perl or other programming languages, where there is more text and less
programming code.

(defvar rcd-template-delimiter-open "⟦")
(defvar rcd-template-delimiter-close "⟧")

(defun insert-rcd-lisp-brackets ()
  (interactive)
  (let ((open "⟦")
	(close "⟧"))
    (insert (format "%s () %s" open close))
    (backward-char 3)))

;; I have it in my key settings, when I press `C-c r i` I get this:
⟦ () ⟧ and then I can immediately write some expression like:
⟦ (+ 2 2) ⟧

if you now run the below function rcd-template-buffer-eval it will
show you 4 on that place. 

;; (define-key rcd-map "i" #'insert-rcd-lisp-brackets)

(defun rcd-template-eval (string &optional delimiters)
  "Evaluates Emacs Lisp enclosed by `rcd-template-delimiter-open' and `rcd-template-delimiter-close'.

Optional DELIMITERS list may be provided to change default
delimiters, first list member has to be the opening delimiter and
second the closing delimiter.

Space or new line has to follow `rcd-template-delimiter-open' and
precede `rcd-template-delimiter-close' for evaluation to get
invoked."
  (let* ((delimiters (or delimiters (list rcd-template-delimiter-open rcd-template-delimiter-close)))
	 (open (car delimiters))
	 (close (cadr delimiters)))
    (with-temp-buffer
      (insert string)
      (goto-char 0)
      (while (re-search-forward (rx (literal open)
				    (one-or-more (or blank "\n"))
				    (group (minimal-match (one-or-more anything)))
				    (one-or-more (or blank "\n"))
				    (literal close))
				nil t)
	;;(message "Found match")
	(let* ((lisp (car (read-from-string
			   (buffer-substring-no-properties
			    (1+ (match-beginning 0)) (1- (match-end 0))))))
	       (value (condition-case nil
			  (eval lisp)
			(error "")))
	       (value (string-or-empty-string value)))
	  ;; (message "HELLO: %s" (eval (format "%s" hello-name)))
	  (delete-region (match-beginning 0) (match-end 0))
	  (insert (format "%s" value))))
      (buffer-string))))

(defun rcd-template-buffer-eval ()
  (interactive)
  (let* ((buffer (buffer-string))
	 (mode major-mode)
	 (point (point)))
    (pop-to-buffer-same-window "Preview")
    (insert (rcd-template-eval buffer))
    (goto-char point)
    (funcall mode)
    ;;(espeak "Preview")
    ))

I have been using various templating systems before, and Perl was very
fast, then I switched to Common Lisp and used CL-EMB, and tried to
emulate it in Emacs Lisp, but syntax was kind of dirty like:

<% (something here) %>

or

<% @var some-variable %>

which I find easier like this:

⟦ some-variable ⟧

One can use any delimiters, I have chosen those ⟦ () ⟧
to input: type "C-x 8 RET 27e7" or "C-x 8 RET MATHEMATICAL RIGHT WHITE SQUARE BRACKET"

In general, this function will be silent with error "" and thus in
case that variable `some-variable' does not exist it will be empty.

Practical use cases:

Mass mailing lists are sent, and I wish to include recipient's
names in the template: ⟦ to-name ⟧ would expand into their full
name.

⟦ (gold-price-kg 1) ⟧ would expand to current gold price, there is
no need for me to change the body text of the email. Prices are
updated automatically, same for EUR/USD/GBP and other currencies.

⟦ (usd (gold-price-kg)) ⟧ would expand into US $57191

⟦ to-email ⟧ would expand into their email address

⟦ unsubscribe-url ⟧ would expand into the per recipient customized
URL to unsubscribe

Time estimates can be caluclated and sent by email automatically
that use current dates expanded in a template, not static dates.

Similar for HTML and WWW publishing, static pages may be generated on
the fly, expanded into Lisp values and transferred to web servers.

Normally in Common Lisp, and CL-EMB templating package I would need to
submit something like variables or list of variables, list alists to
get variables expanded. But I prefer that it works with any variables.

Instead of:
(rcd-template-eval "Template ⟦ variable ⟧" '(variable "VALUE")) 

right now I prefer it this way:

(let* ((variable "VALUE")
       (template "Template ⟦ variable ⟧")
       (expanded (rcd-template-eval template)))
  expanded) ⇒ "Template VALUE"

However, there is problem as my programs are with lexical-binding
⇒ t, so then I have to watch to do following:

(let* ((variable "VALUE")
       (template "Template ⟦ variable ⟧")
       (lexical-binding nil)
       (expanded (rcd-template-eval template)))
  expanded) ⇒ "Template VALUE"

as otherwise it would not necessarily expand, depending where the
function is defined.

In the end this approach allows me to write plain text, Markdown,
Org mode, just anything, and get the Emacs Lisp programs expanded
before as pre-processing before the Markdown, Org mode or
Asciidoc, Restructured Text or other type of processing. It of
course works automatic.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

rcd-template-eval, much is in Org mode

[Jean Louis]

* Jorge P. de Morais Neto <jorge+list@disroot.org> [2021-04-30 17:29]:
> Hi all!
> 
> Em [2021-04-22 qui 10:46:59-0400], Stefan Monnier escreveu:
> 
> >> Is there a way to avoid these warnings?
> >
> > Yes: don't abuse `eval` ;-)
> 
> Jean Louis, could you provide a little more detail on what are you using
> ~eval~ for?  Some tasks accomplished by eval can be done more safely by
> other means.  For example, if you just want symbol indirection, you can
> use ~symbol-value~ (there is also ~symbol-function~).  If you want to
> apply a function object to a sequence of arguments, you can use ~apply~
> or ~funcall~.
> 
> I know little about Elisp; more experienced hackers may know about other
> mechanisms that avoid the need for ~eval~.

Org mode is one of popular modes, and it has 52 matches for "(eval "
on my computer. See below.

For almost 2 decades I use that for WWW publishing. Imagine:

<html>
  <body>
    <title>⟦ (xml-escape "My new <title>") ⟧</title>
  </body>
</html>

it would expand into:

<html>
  <body>
    <title>My new &lt;title&gt;</title>
  </body>
</html>

and I like it that way more than CL-EMB, where I had to do something like:

<html>
  <body>
    <title><% @var title -escape-xml %></title>
  </body>
</html>

We are eval-ing in Emacs more than we think when we use Org mode.

-*- mode: grep; default-directory: "~/.emacs.d/elpa/org-20201216/" -*-
Grep started at Fri Apr 30 22:43:55

grep --color=auto -nH --null -e "(eval " *.el
ob-core.el\0231:	 (eval (or (cdr  (assq :eval headers))
ob-core.el\0617:		      (and (boundp lang-headers) (eval lang-headers t))
ob-core.el\0870:		   (when (boundp lang-headers) (eval lang-headers t))))
ob-core.el\0908:         (lang-headers (when (boundp lang-headers-var) (eval lang-headers-var t)))
ob-core.el\01617:					  (eval raw-result t)))
ob-core.el\02710:				   (if (stringp value) value (eval value t))))))
ob-core.el\02990:	 (eval (read cell) t))
ob-emacs-lisp.el\070:	 (result (eval (read (if (or (member "code" result-params)
ob-shell.el\051:    (eval `(defun ,(intern (concat "org-babel-execute:" name))
ob-shell.el\056:    (eval `(defalias ',(intern (concat "org-babel-variable-assignments:" name))
ob-shell.el\061:    (eval `(defvar ,(intern (concat "org-babel-default-header-args:" name)) '()))))
ob-table.el\0127:                        (eval `(org-babel-parse-header-arguments
ol-bbdb.el\0412:                       ((listp form) (eval form))
ol.el\01324:		   (eval (read path))
org-agenda.el\02910:	      (setq type (nth 2 entry) org-match (eval (nth 3 entry))
org-agenda.el\03241:      (setq match (eval (nth 1 org-cmd)))
org-agenda.el\03400:  (eval (list 'org-batch-store-agenda-views)))
org-agenda.el\03710:    (eval form)))
org-agenda.el\04128:	       (eval form)))))))
org-agenda.el\06728:	(setq rtn (concat (eval formatter) txt))
org-agenda.el\07257:	  (eval (cons 'or org-agenda-sorting-strategy-selected))
org-agenda.el\07511:	(eval series-redo-cmd)
org-agenda.el\08114:	  (unless (eval org-agenda-filter-form)
org-agenda.el\08228:	  (eval org-agenda-jump-prefer-future))
org.el\08421:    (eval `(let ,binds
org.el\09329:			       (or (eval checklist)
org.el\012473:	      (setq scope (eval scope)))
org.el\014231:    (eval form)
org.el\014645:	 (result (if calendar-debug-sexp (eval sexp)
org.el\014647:		       (eval sexp)
org.el\016525:      (eval org-speed-command))
org.el\018337:			(not (equal (symbol-value v) (eval (car (get v 'standard-value)))))))
org-macro.el\0152:	  '("n" . "(eval (org-macro--counter-increment $1 $2))")
org-macro.el\0153:	  '("keyword" . "(eval (org-macro--find-keyword-value $1))")
org-macro.el\0154:	  '("time" . "(eval (format-time-string $1))")
org-macro.el\0155:	  '("property" . "(eval (org-macro--get-property $1 $2))")))))
org-macro.el\0180:          (setq value (eval (condition-case nil (read value)
org-macro.el\0334:	(format "(eval (if (org-string-nw-p $1) %s %S))"
org-macs.el\0176:  `(eval (list 'let ,environment ',form)))
org-macs.el\0626:  (eval (cons 'let (cons list body))))
org-macs.el\0630:  (eval (cons 'let (cons list1 (list (cons 'let (cons list2 body)))))))
org-macs.el\0636:      (eval form)
org-mouse.el\0522:	    (eval `(org-agenda nil (string-to-char ,key))))
org-pcomplete.el\0429:		   (and (boundp lang-headers) (eval lang-headers t)))))
org-table.el\02600:			 (eval (eval (read form)))
ox.el\01572:		 (let ((value (eval (nth 3 cell))))
ox.el\03107:                               (mapconcat #'identity (eval (nth 3 entry)) " ")
ox.el\03108:                             (eval (nth 3 entry)))))
ox.el\03111:                   (push (cons option (eval (nth 3 entry))) options))))))
ox-odt.el\0538:			  (eval (car (widget-get w :args)))))
ox-odt.el\0627:			  (eval (car (widget-get w :args)))))



-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: rcd-template-eval, much is in Org mode

[Tassilo Horn]

Jean Louis <bugs@gnu.support> writes:

> For almost 2 decades I use that for WWW publishing. Imagine:
>
> <html>
>   <body>
>     <title>⟦ (xml-escape "My new <title>") ⟧</title>
>   </body>
> </html>

If I would want to do something like that, I'd just write a lispy HTML
DSL so that I could write

(foo/html
 (foo/body
  (foo/title (xml-escape "My new <title>"))))

which is shorter, easier to type, and doesn't need `eval'.

Bye,
Tassilo

Re: rcd-template-eval, much is in Org mode

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

> Org mode is one of popular modes

Sure but not everyone likes it.

> it has 52 matches for "(eval " on my computer

But that's not the reason, is it? :O

-- 
underground experts united
https://dataswamp.org/~incal

Org mode rant

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-05-01 01:11]:
> Jean Louis wrote:
> 
> > Org mode is one of popular modes
> 
> Sure but not everyone likes it.

For notes I have been using database for long time, then in 2016, I
found about Org mode, and it is quite helpful to quickly without
direct LaTeX usage generate good quality PDF files, so I have been
using it for projects mostly. As I deal with hundreds of people, soon
there were too many Org files for each person tasks and
transactions. That becomes mess, as I cannot query through it, text
files are separate from each other. Sending or sharing tasks is pain,
and I had to extend it myself, collaboration is non-existent. Then
before few months I have decided to switch all Org files that have
some tasks, notes, transactions into the database, and I will not go
back. Now if I wish, I can generate and export to Org file, but not
vice versa. This way I am editing things on meta-level. And I do like
that. There are still 2500 Org files on my computer. But I have
figured out how to move it to database.

I like having the opportunity to reference even a single item in a
list, which is not possible with Org. I like to reference any
paragraph I wish or join and order Org chunks in the database as I
wish, including mix Org chunks or database blobs with other types like
images, videos, PDFs, Markdown, etc.

Org is so much better for writing or document preparation than for
management of procrastination, I mean tasks. One nice thing is that
TODO is highlighted, but that is quite easy to achieve in Emacs
anyway.

There is too much hype about it. Blatant and sensational promotion. Oh
my heaven. The hype draws many people to it, and so many get stuck
into bad design. Text is text, but it wants to be structured text, and
it wants to be database, but it cannot. And programmers try their
best, but will never be a database. And because there is no connection
to any kind of a database, it becomes very tiresome to code even
simple things like tagging with some dates and stamps. Database has
that already built-in and to keep trace of date/time stamps becomes
trivial. Properties in Org file are right there in front of the eyes,
confusing everybody who is confronted with it. My properties are in
the database entry, I can see them on a click, but don't need to.

TASKS - they are too often related to people, we have to do something
for others, others have to do something for us, or for others, people
do tasks, tasks are assigned to people, tasks are shared to people and
groups of people. There are no such concepts in Org, it is left to
user to find for himself, and there is no universaly helpful method.

Org is pleasure, it is interesting, it will help those who otherwise
could not help themselves and Org remains to be extremely useful.

For me it is not. I like to create a task, like today, we receive
Western Union in a group, I duplicate old task, change some
information, press a key and task is sent by email to the assigned
person, SMS notification follows up or even a call. While same can be
done in Org file, due to it not really being structured with separate
data from each other, it is much error prone. I found it surprising
that a TASK cannot be default be assigned to a person and that there
is no built-in function to send it by e-mail but I found my way with
Org.

> > it has 52 matches for "(eval " on my computer
> 
> But that's not the reason, is it? :O

Who knows. Org Babel has to evaluate scripts and insert into
buffer. That is some type of evaluation I am doing for WWW
publishing. I could as well evaluate templates with any other language
specified.

⟦ (perl 2+2) ⟧ would work equally well as anything from "(perl" to ")"
could be evaluated by perl, not Emacs Lisp.

So much hype about Org and literate programming that people don't even
find true beauties of programs such as:

Cherrytree - hierarchical note taking application with rich text and syntax highlighting
https://www.giuspen.com/cherrytree/

Leo editor vs Org-mode
https://leoeditor.com/emacs.html

Leo programmable editor
http://leoeditor.com/



-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: Org mode rant

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>> Sure but not everyone likes it.
>
> For notes I have been using database for long time, then in
> 2016, I found about Org mode, and it is quite helpful to
> quickly without direct LaTeX usage generate good quality PDF
> files

Sorry, you can't do that. Org mode is a way of organizing
data, it is not a generator of documents. Is the answer I got
anyway when I asked how to make the result look civilized. Ha.

re: LaTeX I was once told "you are obsessed with LaTeX"
(Swedish "du är besatt av LaTeX", it literally means that).
I answered "Yeah? What about you?" ("Okej? Du då?") and he
just said "No" ("Nej") and left.

In denial...

> Markdown

Markdown a lot of FOSS people have been in touch with since
the READMEs with all the square brackets you get when you get
FOSS software, that is Markdown.

Unbelievable BTW, I was just using it!

It can look like this to get a 2nd degree header, a quote and
a table with centered columns and a hyperlink.

I guess I like it fine - so far.

## version history

_Try, try again!_
NGE

 version |   start    | teknologi |
:-------:|:----------:|:---------:| -----------------------------------------------------------------------
    1    | 2020-09-13 |    org    |
    2    | 2020-11-01 | HTML5/CSS | *[https://dataswamp.org/~incal/blog](https://dataswamp.org/~incal/blog)
    3    | 2021-03-14 |    md     | (denna)

> There is too much hype about [Org]

One over-hyped piece of software: Org

-- 
underground experts united
https://dataswamp.org/~incal

Re: Org mode rant

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-05-01 03:47]:
> Jean Louis wrote:
> 
> >> Sure but not everyone likes it.
> >
> > For notes I have been using database for long time, then in
> > 2016, I found about Org mode, and it is quite helpful to
> > quickly without direct LaTeX usage generate good quality PDF
> > files
> 
> Sorry, you can't do that. Org mode is a way of organizing
> data, it is not a generator of documents. Is the answer I got
> anyway when I asked how to make the result look civilized. Ha.

Maybe it is not, but I use Org export functions to get nice PDF
documents that people practically use in the field.

> re: LaTeX I was once told "you are obsessed with LaTeX"
> (Swedish "du är besatt av LaTeX", it literally means that).
> I answered "Yeah? What about you?" ("Okej? Du då?") and he
> just said "No" ("Nej") and left.

Who knows which latex you meant... This one is rather spelled as
LaTeH the other one is used for balloons and population minimization.

> > Markdown
> 
> Markdown a lot of FOSS people have been in touch with since
> the READMEs with all the square brackets you get when you get
> FOSS software, that is Markdown.
> 
> Unbelievable BTW, I was just using it!

I have no idea why would README be in Markdown, but never mind. Unless
one wants to expand it into HTML, readme looks better without
Markdown.

First began using it back in the same year when it was published,
today almost everybody uses it. I had static website generation before
any free software came up with it. Today there are many. At that time,
before maybe 16 years, would I mention static website generation I
would almost get stoned.

Markdown and few other markups for HTML have one huge flaw that nobody
talks about, it misses the size for pictures which in the end turns
HTML invalid. That is why I never use any other Markdown but Discount
markdown for images. It is anyway fastest.

Pandoc's markdown I may use sometimes to make nicer syntax
highlighting for code, where there are no images involved.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: Org mode rant

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

> Maybe it is not, but I use Org export functions to get nice
> PDF documents that people practically use in the field.

Are you even allowed to do that?

>> re: LaTeX I was once told "you are obsessed with LaTeX"
>> (Swedish "du är besatt av LaTeX", it literally means that).
>> I answered "Yeah? What about you?" ("Okej? Du då?") and he
>> just said "No" ("Nej") and left.
>
> Who knows which latex you meant... This one is rather
> spelled as LaTeH the other one is used for balloons and
> population minimization.

I don't remember what I used back then but now, if you by
"which latex" mean what compiler and version, the answer is

  $ xelatex --version
  XeTeX 3.14159265-2.6-0.999992 (TeX Live 2020/Debian)

By LaTeX I mean "[the] language [that] is described in the book
LaTeX - A Document Preparation System." (latex(1))

>>> Markdown
>> 
>> Markdown a lot of FOSS people have been in touch with since
>> the READMEs with all the square brackets you get when you
>> get FOSS software, that is Markdown.
>> 
>> Unbelievable BTW, I was just using it!
>
> I have no idea why would README be in Markdown, but never
> mind. Unless one wants to expand it into HTML, readme looks
> better without Markdown.

There are several reasons to do that but yes one is the
possibility to produce the material in various other formats.

I think Markdown looks better than plain text in markdown-mode
(unbelievably not in vanilla Emacs, but I've already written
about that so no more about that), I think the syntax isn't
really disturbing anyone, it is also clear; apart from that
font lock makes it colorful as well.

> First began using it back in the same year when it was
> published, today almost everybody uses it. I had static
> website generation before any free software came up with it.
> Today there are many. At that time, before maybe 16 years,
> would I mention static website generation I would almost
> get stoned.

What is that, it generates static webpages? From some other
data format, when that data is complete? IMO HTML is so
simple one can just as well do it manually so don't have to
bother with anything moving or still.

> Markdown and few other markups for HTML have one huge flaw
> that nobody talks about, it misses the size for pictures
> which in the end turns HTML invalid. That is why I never use
> any other Markdown but Discount markdown for images. It is
> anyway fastest.

What do you mean "misses the size for pictures"?

You are right people don't seem to talk about this problem.

-- 
underground experts united
https://dataswamp.org/~incal

On markdown images

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-05-01 09:35]> > Maybe it is not, but I use Org export functions to get nice
> > PDF documents that people practically use in the field.
> 
> Are you even allowed to do that?

Haha. 

>   $ xelatex --version
>   XeTeX 3.14159265-2.6-0.999992 (TeX Live 2020/Debian)
> 
> By LaTeX I mean "[the] language [that] is described in the book
> LaTeX - A Document Preparation System." (latex(1))

I know that book. 

> > I have no idea why would README be in Markdown, but never
> > mind. Unless one wants to expand it into HTML, readme looks
> > better without Markdown.
> 
> There are several reasons to do that but yes one is the
> possibility to produce the material in various other formats.

The original reason is none other but that Github wanted to display
README files in better way but plain text.

README was always traditionally plain text. Now people think it should
be markdown. It is kind of deceptive. But purpose is to serve Github
vendor.

Like you say, it is readable, but it is also confusing to people who
do now know the markup.

> I think Markdown looks better than plain text in markdown-mode
> (unbelievably not in vanilla Emacs, but I've already written
> about that so no more about that), I think the syntax isn't
> really disturbing anyone, it is also clear; apart from that
> font lock makes it colorful as well.

It is slow, markdown-mode is slow, I don't like it, but I have to deal
with it. For simple documents is fine, for anything more complex it
becomes slow.

> > First began using it back in the same year when it was
> > published, today almost everybody uses it. I had static
> > website generation before any free software came up with it.
> > Today there are many. At that time, before maybe 16 years,
> > would I mention static website generation I would almost
> > get stoned.
> 
> What is that, it generates static webpages? From some other
> data format, when that data is complete? IMO HTML is so
> simple one can just as well do it manually so don't have to
> bother with anything moving or still.

1. There is number of websites; for me or for people who I serve;
   maybe 64 or less, but that is what database says;

2. Each website may have different URLs, different server, different
   username/password, method of copy, different company name,
   maintainer and most importantly different HTML template;

3. Writing HTML by hand is taking more time then writin Markdown
   markup. Or Asciidoc, or Org, restructured text or similar. It saves
   time.

4. HTML templates have its menu, right? It has its heading, title,
   description, some footer stuff, those things are tiresome to be
   written by hand. They are best expanded from the known variables
   which I keep in the database.

Website can have 1000+ pages, imagine writing each footer by hand,
each menu by hand, each header and title by hand. Tiresome.

> > Markdown and few other markups for HTML have one huge flaw
> > that nobody talks about, it misses the size for pictures
> > which in the end turns HTML invalid. That is why I never use
> > any other Markdown but Discount markdown for images. It is
> > anyway fastest.
> 
> What do you mean "misses the size for pictures"?
> 
> You are right people don't seem to talk about this problem.

I think this is minimal valid SRC IMG tag in HTML:

<src imag="file.jpg" height="120" width="100" alt="My file"/>

So far none markdown that I know but Discount markdown does not
provide height, width attributes. This makes HTML invalid and in
general can disturb the HTML display and accessibility. Such pages
will not pass the test here: https://validator.w3.org/

See: http://www.pell.portland.or.us/~orc/Code/discount/

Specifying image sizes
    An image size is defined by adding an additional =widthxheight field to the image tag:

         ![dust mite](http://dust.mite =150x150)

        produces dust mite

But the syntax: ![dust mite](http://dust.mite =150x150) is only common
to this version of Markdown. Anyway, this is fastest Markdown and self
contained, unlike others which are slow or huge in size like pandoc.

Normal markdown expression would be:

![dust mite](http://dust.mite)

But that produced invalid HTML:

(rcd-markdown "![dust mite](http://dust.mite)") ⇒ "<p><img src=\"http://dust.mite\" alt=\"dust mite\" /></p>
"

While this produces valid HTML:

(rcd-markdown "![dust mite](http://dust.mite =150x150)") ⇒ "<p><img src=\"http://dust.mite\" height=\"150\" width=\"150\" alt=\"dust mite\" /></p>
"

As a side note, I have a workflow when I wish to include many images
at once, I can just mark them in Dired, and invoke a function that
constructs the markup, insert into buffer and finished. 


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: On markdown images

[Yuri Khan]

On Sat, 1 May 2021 at 16:43, Jean Louis <bugs@gnu.support> wrote:

> I think this is minimal valid SRC IMG tag in HTML:
>
> <src imag="file.jpg" height="120" width="100" alt="My file"/>
>
> So far none markdown that I know but Discount markdown does not
> provide height, width attributes. This makes HTML invalid and in
> general can disturb the HTML display and accessibility. Such pages
> will not pass the test here: https://validator.w3.org/

You have been misinformed on that point. The width and height
attributes are, and always have been, optional on an img element. See
specs:

HTML 5 — https://www.w3.org/html/wg/spec/the-map-element.html#attr-dim-width
HTML 4.01 — https://www.w3.org/TR/html4/struct/objects.html#adef-width-IMG
HTML 3.2 — https://www.w3.org/TR/2018/SPSD-html32-20180315/#img
HTML 2.0 did not have width and height attributes on img elements.

Re: On markdown images

[Jean Louis]

* Yuri Khan <yuri.v.khan@gmail.com> [2021-05-01 13:00]:
> On Sat, 1 May 2021 at 16:43, Jean Louis <bugs@gnu.support> wrote:
> 
> > I think this is minimal valid SRC IMG tag in HTML:
> >
> > <src imag="file.jpg" height="120" width="100" alt="My file"/>
> >
> > So far none markdown that I know but Discount markdown does not
> > provide height, width attributes. This makes HTML invalid and in
> > general can disturb the HTML display and accessibility. Such pages
> > will not pass the test here: https://validator.w3.org/
> 
> You have been misinformed on that point. The width and height
> attributes are, and always have been, optional on an img element. See
> specs:
> 
> HTML 5 — https://www.w3.org/html/wg/spec/the-map-element.html#attr-dim-width
> HTML 4.01 — https://www.w3.org/TR/html4/struct/objects.html#adef-width-IMG
> HTML 3.2 — https://www.w3.org/TR/2018/SPSD-html32-20180315/#img
> HTML 2.0 did not have width and height attributes on img elements.

You see, after so many years, I get this smacking slap and come back
to reality.

Who knows what is the real reason why I started using it. I think back
in time, before maybe 10 years there were different directions in
creating HTML documents.

Probably I was using page load speed or other type of validator and
have mistaken it for HTML validator, thanks for correction.

https://web.dev/optimize-cls/?utm_source=lighthouse&utm_medium=unknown#images-without-dimensions

For example Google Page speed says:

Image elements do not have explicit width and height
Set an explicit width and height o


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: On markdown images

[Yuri Khan]

On Sat, 1 May 2021 at 17:23, Jean Louis <bugs@gnu.support> wrote:

> Probably I was using page load speed or other type of validator and
> have mistaken it for HTML validator, thanks for correction.

Width and height attributes, when their values are correct, do help
the browser lay out the page. It can reserve a known space before the
image gets downloaded and go on processing the rest of the page.

When width and height are missing, the browser does not know how much
space to reserve, so, when it finally receives the image and finds out
its dimensions, it has to go back and re-layout everything from the
point of the image.

Explicit width and height attributes are not the only way to mitigate
this. For example, one could specify image dimensions in CSS. Or just
redefine some images to be block elements and fit 100% of their
container width.

Re: On markdown images

[Jean Louis]

* Yuri Khan <yuri.v.khan@gmail.com> [2021-05-01 14:10]:
> Width and height attributes, when their values are correct, do help
> the browser lay out the page. It can reserve a known space before the
> image gets downloaded and go on processing the rest of the page.
> 
> When width and height are missing, the browser does not know how much
> space to reserve, so, when it finally receives the image and finds out
> its dimensions, it has to go back and re-layout everything from the
> point of the image.
> 
> Explicit width and height attributes are not the only way to mitigate
> this. For example, one could specify image dimensions in CSS. Or just
> redefine some images to be block elements and fit 100% of their
> container width.

That was it.

Yet to specify image dimensions in CSS within Markdown context becomes
difficult. If it is separate CSS, it would be tiresome, if it would be
inline CSS, alright somehow, but none Markdown support that so
far. One can always write HTML in Markdown documents, of course, but
that does not help.

Some pictures have different proportions and they may come up
dynamically, I would not like displaying such with incorrect
proportions.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: On markdown images

[Emanuel Berg via Users list for the GNU Emacs text editor]

Yuri Khan wrote:

> Width and height attributes, when their values are correct,
> do help the browser lay out the page. It can reserve a known
> space before the image gets downloaded and go on processing
> the rest of the page.
>
> When width and height are missing, the browser does not know
> how much space to reserve, so, when it finally receives the
> image and finds out its dimensions, it has to go back and
> re-layout everything from the point of the image.
>
> Explicit width and height attributes are not the only way to
> mitigate this. For example, one could specify image
> dimensions in CSS. Or just redefine some images to be block
> elements and fit 100% of their container width.

That so?

I have the width here:

  .images_right {
      float:            right;
      padding-top:      6px;
      width:            206px;
  } # [1]

But the height vary from image to image, should I put it
"explicitly" as attributes in the HTML [2] then?

That doesn't feel right because isn't the point of HTML/CSS to
have all that in CSS?

OTOH to have one CSS class for every image feels ... lame.
And a lot of typing.

[1] https://dataswamp.org/~incal/index.css
[2] https://dataswamp.org/~incal/index.html

-- 
underground experts united
https://dataswamp.org/~incal

Re: On markdown images

[Yuri Khan]

On Mon, 3 May 2021 at 02:31, Emanuel Berg via Users list for the GNU
Emacs text editor <help-gnu-emacs@gnu.org> wrote:

> I have the width here:
>
>   .images_right {
>       float:            right;
>       padding-top:      6px;
>       width:            206px;
>   } # [1]
>
> But the height vary from image to image, should I put it
> "explicitly" as attributes in the HTML [2] then?

I wouldn’t. Having width in CSS and height in HTML feels wrong.

> OTOH to have one CSS class for every image feels ... lame.
> And a lot of typing.

True.

I feel there are multiple distinct use cases for images, and they
warrant different approaches:

* Icons. You typically know the size beforehand and use the same size
for all icons. So .icon { width: 24px; height: 24px; }

* Large photos. You prepare them in as high resolution as possible and
let the browser scale them down to fit the window: .photo { display:
block; max-width: 100%; } You let the browser calculate the height on
its own, accepting the layout delay. Fortunately, the visual effect of
re-layout is not as annoying for block images as it would be for
inline images.

* Screenshots, pixel art, and line art. In contrast with photos, you
typically do not want to allow arbitrary scaling for these. So you
either specify the exact dimensions in HTML to help the browser lay
them out, or in CSS (but this leads to one class for every image), or
you let the browser sort it out on its own. (NB: The latter is bad for
HiDPI screenshots, as they will be displayed at twice their natural
size. Also, if displaying normal DPI pixel images at their natural
size, instruct browsers on HiDPI displays to use crisp image scaling
algorithm: .pixelart { image-rendering: -moz-crisp-edges;
image-rendering: -webkit-optimize-contrast; } otherwise it will come
out all blurry.)

Re: On markdown images

[Emanuel Berg via Users list for the GNU Emacs text editor]

Yuri Khan wrote:

>> But the height vary from image to image, should I put it
>> "explicitly" as attributes in the HTML [2] then?
>
> I wouldn’t. Having width in CSS and height in HTML
> feels wrong.

But to have the same value (the width) appear over and over in
HTML isn't good either and there is already a CSS class, and
removing width from it won't remove the class since there are
other settings there as well...

Individual CSS classes for height only option to my mind,
I think the height is the same for a bunch of pics tho, so
that one can put in the base class along with the shared
settings, then the imgs that have different height they get
their own class in addition to the base class which will
override it with respect to height only...

-- 
underground experts united
https://dataswamp.org/~incal

Re: On markdown images

[Emanuel Berg via Users list for the GNU Emacs text editor]

>>> But the height vary from image to image, should I put it
>>> "explicitly" as attributes in the HTML [2] then?
>>
>> I wouldn’t. Having width in CSS and height in HTML
>> feels wrong.
>
> But to have the same value (the width) appear over and over
> in HTML isn't good either and there is already a CSS class,
> and removing width from it won't remove the class since
> there are other settings there as well...
>
> Individual CSS classes for height only option to my mind,
> I think the height is the same for a bunch of pics tho, so
> that one can put in the base class along with the shared
> settings, then the imgs that have different height they get
> their own class in addition to the base class which will
> override it with respect to height only...

Got it to work with CSS only, lines 36-49:

  https://dataswamp.org/~incal/index.css

No reflow! :)

-- 
underground experts united
https://dataswamp.org/~incal

Re: On markdown images

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>> By LaTeX I mean "[the] language [that] is described in the
>> book LaTeX - A Document Preparation System." (latex(1))
>
> I know that book. 

I know right? Everyone should have a buddy like me to teach
them new stuff...

> README was always traditionally plain text. Now people think
> it should be markdown. It is kind of deceptive. But purpose
> is to serve Github vendor.

Files in UPPERCASE are text files and the upcase is a way of
telling this file is intended to be read by a human, not
processed by the computer.

> I think this is minimal valid SRC IMG tag in HTML:
>
> <src imag="file.jpg" height="120" width="100" alt="My file"/>

I think not :)

-- 
underground experts united
https://dataswamp.org/~incal

Re: Org mode rant

[Bastien]

Jean Louis <bugs@gnu.support> writes:

> Org is so much better for writing or document preparation than for
> management of procrastination, I mean tasks.

I like the expression "management of procrastination".

I'm using Org quite a lot and I also think it does not fix discipline,
which is what is ultimately required for managing your tasks.

I write a blog entry about to-do lists that you might find useful, as
it is not directly connected to Org:

https://bzg.fr/en/on-using-to-do-lists-efficiently/

-- 
 Bastien

Re: Org mode rant

[Emanuel Berg via Users list for the GNU Emacs text editor]

Bastien wrote:

> I write a blog entry about to-do lists that you might find
> useful, as it is not directly connected to Org:
>
> https://bzg.fr/en/on-using-to-do-lists-efficiently/

You can include mine if you'd like to, it isn't connected to
Org directly or indirectly - I yank it last.

There are some things that can be improved but they were
actually never put on the TODO list so it is not like they are
tasks that I never did or anything...

;;; -*- lexical-binding: t -*-
;;;
;;; this file:
;;;   http://user.it.uu.se/~embe8573/emacs-init/todo-did.el
;;;   https://dataswamp.org/~incal/emacs-init/todo-did.el

(defun todo-get-buffer ()
  (let*((todo-env-var "TODO_FILE")
        (todo-file     (getenv todo-env-var))
        (todo-buffer   (if todo-file (find-file-noselect todo-file t)
                         (error "Set the env var %s first" todo-env-var) )))
  todo-buffer) )

(defun todo-sort-and-save ()
  (sort-lines nil (point-min) (point-max)) ; not REVERSE
  (whitespace-cleanup)
  (save-buffer) )

(defun todo-insert-entry (what buffer)
  (with-current-buffer buffer
    (goto-char (point-max))
    (insert "\n" what)
    (todo-sort-and-save) ))

(defun todo (what)
  (interactive "sdo what: ")
  (let ((todo-buffer (todo-get-buffer)))
    (todo-insert-entry what todo-buffer) ))

(defun todo-show-file ()
  (interactive)
  (let ((todo-buffer (todo-get-buffer)))
    (with-current-buffer todo-buffer
      (when (buffer-modified-p todo-buffer)
        (todo-sort-and-save) )
      (switch-to-buffer todo-buffer) )))

-- 
underground experts united
https://dataswamp.org/~incal

Re: Org mode rant

[Jean Louis]

* Bastien <bzg@gnu.org> [2021-05-01 08:00]:
> Jean Louis <bugs@gnu.support> writes:
> 
> > Org is so much better for writing or document preparation than for
> > management of procrastination, I mean tasks.
> 
> I like the expression "management of procrastination".
> 
> I'm using Org quite a lot and I also think it does not fix discipline,
> which is what is ultimately required for managing your tasks.
> 
> I write a blog entry about to-do lists that you might find useful, as
> it is not directly connected to Org:
> 
> https://bzg.fr/en/on-using-to-do-lists-efficiently/

I have great affinity to this type of thinking and analyzing. Nice
reference to "freedom of concern"...

It is interesting that there are people who never make any notes,
tasks, whatever written, and they are successful in life and
accomplish so many things. This should say that our success does not
necessarily depend of writing things down.

Do you mean to-do lists become your own enemy when there is much of
them or when they become not easily manageable, bringing person into
confusion?

My personal Org mode use is mostly focused on repeatable
projects. Project has its goals, purposes, operational targets and
production targets and similar, they are printed and assigned to
people who do those projects. This month maybe 50 people attended the
some of such projects and about 5-10 of them are soon to finish it.

The to-do list is thus replicated and given to people in form of a
whole assignment. In this area we have all clarity we need and want.

> Because it gives you a false sense of control, it slowly erodes your
> goals' clarity, which weakens your motivation. You end up being
> controlled by a list of things to do instead of focusing on doing
> them. You hope that your tool will help you achieve self-discipline
> instead of disciplining yourself to use it the right way.

Maybe if you tell your practical use case where you got stuck, then it
would become clearer for me as reader. I am so much interested.

There are those to-do tasks that are immediately actionable, doable,
and those that are not doable. There are those with priority and those
with less priority.

Projects and planning never eroded my goals' clarity as I write goal
in the first place as authoritative to the subordinate assignments
(tasks or to-do). That is maybe method not so much used by Org mode
users, it is like from top to bottom, first defined most important
fundamentals, then define subordinate actions to be done.

If somebody starts putting everything together, like personal tasks,
business tasks, lovers' tasks and notes in once, without having clear
separation, well that may erode totally the goals and purposes.

I wish I could understand it better how it happened to you.

> Write less to-do items and more notes. Notes and "remember" lists are
> always good. But do not blur the line between what you want to
> remember and what you need to do3. Your to-do list should be a list of
> things to do, not to remember. To-do lists should describe what you
> must do, not what you want to do, which belongs on the "remember" list
> until you really must do it.

I wish I could understand it.

If there is action item to do, it is to do, writing notes about it
will never get it done. I cannot write less actions to do then I have
it. Maybe some people never understood the distinction and they write
TODO even where there is nothing to be done.

I am sure you have got a good advise, I just miss to see it. Maybe
there need to be practical example to understand it better.

Let us say I have person to receive Western Union for me, this is my
real world and current task to-do, I click a key, assign the action to
the person and provide instructions, click another key and person is
informed by email, SMS, I could invoke a call from computer as well.

Any time later I can see which are pending tasks for which people, I
can review it. Maybe I should have visual reminders, it will come.

In that particular example I do not see how I can write less to-do
actionable items, and I cannot see how would it be useful to write
notes on the side not related to actionable item. I wish I could
understand.

There are some unfinished notes that lead to action, they have to be
finished, so they have label ACTION (same as TODO), but are not of
highest priority. They may be pending for quite some time, they need
thinkering, focus.

However, all tasks are sorted into some sets or parent nodes. Those
parent nodes can have its own priorities.

For example if I am in working time period, I will look into tasks
related to that working time period.

If I am in leisure time, I could look into tasks which do not relate
to business.

If I focus on relatives, family or friends, let us say I focus on
particular person, I will list tasks for that person as now came the
time to handle things for that particular person.

I wish to understand what you mean, but have hard time. Practical
example is welcome.

Editors are really great and fundamental tool for planning. IMHO even
simple text file, sections of text and search functions just good
enough. I study this subject last months.

In my database I keep ACTION on different types, for example type
could be WWW hyperlink. That requires me to open it up, review the
page, and maybe draw some conclusions or do something with the
information. That may not be highest priority.

There is type "Task" which has definitely higher priority than WWW
type.

There is type "Set" which can have ACTION assigned, that means all
items in the set are actionable, but grouped. If it is some list of
videos for learning on distance, then the play list could be
invoked. But overall Set can be skipped if it is not time for
learning.

I have "Org" type which is whole Org file in the database with ACTION
assigned, for example design and planning of the database system, or
equipment list for particular business site, or various notes that
need yet sorting.

There is FOLLOW-UP type which means I have to follow specific people,
this may be done regularly at specific time intervals.

As TODO items are separate by its types, by its sets or groups, by its
priorities it becomes easy to make intersections.

Intersections help in clarity.

> Take extra care of your agenda. Agendas are necessary, scheduled
> tasks and deadlines are useful. But do not schedule a task unless it
> really needs to be done on a specific date, refrain from using
> deadlines if nothing requires them. Over-scheduling is another way
> of mixing what you want to do and what you need to do. If your
> agenda is growing with many tasks, that's an sign that you should
> rethink if they really need to be scheduled or to have a deadline.

Good advise, understandable, clear. Thank you.

Most of tasks I have I don't schedule, because they are so imminent
that I have to do it right away.

Assigned tasks are almost never scheduled, I let people do it at their
own pace and availability and it works very well apparently. But if
specific task has to be done on Monday, it would tell in the
description. Using scheduled attributes is definitely very useful.

Sometimes I had about 180 appointments in year, appointment could be
scheduled for months in advance, that requires tracking and using
calendar.

Like you said, it is good to minimize confusion.

> Don't overload tasks with details, don't overload your agenda with
> tasks and don't mix notes and tasks—that's basically it.

Yes, sure, but this probably applies to Org mode or text based
stuff. Not to really well structured objects. The database is mixing
everything, but its presentation, queries and views give the precise
clarity.

I can for example say to computer, give me list of people in groups
created after 2021-05-01, who already started on the project I have
assigned them, and provided me their email address, while all others
have only phone number.

As with SQL databases that type of query becomes possible. With text
only, that becomes tiresome and requires programmers to code and code
and code.

That is why I cannot recommend Org mode for some more professional
use, unspoken for collaborative use.

Unless Org mode itself is stored in the database. As that solves those
problems.

I really hope that my breaking down of your article will not bother
you, myself I want to learn more about principles of note taking, it
is my special subject of study now.

> Writing self-contained tasks reinforces your sense of clarity4,
> keeping a minimal agenda lets you in control, and richer notes will
> nurture your motivation, while a minimal list of tasks will help
> boosting your sense of achievement.

Self-contained tasks are understandable, that is how I write
tasks. All instructions are inside. Task is a note too, but "Task"
represents rather type of a note.

Agenda should have those necessary tasks or whatever appointments, and
goal should be to attend those scheduled objects and do them, not to
minimize them. I can only vague idea on what is meant with minimize. I
would not like minimizing my agenda. Maybe you assume that too many
people will decide on their agenda in too broad manner, so to say to
write scheduled items even when they are not necessary. When I have to
meet somebody on Sunday, I have to, I cannot minimize it. Or if I have
to visit some office at Monday, I have to, I wish I could minimize it,
but life obligations are quite strict.

> Now I suspect we are all very bad at using to-do lists. Why?

Oops. Do you really think so?

I have met some elder women who gave so much good value to other
people and society and they never missed an appointment or similar. I
know one close to me, she never put anything on paper, but never
forgets what is to be done, a virtue that was so much common and
present in times before the access to computers and devices.

Then I can remember times when I was using exclusively written lists
for a week, and I have accomplished all of them, or maybe few were
left not done, and were moved to the next week.

I like your article, I wish I could find more of it, some things I did
not understand as for those maybe I do not have or do not find
personal correlation.

Org mode files I have been using mostly to assign tasks related to
people, each person had an Org file. It was quite easy to see who did
what, when, how. And I had money transactions inside. I have not been
writing those Org files, I just made a function, press key, and Org
file is generated with its title, sections, table of transactions,
like template. With number of people growing, that became tiresome,
transactions, instead being on one central place becomes scattered all
around.

That I have moved to database work, invoke function or key, then I say
transaction is from person Happiness to Jean, this much money in this
currency, finished. Instead on focusing on formatting the Org table, I
just enter numbers. If I wish to produce Org table out of the
database, I guess that can be created. But working with structured
data on a meta level seem to give me more organization then working on
non-structured and digitally unrelated data in multiple Org files,
even in one Org file, objects are mostly not digitally related to each
other (they are related in our mind).

Tasks assigned to people, if managed in a central database, avoid the
Org to look for numerous Org files on the system. One can invoke a
key, list people who have pending tasks and list those assignments for
each person separately.


Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: Org mode rant

[Bastien]

Jean Louis <bugs@gnu.support> writes:

>> Because it gives you a false sense of control, it slowly erodes your
>> goals' clarity, which weakens your motivation. You end up being
>> controlled by a list of things to do instead of focusing on doing
>> them. You hope that your tool will help you achieve self-discipline
>> instead of disciplining yourself to use it the right way.
>
> Maybe if you tell your practical use case where you got stuck, then it
> would become clearer for me as reader. I am so much interested.

I cannot go into details about my real use-case here but the gist of
the paragraph you quote is this: once in a while you feel your tool is
not helpful enough for handling your to-do, and then you start trying
to fix your Org configuration, trusting it even more, instead of
sticking to the principles I tried to outline in my post.

-- 
 Bastien

Re: Org mode rant

[Jean Louis]

* Bastien <bzg@gnu.org> [2021-05-01 13:06]:
> Jean Louis <bugs@gnu.support> writes:
> 
> >> Because it gives you a false sense of control, it slowly erodes your
> >> goals' clarity, which weakens your motivation. You end up being
> >> controlled by a list of things to do instead of focusing on doing
> >> them. You hope that your tool will help you achieve self-discipline
> >> instead of disciplining yourself to use it the right way.
> >
> > Maybe if you tell your practical use case where you got stuck, then it
> > would become clearer for me as reader. I am so much interested.
> 
> I cannot go into details about my real use-case here but the gist of
> the paragraph you quote is this: once in a while you feel your tool is
> not helpful enough for handling your to-do, and then you start trying
> to fix your Org configuration, trusting it even more, instead of
> sticking to the principles I tried to outline in my post.

I get it. Yes, that is exactly observation I had too before few
months, it brought me to the condition to escape Org mode and rather
use it more in the sense of markup of the overal instructions.

Maybe the article shall more clearly refer to Org mode, rather than
being general note-taking statement.

Org Hype is fine, I like it.

Not that I find Org mode exponentially useful.

I guess people tend, like me too, to designate or determine one system
"this is it" and build upon it for quite long time, even rest of the
life, even though "this isn't the one".

As a side note, here is the workflow I have for things assigned to
other people or to me:

1. Have Org template ready with instructions and repeatable project in
   the database; all items inside are always "TODO"; there is no file
   on file system;

2. Copy template with single key, assign whole node to single person;
   this is like single TODO tag;

3. Share document by email to that person; share 20 such projects
   assigned to multiple people;

4. Receive reports; update what is "DONE" in their nodes related to
   specific person, in the central file or as central node (Org blob),
   but that does not necessarily complete the overal assignment for
   the person; but it is not necessary to use "DONE/TODO" in such
   project as overall assignment is maybe "Sell service for US
   $100,000" -- maybe single task tells "Prepare list of 2000 people"
   -- but if salesman already got a client for to buy the service, all
   other tasks become redundant, as purpose has been achieved during
   the project execution.

   Some people have serious troubles understanding the purpose and
   single tasks, and will rather go doing single tasks for the sake of
   themselves only while forgetting the purpose. If purpose have been
   achieved, even that single task becomes redundant.

5. When assignment is done, mark with COMPLETE, and write report about
   it;

Instead of pending tasks, I can search for people pending the
assignment.

Agenda becomes list of people and their pending project title.

Archiving becomes redundant too, as the node is anyway archived in the
database and whole instruction marked as COMPLETED, though there are
are maybe redundant tasks inside of node still marked with TODO --
such will never appear in any agenda or list because they are
irrelevant as the overall purpose is COMPLETED.

Person goes out to buy new shoes, and make a list of 10 shops to
visit, but maybe shoe seller on street has already got some nice
shoes, thus all other tasks for the sake of purpose, become redundant.
This is not always so, it depends of task types of course.

Org mode can also be used as presentation of the database backed
information where each heading or section may be edited, as long as
there is right property defined, the information may be updated in the
database.

Then the Org mode, rather Org method of editing could become truly
collaborative, not necessarily in he real time.

A salesman Jimmy in New York, could request information on customer
Joe, information can be displayed as Org mode. Salesman may edit some
personal information of Joe, such is updated to central database in
Boston. Another salesman Mark may be editing a note belonging to
client Joe, and update it, both Jimmy and Mark would get notice of it;
Mark could give assignment to Joe to make a call to customer and
englighten him on new products; Joe's Org file would get a notice and
Org file would get updated; Joe makes a phone call and right notice,
the notes get updated in the database; each of them is collaborating
by using apparently simple Org file while underlying functions connect
each heading to the central database.

People mostly use WWW browsers for that type of Customer Relationship
Management software, yet Emacs can be used with Org mode editing.


Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: Org mode rant

[Bastien]

Jean Louis <bugs@gnu.support> writes:

>> Write less to-do items and more notes. Notes and "remember" lists are
>> always good. But do not blur the line between what you want to
>> remember and what you need to do3. Your to-do list should be a list of
>> things to do, not to remember. To-do lists should describe what you
>> must do, not what you want to do, which belongs on the "remember" list
>> until you really must do it.
>
> I wish I could understand it.

My point here is this one: psychologically, I often find it difficult
to resist the temptation of noting something down *before* I know if
it is something I need to do or just something I need to remember to
check at some point, ending up with my to-do list presenting me with
to many things to do, some of them not really belonging here.

Now I force myself to really ponder: "Do you really need to do it?  
If not, do you want to remember it?" and I do this *before* I write 
or capturing.

-- 
 Bastien

Re: Org mode rant

[Jean Louis]

* Bastien <bzg@gnu.org> [2021-05-01 13:10]:
> Jean Louis <bugs@gnu.support> writes:
> 
> >> Write less to-do items and more notes. Notes and "remember" lists are
> >> always good. But do not blur the line between what you want to
> >> remember and what you need to do3. Your to-do list should be a list of
> >> things to do, not to remember. To-do lists should describe what you
> >> must do, not what you want to do, which belongs on the "remember" list
> >> until you really must do it.
> >
> > I wish I could understand it.
> 
> My point here is this one: psychologically, I often find it difficult
> to resist the temptation of noting something down *before* I know if
> it is something I need to do or just something I need to remember to
> check at some point, ending up with my to-do list presenting me with
> to many things to do, some of them not really belonging here.

When you note something in that example, do you put a tag TODO in Org?

If so, why not just leave it without TODO...

> Now I force myself to really ponder: "Do you really need to do it?  
> If not, do you want to remember it?" and I do this *before* I write 
> or capturing.

It's interesting to exchange.

Constant writing, noting, pondering may definitely lead to moments
like described.

If I may compare to my workflow I have not found myself thinking
beforehand if I need to do something or to remember it (note it), as
that decision has already been made, so I am writing something that
has to be done or I am writing a note.

Like "fetch money from Western Union office" -- it is harder to get
into thinking should I just remember it, or should I do it -- as if I
just remember it, money will not come in my hands.

The distinction on what is ACTION (actionable) and what is NOT ACTION
should be very clear in any note taking system. Refining those objects
further with their types, classes, properties is great feature in any
note taking system.

Since I have switched from Org to database meta level outline where
each node can be something else like Org, enriched text, image, video,
etc. the decision comes first on the type.

Do I want to follow up some person's development, like sales process,
learning process? I press key for that.

Do I want to assign task to somebody? There is special key.

Do I want to create new task for me? There is key for that.

This type of task in Org is just fine to be described as a heading,
especially of there are various tasks related to each other, each
actionable without mixture with notes.

* Pay ticket to town
* Fetch money from Western Union
* Purchase bread, come back home

If there are however notes around, that is where it becomes mixture
that does not give clarity:

* Pay ticket to town
* Prices of tickets to town
* List of Western Union offices
* Bread types I like
* Fetch money from Western Union
* Purchase bread, come back home

That is where nicely marked TODO items make a visual distinction:

* TODO Pay ticket to town
* Prices of tickets to town
* List of Western Union offices
* Bread types I like
* TODO Fetch money from Western Union
* TODO Purchase bread, come back home

Then we get more confusion with priorities as the list could also look like this:

* Prices of tickets to town
* TODO [#B] Fetch money from Western Union
* TODO [#C] Purchase bread, come back home
* List of Western Union offices
* Bread types I like
* TODO [#A] Pay ticket to town

Priorities in Org mode are attributes that may be used in queries, but
I guess there is no sorting or ordering function yet.

If those nodes are in the database, single key is moving one task up
or down in the priorities order. Some things must come before other things.

I recommend separating notes and actionable items:

* Notes

** Prices of tickets to town
** List of Western Union offices
** Bread types I like

* Tasks [0/3] [0%]

** TODO [#A] Pay ticket to town
** TODO [#B] Fetch money from Western Union
** TODO [#C] Purchase bread, come back home

where by if tasks are written in the chronological order, priorities
should also become redundant as the order itself designates
priorities:

* Tasks [0/3] [0%]

** TODO Pay ticket to town
** TODO Fetch money from Western Union
** TODO Purchase bread, come back home

Another problem we have are relations, we do want to know where are
Western Union offices, but the note can be far somewhere even in the
other file. That makes things difficult.

Then we have option to tag things that are related:

* Notes

** Prices of tickets to town                                        :transport:
** List of Western Union offices                                    :western:
** Bread types I like                                                 :bread:

* Tasks [0/3] [0%]

** TODO Pay ticket to town                                        :transport:
** TODO Fetch money from Western Union                              :western:
** TODO Purchase bread, come back home                                :bread:

Then one has to use Agenda function to find items by the tag, to find
what is related. And if we have few more tags, it becomes really messy:

* Notes related to Munich                           :notes:munich:germany:eu:

** Prices of tickets to town                        :transport:bus:tram:taxi:
** List of Western Union offices                       :western:money:family:
** Bread types I like                                  :bread:cakes:birthday:

* Tasks [0/3] [0%]                                                   :action:

** TODO Pay ticket to town                                    :transport:bus:
** TODO Fetch money from Western Union                       :western:munich:
** TODO Purchase bread, come back home                         :bread:mother:

And I have not yet mentioned property lists, which may visually
disturb the user and make it all tiresome.

When working with the database, specific items can simply be related
to each other. Press key and see all related items. But nothing need
to visually bother user.

Number of tags can be unlimited, it is just a string separated with
spaces. Tags could be 20 for a single item and as they are in the
database, such tags need not be displayed visually on the side to the
heading.

If person then wants to find all money related items it becomes
trivial, without searching for files, pressing R remove from Agenda,
etc, what else.


Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

RE: [External] : Re: Org mode rant

[Drew Adams]

> I often find it difficult
> to resist the temptation of noting something down *before* I know if
> it is something I need to do or just something I need to remember to
> check at some point, ending up with my to-do list presenting me with
> to many things to do, some of them not really belonging here.
> 
> Now I force myself to really ponder: "Do you really need to do it?
> If not, do you want to remember it?" and I do this *before* I write
> or capturing.

To-do item 42,424,242:

  Start to figure out if I really need to do
  to-do item 31,425,936.

To-do item 42,424,243:

  Start to figure out if I really need to do
  to-do item 42,424,242.

...

Re: [External] : Re: Org mode rant

[Emanuel Berg via Users list for the GNU Emacs text editor]

Drew Adams wrote:

> To-do item 42,424,242:
>
>   Start to figure out if I really need to do
>   to-do item 31,425,936.
>
> To-do item 42,424,243:
>
>   Start to figure out if I really need to do
>   to-do item 42,424,242.

When you notice an imperfection, do M-x todo RET
two-or-three-technobabble-words-to-describe-it RET. When you
view the TODO file some time later, if you still
understand/remember what a technobabble oneliner refers to,
that means do it or keep it (i.e., do nothing for the time
being). Otherwise remove it has it was probably of very small
importance even to you.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Org mode rant

[Bastien]

Jean Louis <bugs@gnu.support> writes:

>> Now I suspect we are all very bad at using to-do lists. Why?
>
> Oops. Do you really think so?

That's probably an exaggeration of mine, indeed.

-- 
 Bastien

eval myths - Re: How to tame compiler?

[Jean Louis]

* Jorge P. de Morais Neto <jorge+list@disroot.org> [2021-04-30 17:29]:
> Hi all!
> 
> Em [2021-04-22 qui 10:46:59-0400], Stefan Monnier escreveu:
> 
> >> Is there a way to avoid these warnings?
> >
> > Yes: don't abuse `eval` ;-)
> 
> Jean Louis, could you provide a little more detail on what are you using
> ~eval~ for?  Some tasks accomplished by eval can be done more safely by
> other means.  For example, if you just want symbol indirection, you can
> use ~symbol-value~ (there is also ~symbol-function~).  If you want to
> apply a function object to a sequence of arguments, you can use ~apply~
> or ~funcall~.
> 
> I know little about Elisp; more experienced hackers may know about other
> mechanisms that avoid the need for ~eval~.

Good read:
https://en.wikipedia.org/wiki/Eval#Security_risks

My eval-ing is equally dangerous as Org Babel evaling. If I write some
destructive commands, well, it will be destructive.

There is no special security risk there, as eval-ing does not take
place online. But I could as well do it online, even then there need
not be any security risk as I can tell which data goes into eval and
decide not to include dynamic data, but currently I do not see any
need for that.

I could write text with eval-ing markup and have it interpolated by
using Emacs on the fly, it would be pretty fast WWW publishing. Let us
say I wish to say when is document last modified, I could just
include:

Last modified: ⟦ (last-modified) ⟧ and it would interpolate into time
stamp of the file in question.

Eval I use also in the nodes in the PostgreSQL database. I am thinking
also to have various database types define for themselves their
presentation or export functions. Right now I have Emacs hashes in the
database. It is similar to properties in Org mode, though I have
properties also as PostgreSQL array type. Some document entry could be
saved in the hash, it expands PostgreSQL data type into something that
Emacs understands, such hash has to be evaled.

Let us say, I defined "Markdown Flavour X" as type, then the type
could have its functions inside of itself in the database. That would
be evaluated with eval and run. HTML could be generated, or some other
program. Instead of hard coding it statically in a program, a remote
or collaborative program could simply access the database by which
access program would be extended, fetched from remote database and
executed. Of course it gives to the remote resource full control about
which program is there. But then there can be PGP signature for
verification if necessary.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: eval myths - Re: How to tame compiler?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

> Good read:
> https://en.wikipedia.org/wiki/Eval#Security_risks

That's right, there are so many myths with that command.
But what is the truth?

-- 
underground experts united
https://dataswamp.org/~incal

Re: eval myths - Re: How to tame compiler?

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-05-01 01:15]:
> Jean Louis wrote:
> 
> > Good read:
> > https://en.wikipedia.org/wiki/Eval#Security_risks
> 
> That's right, there are so many myths with that command.
> But what is the truth?

It is just one of useful functions.


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: eval myths - Re: How to tame compiler?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>>> Good read:
>>> https://en.wikipedia.org/wiki/Eval#Security_risks
>> 
>> That's right, there are so many myths with that command.
>> But what is the truth?
>
> It is just one of useful functions.

Hahaha, let's put this in the docstring :)

-- 
underground experts united
https://dataswamp.org/~incal

Re: eval myths - Re: How to tame compiler?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 3424 bytes --]

On Fri, Apr 30, 2021 at 11:23:03PM +0300, Jean Louis wrote:
> * Jorge P. de Morais Neto <jorge+list@disroot.org> [2021-04-30 17:29]:
> > Hi all!
> > 
> > Em [2021-04-22 qui 10:46:59-0400], Stefan Monnier escreveu:
> > 
> > >> Is there a way to avoid these warnings?
> > >
> > > Yes: don't abuse `eval` ;-)
> > 
> > Jean Louis, could you provide a little more detail on what are you using
> > ~eval~ for? [...]

> Good read:
> https://en.wikipedia.org/wiki/Eval#Security_risks
> 
> My eval-ing is equally dangerous as Org Babel evaling. If I write some
> destructive commands, well, it will be destructive.

Framing this from the security point of view to then deconstruct
that frame leads up a blind alley. Actually, it's a kind of
strawman [1] :-)

Not that the security aspect isn't important. Actually, it has the
potential to illustrate the underlying problem, but it is /not/
the underlying problem.

What, for me, was elucidating was to try to think about what environment
eval is supposed to "see". Imagine, in a lexical context,

  (let ((x 5))
    (eval '(+ x 2)))

[QUESTION TO THE EXPERTS]
  Why is this code

    (let ((x 5))
      (eval '(+ x 2) t))

  complaining "eval: Symbol’s value as variable is void: x"
  when I `C-x e' it in my scratch buffer?
  Yes, `lexical-binding' is t in the buffer
[END OF QUESTION]

Is eval supposed to "see" x? Now, if you have a compiler which does
constant folding [0] ("ah, x, in this context /can/ only be 5, so we
substitute its value in all expressions...").

Look up the documentation of `eval' in the Emacs Lisp manual for the
path elisp has chosen.

As soon as eval is involved, that is a bad plan. What if deep there
in the eval, there is a (setq x 99)? Which isn't known at compile
time, because... eval?

So whenever there's an eval, the compiler has to drop nearly all its
instruments. And this is bad :-)

This is not really easy to grasp, but you'll always hear people who
have done that tour (Stefan has!) to somewhat discourage the use
of eval. Yes, it's there, is a tool, use it -- but it'll take you
some time and a couple of scars until you fully master it.

Usually, the problem you want to solve isn't nearly general enough
to warrant the full eval.

Put in other words: before you use eval, try to give an account
of the full evaluation environment passed to it: `x' is bound to
5 (see above), `current-buffer' to some funny function, there is
a function `strokes-describe-stroke' (did you know that one? I
didn't)... and so on.

There's that phrase "eval is evil" (I saw it first mentioned by
Christian Queinnec in his book [2a], [2b]). You can use it conveniently
in your search engine to see folks out there arguing the exact
way you do and the answers given (most of the time it's javascript
these days, because it's the language you get paid for currently,
but the underlying problem is similar).

Note that I have very little clue on all those things. Stefan
has. If he says "think before eval", I'd strongly recommend doing
some reading and experimenting. The journey could end up being
panoramic :-)

Cheers

[0] https://en.wikipedia.org/wiki/Constant_folding
[1] The kind I'd call "involuntary strawman". Security often
   has to play involuntary strawman.
[2a] https://christian.queinnec.org/
[2b] https://christian.queinnec.org/WWW/LiSP.html

-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: How to tame compiler?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jorge P. de Morais Neto wrote:

>>> Is there a way to avoid these warnings?
>>
>> Yes: don't abuse `eval` ;-)

Stefan Monnier who wrote that last part, or anyone else for
that matter, what is the rule of thumb when to and when not to
use `eval'?

Maybe something for the docstring?

I have used eval 2 times in all my Elisp. If we apply the rule
of thumb, what will that tell me, is that abuse or not?

For the record I don't get any warnings from either.

(defun wrap-search-again (&optional new-options)
  "Repeat the previous `wrap-search' with NEW-OPTIONS.
NEW-OPTIONS can be supplied with \\[universal-argument].
Otherwise options from the previous search are used."
  (interactive "p")
  (let ((cmd (cl-find-if (lambda (c)
                           (and (eq (car c) #'wrap-search)
                                (not (string= (cl-caddr c) "")) ))
                         command-history) ))
    (if cmd
        (if (= new-options 1)
            (eval cmd)
          (let*((search-str (cl-caddr cmd))
                (final-cmd  (list #'wrap-search new-options search-str)) )
            (setq command-history (cons final-cmd command-history))
            (apply final-cmd) ))
      (message "no previous search") )))

(defun do-repeat-complex-command ()
  (interactive)
  (eval (car command-history) ))

full source:

  https://dataswamp.org/~incal/emacs-init/ide/lisp-incal.el
  https://dataswamp.org/~incal/emacs-init/wrap-search.el

-- 
underground experts united
https://dataswamp.org/~incal

Re: How to tame compiler?

[Stefan Monnier]

Emanuel Berg via Users list for the GNU Emacs text editor [2021-05-01 00:06:30] wrote:
> Jorge P. de Morais Neto wrote:
>>>> Is there a way to avoid these warnings?
>>> Yes: don't abuse `eval` ;-)
> Stefan Monnier who wrote that last part, or anyone else for
> that matter, what is the rule of thumb when to and when not to
> use `eval'?

Use it only when you have to, otherwise don't use it.

>   (let ((cmd (cl-find-if (lambda (c)
>                            (and (eq (car c) #'wrap-search)
>                                 (not (string= (cl-caddr c) "")) ))
>                          command-history) ))
>     (if cmd
>         (if (= new-options 1)
>             (eval cmd)

Here, as in your other example, you have to use `eval`, because
`command-history` was defined to contain a list of ELisp expressions and
those can't be used other than via `eval`.

So I'll grant you a pass for this time.

If you were in a position to control the format of history variable,
then you wouldn't have to use `eval` here, OTOH: you could define that
variable to hold a list of *functions* and you'd just use `funcall`
instead of `eval`.


        Stefan

Re: How to tame compiler?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Stefan Monnier wrote:

> Here, as in your other example, you have to use `eval`,
> because `command-history` was defined to contain a list of
> ELisp expressions and those can't be used other than via
> `eval`.
>
> So I'll grant you a pass for this time.
>
> If you were in a position to control the format of history
> variable, then you wouldn't have to use `eval` here, OTOH:
> you could define that variable to hold a list of *functions*
> and you'd just use `funcall` instead of `eval`.

OK, clear...

-- 
underground experts united
https://dataswamp.org/~incal

Re: How to tame compiler?

[Stefan Monnier]

>> Here, as in your other example, you have to use `eval`,
>> because `command-history` was defined to contain a list of
>> ELisp expressions and those can't be used other than via
>> `eval`.
>>
>> So I'll grant you a pass for this time.
>>
>> If you were in a position to control the format of history
>> variable, then you wouldn't have to use `eval` here, OTOH:
>> you could define that variable to hold a list of *functions*
>> and you'd just use `funcall` instead of `eval`.
>
> OK, clear...

Of course, you can't "search" for functions the way you do with the
`cl-find-if`, and functions can't be printed in a user-friendly way, so
there's a good reason for `command-history` not to contain
just functions.

So if you wanted to replace it with a history that can be used without
`eval`, you'd probably make the new variable contain a list where every
element is of the form (FUNCTION . ARGS), so that you'd replace `eval`
with `apply`, and you'd recommend that the FUNCTION be a symbol rather
than an anonymous function, to be sure that it can be compared and
printed in a useful way.


        Stefan

Re: How to tame compiler?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Stefan Monnier wrote:

>>> Here, as in your other example, you have to use `eval`,
>>> because `command-history` was defined to contain a list of
>>> ELisp expressions and those can't be used other than via
>>> `eval`.
>>>
>>> So I'll grant you a pass for this time.
>>>
>>> If you were in a position to control the format of history
>>> variable, then you wouldn't have to use `eval` here, OTOH:
>>> you could define that variable to hold a list of
>>> *functions* and you'd just use `funcall` instead of
>>> `eval`.
>>
>> OK, clear...
>
> Of course, you can't "search" for functions the way you do
> with the `cl-find-if`, and functions can't be printed in
> a user-friendly way, so there's a good reason for
> `command-history` not to contain just functions.
>
> So if you wanted to replace it with a history that can be
> used without `eval`, you'd probably make the new variable
> contain a list where every element is of the form (FUNCTION
> . ARGS), so that you'd replace `eval` with `apply`, and
> you'd recommend that the FUNCTION be a symbol rather than an
> anonymous function, to be sure that it can be compared and
> printed in a useful way.

Okay, no, we are using `eval' and it works. If someone says we
shouldn't do it we say we do it because of the format of
`command-history'. If they say anything more after that
I don't feel we have to say anything.

-- 
underground experts united
https://dataswamp.org/~incal

Re: How to tame compiler?

[Michael Heerdegen]

Jean Louis <bugs@gnu.support> writes:

> As I have recently implemented new function that use `eval' to expand
> various variables and these variables are not visibly used in the
> program, I would like to tame the compiler, as I get these warnings:
>
> In rcd-send-email:
> rcd-mailing.el:225:62: Warning: Unused lexical variable `unsubscribe-url'
> rcd-mailing.el:231:21: Warning: Unused lexical variable `hello-name'
> rcd-mailing.el:234:72: Warning: Unused lexical variable `unsubscribe-text'
> rcd-mailing.el:252:11: Warning: Unused lexical variable `body'
>
> What is happening here is that those variables are used but inside of
> `eval' form which is expanded dynamically when program runs. Compiler
> cannot see that.
>
> Is there a way to avoid these warnings?

Independent from the question whether your usage of `eval' is good or
valid - there must be some real problem here: if the compiler tells that
the lexical variables are unused, their values will not be available in
you `eval' call - you would have to create dynamical bindings for that.


Regards,

Michael.

Re: How to tame compiler?

[Stefan Monnier]

Michael Heerdegen [2021-05-01 02:44:16] wrote:
> Jean Louis <bugs@gnu.support> writes:
>> As I have recently implemented new function that use `eval' to expand
>> various variables and these variables are not visibly used in the
>> program, I would like to tame the compiler, as I get these warnings:
>>
>> In rcd-send-email:
>> rcd-mailing.el:225:62: Warning: Unused lexical variable `unsubscribe-url'
>> rcd-mailing.el:231:21: Warning: Unused lexical variable `hello-name'
>> rcd-mailing.el:234:72: Warning: Unused lexical variable `unsubscribe-text'
>> rcd-mailing.el:252:11: Warning: Unused lexical variable `body'
>>
>> What is happening here is that those variables are used but inside of
>> `eval' form which is expanded dynamically when program runs. Compiler
>> cannot see that.
>>
>> Is there a way to avoid these warnings?
>
> Independent from the question whether your usage of `eval' is good or
> valid - there must be some real problem here: if the compiler tells that
> the lexical variables are unused, their values will not be available in
> you `eval' call - you would have to create dynamical bindings for that.

I know I sound like a broken clock, but I think a better answer is to
avoid `eval`: instead of taking expressions (that you'd pass to `eval`)
arrange to receive functions (which you'd pass to `funcall` or `apply`).
Then you can pass those functions the data they need (e.g. the value of
things like `unsubscribe-text`).


        Stefan

Re: How to tame compiler?

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I know I sound like a broken clock, but I think a better answer is to
> avoid `eval`: instead of taking expressions (that you'd pass to `eval`)
> arrange to receive functions (which you'd pass to `funcall` or `apply`).
> Then you can pass those functions the data they need (e.g. the value of
> things like `unsubscribe-text`).

How could that be implemented when he wants his templates to accept
arbitrary Lisp code to be evaluated?

Regards,

Michael.

Re: How to tame compiler?

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2021-05-01 06:50]:
> > Independent from the question whether your usage of `eval' is good or
> > valid - there must be some real problem here: if the compiler tells that
> > the lexical variables are unused, their values will not be available in
> > you `eval' call - you would have to create dynamical bindings for that.
> 
> I know I sound like a broken clock, but I think a better answer is to
> avoid `eval`: instead of taking expressions (that you'd pass to `eval`)
> arrange to receive functions (which you'd pass to `funcall` or `apply`).
> Then you can pass those functions the data they need (e.g. the value of
> things like `unsubscribe-text`).

  I have to understand it in this example:
  
  The HTML template is following and it is string:

  <html>
    <head>
      <title>⟦ xml-escape title ⟧</title>
    </head>
    <body>
      <p>Today is one US dollar this many euros: ⟦ usd-eur 1 ⟧</p>
    </body>
  </html>

> I know I sound like a broken clock, but I think a better answer is to
> avoid `eval`: instead of taking expressions (that you'd pass to `eval`)
> arrange to receive functions (which you'd pass to `funcall` or `apply`).
> Then you can pass those functions the data they need (e.g. the value of
> things like `unsubscribe-text`).

  Does that mean I should now extract maybe function `xml-escape' and use apply with rest of arguments?

  Maybe instead of:
  
  <title>⟦ xml-escape title ⟧</title>

  I should make:

  <title>⟦ xml-escape title ⟧</title>

  And then apply xml-escape with "title".

Then I would convert first word into function, that somehow
works, but all the remaining parts of string still need to be
converted. How is program to know that things in a string are for
example other variables or other Lisp functions?

(let* ((lisp (buffer-substring-no-properties
	      (1+ (match-beginning 0)) (1- (match-end 0))))
       (lisp (split-string lisp))
       (function (pop lisp)) 
       (_ (message "%s" function))
       (value (apply (intern function) lisp)) ;; I am stuck here
       (value (string-or-empty-string value)))


For simplest use case like converting string to variable, or something like:

⟦ var hello-name ⟧

I think that would work, but then if I only use variables, I
would not need "var" there, I could just interpolate it into
variable without eval.

For the case where there is even slightly complex Lisp like ⟦ (usd-eur (gold-price-kg)) ⟧
I would not know what to do there.

Maybe it could look like:

⟦ usd-eur (gold-price-kg) ⟧

But then again program would receive string "(gold-price-kg)" and
how would I know that string is another function? I would need to
extract that string and basically make small Lisp reader, again
we come to kind of eval-ing it.

I want to know if there is way to do it.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: How to tame compiler?

[Stefan Monnier]

>   I have to understand it in this example:
>   
>   The HTML template is following and it is string:
>
>   <html>
>     <head>
>       <title>⟦ xml-escape title ⟧</title>
>     </head>
>     <body>
>       <p>Today is one US dollar this many euros: ⟦ usd-eur 1 ⟧</p>
>     </body>
>   </html>

I think the way I'd attack this is similar to what was suggested
recently.  I think I'd introduce a macro (call it `xml-template`) which
you'd use maybe as follows:

    (xml-template
     (html
      (head (title ,title))
      (body (p "Today is one US dollar this many euros: " ,@(usd-eur 1)))))

where `,` is used to insert a random chunk of plain text,
whereas `,@` is used to insert a chunk of XML.


        Stefan

Re: How to tame compiler?

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2021-05-01 16:39]:
> >   I have to understand it in this example:
> >   
> >   The HTML template is following and it is string:
> >
> >   <html>
> >     <head>
> >       <title>⟦ xml-escape title ⟧</title>
> >     </head>
> >     <body>
> >       <p>Today is one US dollar this many euros: ⟦ usd-eur 1 ⟧</p>
> >     </body>
> >   </html>
> 
> I think the way I'd attack this is similar to what was suggested
> recently.  I think I'd introduce a macro (call it `xml-template`) which
> you'd use maybe as follows:
> 
>     (xml-template
>      (html
>       (head (title ,title))
>       (body (p "Today is one US dollar this many euros: " ,@(usd-eur 1)))))
> 
> where `,` is used to insert a random chunk of plain text,
> whereas `,@` is used to insert a chunk of XML.

Approach is known to me from, is it Scribilo?
https://www.nongnu.org/skribilo/ and Skribe
https://www.nongnu.org/skribilo/#self

I use similar approach for CGI document generation.

My system does allow such inclusion as I can introduce any type of
document.

But let me think:

- HTML template would need to be converted into sex-pressions; (don't
  stone me); it would increase the startup time of preparing the
  template, there are just 127 templates in the database;

- then Markdown documents, what I do with it? Maybe all of 4057 should
  be converted into sex-pressions... with very careful attention to
  detail;  

- then Markdown is not any more Markdown, it would still need to be
  expanded by Emacs Lisp;

- then everything expanded embedded into HTML as Emacs Lisp;

Problems with the process:

- it is not readable any more;

- becomes tiresome, much more work is involved to write simple text;
  HTML template preparation alone involves programming skills; not
  easily scalable;

- it becomes very much error prone, one mistake and all page will not
  interpolate;

- author, who only slighly was thinking about the markup, has now to
  go back into thinking harder about the markup, instead of just
  writing text, author has to mark it up with (p "something") or
  similar; author is not any more in the "lightweight" markup as
  described here:
  https://en.wikipedia.org/wiki/Lightweight_markup_language

That approach would be definitely usable for:

- specific pages where large need arise for interpolation;

- with complex code where a lot of interpolation is taking place



-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: How to tame compiler?

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I know I sound like a broken clock, but [...]

I think you would sound more like an unbroken clock if you would shortly
tell the disadvantages of using `eval', then people would be more open
to alternatives.

Michael.

Re: How to tame compiler?

[Jean Louis]

* Michael Heerdegen <michael_heerdegen@web.de> [2021-05-02 08:43]:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
> > I know I sound like a broken clock, but [...]
> 
> I think you would sound more like an unbroken clock if you would shortly
> tell the disadvantages of using `eval', then people would be more open
> to alternatives.

I have now just isolated the `rcd-template-eval' function into
separate package without `lexical-binding' and it works well:
https://hyperscope.link/3/7/1/3/3/RCD-Template-Interpolation-System-for-Emacs.html

This is one special case where lexical-binding should be nil, as me, I
wish to be able to expand any variables in the template. I could use
`lexical-binding t' to see maybe some compiler errors, but then I have
removed it.

Of course I have heard of dangers of using `eval' in various
programming languages. But we have to put it in the specific contexts
as we do use `eval' so many times.

Example is Org mode source blocks, eval is used there, including it is
used to evaluate other or any kind of language. And people publish Org
mode documents after doing eval. Just same thing is taking place here.

The RCD Template Interpolation system uses `eval' to expand the
embedded Emacs Lisp into text. It is just same as using Org mode with
source blocks to expand Emacs Lisp or other programming languages into
text.

Just as with writing text, I could accidently write my passwords and
publish them online, or I could accidentaly or unknowingly remove all
my files with `rm -rf /home/myhome' -- and I remember doing that first
time when I met MS-DOS, but command name maybe was different, who
cares.

When `eval' is used with parameters that arrive from website visitor,
that is where one should be careful maybe to escape the parameters and
make sure there is nothing that could disturb or be malicious. Of
course one should not `eval' the parameter rather accept strings and
process strings. 

I guess hundreds of not thousands of people already publish Org files
that have such embedded programming languages eval-ed and expanded,
and now such Org files are published online.

What I am doing here is the same as Org, just that I like to use any
kind of markup to pre-process it before the conversion into HTML.

Back in time I have been testing various templating systems with Perl,
and I found this one was the fastest:
https://metacpan.org/pod/Text::NeatTemplate because I have tested all
templating systems on thousands page expansions and found that one
being fastest, I used it for years.

One can see how variables are being carefully prepared to be passed to
the function. And there are many similar templating systems used for
HTML generation.

Then I have been using for another bunch of years the CL-EMB Common
Lisp templating system:
https://40ants.com/lisp-project-of-the-day/2020/09/0192-cl-emb.html

https://www.common-lisp.net/project/cl-emb/

I think CL-EMB does not use `eval' directly but it uses
`read-from-string' and macro to expand the values. Anything can take
place there, as it is equivalent to eval, any code can be placed in
the text, and of course somebody can insert malicious code and do
something bad.

Let us say I make system in the functional style, that I don't write
plain text but rather Lisp expressions like (html (p "Something"))
then it is also open to inserting malicious code there, it is even
easier to insert malicious code into the code as it camouflages
itself, then inserting malicious code into the plain text.

People like to use embedded programming languages:
https://github.com/dbohdan/embedded-scripting-languages

There are many web template systems:
https://en.wikipedia.org/wiki/Web_template_system

And those hyperlinks demonstrate the demand for that.

Maybe not many people generate HTML pages or emails with Emacs Lisp,
but I need it. I have SMS communication, email communication, letters,
and I like expanding templates on the fly. When working with 1500+
people, it brings better understanding if I send SMS which expands
into personalized messages:

Hello John, I have not get answer on my message to you from 4th May,
did you read it? You still have US $150 pending with us.

What really matters?

- that I can send mass SMS communication, hire people and engage them
  on projects;

- that I can use templating system to send hundreds of thousands of
  emails and thus close sales, as they are personalized;

- that I can use both HTML pages with embedded personalization for
  emails; send HTML pages, it appears personalized, or publish it, it
  is not personalized, with dynamically expanded values, such as
  prices, listings of products, and similar;

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: How to tame compiler?

[Jean Louis]

* Michael Heerdegen <michael_heerdegen@web.de> [2021-05-02 08:43]:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
> > I know I sound like a broken clock, but [...]
> 
> I think you would sound more like an unbroken clock if you would shortly
> tell the disadvantages of using `eval', then people would be more open
> to alternatives.

If possible, let me know one bad example in my context of expanding
plain text with embedded Emacs Lisp.

Like how it can go wrong that in same time cannot go wrong in same
replacement method?

And which replacement method could I use?

Like why did Lisp authors the `read-from-string' function if it should
not be used? It is quite common that its result will be given to `eval'.


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: How to tame compiler?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1848 bytes --]

On Sun, May 02, 2021 at 10:45:10AM +0300, Jean Louis wrote:
> * Michael Heerdegen <michael_heerdegen@web.de> [2021-05-02 08:43]:
> > Stefan Monnier <monnier@iro.umontreal.ca> writes:
> > 
> > > I know I sound like a broken clock, but [...]
> > 
> > I think you would sound more like an unbroken clock if you would shortly
> > tell the disadvantages of using `eval', then people would be more open
> > to alternatives.
> 
> If possible, let me know one bad example in my context of expanding
> plain text with embedded Emacs Lisp.

I tried to offer an explanation which you chose to ignore [1] . I
think the problem is more subtle, and thus difficult to explain in
just a few words.

If it were that easy to explain clearly, I think Stefan would have
done that already.

> Like how it can go wrong that in same time cannot go wrong in same
> replacement method?
> 
> And which replacement method could I use?
> 
> Like why did Lisp authors the `read-from-string' function if it should
> not be used? It is quite common that its result will be given to `eval'.

It's not as clear-cut as "should not be used". Rather something along
the lines of "should be used with care" [2]. Template expansion is one
of those borderline cases, but if you look at all those modern template
expanders out there [3], you'll realise that they all have some kind of
"custom evaluator", where you explicitly provide an environment, instead
of just punting to `eval' and saying "use... uh, whatever".

For a reason [4].

Cheers

[1] Which, to be honest, is somewhat confusing to me.
[2] Yes, do it: enter the phrase "eval is evil" into your favourite
   search engine.
[3] e.g. Perl's Text::Template, Python's Jinja2, you name it.
[4] Perhaps one short way to express it would be "this havocs
   the compiler". Dunno.

 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: How to tame compiler?

[Jean Louis]

* tomas@tuxteam.de <tomas@tuxteam.de> [2021-05-02 12:08]:
> On Sun, May 02, 2021 at 10:45:10AM +0300, Jean Louis wrote:
> > * Michael Heerdegen <michael_heerdegen@web.de> [2021-05-02 08:43]:
> > > Stefan Monnier <monnier@iro.umontreal.ca> writes:
> > > 
> > > > I know I sound like a broken clock, but [...]
> > > 
> > > I think you would sound more like an unbroken clock if you would shortly
> > > tell the disadvantages of using `eval', then people would be more open
> > > to alternatives.
> > 
> > If possible, let me know one bad example in my context of expanding
> > plain text with embedded Emacs Lisp.
> 
> I tried to offer an explanation which you chose to ignore [1] . I
> think the problem is more subtle, and thus difficult to explain in
> just a few words.

As I am asking questions it is demonstration that I am not ignoring
it. In background I am running business, so the templating engine does
help me, it works, I have sent now quite number of messages to
people. But in foreground, here, I am interested for any kinds of
suggestions.

It is not the option to change text to sex-pressions, as that would
cause way too much work and minimize collaborative capacity, as that
demands teaching people not only how to write text, but how to write
kind of Lisp.

Purpose of it is to interpolate values into text, to pre-process text
or templates. Usually both are pre-processed, the text first, then
text and variables are interpolated into the template. It could be for
letters, it could be like for LaTeX or other markup, mostly HTML,
emails and SMS message.

Related keywords are "web template system" and "templating engines":

https://en.wikipedia.org/wiki/Web_template_system

https://en.wikipedia.org/wiki/Comparison_of_web_template_engines

It is by the same principle as:

- Org Babel, which uses `eval'

- Emacs Muse.

Both are used for publishing.

> > Like how it can go wrong that in same time cannot go wrong in same
> > replacement method?
> > 
> > And which replacement method could I use?
> > 
> > Like why did Lisp authors the `read-from-string' function if it should
> > not be used? It is quite common that its result will be given to `eval'.
> 
> It's not as clear-cut as "should not be used". Rather something along
> the lines of "should be used with care" [2].

It may be evil, just that I don't find practical evilness, but I would
like one simple example how it can be evil.

IMHO, functions are here, if you see something wrong with it, let me
know. It is in a package that does not have lexical-binding t, and it
is one particular case where it should stay so, as I wish to
interpolate any variables.

One problem I had is with (match-beginning 0) and (match-end
(match-end 0)) as with some functions it would not work, maybe because
`eval' was using it deeply somewhere.

That is why I have captured the values in `let':

(let* ((match-beginning (match-beginning 0))
       (match-end (match-end 0))

as otherwise it would not be able to:
(delete-region (match-beginning 0) (match-end 0))
that is why this works:
(delete-region match-beginning match-end) with captured values.

If you see something else, let me know.

(defcustom rcd-template-delimiter-open "⟦"
  "The opening delimiter for RCD Template Interpolation System."
  :group 'rcd
  :type 'string)

(defcustom rcd-template-delimiter-close "⟧"
  "The closing delimiter for RCD Template Interpolation System."
  :group 'rcd
  :type 'string)

(defun rcd-template-eval (string &optional delimiters)
  "Evaluates Emacs Lisp enclosed by `rcd-template-delimiter-open' and `rcd-template-delimiter-close'.

Optional DELIMITERS list may be provided to change default
delimiters, first list member has to be the opening delimiter and
second the closing delimiter.

Space or new line has to follow `rcd-template-delimiter-open' and
precede `rcd-template-delimiter-close' for evaluation to get
invoked."
  (let* ((delimiters (or delimiters (list rcd-template-delimiter-open rcd-template-delimiter-close)))
	 (open (car delimiters))
	 (close (cadr delimiters))
	 (add (length open))
	 (minus (length close)))
    (with-temp-buffer
      (insert string)
      (goto-char 0)
      (while (re-search-forward (rx (literal open)
				    (one-or-more (or blank "\n"))
				    (group (minimal-match (one-or-more anything)))
				    (one-or-more (or blank "\n"))
				    (literal close))
				nil t)
	(let* ((match-beginning (match-beginning 0))
	       (match-end (match-end 0))
	       (lisp (condition-case nil
			 (read-from-string
			  (buffer-substring-no-properties
			   (+ add match-beginning) (- match-end minus)))
		       (error "")))
	       (lisp (if (listp lisp) (car lisp) lisp))
	       (value (condition-case nil
			  (eval lisp)
			(error "")))
	       (value (cond ((null value) "")
			    (t (format "%s" value)))))
	  (delete-region match-beginning match-end)
	  (insert (format "%s" value))))
      (buffer-string))))

(defun rcd-template-buffer-preview ()
  (interactive)
  (let* ((current (current-buffer))
	 (buffer (format "Preview of buffer %s" current))
	 (string (buffer-string))
	 (mode major-mode)
	 (point (point)))
    (pop-to-buffer-same-window buffer)
    (insert (rcd-template-eval string))
    (goto-char point)
    (funcall mode)
    (message buffer)))

(defun rcd-template-insert-delimiters ()
  "Quickly insert the template evaluation snippet such as ⟦ () ⟧
and move cursor into parenthesis."
  (interactive)
  (insert (format "%s () %s" rcd-template-delimiter-open rcd-template-delimiter-close))
  (backward-char (+ 2 (length rcd-template-delimiter-close))))

> Template expansion is one of those borderline cases, but if you look
> at all those modern template expanders out there [3], you'll realise
> that they all have some kind of "custom evaluator", where you
> explicitly provide an environment, instead of just punting to `eval'
> and saying "use... uh, whatever".

For string expansion they will provide environment and "manually"
replace some words into provided alist for example. CL-EMB in Common
Lisp uses `read-from-string'.

So far I see the use of it and I used it years before, just with
Common Lisp.

Here is one example page with its source that uses `rcd-template-eval':

https://hyperscope.link/3/7/1/3/6/Emacs-Lisp-interpolation-of-embedded-PostgreSQL-queries-within-plain-text.html

Purpose of it is to give liberation to HTML, SMS, email templates and
make static HTML pages kind of dynamic, as some dynamically generated
data can be expanded before publishing HTML pages.

I understand so far:

- there are opposing arguments to `eval', but...

- there is no replacement for `eval' in the context of templates with
  programming code; or other contexts;

- `eval' is already heavily used in Org mode and Muse, so many other
  large number packages;

Those contradictions are confusing.


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: How to tame compiler?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 4171 bytes --]

On Sun, May 02, 2021 at 02:18:23PM +0300, Jean Louis wrote:
> * tomas@tuxteam.de <tomas@tuxteam.de> [2021-05-02 12:08]:

[...]

> > I tried to offer an explanation which you chose to ignore [1] . I
> > think the problem is more subtle, and thus difficult to explain in
> > just a few words.
> 
> As I am asking questions it is demonstration that I am not ignoring
> it. In background I am running business, so the templating engine does
> help me, it works, I have sent now quite number of messages to
> people. But in foreground, here, I am interested for any kinds of
> suggestions.

Well, I offered a suggestion. Write your limited version of eval,
which makes available the substitutions (values or functions)
you intend to use in your template, for example in a dictionary
(aka hash). That's what other templating systems do, even in
languages which do have eval (that's why I provided examples in
Perl and Python).

> It is not the option to change text to sex-pressions, as that would
> cause way too much work and minimize collaborative capacity, as that
> demands teaching people not only how to write text, but how to write
> kind of Lisp.

The S-expression proposal is orthogonal to the problem at hand.
That's why I didn't propose it (although it does have some advantages
of its own).

> Purpose of it is to interpolate values into text, to pre-process text
> or templates. Usually both are pre-processed, the text first, then
> text and variables are interpolated into the template. It could be for
> letters, it could be like for LaTeX or other markup, mostly HTML,
> emails and SMS message.

[...]

> > It's not as clear-cut as "should not be used". Rather something along
> > the lines of "should be used with care" [2].
> 
> It may be evil, just that I don't find practical evilness, but I would
> like one simple example how it can be evil.

It's too subtle to be covered by "just one simple example", that's why
I proposed you do some research.

> IMHO, functions are here, if you see something wrong with it, let me
> know. It is in a package that does not have lexical-binding t, and it
> is one particular case where it should stay so, as I wish to
> interpolate any variables.

[...]

I haven't the time to go through your code, sorry.

> > Template expansion is one of those borderline cases, but if you look
> > at all those modern template expanders out there [3], you'll realise
> > that they all have some kind of "custom evaluator", where you
> > explicitly provide an environment, instead of just punting to `eval'
> > and saying "use... uh, whatever".
> 
> For string expansion they will provide environment and "manually"
> replace some words into provided alist for example. CL-EMB in Common
> Lisp uses `read-from-string'.
> 
> So far I see the use of it and I used it years before, just with
> Common Lisp.
> 
> Here is one example page with its source that uses `rcd-template-eval':
> 
> https://hyperscope.link/3/7/1/3/6/Emacs-Lisp-interpolation-of-embedded-PostgreSQL-queries-within-plain-text.html
> 
> Purpose of it is to give liberation to HTML, SMS, email templates and
> make static HTML pages kind of dynamic, as some dynamically generated
> data can be expanded before publishing HTML pages.
> 
> I understand so far:
> 
> - there are opposing arguments to `eval', but...

Let me put that this way: to do a good job, the compiler makes assumptions
on things like variable bindings ("this variable can only be seen in
this part of the code I analysed right now"). When an eval is in the
middle of things, the compiler can't "see through": the code therein
can change at run time. All bets are off.

Just imagine a (let ((x 5) ...) and then an eval whithin that with
a form (setq x 6).

> - there is no replacement for `eval' in the context of templates with
>   programming code; or other contexts;

There is. See my examples.

> - `eval' is already heavily used in Org mode and Muse, so many other
>   large number packages;
> 
> Those contradictions are confusing.

Because the topic is not that simple :)

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: How to tame compiler?

[Jean Louis]

* tomas@tuxteam.de <tomas@tuxteam.de> [2021-05-02 15:25]:
> Well, I offered a suggestion. Write your limited version of eval,
> which makes available the substitutions (values or functions)
> you intend to use in your template, for example in a dictionary
> (aka hash). That's what other templating systems do, even in
> languages which do have eval (that's why I provided examples in
> Perl and Python).

I did use that for years. The new approach seem so much simpler and
liberating, it lessens the work.

> The S-expression proposal is orthogonal to the problem at hand.
> That's why I didn't propose it (although it does have some advantages
> of its own).

As my system supports any type of markup, I can include that as
well. Again I would be reading it with `read-from-string' straight
from the database, and then using `eval'.

S-expression text has to be stored somewhere, right? It has to be
loaded from somewhere, so majority would be loading it from file I
guess, I am loading it from the database and again I have to use
`eval'. The purpose not to use `eval' defeats itself how I see it as I
have to use `eval'.

Let us say database table is named: pages, there is column pages_text,
if it contains string like "(p \"Hello there\")" -- I have to
`read-from-string` and `eval' or maybe there exists some other way?

What would you do? Would you load it from file?

I am thinking, what would be the difference of loading it from file,
or using `eval', I cannot find the difference, I wish I could.

> It's too subtle to be covered by "just one simple example", that's why
> I proposed you do some research.

I did that research years ago. Do we have any practical use of
declining to use specific function without being able to tell why?

On the other hand, thousands of people use `eval' in Org mode as a
good example, and not for Emacs Lisp only.

If we wish to remedy the situation we would need maybe to find safer
`eval' but before finding safer `eval' we have to know why exactly is
this one not safe as compared to whatever proposed replacement
methods. 

> > I understand so far:
> > 
> > - there are opposing arguments to `eval', but...
> 
> Let me put that this way: to do a good job, the compiler makes
> assumptions on things like variable bindings ("this variable can
> only be seen in this part of the code I analysed right now"). When
> an eval is in the middle of things, the compiler can't "see
> through": the code therein can change at run time. All bets are off.

I understand, compiler will not be able to see through.

> Just imagine a (let ((x 5) ...) and then an eval whithin that with
> a form (setq x 6).

Good example, thank you.

That would be `eval' within a program. My `eval' takes place within a
text. It does not necessarily change program, I do not know if it
would be possible, I can try.

(let ((variable "Gully")
      (my-value 0)
      (template "⟦ variable ⟧ ⟦ (setq my-value 1) ⟧"))
  (rcd-template-eval template)) ⇒ "Gully 1"

Now I can see that I can change program values from within `eval'
in the template. On the other hand, I can change values also
within the program. The safety is thus in the decision making of
authors, not in `eval' itself.

> > - there is no replacement for `eval' in the context of templates with
> >   programming code; or other contexts;
> 
> There is. See my examples.

Do you mean example of passing a hash?


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Stages of WWW development compared to Emacs Lisp development

[Jean Louis]

As in the context of templating engines, WWW development is very
similar to Emacs Lisp development or any programming language.

One has to try it out, there are maybe some trials and errors, there
is debugging involved, preview or testing, editing, preview or
testing, until it becomes what author wants it to be.

One bad example that I can think of in using `read-from-string' and
`eval' in templates is when strings from visitor's input would be
processed with it.

For example, if visitor would be asked for email, and that email
address is then, instead of obtained from environment like POST,
processed by `read-from-string' -- at this moment I would not know why
it should be so, unless website wants to evaluate some programming
code. There are today so many websites evaluating programming code in
browser. Who knows how secure they are...

By principle, I do not keep the database online, at least privacy is
ensured. Attacker on a website could eventually send spam or use some
band width, but nothing more than that.

The comparison of web templating engines shows that many of them are
using "functions":
https://en.wikipedia.org/wiki/Comparison_of_web_template_engines

I see that Perl's Template Toolkit probably uses `eval':
https://github.com/abw/Template2/search?q=eval&type=

In this list for Python templating engines, two of them are mentioned
using `eval' I rather think there are more than two:
https://wiki.python.org/moin/Templating

Or this Tiny Template, that says not to use `eval' but it is
restricted in operation:
https://github.com/yuanchuan/tiny-template

And my intention is not to be restricted in operation.

I was doing for years preparation of variables, etc. so that I can
pass it to the templating engine like CL-EMB, but I think that was
more because of Common Lisp nature of packages, I would need to call
functions with their prefixes. Global variables were anyway present,
and I could anyway evaluate any Common Lisp code in CL-EMB.

From CL-EMB:

- `<% ... %>` is a scriptlet tag, and wraps Common Lisp code.
- `<%= ... %>` is an expression tag. Its content gets evaluated and fed as a
            parameter to `(FORMAT T "~A" ...)`.
- `<%# ... #%>` is a comment. Everything within will be removed/ignored.  Can't
            be nested!

I have just minimized it to the system that can use various
delimiters, including just parenthesis (would the remaining text not
use parenthesis with spaces):

( anything ) would be evaluated while (anything) would not. And I can
skip the CL-EMB's tag <%= something %> into ( something ) or ⟦ something ⟧
and I skip using complex tags like <% @var VARIABLE %> as ⟦ VARIABLE ⟧
will work or expand empty.

In CL-EMB there are some template tags, while any Lisp can be embedded anyway:

Currently supported: `@if`, `@else`, `@endif`, `@ifnotempty`, `@unless`, 
`@endunless`, `@var`, `@repeat`, `@endrepeat`, `@loop`, `@endloop`, `@include`,
`@includevar`, `@call`, `@with`, `@endwith`, `@set`, `@genloop`, `@endgenloop`,
`@insert`.

It is more complex templating engine, one can have separate chunks of
embedded templating tags which then work together with the mixed text
inside, great, but also complex. It mixes 2 languages, one is
templating language and one is Lisp. Instead, why not just use Lisp.

Process of preparing pages for evaluation is simple:

- edit page
- preview
- edit and preview until satisfied
- publish

In that sense whatever can go wrong with sex-pression templates or
Scribe style of preparation, it can also go wrong with `eval' based
templating engine.

Muse can embed any kind of Emacs Lisp and anyway does not have good
visualization of it. Org mode embeds any kind of programming.

I am using Org mode and expanding it from outside of Emacs by using
Common Lisp which invokes Emacs each time, and I guess that makes
things slower. If I manage the Website Revision System directly from
Emacs, it will be somewhat faster to expand Org blobs from database.

Following Common Lisp is used to convert stream of Org data to
Markdown markup:

$ cat file.org | pipe-org-to-markdown.lisp



#!/home/data1/protected/bin/lisp
(defparameter memdir "/dev/shm/")
(defparameter tmp-file "rcd-wrs-XXXXXX")
(defparameter tmp-org (format nil "~a~a.org" memdir tmp-file))
(defparameter tmp-md (format nil "~a~a.md" memdir tmp-file))
;;(delete-file tmp-org)
;;(delete-file tmp-md)
(load "/home/data1/protected/Programming/git/RCDBusiness/lib/lisp/streamtools.lisp")
;; (require "syscalls")

(defparameter org-autoloads.el
  (concatenate
   'string
   (namestring (car (directory "/home/data1/protected/.emacs.d/elpa/org-20??????/")))
   "org-autoloads.el"))

(defun main ()
  (let ((input '())
        (output '()))
    (with-open-file (out tmp-org :direction :output :external-format "utf-8" :if-exists :supersede)
      (loop for line = (read-line *standard-input* nil nil)
         while line do
           (write-line line out)))
    (shell (format nil "emacs -Q -l ~a \"~a\" --batch -f org-mode -f org-md-export-to-markdown --kill" org-autoloads.el tmp-org))
    (setf output
            (with-open-file (stream tmp-md :direction :input :external-format "utf-8")
              (loop for line = (read-line stream nil nil)
                 while line collect line)))
    (setf output (format nil "~{~a~^~%~}" output))
    (setf output (slurp-stream-io-command "markdown -F 0x4" output))
    (princ output))
  (exit))

;; ;; (saveinitmem "org2html" :quiet t :init-function 'main :verbose nil :norc t :documentation "Converts Org standard input into markdown" :executable t)

(main)


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: Stages of WWW development compared to Emacs Lisp development

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

> As in the context of templating engines, WWW development is
> very similar to Emacs Lisp development or any
> programming language.

Well...

> One has to try it out, there are maybe some trials and
> errors, there is debugging involved, preview or testing,
> editing, preview or testing, until it becomes what author
> wants it to be.

... rather I'd say what you describe holds for everything that
is difficult to do, almost.

> (concatenate

?

> (loop for line = (read-line
>    while line do
>      (write-line

???

> (slurp-stream-io-command 
> (saveinitmem

Where you get all this stuff?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Stages of WWW development compared to Emacs Lisp development

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-05-02 19:53]:
> Jean Louis wrote:
> 
> > As in the context of templating engines, WWW development is
> > very similar to Emacs Lisp development or any
> > programming language.
> 
> Well...
> 
> > One has to try it out, there are maybe some trials and
> > errors, there is debugging involved, preview or testing,
> > editing, preview or testing, until it becomes what author
> > wants it to be.
> 
> ... rather I'd say what you describe holds for everything that
> is difficult to do, almost.
> 
> > (concatenate
> 
> ?
> 
> > (loop for line = (read-line
> >    while line do
> >      (write-line

I don't use it anymore, that is not from me, at least not currently. 

> ???
> 
> > (slurp-stream-io-command 
> > (saveinitmem
> 
> Where you get all this stuff?

Isn't that Common Lisp?

(require "asdf")

(defun slurp-stream-io-command (command string)
  "Returns the output of a command to which string has been fed, very
usable for markdown, emacs Org mode processing and similar"
  (with-input-from-string (in string)
    (uiop:run-program command :input in :output :string :ignore-error-status t)))

(slurp-stream-io-command "markdown" "## hello")

"<h2>hello</h2>
"



-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: How to tame compiler?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Michael Heerdegen wrote:

> I think you would sound more like an unbroken clock if you
> would shortly tell the disadvantages of using `eval', then
> people would be more open to alternatives.

That would be an even better clock yes...

-- 
underground experts united
https://dataswamp.org/~incal

Re: How to tame compiler?

[Stefan Monnier]

>> I know I sound like a broken clock, but [...]
> I think you would sound more like an unbroken clock if you would shortly
> tell the disadvantages of using `eval', then people would be more open
> to alternatives.

Code always starts in the form of a string somewhere, i.e. data, and
somewhere along the lines it gets transformed into something that can be
executed, i.e. code.

`eval` is one way to do that, but it's a quite restrictive, because it
doesn't give any opportunity to do something useful with the data
representation of the code before we run it.  I.e. it makes it hard to
compile the code to make it more efficient, it makes it virtually
impossible to give feedback about problems in the code (except for those
problems which lead to "hard errors"), and for the same reason it makes
it hard to provide any other kind of help in writing the code (like code
completion).

In the current context of Emacs, places that use `eval` tend to presume
that variables have names and that those names are significant,
i.e. they presume dynamic scoping, and for the same reason that it's
hard to provide feedback about problems in code that is passed to `eval`
it's very hard (both for tools and for humans) to figure out what needs
to be done in order for the code to work correctly in the lexical-scoped
dialect of ELisp.


        Stefan

Re: How to tame compiler?

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2021-05-03 01:30]:
> >> I know I sound like a broken clock, but [...]
> > I think you would sound more like an unbroken clock if you would shortly
> > tell the disadvantages of using `eval', then people would be more open
> > to alternatives.
> 
> Code always starts in the form of a string somewhere, i.e. data, and
> somewhere along the lines it gets transformed into something that can be
> executed, i.e. code.
> 
> `eval` is one way to do that, but it's a quite restrictive, because it
> doesn't give any opportunity to do something useful with the data
> representation of the code before we run it.  I.e. it makes it hard to
> compile the code to make it more efficient, it makes it virtually
> impossible to give feedback about problems in the code (except for those
> problems which lead to "hard errors"), and for the same reason it makes
> it hard to provide any other kind of help in writing the code (like code
> completion).
> 
> In the current context of Emacs, places that use `eval` tend to presume
> that variables have names and that those names are significant,
> i.e. they presume dynamic scoping, and for the same reason that it's
> hard to provide feedback about problems in code that is passed to `eval`
> it's very hard (both for tools and for humans) to figure out what needs
> to be done in order for the code to work correctly in the lexical-scoped
> dialect of ELisp.
> 
> 
>         Stefan

Thank you. I agree with compiler issues.

In the context of template expansion, would there be some larger
program and they are in background somewhat larger, those programs or
functions may and do reside in packages with lexical scoping, rather
for verification of the code. When certain function runs well, they
are proven and they may be then embedded into the text. It could be as
simple as:

⟦ (sql "SELECT count(1) FROM people") ⟧ 

or

⟦ (clisp-macro (princ (format nil "~R" (expt 2 50)))) ⟧

Those are usually smaller snippets that are verified outside of the
text template beforehand, and previewed before publishing. If anything
goes wrong, they may evaluate to empty string and continue with
parsing of the text. Previewer may decide if function works or not,
basically, if the text is ready to be published or not.

Would there be larger or more complex code, then such can be invoked
beforehand in form of a report, or file, and its result could be
included into the page by one of the methods. In general the embedded
Emacs Lisp will never be used or installed as a package, neither byte
compiled or otherwise verified for correctness as how we usually do it
with Emacs Lisp files.

It is one of multiple programming paradigms of Lisp.



-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: How to tame compiler?

[Eduardo Ochs]

On Sun, 2 May 2021 at 20:32, Jean Louis <bugs@gnu.support> wrote:
>
> * Stefan Monnier <monnier@iro.umontreal.ca> [2021-05-03 01:30]:
> > >> I know I sound like a broken clock, but [...]
> > > I think you would sound more like an unbroken clock if you would shortly
> > > tell the disadvantages of using `eval', then people would be more open
> > > to alternatives.
> >
> > Code always starts in the form of a string somewhere, i.e. data, and
> > somewhere along the lines it gets transformed into something that can be
> > executed, i.e. code.
> >
> > `eval` is one way to do that, but it's a quite restrictive, because it
> > doesn't give any opportunity to do something useful with the data
> > representation of the code before we run it.  I.e. it makes it hard to
> > compile the code to make it more efficient, it makes it virtually
> > impossible to give feedback about problems in the code (except for those
> > problems which lead to "hard errors"), and for the same reason it makes
> > it hard to provide any other kind of help in writing the code (like code
> > completion).
> >
> > In the current context of Emacs, places that use `eval` tend to presume
> > that variables have names and that those names are significant,
> > i.e. they presume dynamic scoping, and for the same reason that it's
> > hard to provide feedback about problems in code that is passed to `eval`
> > it's very hard (both for tools and for humans) to figure out what needs
> > to be done in order for the code to work correctly in the lexical-scoped
> > dialect of ELisp.
> >
> >
> >         Stefan
>
> Thank you. I agree with compiler issues.
>
> In the context of template expansion, would there be some larger
> program and they are in background somewhat larger, those programs or
> functions may and do reside in packages with lexical scoping, rather
> for verification of the code. When certain function runs well, they
> are proven and they may be then embedded into the text. It could be as
> simple as:
>
> ⟦ (sql "SELECT count(1) FROM people") ⟧
>
> or
>
> ⟦ (clisp-macro (princ (format nil "~R" (expt 2 50)))) ⟧
>
> Those are usually smaller snippets that are verified outside of the
> text template beforehand, and previewed before publishing. If anything
> goes wrong, they may evaluate to empty string and continue with
> parsing of the text. Previewer may decide if function works or not,
> basically, if the text is ready to be published or not.
>
> Would there be larger or more complex code, then such can be invoked
> beforehand in form of a report, or file, and its result could be
> included into the page by one of the methods. In general the embedded
> Emacs Lisp will never be used or installed as a package, neither byte
> compiled or otherwise verified for correctness as how we usually do it
> with Emacs Lisp files.
>
> It is one of multiple programming paradigms of Lisp.
>
>
>
> --
> Jean


Hi Jean-Louis,

just as a curiosity, here is how eev implements template
interpolation:

  http://angg.twu.net/eev-current/eev-template0.el.html
  http://angg.twu.net/eev-current/eev-template0.el

I tried several other implementations of template interpolation
functions before that one, and from time to time I would find another
way that would be both simpler and more powerful than the previous
one... the one above is so simple that in one of the times that I gave
a minicourse on LaTeX and Emacs I followed the code of `ee-template0'
with the students in the third day of the minicourse - one day after
teaching them briefly how to understand code with `defun's and `let's.

My trick to handle lexical binding was to tell the students that
dynamic binding is much easier to understand than lexical binding, and
that they should treat dynamic binding as an advanced topic that would
make more sense when they had at least one week of experience with
Emacs - and my minicourse was only five days long, from monday to
friday... but the students in that minicourse were mathematicians with
little experience in programming, not compsci people who would see
immediately that dynamic binding is "wrong".

  Cheers =),
    Eduardo Ochs
    http://angg.twu.net/#eev

Eval in templates - Re: How to tame compiler?

[Jean Louis]

* Eduardo Ochs <eduardoochs@gmail.com> [2021-05-03 04:59]:
> just as a curiosity, here is how eev implements template
> interpolation:
> 
>   http://angg.twu.net/eev-current/eev-template0.el.html
>   http://angg.twu.net/eev-current/eev-template0.el
> 
> I tried several other implementations of template interpolation
> functions before that one, and from time to time I would find another
> way that would be both simpler and more powerful than the previous
> one... the one above is so simple that in one of the times that I gave
> a minicourse on LaTeX and Emacs I followed the code of `ee-template0'
> with the students in the third day of the minicourse - one day after
> teaching them briefly how to understand code with `defun's and
> `let's.

That is so interesting. This templating system has slightly different
concept. In my eval-ing, I have to get some value no matter what, I am
also thinking I have freedom to make errors, to the system is not
allowed to fail.

(setq rcd-template-delimiter-open "(") ⇒ "("
(setq rcd-template-delimiter-close ")") ⇒ ")"
(setq text1 "We will evaluate (+ 2 2) to result ((+ 2 2))") ;; won't interpolate
(setq text2 "We will evaluate (+ 2 2) to result ( (+ 2 2) )")
(setq text3 "We will evaluate (+ 2 2) to result ( (+ 2 2 )") ;; error allowed

This does not interpolate, as I demand spaces after delimiters:
(rcd-template-eval text1) ⇒ "We will evaluate (+ 2 2) to result ((+ 2 2))"

This does interpolate:
(rcd-template-eval text2) ⇒ "We will evaluate (+ 2 2) to result 4"

Error is interpolated to "", but I could log it in future:
(rcd-template-eval text3) ⇒ "We will evaluate (+ 2 2) to result "

Compared to this that works:
(ee-template0 "{<} a{(+ 2 3)} {>}") ⇒ "{ a5 }"

And this one giving error:
(ee-template0 "{<} a{(+ 2 3} {>}")

Concept is the same. You also do not use dynamic binding, which
is quite clear why.

In the sense of teaching is fine. I guess you almost never had a
problem when evaluation fails for some reason.

In the sense of generation of hundreds of emails for people, or
generation of thousands of HTML pages, the template system IMHO
is not allowed to fail. Errors however, could be reported,
templating system could then tell me in which page or email the
error has been found. But it has to move silently over the failure.

Thus I would expect also this one, to produce following:
(ee-template0 "{<} a{(+ 2 3} {>}")  ⇒ "{ a }"

And more perfect would be if error would be reported somewhere in silent.

> My trick to handle lexical binding was to tell the students that
> dynamic binding is much easier to understand than lexical binding, and
> that they should treat dynamic binding as an advanced topic that would
> make more sense when they had at least one week of experience with
> Emacs - and my minicourse was only five days long, from monday to
> friday... but the students in that minicourse were mathematicians with
> little experience in programming, not compsci people who would see
> immediately that dynamic binding is "wrong".

Thus you had the luck to have a different type of audience.

Here is also demonstrated how eev tools have different context,
programs are invoked from within text manuall. Your tools are in
general, beyond templating, meant to be evaluated in the text or
within the text and in that concept there is also no need for
lexical binding. Small snippets are evaluated one after each
other by user's demand, functions evaluated may come originally
from programs with lexical binding.

Thus I have improved my function to evalute with errors reported,
as it makes so much more sense. It is made to interpolate
thousands of times within minutes automatically.

Source: https://gnu.support/files/sources/rcd-template.el

(defun rcd-template-eval (string &optional delimiters)
  "Evaluates Emacs Lisp enclosed by `rcd-template-delimiter-open' and `rcd-template-delimiter-close'.

Optional DELIMITERS list may be provided to change default
delimiters, first list member has to be the opening delimiter and
second the closing delimiter.

Space or new line has to follow `rcd-template-delimiter-open' and
precede `rcd-template-delimiter-close' for evaluation to get
invoked."
  (let* ((delimiters (or delimiters (list rcd-template-delimiter-open rcd-template-delimiter-close)))
	 (open (car delimiters))
	 (close (cadr delimiters))
	 (add (length open))
	 (minus (length close)))
    (with-temp-buffer
      (insert string)
      (goto-char 0)
      (while (re-search-forward (rx (literal open)
				    (one-or-more (or blank "\n"))
				    (group (minimal-match (one-or-more anything)))
				    (one-or-more (or blank "\n"))
				    (literal close))
				nil t)
	(let* ((match-beginning (match-beginning 0))
	       (match-end (match-end 0))
	       (match (buffer-substring-no-properties
		       (+ add match-beginning) (- match-end minus)))
	       (lisp (condition-case error
			 (read-from-string match)
		       (error (princ (format "rcd-template-eval: `read-from-string' error: %s" error))
			      "")))
	       (lisp (if (listp lisp) (car lisp) lisp))
	       (value (condition-case error
			  (eval lisp)
			(error (princ (format "rcd-template-eval: `eval' error: %s" error)))))
	       (value (cond ((null value) "")
			    (t (format "%s" value)))))
	  (delete-region match-beginning match-end)
	  (insert (format "%s" value))))
      (buffer-string))))

(rcd-template-eval "( (+ 2 2) )" '("(" ")")) ⇒ "4"

(rcd-template-eval "( (+ 2 2 )" '("(" ")")) ⇒ "" 
     with error reporting in *Messages* buffer:
     rcd-template-eval: `read-from-string' error: (end-of-file)

(rcd-template-eval "We expect 4 but fail: ( (+2 2) )" '("(" ")")) ⇒ "We expect 4 but fail: "
     with error reporting in *Messages* buffer:
     rcd-template-eval: `eval' error: (invalid-function 2)

(defun ee-template-test (text)
  (rcd-template-eval text '("{" "}")))

My regexp asks for spaces or new lines between delimiters:

(ee-template-test "{<} a{ (+ 2 3 } {>}") ⇒ "{<} a {>}"
     with error reporting in *Messages* buffer:
     rcd-template-eval: `read-from-string' error: (end-of-file)

but of course, it does not do your trick:
(ee-template-test "{<} a{ (+ 2 3) } {>}") ⇒ "{<} a5 {>}"

If I would like to enclose it with open delimiters in a result, this would work:
(ee-template-test "{ \"{\" } a{ (+ 2 3) } { \"}\" }") ⇒ "{ a5 }"

And I use the habit that people would not tend to write space
after parenthesis.

I could write: (maybe I am wrong), but I would never write 
( maybe I am not wrong ).

That may allow using parenthesis in templates which makes it more
Lisp-like:

(rcd-template-eval "We know that (+ 2 2) evaluates to ( (+ 2 2) )") ⇒ "We know that (+ 2 2) evaluates to 4"

Then `eval' within `eval' help in avoiding repetition of demonstrated code:
(rcd-template-eval "We know that ( (setq code '(+ 2 2)) ) evaluates to ( (eval code) )") ⇒ "We know that (+ 2 2) evaluates to 4"

Regular expression is I see hard coded:
(defvar ee-template00-re "{\\([^{}]+\\)}")

And I have envisioned it to rather have flexible delimiters:

(rcd-template-eval "<% user-login-name %>" '("<%" "%>")) ⇒ "jimmy"

as it emulates some other templating engines and allows re-use of templates:
(rcd-template-eval "<% @var user-login-name %>" '("<% @var" "%>")) ⇒ "jimmy"

Maybe you wish to include error reporting and silent failure as
well, or no?

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: How to tame compiler?

[Michael Heerdegen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> you would have to create dynamical bindings for that.

Oh, you could also use the second argument of `eval' to specify a
lexical environment.  That would also help to get rid of the warnings.

Michael.

Re: How to tame compiler?

[Jean Louis]

[-- Attachment #1: Type: text/plain, Size: 1610 bytes --]

* Michael Heerdegen <michael_heerdegen@web.de> [2021-05-01 07:54]:
> Michael Heerdegen <michael_heerdegen@web.de> writes:
> 
> > you would have to create dynamical bindings for that.
> 
> Oh, you could also use the second argument of `eval' to specify a
> lexical environment.  That would also help to get rid of the
> warnings.

How I understand it, I would need to use dynamical bindings, but in
smallest example with lexical-binding t on the top of file, it does
not work.

So I do not know how to use dynamical bindings.

As function `rcd-template-eval' is in other file, when I provide `t'
to (eval) in that other file, that still gives warnings in the file
where variables are used, and then it will not expand.

I have lost control.

My program works, but it should not work.

In fact if I remove:

    (ignore "Avoid compiler warnings" to-name hello-name body
    unsubscribe-text unsubscribe-url)

than nothing works, and template does not get expanded, probably
because of lexical bindings. (eval) I use without t

If I comment `ignore' or I do not use ignore, it stops working, and
nothing expands.

If I do use ignore as above, it does work.

But in the smallest example as in attached files, it does not work.

Maybe it is some bug, and program works due to bug.

I cannot bring it to minimum example, it just works, exactly how it is
in these attached exampled (which don't work).

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/


[-- Attachment #2: test.el --]
[-- Type: text/plain, Size: 370 bytes --]

;; -*- lexical-binding: t; -*-

;; (add-to-list 'load-path "~/Programming/emacs-lisp")
(require 'rcd-template)

(defun my-fun ()
  (let* ((hello-name "Joe")
	 ;; (lexical-binding nil)
	 (template "⟦ hello-name ⟧")
	 (expanded-body (rcd-template-eval template)))
    (ignore "Avoid compiler warnings" hello-name template)
    (message "%s" expanded-body)))

(my-fun)

[-- Attachment #3: rcd-template.el --]
[-- Type: text/plain, Size: 2126 bytes --]

;;; rcd-template.el --- template
;; -*- lexical-binding: t; -*-


;;; Commentary:
;; 

;;; Code:

(defvar rcd-template-delimiter-open "⟦")
(defvar rcd-template-delimiter-close "⟧")

(defun rcd-template-eval (string &optional delimiters)
  "Evaluates Emacs Lisp enclosed by `rcd-template-delimiter-open' and `rcd-template-delimiter-close'.

Optional DELIMITERS list may be provided to change default
delimiters, first list member has to be the opening delimiter and
second the closing delimiter.

Space or new line has to follow `rcd-template-delimiter-open' and
precede `rcd-template-delimiter-close' for evaluation to get
invoked."
  (let* ((delimiters (or delimiters (list rcd-template-delimiter-open rcd-template-delimiter-close)))
	 (open (car delimiters))
	 (close (cadr delimiters)))
    (with-temp-buffer
      (insert string)
      (goto-char 0)
      (while (re-search-forward (rx (literal open)
				    (one-or-more (or blank "\n"))
				    (group (minimal-match (one-or-more anything)))
				    (one-or-more (or blank "\n"))
				    (literal close))
				nil t)
	;;(message "Found match")
	(let* ((lisp (car (read-from-string
			   (buffer-substring-no-properties
			    (1+ (match-beginning 0)) (1- (match-end 0))))))
	       (value (condition-case nil
			  (eval lisp)
			(error "")))
	       (value (string-or-empty-string value)))
	  ;; (message "HELLO: %s" (eval (format "%s" hello-name)))
	  (delete-region (match-beginning 0) (match-end 0))
	  (insert (format "%s" value))))
      (buffer-string))))

(defun rcd-template-buffer-eval ()
  (interactive)
  (let* ((buffer (buffer-string))
	 (mode major-mode)
	 (point (point)))
    (pop-to-buffer-same-window "Preview")
    (insert (rcd-template-eval buffer))
    (goto-char point)
    (funcall mode)
    (espeak "Preview")))

(defun string-or-empty-string (i)
  "Returns empty string for nil"
  (let* ((type (type-of i)))
    (cond ((or (eql type 'integer)
	       (eql type 'float)) 
	   (number-to-string i))
	  ((null i) "")
	  ((eql type 'symbol) (prin1-to-string i))
	  ((eql type 'string) i))))

(provide 'rcd-template)

;;; rcd-template.el ends here

Re: How to tame compiler?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

> So I do not know how to use dynamical bindings

Even more reason not to do it...

-- 
underground experts united
https://dataswamp.org/~incal

Re: How to tame compiler?

[Jean Louis]

* Michael Heerdegen <michael_heerdegen@web.de> [2021-05-01 03:44]:
> Jean Louis <bugs@gnu.support> writes:
> 
> > As I have recently implemented new function that use `eval' to expand
> > various variables and these variables are not visibly used in the
> > program, I would like to tame the compiler, as I get these warnings:
> >
> > In rcd-send-email:
> > rcd-mailing.el:225:62: Warning: Unused lexical variable `unsubscribe-url'
> > rcd-mailing.el:231:21: Warning: Unused lexical variable `hello-name'
> > rcd-mailing.el:234:72: Warning: Unused lexical variable `unsubscribe-text'
> > rcd-mailing.el:252:11: Warning: Unused lexical variable `body'
> >
> > What is happening here is that those variables are used but inside of
> > `eval' form which is expanded dynamically when program runs. Compiler
> > cannot see that.
> >
> > Is there a way to avoid these warnings?
> 
> Independent from the question whether your usage of `eval' is good or
> valid - there must be some real problem here: if the compiler tells that
> the lexical variables are unused, their values will not be available in
> you `eval' call - you would have to create dynamical bindings for
> that.

How do I go to create dynamical bindings? Help me.



This file works:

(let* ((hello-name "Joe")
       (template "⟦ hello-name ⟧")
       (expanded (rcd-template-eval template)))
  expanded) ⇒ "Joe"

while this one does not work:

;; -*- lexical-binding: t; -*-

(let* ((hello-name "Joe")
       (template "⟦ hello-name ⟧")
       (expanded (rcd-template-eval template)))
  expanded) ⇒ ""

and to make it work I did this:

;; -*- lexical-binding: t; -*-

(let* ((hello-name "Joe")
       (template "⟦ hello-name ⟧")
       (lexical-binding nil)
       (expanded (rcd-template-eval template)))
  expanded)

(ignore 'hello-name)

however, that simple example does not work, but what is astonishing I
have sent 900+ emails yesterday and they were expanded just fine.

The file test.el with above code is not working.

My file with lexical-binding: t is working.

I am using it and I do not know why it does work.

Without (lexical-binding nil) it did not work. It seems I have to use
that to make it work, but in smallest example it does not.

Maybe it is something completely else.


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: How to tame compiler?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

> How do I go to create dynamical bindings? Help me.

Lexical is much better, so put

  ;; -*- lexical-binding: t; -*-

in the beginning (first line) of every file including ~/.emacs

-- 
underground experts united
https://dataswamp.org/~incal

Re: How to tame compiler?

[Michael Heerdegen]

Jean Louis <bugs@gnu.support> writes:

> How do I go to create dynamical bindings? Help me.

Newest Emacs versions have `dlet' to create dynamical bindings.  Here is
the definition:

#+begin_src emacs-lisp
(defmacro dlet (binders &rest body)
  "Like `let*' but using dynamic scoping."
  (declare (indent 1) (debug let))
  `(let (_)
     ,@(mapcar (lambda (binder)
                 `(defvar ,(if (consp binder) (car binder) binder)))
               binders)
     (let* ,binders ,@body)))
#+end_src

Use that macro to create dynamical bindings where you really need, and
rely on lexical binding else.

Apart from that, I recommend to take Stefan's suggestions into
consideration, he knows what he is talking about.

Regards,

Michael.

Re: How to tame compiler?

[Jean Louis]

* Michael Heerdegen <michael_heerdegen@web.de> [2021-05-02 08:59]:
> Jean Louis <bugs@gnu.support> writes:
> 
> > How do I go to create dynamical bindings? Help me.
> 
> Newest Emacs versions have `dlet' to create dynamical bindings.  Here is
> the definition:
> 
> #+begin_src emacs-lisp
> (defmacro dlet (binders &rest body)
>   "Like `let*' but using dynamic scoping."
>   (declare (indent 1) (debug let))
>   `(let (_)
>      ,@(mapcar (lambda (binder)
>                  `(defvar ,(if (consp binder) (car binder) binder)))
>                binders)
>      (let* ,binders ,@body)))
> #+end_src
> 
> Use that macro to create dynamical bindings where you really need, and
> rely on lexical binding else.
> 
> Apart from that, I recommend to take Stefan's suggestions into
> consideration, he knows what he is talking about.

Thankful for dlet tip!

It would help if I wish to expand few variables dynamically bound.

Though I found the solution for `rcd-template-eval' as I want global
variables, I am simply not using lexical-binding t in the package.

All I had to do is to isolate the function for template evaluation
into separate package without lexical-binding and now there are no
problems whatsoever, interpolation works well.

It is here:
https://hyperscope.link/3/7/1/3/3/RCD-Template-Interpolation-System-for-Emacs.html

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: How to tame compiler?

[Jean Louis]

* Michael Heerdegen <michael_heerdegen@web.de> [2021-05-02 08:59]:
> Jean Louis <bugs@gnu.support> writes:
> 
> > How do I go to create dynamical bindings? Help me.
> 
> Newest Emacs versions have `dlet' to create dynamical bindings.  Here is
> the definition:
> 
> #+begin_src emacs-lisp
> (defmacro dlet (binders &rest body)
>   "Like `let*' but using dynamic scoping."
>   (declare (indent 1) (debug let))
>   `(let (_)
>      ,@(mapcar (lambda (binder)
>                  `(defvar ,(if (consp binder) (car binder) binder)))
>                binders)
>      (let* ,binders ,@body)))
> #+end_src
> 
> Use that macro to create dynamical bindings where you really need, and
> rely on lexical binding else.

That really helped me, or I would be lost today. 

As my `rcd-template-eval' function did work, but somehow, for unknown
reasons as indicated already by developers, it started misbehaving,
and variables sometimes interpolated, sometimes not, or maybe at
second invokation not.

And I had to enclose the variables in the function within `dlet' and
now it works well. But I have to tame compiler...

This function can now generate thousands of pages, pre-process it,
process with Markdown, Asciidoctor, txt2tags, Org mode, or however,
page is written (or not process...) and produce the output.

I guess it is very inefficient. Important is that it works.

(defun wrs-generate-page (page-id)
  (dlet ((page (rcd-db-table-id-hash "pages" page-id cf-db))
	 (area (rcd-db-table-id-hash "areas" (gethash "pages_area" page) cf-db))
	 (template (rcd-db-table-id-hash "templates" 
					 (or (gethash "pages_templates" page)
					     (gethash "areas_template" area))
					 cf-db))
	 (pages_title (gethash "pages_title" page))
	 (pages_keywords (gethash "pages_keywords" page))
	 (pages_description (gethash "pages_description" page))
	 (variables (make-hash-table :test 'equal))
	 (_ (hash-append variables (wrs-variables-hash nil nil nil 3)))
	 (_ (hash-append variables (wrs-variables-hash (gethash "pages_area" page))))
	 (_ (hash-append variables (wrs-variables-hash nil (gethash "templates_id" template))))
	 (language (rcd-db-get-entry "languages" "languages_extension" 
				     (or (gethash "pages_language" page)
					 (gethash "areas_defaultlanguage" area))
				     cf-db))
	 (_ (puthash "languages_extension" language variables))
	 (_ (when (gethash "areas_company" area)
	      (puthash "areas_company"
		       (rcd-db-get-entry "accounts" "accounts_name"
					 (gethash "areas_company" area)
					 cf-db)
		       variables)))
	 (_ (hash-append variables page area template))
	 (text (rcd-template-eval (gethash "pages_content" page "") '("⟦" "⟧") variables))
	 (processor (gethash "templates_processor" template))
	 (text (wrs-process-text processor text))
	 (_ (puthash "pages_content" text variables))
	 (template (gethash "templates_content" template))
	 (open-graph-type "Article")
	 (html (rcd-template-eval template '("⟦" "⟧") variables))
	 (local-file (wrs-page-url page-id t)))
    (string-to-file-force html local-file)))

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/