Emacs Modular Configuration: the preferable way.

Emacs Modular Configuration: the preferable way.

[Hongyi Zhao]

Dear all,

I noticed the following instructions/tools for Emacs modular configuration:

https://www.emacswiki.org/emacs/DotEmacsModular
https://www.emacswiki.org/emacs/DotEmacsStructuring
https://github.com/emacs-jp/init-loader

There are so many ways to do this. What's the preferable way?

Regards
-- 
Assoc. Prof. Hongyi Zhao <hongyi.zhao@gmail.com>
Theory and Simulation of Materials
Hebei Vocational University of Technology and Engineering
NO. 552 North Gangtie Road, Xingtai, China

Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

Hongyi Zhao wrote:

> I noticed the following instructions/tools for Emacs modular
> configuration:
>
> https://www.emacswiki.org/emacs/DotEmacsModular
> https://www.emacswiki.org/emacs/DotEmacsStructuring
> https://github.com/emacs-jp/init-loader
>
> There are so many ways to do this. What's the
> preferable way?

My personal favorite is the method below, I don't know if
that's mentioned on any of your links.

Note that every file is mentioned explicitly, that means if
you are to search for an error (say do the so-called binary
search) you don't have to comment in and out code that might
have comments, headers, footers, and so on, instead you just
comment out one or more filenames, and when the bug or
misfeature is localized, it is as easy to enable them again.

(let*((emacs-dir "~/.emacs.d")
      (lisp-dir  (format "%s/lisp"       emacs-dir))
      (init-dir  (format "%s/emacs-init" emacs-dir))
      (erc-dir   (format "%s/erc"        init-dir))
      (gnus-dir  (format "%s/gnus"       init-dir))
      (ide-dir   (format "%s/ide"        init-dir))
      (w3m-dir   (format "%s/w3m"        init-dir))
      (dirs     (list
                 lisp-dir
                 init-dir
                 erc-dir
                 gnus-dir
                 ide-dir
                 w3m-dir
                 )))
  (dolist (d dirs)
    (push d load-path) )
  (let ((emacs-init-files '(
                            abc.el
                            align-incal.el
                            batch.el
                            bibtex-incal.el
                            bike.el
                            bmi.el
                            buc.el
                            buffer-menu.el
                            caps-back.el
                            close.el
                            console-keys.el
                            count.el
                            custom-vars.el
                            day.el
                            dired-incal.el
                            ecat-incal.el
                            echo-message.el
                            edit.el
                            elpa.el
                            emacs-shell.el
                            face.el
                            file-write-to.el
                            file.el
                            fill-incal.el
                            frame-size.el
                            geh.el
                            get-search-string.el
                            help-incal.el
                            info-incal.el
                            isbn-verify.el
                            issn-verify.el
                            iterate-files.el
                            jean.el
                            keys.el
                            kill-path.el
                            kill.el
                            latex.el
                            lights.el
                            linux-shell.el
                            list-quoted-functions.el
                            man-incal.el
                            match-data-format.el
                            math.el
                            measure.el
                            minor-modes.el
                            misc.el
                            mode-by-filename.el
                            mode-line.el
                            navigate-fs-keys.el
                            negative-subtraction.el
                            perm.el
                            printer.el
                            quit.el
                            random.el
                            re-make-list.el
                            replace-list.el
                            revert-buffer.el
                            scale.el
                            scroll.el
                            search-regexp-in-files.el
                            sequence-string.el
                            shell-cli.el
                            show-command.el
                            signal.el
                            sort-incal.el
                            spell.el
                            street.el
                            string.el
                            sudo-user-path.el
                            super.el
                            survivor.el
                            switch-to-buffer-regexp.el
                            switch-to-buffer.el
                            tabs.el
                            test-face.el
                            test.el
                            time-cmp.el
                            time-incal.el
                            time-insert.el
                            todo-did.el
                            tramp-incal.el
                            variance.el
                            vt.el
                            window-incal.el
                            wood.el
                            wrap-search.el
                            xsel.el
                            yank.el
                            )))
    (dolist (f emacs-init-files)
      (load-file (format "%s/%s" init-dir f) )))

  (let ((emacs-erc-init-files '(
                                erc-connect.el
                                erc-incal.el
                                erc-iterate.el
                                erc-kill.el
                                erc-misc.el
                                erc-spell.el
                                )))
    (dolist (f emacs-erc-init-files)
      (load-file (format "%s/%s" erc-dir f) )))

  (let ((emacs-gnus-init-files '(
                                 article.el
                                 browse.el
                                 gnus-incal.el
                                 gnus-server.el
                                 group-summary.el
                                 group.el
                                 mail-to-many.el
                                 mail.el
                                 mailrc.el
                                 message-dynamic.el
                                 message-incal.el
                                 moggle.el
                                 summary.el
                                 )))
    (dolist (f emacs-gnus-init-files)
      (load-file (format "%s/%s" gnus-dir f) )))

  (let ((emacs-ide-init-files '(
                                c-and-cpp.el
                                compile-incal.el
                                elisp.el
                                find-command.el
                                html.el
                                ide.el
                                lisp-incal.el
                                )))
    (dolist (f emacs-ide-init-files)
      (load-file (format "%s/%s" ide-dir f) )))

  (let ((emacs-w3m-init-files '(
                                bookmarks.el
                                dl.el
                                history.el
                                w3m-incal.el
                                w3m-keys.el
                                w3m-sem.el
                                w3m-tabs.el
                                w3m-unisearch.el
                                )))
    (dolist (f emacs-w3m-init-files)
      (load-file (format "%s/%s" w3m-dir f) )))
  )

https://dataswamp.org/~incal/conf/.emacs

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Modular Configuration: the preferable way.

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-06-21 05:57]:
> (let*((emacs-dir "~/.emacs.d")
>       (lisp-dir  (format "%s/lisp"       emacs-dir))
>       (init-dir  (format "%s/emacs-init" emacs-dir))
>       (erc-dir   (format "%s/erc"        init-dir))
>       (gnus-dir  (format "%s/gnus"       init-dir))
>       (ide-dir   (format "%s/ide"        init-dir))
>       (w3m-dir   (format "%s/w3m"        init-dir))
>       (dirs     (list
>                  lisp-dir
pp>                  init-dir
>                  erc-dir
>                  gnus-dir
>                  ide-dir
>                  w3m-dir
>                  )))
>   (dolist (d dirs)
>     (push d load-path) )
>   (let ((emacs-init-files '(
>                             abc.el
>                             align-incal.el
>                             batch.el
>                             bibtex-incal.el
>                             bike.el
>                             bmi.el
>                             buc.el

(✿╹◡╹) that does not seem to easy anything and it can be replaced with few lines. 

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>> (let*((emacs-dir "~/.emacs.d")
>>       (lisp-dir  (format "%s/lisp"       emacs-dir))
>>       (init-dir  (format "%s/emacs-init" emacs-dir))
>>       (erc-dir   (format "%s/erc"        init-dir))
>>       (gnus-dir  (format "%s/gnus"       init-dir))
>>       (ide-dir   (format "%s/ide"        init-dir))
>>       (w3m-dir   (format "%s/w3m"        init-dir))
>>       (dirs     (list
>>                  lisp-dir
>>                  init-dir
>>                  erc-dir
>>                  gnus-dir
>>                  ide-dir
>>                  w3m-dir
>>                  )))
>>   (dolist (d dirs)
>>     (push d load-path) )
>>   (let ((emacs-init-files '(
>>                             abc.el
>>                             align-incal.el
>>                             batch.el
>>                             bibtex-incal.el
>>                             bike.el
>>                             bmi.el
>>                             buc.el
>
> that does not seem to easy anything and it can be replaced
> with few lines.

?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Modular Configuration: the preferable way.

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-06-21 20:21]:
> Jean Louis wrote:
> 
> >> (let*((emacs-dir "~/.emacs.d")
> >>       (lisp-dir  (format "%s/lisp"       emacs-dir))
> >>       (init-dir  (format "%s/emacs-init" emacs-dir))
> >>       (erc-dir   (format "%s/erc"        init-dir))
> >>       (gnus-dir  (format "%s/gnus"       init-dir))
> >>       (ide-dir   (format "%s/ide"        init-dir))
> >>       (w3m-dir   (format "%s/w3m"        init-dir))
> >>       (dirs     (list
> >>                  lisp-dir
> >>                  init-dir
> >>                  erc-dir
> >>                  gnus-dir
> >>                  ide-dir
> >>                  w3m-dir
> >>                  )))
> >>   (dolist (d dirs)
> >>     (push d load-path) )

You could put all configuration directories under unified one, for
example:

~/.emacs.d/my-config/lisp
~/.emacs.d/my-config/erc-dir
~/.emacs.d/my-config/gnus-dir

etc.

Then just do:

(setq load-path (append load-path (delq nil (mapcar (lambda (f) (when (file-directory-p f) f))
                                      (directory-files "/home/admin/.emacs.d/my-config" t)))))

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>>>> (let*((emacs-dir "~/.emacs.d")
>>>>       (lisp-dir  (format "%s/lisp"       emacs-dir))
>>>>       (init-dir  (format "%s/emacs-init" emacs-dir))
>>>>       (erc-dir   (format "%s/erc"        init-dir))
>>>>       (gnus-dir  (format "%s/gnus"       init-dir))
>>>>       (ide-dir   (format "%s/ide"        init-dir))
>>>>       (w3m-dir   (format "%s/w3m"        init-dir))
>>>>       (dirs     (list
>>>>                  lisp-dir
>>>>                  init-dir
>>>>                  erc-dir
>>>>                  gnus-dir
>>>>                  ide-dir
>>>>                  w3m-dir
>>>>                  )))
>>>>   (dolist (d dirs)
>>>>     (push d load-path) )
>
> You could put all configuration directories under unified
> one, for example:
>
> ~/.emacs.d/my-config/lisp
> ~/.emacs.d/my-config/erc-dir
> ~/.emacs.d/my-config/gnus-dir

What are you talking about?

> etc.
>
> Then just do:
>
> (setq load-path (append load-path (delq nil (mapcar (lambda
> (f) (when (file-directory-p f) f))
>                                       (directory-files
> "/home/admin/.emacs.d/my-config" t)))))

?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Modular Configuration: the preferable way.

[Arthur Miller]

Emanuel Berg via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

> Hongyi Zhao wrote:
>
>> I noticed the following instructions/tools for Emacs modular
>> configuration:
>>
>> https://www.emacswiki.org/emacs/DotEmacsModular
>> https://www.emacswiki.org/emacs/DotEmacsStructuring
>> https://github.com/emacs-jp/init-loader
>>
>> There are so many ways to do this. What's the
>> preferable way?
>
> My personal favorite is the method below, I don't know if
> that's mentioned on any of your links.
>
> Note that every file is mentioned explicitly, that means if
> you are to search for an error (say do the so-called binary
> search) you don't have to comment in and out code that might
> have comments, headers, footers, and so on, instead you just
> comment out one or more filenames, and when the bug or
> misfeature is localized, it is as easy to enable them again.
>
> (let*((emacs-dir "~/.emacs.d")
>       (lisp-dir  (format "%s/lisp"       emacs-dir))
>       (init-dir  (format "%s/emacs-init" emacs-dir))
>       (erc-dir   (format "%s/erc"        init-dir))
>       (gnus-dir  (format "%s/gnus"       init-dir))
>       (ide-dir   (format "%s/ide"        init-dir))
>       (w3m-dir   (format "%s/w3m"        init-dir))

Isn't it better to use path formatting function instead of string
formatting to ensure platform indepent paths and such? Something like
this:

(let ((etc (expand-file-name "etc" user-emacs-directory)))
             (unless (file-directory-p etc)
               (make-directory etc))
             (setq show-paren-style 'expression

             .....
              
              backup-directory-alist `(("." . ,etc))
              custom-file (expand-file-name "emacs-custom.el" etc)
              abbrev-file-name (expand-file-name "abbrevs.el" etc)
              bookmark-default-file (expand-file-name "bookmarks" etc)))


https://github.com/amno1/.emacs.d/blob/main/init.org

Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

Arthur Miller wrote:

> Isn't it better to use path formatting function instead of
> string formatting to ensure platform indepent paths

If the path is a string and the filename is a string joining
them with "string formatting" is pretty natural I think?

Or what do you think one should do to put together for example
the complete path ~/.emacs.d/emacs-init/erc/erc-iterate.el ?

That path (and the entire) file isn't platform independent to
begin with and has never been used outside the Unix world BTW,
but why not add that for fun if there is a way that is
better...

-- 
underground experts united
https://dataswamp.org/~incal

RE: [External] : Re: Emacs Modular Configuration: the preferable way.

[Drew Adams]

> > Isn't it better to use path formatting function instead of
> > string formatting to ensure platform indepent paths
> 
> If the path is a string and the filename is a string joining
> them with "string formatting" is pretty natural I think?
> 
> Or what do you think one should do to put together for example
> the complete path ~/.emacs.d/emacs-init/erc/erc-iterate.el ?

Put together how? from what?

https://www.gnu.org/software/emacs/manual/html_node/elisp/File-Names.html

https://www.gnu.org/software/emacs/manual/html_node/elisp/File-Name-Components.html

> That path (and the entire) file isn't platform independent to
> begin with and has never been used outside the Unix world BTW,
> but why not add that for fun if there is a way that is
> better...

Actually, ~/.emacs.d/emacs-init/erc/erc-iterate.el
is completely _platform independent_ when used in
Emacs.  Emacs takes care of platform differences /
oddities automatically.

Automatically, that is, provided code uses Elisp
file-name functions (or the predefined commands
that use them)...

Yes, this is one of the things that the file-name
functions you seem so terrorized by do for you.

And this is presumably why the OP specifically
mentioned that for his own use he didn't need to
worry about such differences.  (Which is true, as
far as it goes.)

Is Emacs "outside the Unix world" because it does
this for you?  If so, so be it.

Re: [External] : Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

Drew Adams wrote:

> Actually, ~/.emacs.d/emacs-init/erc/erc-iterate.el is
> completely _platform independent_ when used in Emacs.
> Emacs takes care of platform differences /
> oddities automatically.
>
> Automatically, that is, provided code uses Elisp file-name
> functions (or the predefined commands that use them)...

But why don't put that in the functions where something can go
wrong because they are the ones that should be responsible for
that, using string functions on a string doesn't imply any
danger whatsoever so if one wants to be safe that doesn't make
it any more safe, because it is already safe.

-- 
underground experts united
https://dataswamp.org/~incal

Re: [External] : Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

>> Actually, ~/.emacs.d/emacs-init/erc/erc-iterate.el is
>> completely _platform independent_ when used in Emacs.
>> Emacs takes care of platform differences /
>> oddities automatically.
>>
>> Automatically, that is, provided code uses Elisp file-name
>> functions (or the predefined commands that use them)...
>
> But why don't put that in the functions where something can
> go wrong because they are the ones that should be
> responsible for that, using string functions on a string
> doesn't imply any danger whatsoever so if one wants to be
> safe that doesn't make it any more safe, because it is
> already safe.

Let's take a more practical approach, you mention file-name,
what I can see there are these function, some of them what
they do one can understand, assume, or guess, but I don't
understand 100% what one is supposed to do, send the path as
one fancies it to a function and get the same thing back, only
on some other system it is changed to work there? If so, what
function is that?

I still don't understand why this is done manually but it is
just a detail, I'll enclose it if that's how it is supposed to
look...

file-name-absolute-p
file-name-all-completions
file-name-as-directory
file-name-base
file-name-case-insensitive-p
file-name-completion
file-name-directory
file-name-extension
file-name-non-special
file-name-nondirectory
file-name-quote
file-name-quoted-p
file-name-sans-extension
file-name-sans-versions
file-name-shadow-mode
file-name-unquote

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Modular Configuration: the preferable way.

[Arthur Miller]

Emanuel Berg via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

> Arthur Miller wrote:
>
>> Isn't it better to use path formatting function instead of
>> string formatting to ensure platform indepent paths
>
> If the path is a string and the filename is a string joining
> them with "string formatting" is pretty natural I think?

In some functions it matters if you end filename with a slash or
not when you work with directories. Path formatting functions take care
of that for example. Personally I think it is less error prone, than
using string functions.

> Or what do you think one should do to put together for example
> the complete path ~/.emacs.d/emacs-init/erc/erc-iterate.el ?

I pasted you an example from my init file.

> That path (and the entire) file isn't platform independent to
> begin with and has never been used outside the Unix world BTW,
> but why not add that for fun if there is a way that is
> better...

It is up to you how you write your init file, but they say never say
never ... 

Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

> In some functions it matters if you end filename with
> a slash or not when you work with directories.

I can imagine the situation but if it should do different
things depending on if it is a directory or a regular file, it
is better for these functions themselves to determine what it
is, and not rely on the user of the function to insert a slash
to denote a directory...

>> Or what do you think one should do to put together for
>> example the complete path
>> ~/.emacs.d/emacs-init/erc/erc-iterate.el ?
>
> I pasted you an example from my init file.

All that can't possibly be less error prone than a simple
string concat with `format', besides what errors exactly are
we trying to avoid?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Modular Configuration: the preferable way.

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-06-22 03:04]:
> > In some functions it matters if you end filename with
> > a slash or not when you work with directories.
> 
> I can imagine the situation but if it should do different
> things depending on if it is a directory or a regular file, it
> is better for these functions themselves to determine what it
> is, and not rely on the user of the function to insert a slash
> to denote a directory...

(setq file "") ⇒ ""
(file-exists-p file) ⇒ t ;; this is funny to me, but I guess both
			 ;; directory and file are files...

(file-exists-p "/") ⇒ t  ;; aha, directory is file too...

Then:

(expand-file-name file) ⇒ "/home/data1/protected/tmp" ;; aha, this one is
						      ;; directory,
						      ;; that is why
						      ;; empty string
						      ;; is considered
						      ;; file.

(file-name-as-directory "") ⇒ "./" ;; now it makes more sense

and now we get it as directory with a slash:

(file-name-as-directory (expand-file-name "")) ⇒ "/home/data1/protected/tmp/"

(defun directory (file)
  (when (and (stringp file) (file-exists-p file))
    (file-name-as-directory (expand-file-name file))))

(directory "") ⇒ "/home/data1/protected/tmp/" ;; because that is my current directory

(directory "/home") ⇒ "/home/"

If directory does not exist, return NIL:

(directory "/home-is-not-here") ⇒ nil


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Emacs Modular Configuration: the preferable way.

[Arthur Miller]

Emanuel Berg via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

>> In some functions it matters if you end filename with
>> a slash or not when you work with directories.
>
> I can imagine the situation but if it should do different
> things depending on if it is a directory or a regular file, it
> is better for these functions themselves to determine what it
> is, and not rely on the user of the function to insert a slash
> to denote a directory...
>
>>> Or what do you think one should do to put together for
>>> example the complete path
>>> ~/.emacs.d/emacs-init/erc/erc-iterate.el ?
>>
>> I pasted you an example from my init file.
>
> All that can't possibly be less error prone than a simple
> string concat with `format', besides what errors exactly are
> we trying to avoid?

"All that" was far less than what you had :). I do think it is less error
prone.

It lets me type less of long paths, then if I was typing and concating
strings. As you see I have setup one directory, "etc", and everything is
relative to that one. Instead of typing and concating every file name
for every directory and file, I can use that parth as a variable. If i
decide to change that directory I don't need to do it in 20 different
places and re-type everything, I can just type it in one place. Even with
search-replace I think it is more error prone to have it the way you do,
than to have just one place to edit.

For me it is important to be able to clone my setup from git and just
generate init files without having to tweak much. Currently I am able to
actually do that on both my stationary gnu/linux computer and a Windows
laptop. Not hardcoding paths as you did is one part of that setup. Even
if you personally don't care about other OS:s then *nix, you may still
have different physical machines with slightly different setup, say some
simple Pi computer and more powerful desktop computer which might have
different capabilities and your Emacs init doing differnt things at
startup, with different paths, hardware etc.

I don't know, just how I am using my computer(s), you may of course
setup your init file the way you prefer. 

Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

Arthur Miller wrote:

>> All that can't possibly be less error prone than a simple
>> string concat with `format', besides what errors exactly
>> are we trying to avoid?
>
> "All that" was far less than what you had :). I do think it
> is less error prone.
>
> It lets me type less of long paths, then if I was typing and
> concating strings. As you see I have setup one directory,
> "etc", and everything is relative to that one. Instead of
> typing and concating every file name for every directory and
> file, I can use that parth as a variable. If i decide to
> change that directory I don't need to do it in 20 different
> places and re-type everything, I can just type it in one
> place. Even with search-replace I think it is more error
> prone to have it the way you do, than to have just one place
> to edit.

?

I'm not retyping any paths either, that's the whole point

      (lisp-dir  (format "%s/lisp"       emacs-dir))
      (init-dir  (format "%s/emacs-init" emacs-dir))
      (erc-dir   (format "%s/erc"        init-dir))
      (gnus-dir  (format "%s/gnus"       init-dir))
      (ide-dir   (format "%s/ide"        init-dir))
      (w3m-dir   (format "%s/w3m"        init-dir))

I retype variable names of the variables that hold them but
that's another thing, if I make a typo there Emacs should warn
me somewhere along the way.

And more than that, it is just a principle which is sound to
always uphold not duplicate the same data.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Modular Configuration: the preferable way.

[Eli Zaretskii]

> Date: Mon, 21 Jun 2021 04:56:04 +0200
> From:  Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org>
> 
> (let*((emacs-dir "~/.emacs.d")
>       (lisp-dir  (format "%s/lisp"       emacs-dir))
>       (init-dir  (format "%s/emacs-init" emacs-dir))
>       (erc-dir   (format "%s/erc"        init-dir))
>       (gnus-dir  (format "%s/gnus"       init-dir))
>       (ide-dir   (format "%s/ide"        init-dir))
>       (w3m-dir   (format "%s/w3m"        init-dir))

Why are you using 'format' where expand-file-name should be used?

Re: Emacs Modular Configuration: the preferable way.

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Mon, 21 Jun 2021 04:56:04 +0200
>> From:  Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org>
>> 
>> (let*((emacs-dir "~/.emacs.d")
>>       (lisp-dir  (format "%s/lisp"       emacs-dir))
>>       (init-dir  (format "%s/emacs-init" emacs-dir))
>>       (erc-dir   (format "%s/erc"        init-dir))
>>       (gnus-dir  (format "%s/gnus"       init-dir))
>>       (ide-dir   (format "%s/ide"        init-dir))
>>       (w3m-dir   (format "%s/w3m"        init-dir))
>
> Why are you using 'format' where expand-file-name should be used?

Does it really make a difference in a personal configuration, especially
when you don't use Windows or other non /-delimiting systems?

-- 
	Philip K.

Re: Emacs Modular Configuration: the preferable way.

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: help-gnu-emacs@gnu.org
> Date: Mon, 21 Jun 2021 12:45:49 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Date: Mon, 21 Jun 2021 04:56:04 +0200
> >> From:  Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org>
> >> 
> >> (let*((emacs-dir "~/.emacs.d")
> >>       (lisp-dir  (format "%s/lisp"       emacs-dir))
> >>       (init-dir  (format "%s/emacs-init" emacs-dir))
> >>       (erc-dir   (format "%s/erc"        init-dir))
> >>       (gnus-dir  (format "%s/gnus"       init-dir))
> >>       (ide-dir   (format "%s/ide"        init-dir))
> >>       (w3m-dir   (format "%s/w3m"        init-dir))
> >
> > Why are you using 'format' where expand-file-name should be used?
> 
> Does it really make a difference in a personal configuration, especially
> when you don't use Windows or other non /-delimiting systems?

(a) It's definitely slower; (b) it teaches you bad habits, which you
are likely to apply even in non-personal cases; (c) this here is a
public place where you affect others, not just your personal code on a
known flavor of systems.

RE: [External] : Re: Emacs Modular Configuration: the preferable way.

[Drew Adams]

> > > Why are you using 'format' where expand-file-name should be used?
> >
> > Does it really make a difference in a personal configuration, especially
> > when you don't use Windows or other non /-delimiting systems?
> 
> (a) It's definitely slower; (b) it teaches you bad habits, which you
> are likely to apply even in non-personal cases; (c) this here is a
> public place where you affect others, not just your personal code on a
> known flavor of systems.

+1.

It's common for those new to Elisp to look for, find,
and use string-manipulation functions to manipulate
file names and their components.  Unfortunately (but
understandably).

Those things are not just strings - they're strings
with particular meaning and behavior.  Which is why
Elisp has particular functions to support them.

Just-strings naivete is hard enough to guide folks
around (away from), without help sites such as this
reinforcing it, however inadvertently.  Guidance
and discussion here that prevents or corrects such
misunderstanding is helpful.

Once pointed to things like `expand-file-name', it
behooves you to use them.  You won't regret it, and
doing so will help others by passing along the tip.

Re: [External] : Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

Drew Adams wrote:

> It's common for those new to Elisp to look for, find, and
> use string-manipulation functions to manipulate file names
> and their components. Unfortunately (but understandably).

Why are filenames strings then?

Make a new type for them if you don't like one to use and
modify them as strings.

-- 
underground experts united
https://dataswamp.org/~incal

Re: [External] : Re: Emacs Modular Configuration: the preferable way.

[Eli Zaretskii]

> Date: Mon, 21 Jun 2021 18:51:38 +0200
> From:  Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org>
> 
> Drew Adams wrote:
> 
> > It's common for those new to Elisp to look for, find, and
> > use string-manipulation functions to manipulate file names
> > and their components. Unfortunately (but understandably).
> 
> Why are filenames strings then?

They aren't.  We represent file names as strings, but they aren't
_just_ strings.  In a way analogous to characters, which are not
_just_ integers.

FW: [External] : Re: Emacs Modular Configuration: the preferable way.

[Drew Adams]

> > It's common for those new to Elisp to look for, find, and
> > use string-manipulation functions to manipulate file names
> > and their components. Unfortunately (but understandably).
> 
> Why are filenames strings then?

A file name is a string.  A string is not necessarily
a file name.
 
> Make a new type for them if you don't like one to use
> and modify them as strings.

I don't care whether you use and modify file-name
strings as arbitrary strings.  Feel free to do that.

My advice was for anyone who's open to it.  It will
generally help people to use the file-name functions,
in my experience.  You can likely find stories of
problems users have run into by not doing so.  YMMV.

Anyone (including you) can submit an enhancement
request (`M-x report-emacs-bug') to do <whatever>,
including to add a file-name type.

Re: FW: [External] : Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

Drew Adams wrote:

> Anyone (including you) can submit an enhancement request
> (`M-x report-emacs-bug') to do <whatever>, including to add
> a file-name type.

Hah, I don't think one should do that, I think the functions
that are sensitive to, in your words, <whatever>, should - and
only at that point - verify their inputs - and not rely on the
user to do that way before and there is no saying what will
happen after that and in between.

If several functions have this sensitivity to people string
manipulating the string indata, maybe they can share
a function to sanitize it - I mean, wouldn't that make it
modular, even?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Modular Configuration: the preferable way.

[tomas]

[-- Attachment #1: Type: text/plain, Size: 888 bytes --]

On Mon, Jun 21, 2021 at 12:45:49PM +0000, Philip Kaludercic wrote:
> Eli Zaretskii <eliz@gnu.org> writes:

[...]

> > Why are you using 'format' where expand-file-name should be used?
> 
> Does it really make a difference in a personal configuration, especially
> when you don't use Windows or other non /-delimiting systems?

Of course, the OP can do what (s)he wants at home, but once the
code ends up in public, as above, such a hint as Eli's is invaluable.

If you want to see what happens otherwise, have a look at PHP.

The language itself has evolved a lot since its beginnings (to
the better, IMO). But you still see extremely bad habits "out
there" which wouldn't be necessary these days -- because, well,
they are "out there" (for example: assebling SQL queries with
sprintf [1]). They take a life of their own :-)

Cheers
[1] https://xkcd.com/327/
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

tomas wrote:

> Of course, the OP can do what (s)he wants at home, but once
> the code ends up in public, as above, such a hint as Eli's
> is invaluable.

It stinks.

> If you want to see what happens otherwise, have a look
> at PHP.
>
> The language itself has evolved a lot since its beginnings
> (to the better, IMO). But you still see extremely bad habits
> "out there" which wouldn't be necessary these days --
> because, well, they are "out there" (for example: assebling
> SQL queries with sprintf [1]). They take a life of their own
> :-)

If it is string to begin with and the end result is a string
one should be able to use string functions to "assemble" it.

It is just as natural as using addition and subtraction with
integers.

Read the book on type theory if you don't believe me! And the
Wikipedia page on type systems.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Modular Configuration: the preferable way.

[Eli Zaretskii]

> Date: Mon, 21 Jun 2021 18:47:17 +0200
> From:  Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org>
> 
> If it is string to begin with and the end result is a string
> one should be able to use string functions to "assemble" it.

Repeat after me: "filenames are not strings", "filenames are not
strings", "filenames...

Bonus points for coming up with at least one use case which shows how
filenames are not just strings.

Re: Emacs Modular Configuration: the preferable way.

[Jean Louis]

* Eli Zaretskii <eliz@gnu.org> [2021-06-21 21:07]:
> > Date: Mon, 21 Jun 2021 18:47:17 +0200
> > From:  Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org>
> > 
> > If it is string to begin with and the end result is a string
> > one should be able to use string functions to "assemble" it.
> 
> Repeat after me: "filenames are not strings", "filenames are not
> strings", "filenames...
> 
> Bonus points for coming up with at least one use case which shows how
> filenames are not just strings.

As an irrelevant side note, this one bothers me:

(file-exists-p "") ⇒ t

Is that supposed to be so?

Re: Emacs Modular Configuration: the preferable way.

[Eli Zaretskii]

> Date: Tue, 22 Jun 2021 00:09:40 +0300
> From: Jean Louis <bugs@gnu.support>
> Cc: help-gnu-emacs@gnu.org
> 
> As an irrelevant side note, this one bothers me:
> 
> (file-exists-p "") ⇒ t
> 
> Is that supposed to be so?

Yes, there are good practical reasons for this behavior.

Repeat after me: "file names are not just strings"...

Re: Emacs Modular Configuration: the preferable way.

[Jean Louis]

* Eli Zaretskii <eliz@gnu.org> [2021-06-22 14:46]:
> > Date: Tue, 22 Jun 2021 00:09:40 +0300
> > From: Jean Louis <bugs@gnu.support>
> > Cc: help-gnu-emacs@gnu.org
> > 
> > As an irrelevant side note, this one bothers me:
> > 
> > (file-exists-p "") ⇒ t
> > 
> > Is that supposed to be so?
> 
> Yes, there are good practical reasons for this behavior.
> 
> Repeat after me: "file names are not just strings"...

I understand that "" is meant to represent ./

Re: Emacs Modular Configuration: the preferable way.

[Eli Zaretskii]

> Date: Tue, 22 Jun 2021 15:29:38 +0300
> From: Jean Louis <bugs@gnu.support>
> Cc: help-gnu-emacs@gnu.org
> 
> * Eli Zaretskii <eliz@gnu.org> [2021-06-22 14:46]:
> > > Date: Tue, 22 Jun 2021 00:09:40 +0300
> > > From: Jean Louis <bugs@gnu.support>
> > > Cc: help-gnu-emacs@gnu.org
> > > 
> > > As an irrelevant side note, this one bothers me:
> > > 
> > > (file-exists-p "") ⇒ t
> > > 
> > > Is that supposed to be so?
> > 
> > Yes, there are good practical reasons for this behavior.
> > 
> > Repeat after me: "file names are not just strings"...
> 
> I understand that "" is meant to represent ./

Yes, that's true.

Re: Emacs Modular Configuration: the preferable way.

[Stefan Monnier via Users list for the GNU Emacs text editor]

> If it is string to begin with and the end result is a string
> one should be able to use string functions to "assemble" it.

Yet `lisp/emacs-lisp/bytecomp.el` doesn't use this strategy in order
to transform the string found in `foo.el` into the string we want in
`foo.elc`.


        Stefan

Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

Stefan Monnier via Users list for the GNU Emacs text editor wrote:

>> If it is string to begin with and the end result is
>> a string one should be able to use string functions to
>> "assemble" it.
>
> Yet `lisp/emacs-lisp/bytecomp.el` doesn't use this strategy
> in order to transform the string found in `foo.el` into the
> string we want in `foo.elc`.

It doesn't?

~/src/emacs-28.0.50/lisp/emacs-lisp/bytecomp.el lines 182-186:

    (setq filename (file-name-sans-versions
		    (byte-compiler-base-file-name filename)))
    (cond ((string-match emacs-lisp-file-regexp filename)
	   (concat (substring filename 0 (match-beginning 0)) ".elc"))
	  (t (concat filename ".elc")))))

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Modular Configuration: the preferable way.

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-06-21 20:07]:
> > The language itself has evolved a lot since its beginnings
> > (to the better, IMO). But you still see extremely bad habits
> > "out there" which wouldn't be necessary these days --
> > because, well, they are "out there" (for example: assebling
> > SQL queries with sprintf [1]). They take a life of their own
> > :-)
> 
> If it is string to begin with and the end result is a string
> one should be able to use string functions to "assemble" it.

I am thinking how can I make it safer for SQL queries. It seem
not an easy task. Major updating function is using this:

(let* ((table "new")
       (column "new_name")
       (new-value "'Joe'")
       (id 1)
       (sql (format "UPDATE %s SET %s = %s WHERE %s_id = %s RETURNING %s_id" table column new-value table id table)))
  (message sql)
  (rcd-sql-first sql db)) ⇒ 1

Then I have to convert it to following by its meaning:

(let* ((table "new")
       (column "new_name")
       (new-value "'Joe'")
       (id 1)
       (parameters (list table column new-value id))
       (sql "UPDATE $1 SET $2 = $3 WHERE $1_id = $4 RETURNING $1_id"))
  (message sql)
  (rcd-sql-first sql db parameters))

But no, that does not work:

if: Wrong type argument: stringp, ("ERROR:  syntax error at or near \"$1\"
LINE 1: UPDATE $1 SET $2 = $3 WHERE $1_id = $4 RETURNING $1_id
               ^
" "42601")

As those paramters are probably converted to strings. Thus I
cannot avoid using the function `format' just everywhere, but I
can minimize it wherever there is possible danger for SQL
injection (though this below is not working):

(let* ((table "new")
       (column "new_name")
       (new-value "'Joe'")
       (id 1)
       (parameters (list new-value id))
       (sql (format "UPDATE %s SET %s = $1 WHERE %s_id = $2 RETURNING %s_id" table column table table)))
  (message sql)
  (rcd-sql-first sql db parameters))

Maybe solution would be to use `format' in steps, so that final
step can accept users' input.

Issue is not solved. First I have to contact developers of
`emacs-libpq' package to see if this is error, as it returns
string by supplying integer parameter:

This is not expected:

(pq:query db "SELECT $1" 100) ⇒ ("100")

While this is expected:

(pq:query db "SELECT $1" "100") ⇒ ("100")

So the issue is pending on Github:
https://github.com/anse1/emacs-libpq/issues/19

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>>> The language itself has evolved a lot since its beginnings
>>> (to the better, IMO). But you still see extremely bad
>>> habits "out there" which wouldn't be necessary these days
>>> -- because, well, they are "out there" (for example:
>>> assebling SQL queries with sprintf [1]). They take a life
>>> of their own :-)
>> 
>> If it is string to begin with and the end result is
>> a string one should be able to use string functions to
>> "assemble" it.
>
> I am thinking how can I make it safer for SQL queries.

SQL injection isn't avoided by not assembling queries with
string functions but by quoting user input.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Printf and quoting in general, SQL injection in particular

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-06-22 03:35]:
> > I am thinking how can I make it safer for SQL queries.
> 
> SQL injection isn't avoided by not assembling queries with
> string functions but by quoting user input.

It is impossible in `emacs-libpq' package to avoid formatting strings
and passing it to database.

What is possible is to minimize it so that users' input is
automatically quoted by the database by passing it as parameters
instead of passing data as parameters to `format'.

I prefer the latter. There is less code. I have improved after
Thomas's suggestions.

Now I am preparings statements:

(defun rcd-db-prepare-statement (name prepared pg)
  (unless
      (rcd-sql-first "SELECT statement FROM pg_prepared_statements WHERE name = $1" pg name)
    (rcd-sql prepared pg)))

(defun rcd-db-prepare-statements ()
  (rcd-db-prepare-statement "persons_emails" "PREPARE persons_emails(int) AS SELECT DISTINCT unnest(array[people_email1, people_email2, people_email3] || people_emailsobsolete) FROM people WHERE people_id = $1"))

Now function is small and nice:

(defun cf-emails-by-id (id)
  "Returns list of emails for contact ID"
  (delq nil (pq:query cf-db (format "EXECUTE persons_emails(%s)" id))))

and it was this big and without true necessity complex:

(defun cf-emails-by-id (id)
  "Returns list of emails for contact ID"
  (let* ((sql (format "SELECT people_email1, people_email2, people_email3 FROM people WHERE people_id = %s" id))
	 (emails (rcd-sql-first sql cf-db))
	 (obsolete-emails (rcd-db-array-value-as-list "people" "people_emailsobsolete" id cf-db))
	 (emails (append emails obsolete-emails))
	 (emails (seq-remove 'seq-empty-p emails))
	 (emails (mapcar (lambda (e) (when (string-match "@" e) e)) emails))
	 (emails (remove nil emails)))
    emails))


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Printf and quoting in general, SQL injection in particular

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>>> I am thinking how can I make it safer for SQL queries.
>> 
>> SQL injection isn't avoided by not assembling queries with
>> string functions but by quoting user input.
>
> It is impossible in `emacs-libpq' package to avoid
> formatting strings and passing it to database.
>
> What is possible is to minimize it so that users' input is
> automatically quoted by the database by passing it as
> parameters instead of passing data as parameters to `format'
> [...]

Relax, this notion that you shouldn't construct file paths by
string functions, nor SQL queries for that matter, and what
more? hyperlinks? or are you allowed to do that?

These opinions are "arguably" correct at best - and that means
some people will insist (argue) they are. And maybe that's
what's happening right now?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Printf and quoting in general, SQL injection in particular

[Yuri Khan]

On Sat, 26 Jun 2021 at 13:56, Emanuel Berg via Users list for the GNU
Emacs text editor <help-gnu-emacs@gnu.org> wrote:

> Relax, this notion that you shouldn't construct file paths by
> string functions, nor SQL queries for that matter, and what
> more? hyperlinks?

Hyperlinks, too.

One of the requirements of URLs is that all non-ascii and some ascii
characters be %-encoded when used in the path or query string, or
punycode-encoded when used in the host name:

    (let ((base "http://ru.wikipedia.org/wiki/")
          (term "Гиперссылка")
          (joined (concat base term)))
      (assert (string= joined
"https://ru.wikipedia.org/wiki/%D0%93%D0%B8%D0%BF%D0%B5%D1%80%D1%81%D1%81%D1%8B%D0%BB%D0%BA%D0%B0")))
 ;; alas, no

Another rule is that resolving a relative reference containing a path
against a base URL will drop the last segment of the base:

    (let ((base "http://example.org/foo")
          (href "bar")
          (resolved (concat base href)))
      (assert (string= resolved "http://example.org/bar")))  ;; also no

String concat does not know any of these rules.


It is okay to represent file names, SQL queries, and URLs as strings
*internally*. It is okay to use string functions to implement
high-level domain-specific functions. In many cases, interoperating
with external code will also require these things represented as
strings. But it is a good idea to use domain-specific functions to
manipulate file names, queries, and URLs, rather than string
functions, because this way you are less likely to violate those
types’ invariants.

Re: Printf and quoting in general, SQL injection in particular

[Emanuel Berg via Users list for the GNU Emacs text editor]

Yuri Khan wrote:

>> Relax, this notion that you shouldn't construct file paths
>> by string functions, nor SQL queries for that matter, and
>> what more? hyperlinks?
>
> Hyperlinks, too.
>
> One of the requirements of URLs is that all non-ascii and
> some ascii characters be %-encoded when used in the path or
> query string, or punycode-encoded when used in the host
> name:
>
>     (let ((base "http://ru.wikipedia.org/wiki/")
>           (term "Гиперссылка")
>           (joined (concat base term)))
>       (assert (string= joined
> "https://ru.wikipedia.org/wiki/%D0%93%D0%B8%D0%BF%D0%B5%D1%80%D1%81%D1%81%D1%8B%D0%BB%D0%BA%D0%B0")))
>  ;; alas, no
>
> Another rule is that resolving a relative reference
> containing a path against a base URL will drop the last
> segment of the base:
>
>     (let ((base "http://example.org/foo")
>           (href "bar")
>           (resolved (concat base href)))
>       (assert (string= resolved "http://example.org/bar")))  ;; also no
>
> String concat does not know any of these rules.
>
> It is okay to represent file names, SQL queries, and URLs as
> strings *internally*. It is okay to use string functions to
> implement high-level domain-specific functions. In many
> cases, interoperating with external code will also require
> these things represented as strings. But it is a good idea
> to use domain-specific functions to manipulate file names,
> queries, and URLs, rather than string functions, because
> this way you are less likely to violate those
> types’ invariants.

Interesting, what about typing? Not as error prone, because
you see the result?

So it is like a three stage hierarchy or procedure rather,

  1. typing (just type, you see it anyway)
  2. manipulate (domain-specific functions)
  3. use

(For hyperlinks, step 1 could be done with a lot of functions
(protocol (domain (path (page (page-type ...) ? That'd just be
grotesque. I don't know if that'd be more safe, really?)

But OK, so instead of "anything goes, then verify input at
step 3" - which is where it matters BTW - the approach is
_type carefully_, then just exclusively use domain-specific
manipulators. So all you have to care about is the typing
part, because after that, it won't get messed up!

I see the point, but - to be honest, string manipulation at
this very basic level - and typing - I don't know, I feel kind
of comfortable doing BOTH by now :) But never say never...

I still don't understand what to replace my stuff with tho,

(let*((emacs-dir "~/.emacs.d")
      (lisp-dir  (format "%s/lisp"       emacs-dir))
      (init-dir  (format "%s/emacs-init" emacs-dir))
      (erc-dir   (format "%s/erc"        init-dir))
      (gnus-dir  (format "%s/gnus"       init-dir))
      (ide-dir   (format "%s/ide"        init-dir))
      (w3m-dir   (format "%s/w3m"        init-dir))
      (dirs     (list
                 lisp-dir
                 init-dir
                 erc-dir
                 gnus-dir
                 ide-dir
                 w3m-dir
                 ))) ...

-- 
underground experts united
https://dataswamp.org/~incal

Re: Printf and quoting in general, SQL injection in particular

[tomas]

[-- Attachment #1: Type: text/plain, Size: 945 bytes --]

On Sat, Jun 26, 2021 at 02:30:59PM +0700, Yuri Khan wrote:
> On Sat, 26 Jun 2021 at 13:56, Emanuel Berg via Users list for the GNU
> Emacs text editor <help-gnu-emacs@gnu.org> wrote:
> 
> > Relax, this notion that you shouldn't construct file paths by
> > string functions, nor SQL queries for that matter, and what
> > more? hyperlinks?
> 
> Hyperlinks, too.

Mmm. Yummy hyperlinks. You just have to enter "URL parsing injection" to
enjoy a colourful bestiary. This is user-provided stuff which is parsed
server-side. Creativity!

Two nice links (of... thousands?)

  https://s1gnalcha0s.github.io/node/2015/01/31/SSJS-webshell-injection.html
  https://www.blackhat.com/docs/us-17/thursday/us-17-Tsai-A-New-Era-Of-SSRF-Exploiting-URL-Parser-In-Trending-Programming-Languages.pdf

I'm all for DIY, but in this case, it comes with one caveat. Know your
stuff. Read. Have good data models. Read. Test. Read.

Have fun
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Printf and quoting in general, SQL injection in particular

[Jean Louis]

* Emanuel Berg <moasenwood@zoho.eu> [2021-06-26 02:55]:
> Relax, this notion that you shouldn't construct file paths by
> string functions, nor SQL queries for that matter, and what
> more? hyperlinks? or are you allowed to do that?
> 
> These opinions are "arguably" correct at best - and that means
> some people will insist (argue) they are. And maybe that's
> what's happening right now?

We can observe the phenomena of raised warnings for various particular
issues in programming. I am very fine with warnings and learn out of
it.

What I don't agree easily is sometimes how those warnings are
formulated as there is usually deeper truthful meaning that could not
express by poster clearly in the email. 

Sometimes warnings are in a wrong context.

Whatever it may be, one thing is sure, there must be some truth there
as warnings are given by experienced people.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Printf and quoting in general, SQL injection in particular

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

> Whatever it may be, one thing is sure, there must be some
> truth there as warnings are given by experienced people.

There is the Windows compatibility issue, that I accept.

As for extra security in general that should be put in
manually and in-between when what happens is as undramatic as
a manipulation of a string - ha! - i.e., something that is
100% safe - I'm not convinced that actually adds security ...

Maybe a false sense of security :P

Maybe the functions are useful for other things as well tho,
that's another thing, if so.

Don't know what functions this refers to specifically, please
provide a list for paths and URLs and I'll check my material...

-- 
underground experts united
https://dataswamp.org/~incal

Re: Printf and quoting in general, SQL injection in particular

[Emanuel Berg via Users list for the GNU Emacs text editor]

You guys spoke of symbols the other day and if they were
mutable or not.

Maybe here we have a use case... for, uhm, symbols!

Maybe one can do this like this with properties, tag strings
that are intended for or already paths and URLs, and they can
only be used by safe functions!

That I'd like more because that would imply a higher degree of
automation, perhaps, but if nothing else the policy would
be enforced, done deal.

So when you sent an ordinary string to the file opener -
¡No pasarán! ha :)

Compare

(defun what-face (pos)
  (interactive "d")
  (let ((face (or (get-char-property pos 'face)
                  (get-char-property pos 'read-cf-name) )))
    (message "face: %s" (or face "no face")) ))

or

(defun gnus-article-show-filled-citation ()
  "Fill the citation opened on a hidden-citation button press."
  (interactive)
  (let*((pos    (point))
        (button (get-char-property pos 'button)) )
    (when button
      (widget-button-press pos)
      (save-excursion
        (gnus-article-fill-cited-article) ))))

;; source: 
;;   https://dataswamp.org/~incal/emacs-init/face.el
;;   https://dataswamp.org/~incal/emacs-init/gnus/article.el

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Modular Configuration: the preferable way.

[Jean Louis]

* tomas@tuxteam.de <tomas@tuxteam.de> [2021-06-21 17:12]:
> But you still see extremely bad habits "out there" which wouldn't be
> necessary these days -- because, well, they are "out there" (for
> example: assebling SQL queries with sprintf [1]). They take a life
> of their own :-)
> 
> Cheers
> [1] https://xkcd.com/327/

Your small reference is definitely a possible danger if SQL input is
anyhow exposed to public input. Within a close group or within a team
the danger mentioned on the funny comic is practically non-existent as
it will never take place on my side. It is highly unlikely to take
place within third party Emacs Lisp collection of programs which are
so much single user oriented. But then again, we never know it, and it
is a bad habit.

I am heavy user of the Emacs package: emacs-libpq @ Github
https://github.com/anse1/emacs-libpq

Your comment is important.
━━━━━━━━━━━━━━━━━━━━━━━━━━

 I just guess that the package's original
command: `pq:query' is so much safer than what I re-wrote:

(defun rcd-sql (sql pg)
  "Sends SQL queries to PostgreSQL database and return results.
Argument PG is database handle."
  (prog1
      (condition-case err
	    (pq:query pg sql)
	(error
	 (if (string-match "^ERROR:  syntax error" (cdr err))
	     (progn
	       (if (fboundp 'speak) (speak (cdr err)))
	       (message (cdr err)))
	   ;; re-throw
	   (signal (car err) (cdr err)))))
    (when rcd-db-sql-logging
      (funcall rcd-db-sql-message-function (string-replace "\n" " " sql)))))

Thus I guess I would need to skip in some functions usage of function
`format' and rather use the `pq:query' parameters:

Then function should begin with:

(defun rcd-sql (sql pg &rest parameters)
  "Sends SQL queries to PostgreSQL database and return results.
Argument PG is database handle."
  (prog1
      (condition-case err
	    (apply 'pq:query pg sql parameters)

(setq db (rcd-db-connect "admin"))
db ⇒ #<user-ptr ptr=0x56037dece650 finalizer=0x7fafbd3dabb6>

Then for the following, where both tables `data` and `data1' exist:

(rcd-sql-first
 (format "INSERT INTO data (data_name) VALUES (%s) RETURNING data_id" (sql-escape-string "John"))
 db) ⇒ 16 as ID

Attempt to ruin the table did not really work as there is error,
and I don't know how to drop it maliciously. If you have idea let
me know.

(rcd-sql-first
 (format "INSERT INTO data (data_name) VALUES (%s)" "'John'); DROP TABLE data1;")
 db)

But the idea is to use the arguments as they are automatically
quoted by `pq:query' and I just hope there is some
more "protection":

(defun rcd-sql (sql pg &rest parameters)
  "Sends SQL queries to PostgreSQL database and return results.
Argument PG is database handle."
  (prog1
      (condition-case err
	    (apply 'pq:query pg sql parameters)
	(error
	 (if (string-match "^ERROR:  syntax error" (cdr err))
	     (progn
	       (if (fboundp 'speak) (speak (cdr err)))
	       (message (cdr err)))
	   ;; re-throw
	   (signal (car err) (cdr err)))))
    (when rcd-db-sql-logging
      (funcall rcd-db-sql-message-function (string-replace "\n" " " sql)))))

That it works preliminary:

(rcd-sql "SELECT 1" db) ⇒ (1)

And now with parameters, I see I am getting a string which was
meant to be integer, this may be bug in the package:

(rcd-sql "SELECT $1" db 1) ⇒ ("1")

But then I can cast it to integer:

(rcd-sql "SELECT $1::integer" db 1) ⇒ (1)

Now again the attempt to drop the table:

(rcd-sql "SELECT $1::integer" db "1; DROP TABLE data1;") - invalid input syntax

New attempt, it did not work:

(rcd-sql "SELECT $1" db "1; DROP TABLE data1;") ⇒ ("1; DROP TABLE data1;")

Let us try with function `format' instead: ⛳ ⛳ ⛳ ⛳ ⛳

(rcd-sql (format "SELECT %s" "1; DROP TABLE data1;") db) ⇒ nil

Bingo! This worked well. Let me try to destroy it by using parameters, again:

(rcd-sql "SELECT $1" db "1; DROP TABLE data1;") ⇒ ("1; DROP TABLE data1;")

That gives me only 249 `format' issues to verify and sanitize in
a major file and probably about 200 other functions.

Not that I was not thinking about this, I was thinking and I knew
it is waiting for me. But I did not ackle it. Now when you
mentioned it I feel I have to do it and use the parameters to the
C function exposed in Emacs Lisp instead of the function
`format'.

249 matches for "(sql (format" in buffer: rcd-cf.el
    222:	  (let* ((sql (format "INSERT INTO people (people_firstname, people_middlenames, people_lastname, people_email1, people_account1, people_description) VALUES (%s, %s, %s, '%s', %s, '%s')" first-name middle-names last-name email account description)))
    229:	 (sql (format "SELECT a.attname,
    378:  (let* ((sql (format "SELECT people_email1, people_email2, people_email3 FROM people WHERE people_id = %s" id))
    525:	 (sql (format "SELECT get_full_contacts_name(%s) FROM people WHERE people_id = %s" id id))
    549:  (let* ((sql (format "SELECT people_id FROM people WHERE people_email1 ILIKE '%s' OR people_email2 ILIKE '%s' OR people_email3 ILIKE '%s' OR '%s' = ANY (people_emailsobsolete)" email email email email))
    562:  (let* ((sql (format "SELECT people_id FROM people WHERE people_email1 ILIKE '%s' OR people_email2 ILIKE '%s' OR people_email3 ILIKE '%s' OR '%s' = ANY (people_emailsobsolete)" email email email email)))
    568:	 (sql (format "SELECT people_id FROM people WHERE people_officephone ~ '%s' OR people_mobilephone ~ '%s' OR people_homephone ~ '%s' OR people_otherphone ~ '%s' OR people_fax ~ '%s' OR '%s' = ANY (people_phoneobsolete)" number number number number number original-number))
    579:  (let ((sql (format "INSERT INTO contacts (people_lastname, people_mobilephone) VALUES (%s, %s) RETURNING people_id" (sql-escape-string number) (sql-escape-string number))))
    622:  (let ((sql (format "SELECT people_id FROM people WHERE (people_account1 = %s OR people_account2 = %s OR people_account3 = %s) AND %s ~* %s ORDER BY people_id" account account account column (sql-escape-string query))))
    661:  (let ((sql (format "SELECT count(notes_id) FROM notes WHERE notes_contact = %s" id)))
    665:  (let ((sql (format "SELECT count(markassignments_id) FROM markassignments WHERE markassignments_contact = %s" id)))
    669:  (let ((sql (format "SELECT count(1) FROM hyobjects WHERE hyobjects_people = %s OR hyobjects_assignedperson = %s" id id)))
    673:  (let ((sql (format "SELECT count(people_id) FROM people WHERE people_introducedby = %s" id)))
    677:  (let ((sql (format "SELECT count(calls_id) FROM calls WHERE calls_contact = %s" id)))
    681:  (let ((sql (format "SELECT count(sms_id) FROM sms WHERE sms_contacts = %s" id)))
    711:	 (sql (format "INSERT INTO interactions (interactions_contacts, interactions_interactiontypes, interactions_count) VALUES (%s, %s, %s) ON CONFLICT (interactions_contacts,interactions_interactiontypes) DO UPDATE SET interactions_count = %s WHERE interactions.interactions_contacts = %s AND interactions.interactions_interactiontypes = %s;" id type count count id type)))
    760:  (let* ((sql (format "SELECT DISTINCT people_id as id FROM
    814:  (let* ((sql (format "SELECT tags_name FROM peopletags, tags WHERE tags_id = peopletags_tags AND peopletags_%s = %s" table id))
    820:  (let ((sql (format "INSERT INTO peopletags (peopletags_%s, peopletags_tags) VALUES (%d, %d) ON CONFLICT (peopletags_%s, peopletags_tags) DO NOTHING RETURNING peopletags_id " table id tag table)))
    935:  (let ((sql (format "SELECT people_id, get_full_contacts_name(people_id), coalesce(get_accounts_name(people_account1),'UNKNOWN') FROM people WHERE people_id IN (%s) ORDER BY people_id" (rcd-sql-id-list list))))
    996:  (let* ((sql (format "SELECT CASE WHEN people_invalid1 IS NOT TRUE AND people_email1 ~ '@' THEN people_email1 WHEN people_invalid2 IS NOT TRUE AND people_email2 ~ '@' THEN people_email2 WHEN people_invalid3 IS NOT TRUE AND people_email3 ~ '@' THEN people_email3 ELSE NULL END AS email FROM people WHERE people_id = %s ORDER BY people_id LIMIT 1" contact)))
   1008:  (let* ((sql (format "SELECT CASE WHEN people_invalid1 IS NOT TRUE AND people_email1 ~ '@' THEN people_email1 ELSE NULL END AS email1, CASE WHEN people_invalid2 IS NOT TRUE AND people_email2 ~ '@' THEN people_email2 ELSE NULL END AS email2, CASE WHEN people_invalid3 IS NOT TRUE AND people_email3 ~ '@' THEN people_email3 ELSE NULL END AS email FROM people WHERE people_id = %s ORDER BY people_id LIMIT 1" contact)))
   1013:  (let* ((sql (format "SELECT people_officephone, people_mobilephone, people_homephone, people_otherphone, people_fax FROM people WHERE people_id = %s" contact)))
   1056:	 (sql (format "INSERT INTO sms (sms_contacts, sms_smsstatus, sms_body, sms_phone) VALUES (%s, %s, %s, '%s') RETURNING sms_id" contact status (sql-escape-string body) phone)))
   1061:      (let* ((sql (format "SELECT sms_datecreated, sms_body, sms_phone FROM sms WHERE sms_datecreated = '%s' AND sms_body = %s" date (sql-escape-string text)))
   1071:      (let ((sql (format "INSERT INTO sms (sms_datecreated, sms_contacts, sms_smsstatus, sms_body, sms_phone) VALUES ('%s', %s, %s, %s, '%s') RETURNING sms_id" date contact sms-type (sql-escape-string text) phone)))
   1174:      (let* ((sql (format "INSERT INTO fromidentities VALUES (DEFAULT, %s, %s, NULL, NULL, NULL) ON CONFLICT(fromidentities_contacts) DO UPDATE SET fromidentities_identities = %s WHERE fromidentities.fromidentities_contacts = %s RETURNING fromidentities_id;" contact id id contact)))
   1197:	 (sql (format "SELECT identities_id, concat_ws(', ',identities_name, identities_firstname, identities_lastname, identities_email) FROM identities WHERE identities_id IN (%s)" list))
   1208:	(let* ((sql (format "UPDATE accounts SET accounts_identity = %s WHERE accounts_id = %s" identity id))
   1246:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id) || ', ' || interactions_count FROM people, interactions WHERE interactions_count >= %s %s AND people_id = interactions_contacts ORDER BY interactions_count DESC LIMIT %s" interactions-min account limit)))
   1252:	 (sql (format "SELECT accounts_id, accounts_name FROM accounts WHERE accounts_name ~* %s" query)))
   1277:	 (sql (format "SELECT * FROM %s_combo" table)))
   1310:	 (sql (format "SELECT * FROM %s_combo ORDER BY id DESC" table))
   1327:	 (sql (format "SELECT * FROM %s_combo ORDER BY id DESC" table)))
   1397:	     (sql (format "INSERT INTO litems (litems_name, litems_currency, litems_purchasingvalue, litems_marketvalue, litems_salesvalue, litems_count, litems_lists) VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING litems_id" name currency purchasing-value market-value sales-value count list))
   1406:      (let ((sql (format "SELECT litems_id, litems_name FROM litems, lists WHERE litems_lists = lists_id AND litems_lists = %s" id)))
   1471:	 (sql (format "SELECT litems_name, litems_description, litems_url, litems_subtitle, litems_nofollow, litems_dateeffective FROM litems WHERE litems_lists = %s ORDER BY litems_priority, litems_id" id))
   1534:	(let* ((sql (format "INSERT INTO peoplegroupmembers (peoplegroupmembers_person, peoplegroupmembers_peoplegroups) VALUES (%s, %s) RETURNING peoplegroupmembers_id" (pop marked) group))
   1669:	 (sql (format "SELECT contactskills_contacts, get_full_contacts_name(contactskills_contacts) FROM contactskills WHERE contactskills_skills = %s" skill))
   1677:    (let* ((sql (format "SELECT contactskills_contacts, get_full_contacts_name(contactskills_contacts) FROM contactskills WHERE contactskills_skills = %s" id)))
   1695:  (let* ((sql (format "SELECT people_id, get_full_contacts_name(people_id) || ' ' || people_fax FROM people WHERE people_fax ~ '[0-9]' AND ((people_account1 = %s OR people_account2 = %s OR people_account3 = %s) OR (SELECT mailingsubscriptions_contacts FROM mailingsubscriptions WHERE mailingsubscriptions_contacts = people_id AND mailingsubscriptions_accounts = %s) = 1);" id id id id))
   1709:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id), coalesce(get_accounts_name(people_account1),'UNKNOWN') FROM people WHERE people_description ~* %s" query)))
   1717:	(let ((sql (format "UPDATE people SET people_account1 = %s WHERE people_account1 = %s" id other-account)))
   1786:	     (sql (format "SELECT people_id, get_full_contacts_name(people_id), coalesce(get_accounts_name(people_account1),'UNKNOWN') FROM people WHERE %s" where))
   1852:      (let* ((sql (format "DELETE FROM %s WHERE %s_%s = %s AND %s_tags = %s" table table foreign id table tag-id)))
   1860:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id), coalesce(get_accounts_name(people_account1),'UNKNOWN') FROM people WHERE people_description ~* %s AND (people_account1 = %s OR people_account2 = %s OR people_account3 = %s)" query account account account)))
   1924:  (let ((sql (format
   2189:  (let* ((sql (format "SELECT mailingsubscriptions_id FROM mailingsubscriptions WHERE mailingsubscriptions_accounts = %s AND mailingsubscriptions_contacts = %s" mid cid))
   2195:         (sql (format "UPDATE mailingsubscriptions SET mailingsubscriptions_donotemail = TRUE, mailingsubscriptions_email = '%s', mailingsubscriptions_relatedemail = '%s', mailingsubscriptions_dateunsubscribed = now() WHERE mailingsubscriptions_accounts = %s AND mailingsubscriptions_contacts = %s" email eid mid cid)))
   2200:  (let* ((sql (format "UPDATE mailingsubscriptions SET mailingsubscriptions_donotemail = TRUE, mailingsubscriptions_email = '%s', mailingsubscriptions_dateunsubscribed = now() WHERE mailingsubscriptions_accounts = %s AND mailingsubscriptions_contacts = %s" email mid cid)))
   2205:         (sql (format "INSERT INTO mailingsubscriptions (mailingsubscriptions_donotemail, mailingsubscriptions_email, mailingsubscriptions_dateunsubscribed, mailingsubscriptions_accounts, mailingsubscriptions_contacts, mailingsubscriptions_relatedemail) VALUES (TRUE, '%s', now(), %s, %s, %s)"  email mid cid eid)))
   2209:  (let* ((sql (format "INSERT INTO mailingsubscriptions (mailingsubscriptions_donotemail, mailingsubscriptions_dateunsubscribed, mailingsubscriptions_accounts, mailingsubscriptions_contacts, mailingsubscriptions_email) VALUES (TRUE, now(), %s, %s, '%s')"  mid cid email)))
   2220:  (let* ((sql (format "SELECT mailingsubscriptions_id FROM mailingsubscriptions WHERE mailingsubscriptions_accounts = %s AND mailingsubscriptions_contacts = %s AND (mailingsubscriptions_donotemail IS NOT TRUE OR mailingsubscriptions_holdemail IS NOT TRUE)" account id))
   2226:      (let ((sql (format "UPDATE mailingsubscriptions SET mailingsubscriptions_donotemail = FALSE WHERE mailingsubscriptions_accounts = %s AND mailingsubscriptions_contacts = %s AND mailingsubscriptions_donotemail IS TRUE" account id)))
   2232:	   (sql (format "INSERT INTO mailingsubscriptions (mailingsubscriptions_accounts, mailingsubscriptions_contacts, mailingsubscriptions_email, mailingsubscriptions_referer, mailingsubscriptions_ip, mailingsubscriptions_assignedto, mailingsubscriptions_datecreated) VALUES (%s, %s, '%s', %s, %s, %s, '%s')" account id email (sql-escape-string referer) (sql-escape-string ip) assigned timestamp)))
   2236:  (let* ((sql (format "SELECT mailingsubscriptions_id FROM mailingsubscriptions WHERE mailingsubscriptions_accounts = %s AND mailingsubscriptions_contacts = %s AND mailingsubscriptions_donotemail IS NOT TRUE AND mailingsubscriptions_holdemail IS NOT TRUE" account id))
   2257:      (let ((sql (format "INSERT INTO contactskills (contactskills_contacts, contactskills_skills) VALUES (%s, %s)" contact skill)))
   2261:  (let ((sql (format "INSERT INTO contactskills (contactskills_contacts, contactskills_skills) VALUES (%s, %s)" contact skill)))
   2272:      (let ((sql (format "INSERT INTO markassignments (markassignments_mark, markassignments_contact, markassignments_account2, markassignments_date) VALUES (%s, %s, %s, '%s')" mark id account date)))
   2279:	 (sql (format "SELECT concat(markassignments_contact, ' ', get_full_contacts_name(markassignments_contact)) FROM markassignments WHERE markassignments_mark = %s AND markassignments_contact IS NOT NULL" mark))
   2287:	 (sql (format "SELECT markassignments_contact, get_full_contacts_name(markassignments_contact) FROM markassignments WHERE markassignments_mark = %s AND markassignments_contact IS NOT NULL" mark))
   2292:  (let* ((sql (format "SELECT markassignments_id, marks_hid || ', ' || get_contacts_name(%s) FROM markassignments, marks WHERE marks_id = markassignments_mark AND (markassignments_contact = %s OR markassignments_contact2 = %s)" id id id))
   2302:      (let ((sql (format "DELETE FROM markassignments WHERE markassignments_id = %s" mark-assignment)))
   2325:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id), coalesce(get_accounts_name(people_account1),'') FROM people WHERE people_country1 = %s OR people_country2 = %s" country country)))
   2329:  (let* ((sql (format "SELECT people_id FROM people WHERE (people_account1 = %s OR people_account2 = %s OR people_account3 = %s) OR (SELECT mailingsubscriptions_contacts FROM mailingsubscriptions WHERE mailingsubscriptions_contacts = people_id AND mailingsubscriptions_accounts = %s) = 1;" id id id id)))
   2333:  (let* ((sql (format "SELECT people_id FROM people WHERE people_fax ~ '[0-9]' AND ((people_account1 = %s OR people_account2 = %s OR people_account3 = %s) OR (SELECT mailingsubscriptions_contacts FROM mailingsubscriptions WHERE mailingsubscriptions_contacts = people_id AND mailingsubscriptions_accounts = %s) = 1);" id id id id)))
   2338:  (let* ((sql (format "SELECT people_id || ' ' || get_full_contacts_name(people_id) || ', ' || coalesce(people_title,'') || ', ' || get_accounts_name(%s) || ', ' || coalesce(country_name(people_country1), 'Unknown country') || ', ' || coalesce(country_name(people_country2),'') FROM people WHERE (people_account1 = %s OR people_account2 = %s OR people_account3 = %s) OR (SELECT mailingsubscriptions_contacts FROM mailingsubscriptions WHERE mailingsubscriptions_contacts = people_id AND mailingsubscriptions_accounts = %s) = 1;" id id id id id)))
   2348:      (let* ((sql (format "SELECT people_id FROM people WHERE people_introducedby = %s ORDER BY people_id" id))
   2361:  (let* ((sql (format "SELECT people_id || ' ' || get_contacts_name(people_id) FROM people WHERE people_introducedby = %s ORDER BY people_id" id)))
   2376:	 (sql (format "INSERT INTO generallog (generallog_accounts, generallog_assignedto, generallog_date, generallog_time, generallog_title, generallog_description, generallog_publish) VALUES (%s, %s, %s, %s, %s, %s, TRUE) RETURNING generallog_id" account assigned-to date time title description)))
   2387:             (sql (format "INSERT INTO generallog (generallog_contacts, generallog_title) VALUES (1, %s)" title-2)))
   2393:  (let ((sql (format "SELECT generallog_id, generallog_title, coalesce(generallog_description,'') 
   2402:  (let ((sql (format "SELECT people_id FROM people WHERE people_id != %s AND (people_email1 ILIKE '%s' OR people_email2 ILIKE '%s' OR people_email3 ILIKE '%s') ORDER BY people_id" id email email email)))
   2421:  (let* ((sql (format "SELECT people_id FROM people WHERE people_email1 ~* '%s' OR people_email2 ~* '%s' OR people_email3 ~* '%s' OR ((people_contacttype1 = 9 AND people_contact1 ~* '%s') OR (people_contacttype2 = 9 AND people_contact2 ~* '%s') OR (people_contacttype3 = 9 AND people_contact3 ~* '%s')) ORDER BY people_id" email email email email email email)))
   2434:	 (sql (format "INSERT INTO notes (notes_contact, notes_name, notes_note) VALUES (%s, %s, %s) RETURNING notes_id" id name note))
   2462:	     (sql (format "UPDATE people SET %s = trim(both %s);\n" column column)))
   2494:  (let* ((sql (format "SELECT people_prefix, people_suffix FROM people WHERE people_id = %s;" id))
   2506:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id), coalesce(get_accounts_name(people_account1),'UNKNOWN') FROM people ORDER BY people_id DESC LIMIT %s" limit))
   2513:      (let ((sql (format "SELECT people_id, get_full_contacts_name(people_id), coalesce(get_accounts_name(people_account1),'UNKNOWN') FROM people WHERE people_id = %s" id))
   2538:		       (sql (format "SELECT people_id, get_full_contacts_name(people_id), coalesce(get_accounts_name(people_account1),'UNKNOWN') FROM people WHERE people_id in (%s)" id-list))
   2591:  (let ((sql (format "SELECT people_id || ' ' || get_full_contacts_name(people_id) || ' ' || coalesce(country_name(people_country1),' ') || coalesce(country_name(people_country2),' ') || contact_interactions(people_id) AS entry FROM people WHERE people_account1 = '%s' OR people_account2 = '%s' OR people_account3 = '%s' ORDER BY entry" id id id)))
   2629:	   (sql (format "INSERT INTO emacsplaces (emacsplaces_hostname, emacsplaces_database, emacsplaces_table, emacsplaces_column, emacsplaces_dbid, emacsplaces_place) VALUES ('%s','%s','%s','%s',%s,%s) ON CONFLICT (emacsplaces_hostname, emacsplaces_database, emacsplaces_schema, emacsplaces_table, emacsplaces_table, emacsplaces_column, emacsplaces_dbid) DO UPDATE SET emacsplaces_place = %s WHERE emacsplaces.emacsplaces_hostname = '%s' AND emacsplaces.emacsplaces_database = '%s' AND emacsplaces.emacsplaces_table = '%s' AND emacsplaces.emacsplaces_column = '%s' AND emacsplaces.emacsplaces_dbid = %s;" hostname cf-database-name rcd-current-table rcd-current-column rcd-current-table-id (point) (point) hostname cf-database-name rcd-current-table rcd-current-column rcd-current-table-id)))
   2636:	   (sql (format "SELECT emacsplaces_place FROM emacsplaces WHERE emacsplaces_hostname = '%s' AND emacsplaces_database = '%s' AND emacsplaces_schema = '%s' AND emacsplaces_table = '%s' AND emacsplaces_column = '%s' AND emacsplaces_dbid = '%s'" hostname cf-database-name "public" table column id)))
   2649:	 (sql (format "SELECT accounts_id, accounts_name FROM accounts %s ORDER BY accounts_name" where)))
   2666:	 (sql (format "SELECT accounts_id, accounts_name FROM accounts WHERE accounts_id IN (%s)" accounts)))
   2671:  (let* ((sql (format "SELECT CASE WHEN accounts_email1 ~ '@' THEN accounts_email1 ELSE NULL END AS email1, CASE WHEN accounts_email2 ~ '@' THEN accounts_email2 ELSE NULL END AS email2, CASE WHEN accounts_email3 ~ '@' THEN accounts_email3 ELSE NULL END AS email3 FROM accounts WHERE accounts_id = %s ORDER BY accounts_id LIMIT 1" account)))
   2678:	 (sql (format "INSERT INTO notes (notes_account, notes_name, notes_note) VALUES (%s, %s, %s) RETURNING notes_id" id name note))
   2718:	(let* ((sql (format "UPDATE people SET people_account1 = %s WHERE people_id = %s" account contact)))
   2725:	(let* ((sql (format "UPDATE people SET people_account2 = %s WHERE people_id = %s" account contact)))
   2732:	(let* ((sql (format "UPDATE people SET people_account3 = %s WHERE people_id = %s" account contact)))
   2738:	 (sql (format "SELECT accounts_id, accounts_name FROM accounts WHERE accounts_accounttypes = %s" type)))
   2748:  (let* ((sql (format "SELECT emails_id, emails_subject FROM emails WHERE emails_mailinglist = %s ORDER BY emails_priority DESC" mid)))
   2775:  (let* ((sql (format "SELECT accounts_id, accounts_name || ' ' || CASE WHEN mailingsubscriptions_holdemail IS TRUE THEN ', ON HOLD' ELSE '' END AS hold FROM accounts, mailingsubscriptions WHERE mailingsubscriptions_accounts = accounts_id AND mailingsubscriptions_contacts = %s AND mailingsubscriptions_donotemail IS NOT TRUE" contact))
   2858:  (let ((sql (format "SELECT CASE WHEN (SELECT count(mailingsubscriptions_id) FROM mailingsubscriptions WHERE mailingsubscriptions_contacts = %s) = 0 THEN NULL ELSE mailingsubscriptions_accounts || ' ' || get_accounts_name(mailingsubscriptions_accounts) END FROM mailingsubscriptions WHERE mailingsubscriptions_contacts = %s AND mailingsubscriptions_donotemail IS NOT TRUE" id id)))
   2874:  (let ((sql (format "SELECT date(mailings_datecreated) || ' ' || mailings_subject || ', ' || get_accounts_name(mailings_fromcompany) FROM mailings WHERE mailings_contacts = %s" id)))
   2879:  (let* ((sql (format "SELECT interactiontypes_name || ': ' || interactions_count FROM interactiontypes, interactions WHERE interactions_contacts = %s AND interactions_interactiontypes = interactiontypes_id" id)))
   2884:  (let* ((sql (format "SELECT interactiontypes_name, interactions_count FROM interactiontypes, interactions WHERE interactions_contacts = %s AND interactions_interactiontypes = interactiontypes_id" id))
   3039:  (let ((sql (format "SELECT notes_id FROM notes WHERE notes_contact = %s ORDER BY notes_id" id)))
   3043:  (let ((sql (format "SELECT notes_id, notes_name, notes_note FROM notes WHERE notes_id = %s" id)))
   3062:  (let* ((sql (format "SELECT '\n** ' || sms_datecreated || '\n\n' || smsstatus_name || ' by number ' || sms_phone || '\n\n' || sms_body || '\n' FROM sms, smsstatus WHERE smsstatus_id = sms_smsstatus AND sms_contacts = %s ORDER BY sms_datecreated" id))
   3076:  (let ((sql (format "SELECT DISTINCT interactions_contacts || ' ' || get_full_contacts_name(interactions_contacts) || ', ' || interactions_count FROM interactions WHERE interactions_count > %s" min)))
   3081:  (let* ((sql (format "SELECT * FROM people_by_interactions ORDER BY \"Interactions\"::integer DESC LIMIT %s" number-of-people)))
   3088:	 (sql (format "SELECT people_id FROM people WHERE people_account1 IN (%s) ORDER BY people_id" accounts-greater-than))
   3091:	 (sql (format "SELECT * FROM people_by_interactions WHERE \"ID\" IN (%s) ORDER BY \"Interactions\"::integer DESC" list)))
   3115:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id), coalesce(get_accounts_name(coalesce(people_account1,people_account2,people_account3)),'UNKNOWN') FROM people WHERE people_country1 = %s OR people_country2 = %s" country country))
   3125:	 (sql (format "SELECT people_id, interactions_count_people(people_id)::text AS count, get_full_contacts_name(people_id) FROM people WHERE people_id in (%s) ORDER BY count DESC" people)))
   3135:	     (sql (format "INSERT INTO dbtranslations (dbtranslations_table, dbtranslations_field, dbtranslations_tableid, dbtranslations_language, dbtranslations_translation) VALUES ('%s', '%s', %s, %s, %s) ON CONFLICT DO NOTHING RETURNING dbtranslations_translation" table column id language-id (sql-escape-string translation))))
   3141:  (let* ((sql (format "SELECT people_id FROM people WHERE people_email1 ILIKE '%s' OR people_email2 ILIKE '%s' OR people_email3 ILIKE '%s'" email email email))
   3184:  (let ((sql (format "SELECT accounts_id, accounts_name, coalesce(country_name(accounts_billingcountry),'UNKNOWN') FROM accounts ORDER BY accounts_datecreated DESC LIMIT 200")))
   3190:  (let* ((sql (format "SELECT mininglands_contacts, get_full_contacts_name(mininglands_contacts), mininglands_code, coalesce(country_name(people_country1), country_name(people_country2), 'UNKNOWN') FROM mininglands, people WHERE mininglands_contacts = people_id ORDER BY mininglands_id DESC")))
   3452:	       (sql (format "INSERT INTO people (people_lastname, people_account1, people_email1) VALUES ('%s',%s,'%s')" email account email)))
   3474:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id), get_accounts_name(people_account1) FROM people WHERE (people_invalid1 IS TRUE or people_invalid2 IS TRUE or people_invalid3) IS TRUE AND people_datecreated > current_timestamp - interval '%s days'" days)))
   3482:	 (sql (format "SELECT people_id, sum(interactions_count)::text as sum, get_full_contacts_name(people_id) AS name FROM people, interactions WHERE interactions_contacts = people_id AND (people_account1 = %s OR people_account2 = %s OR people_account3 = %s) GROUP BY people_id, name ORDER BY sum DESC" account account account)))
   3509:	 (sql (format "SELECT sum(interactions_count) FROM interactions WHERE %s" or-clause))
   3517:	     (sql (format "INSERT INTO interactions (interactions_interactiontypes, interactions_accounts, interactions_count) VALUES (11, %s, %s) ON CONFLICT (interactions_accounts,interactions_interactiontypes) DO UPDATE SET interactions_count = %s WHERE interactions.interactions_accounts = %s AND interactions.interactions_interactiontypes = 11;" id count count id)))
   3557:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id), get_accounts_name(people_account1) FROM people WHERE people_id in (%s)" emails))
   3578:	 (sql (format "INSERT INTO peopleactivities (peopleactivities_languages, peopleactivities_people, peopleactivities_activity, peopleactivities_locationtext, peopleactivities_contactline) VALUES (%s, %s, %s, %s, %s) RETURNING peopleactivities_id" language id activity location contact-line)))
   3619:	(let* ((sql (format "INSERT INTO relations (relations_contacts, relations_relationtypes, relations_tocontact, relations_description) VALUES (%s, %s, %s, %s)" contact type related-to-contact (sql-escape-string description)))
   3628:	   (sql (format "SELECT relations_id, get_full_contacts_name(relations_contacts), relationtypes_name, get_full_contacts_name(relations_tocontact), relations_description FROM relations, relationtypes WHERE relationtypes_id = relations_relationtypes AND (relations_contacts = %s OR relations_tocontact = %s)" id id)))
   3655:      (let ((sql (format "INSERT INTO accounts (accounts_name) VALUES (%s) RETURNING accounts_id" name)))
   3679:  (let* ((sql (format "SELECT %s FROM %s WHERE %s" (string-join columns  ", ") table where)))
   3748:  (let* ((sql (format "INSERT INTO domains (domains_name, domains_tlds, domains_ownercontact) VALUES ('%s', %s, %s)" domain tld contact)))
   3763:	 (sql (format "SELECT attname, atttypid::regtype, attnotnull FROM pg_attribute WHERE attrelid = '%s.%s'::regclass AND attnum > 0 AND NOT attisdropped ORDER BY attnum" schema table))
   3769:  (let* ((sql (format "SELECT description FROM pg_shdescription JOIN pg_database ON objoid = pg_database.oid WHERE datname = '%s'" table))
   3774:  (let ((sql (format "SELECT pgd.description FROM pg_catalog.pg_statio_all_tables AS st INNER JOIN pg_catalog.pg_description pgd ON (pgd.objoid=st.relid) INNER JOIN information_schema.columns c ON (pgd.objsubid=c.ordinal_position AND c.table_schema=st.schemaname AND c.table_name=st.relname AND c.table_name = '%s' AND c.table_schema = 'public' AND c.column_name = '%s')" table column)))
   3779:	 (sql (format "SELECT atttypid, attname FROM pg_attribute WHERE attrelid = '%s.%s'::regclass AND attnum > 0 AND NOT attisdropped ORDER BY attnum" schema table))
   3804:	 (sql (format "INSERT INTO %s (%s) SELECT %s FROM %s WHERE %s_id = %d RETURNING %s_id"
   3816:	 (sql (format "SELECT '%s.%s'::regclass::oid" schema table))
   3822:	 (sql (format "SELECT 
   3864:  (let* ((sql (format "SELECT a.attname, pg_catalog.format_type(a.atttypid, a.atttypmod), (SELECT substring(pg_catalog.pg_get_expr(d.adbin, d.adrelid) for 128) FROM pg_catalog.pg_attrdef d WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef), a.attnotnull, a.attnum, (SELECT c.collname FROM pg_catalog.pg_collation c, pg_catalog.pg_type t WHERE c.oid = a.attcollation AND t.oid = a.atttypid AND a.attcollation <> t.typcollation) AS attcollation, a.attidentity, NULL AS indexdef, NULL AS attfdwoptions, a.attstorage, CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget, pg_catalog.col_description(a.attrelid, a.attnum) FROM pg_catalog.pg_attribute a WHERE a.attrelid = '%s' AND a.attnum > 0 AND NOT a.attisdropped ORDER BY a.attnum" oid))
   3894:	 (sql (format "SELECT description FROM pg_shdescription JOIN pg_database ON objoid = pg_database.oid WHERE datname = '%s'" database-name)))
   3912:	 (sql (format "COMMENT ON COLUMN %s.%s IS %s" table column comment)))
   3998:	 (sql (format "SELECT * FROM %s WHERE to_tsvector(%s::text) @@ to_tsquery('%s')" table table query)))
   4003:  (let* ((sql (format"SELECT n.nspname as \"Schema\",
   4022:  (let* ((sql (format"SELECT c.oid, c.relname as \"Name\",
   4055:  (let* ((sql (format"SELECT c.relname FROM pg_catalog.pg_class c LEFT JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace WHERE c.relkind IN ('r','p','') AND n.nspname <> 'pg_catalog' AND n.nspname <> 'information_schema' AND n.nspname !~ '^pg_toast' AND pg_catalog.pg_table_is_visible(c.oid) ORDER BY c.relname")))
   4191:	 (sql (format "SELECT %s_id FROM %s WHERE %s = %s" table table column value))
   4215:	 (sql (format "UPDATE %s SET %s = %s WHERE %s_id = %s RETURNING %s" table column nvalue table id column)))
   4245:  (let* ((sql (format "SELECT * FROM meta_fields WHERE meta_fields_table = '%s' AND meta_fields_field = '%s'" table column)))
   4410:  (let ((sql (format "DELETE FROM %s WHERE %s_id = %s" table table id)))
   4418:  (let ((sql (format "DELETE FROM %s WHERE %s = %s" table where value)))
   4426:  (let ((sql (format "SELECT EXISTS (
   4436:;;   (let ((sql (format "CREATE VIEW %s_combo AS SELECT %s_id AS id FROM %s ORDER BY %s" table table column table column)))
   4445:      (let ((sql (format "SELECT people_id, get_full_contacts_name(people_id), get_accounts_name(people_account1) FROM people WHERE people_leadsource = %s" lead-source)))
   4474:  (let ((sql (format "SELECT (SELECT string_agg(regexp_replace(x.v,'\n',' ','g'), ' ') FROM jsonb_each_text(to_jsonb(t)) AS x(k,v)) AS all_columns FROM %s t ORDER BY %s_id;" table table)))
   4478:  (let ((sql (format "SELECT concat_ws(' ', id, text) FROM %s_combo ORDER BY id" table)))
   4482:  (let* ((sql (format "SELECT concat(%s_list.*) FROM %s_list ORDER BY %s_id" table table table))
   4487:  (let* ((sql (format "SELECT concat(%s.*) FROM %s ORDER BY %s_id" table table table))
   4516:	 (sql (format "UPDATE %s SET %s = regexp_replace(%s, %s, %s, 'g') WHERE %s ~ %s" table column column pattern replacement column pattern)))
   4524:	 (sql (format "UPDATE %s SET %s = regexp_replace(%s, %s, %s, 'g') WHERE %s ~ %s" table column column pattern replacement column pattern)))
   4539:  (let* ((sql (format "SELECT %s FROM %s WHERE %s_id = %s" (string-join columns  ", ") table table id)))
   4589:  (let* ((sql (format "UPDATE %s SET %s = NULL WHERE %s_id = %s" table column table id)))
   4604:	 (sql (format "DELETE FROM %s a USING %s b WHERE a.%s_id > b.%s_id AND a.%s = b.%s %s" table table table table column column and-where)))
   4641:  (let* ((sql (format "SELECT * FROM %s" view))
   4714:	 (sql (format "UPDATE people SET people_tokens = to_tsvector(concat_ws(' ', people_firstname, people_middlenames, people_lastname, people_email1, people_email2, people_email3, get_accounts_name(people_account1), get_accounts_name(people_account2), get_accounts_name(people_account3), people_city1, CASE WHEN people_country1 IS NOT NULL THEN country_name(people_country1) ELSE '' END, coalesce((SELECT string_agg(tags_name,' ') FROM tags, peopletags WHERE peopletags_tags = tags_id AND peopletags_people = people_id),''), CASE WHEN people_country2 IS NOT NULL THEN country_name(people_country2) ELSE '' END, people_description, (select string_agg(sms_body,' ') from sms where sms_contacts = people_id))) %s" where)))
   4725:	   (sql (format "SELECT documents_id, documents_name || ' ' || ts_rank_cd(to_tsvector(documents_name || ' ' || documents_description || ' ' || documents_document),%s,32 /* rank/(rank+1) */) AS rank FROM documents, to_tsquery(%s) query WHERE query @@ to_tsvector(documents_name || ' ' || documents_description || ' ' || documents_document) ORDER BY rank DESC LIMIT 30;" query query)) ;; TODO this cannot order by rank
   4736:  (let ((sql (format "SELECT unnest(%s) FROM %s WHERE %s_id = %s" column table table id)))
   4842:	 (sql (format "INSERT INTO markassignments (markassignments_mark, markassignments_account, markassignments_contact, markassignments_date) VALUES (%s, %s, %s, '%s') RETURNING markassignments_id" mark account contact date)))
   4894:    (let* ((sql (format "SELECT people_id, get_full_contacts_name(people_id) FROM people WHERE people_introducedby = %s" id))
   4902:    (let* ((sql (format "SELECT mailings_id, mailings_subject, date(mailings_datecreated), get_accounts_name(mailings_fromcompany) FROM mailings WHERE mailings_contacts = %s ORDER by mailings_datecreated" id)))
   4922:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id), get_accounts_name(people_account1) FROM people WHERE people_officephone ~ '%s' OR people_mobilephone ~ '%s' OR people_homephone ~ '%s' OR people_otherphone ~ '%s' OR people_fax ~ '%s'" number number number number number))
   4954:  (let* ((sql (format "SELECT accounts_id, accounts_name FROM accounts WHERE accounts_id = %s" id)))
   5002:    (let* ((sql (format "SELECT notes_id, notes_name, notes_note FROM notes WHERE notes_contact = %s" id))
   5010:    (let* ((sql (format "SELECT notes_id, notes_name FROM notes WHERE notes_account = %s" id))
   5022:	   (sql (format "INSERT INTO notes (notes_name, notes_contact, notes_note) VALUES (%s, %s, %s) RETURNING notes_id" name id note))
   5034:	   (sql (format "INSERT INTO notes (notes_name, notes_account, notes_note) VALUES (%s, %s, %s) RETURNING notes_id" name id note))
   5042:    (let* ((sql (format "SELECT sms_id, sms_datecreated::date, smsstatus_name, sms_body FROM sms, smsstatus WHERE sms_contacts = %s AND smsstatus_id = sms_smsstatus" id))
   5071:	(let* ((sql (format "INSERT INTO markassignments (markassignments_mark, markassignments_contact, markassignments_date) VALUES (%s, %s, '%s') RETURNING markassignments_id" mark id  date))
   5078:  (let ((sql (format "SELECT addressbookentries_people FROM addressbookentries WHERE addressbookentries_people = %s AND addressbookentries_addressbooks = %s" person-id addressbook)))
   5084:    (let ((sql (format "INSERT INTO addressbookentries (addressbookentries_addressbooks, addressbookentries_people) VALUES (%s, %s) RETURNING addressbookentries_id" addressbook person-id)))
   5092:      (let ((sql (format "DELETE FROM addressbookentries WHERE addressbookentries_addressbooks = %s AND addressbookentries_people = %s" addressbook person-id)))
   5135:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id) FROM people WHERE (people_mobilephone ~ '\\+%s' OR people_homephone ~ '\\+%s' OR people_homephone ~ '\\+%s' OR people_fax ~ '\\+%s') AND people_country1 IS NULL" prefix prefix prefix prefix)))
   5142:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id), coalesce(get_accounts_name(people_account1),get_accounts_name(people_account2)) FROM people WHERE (people_mobilephone ~ '\\+%s' OR people_homephone ~ '\\+%s' OR people_homephone ~ '\\+%s' OR people_fax ~ '\\+%s') OR people_country1 = %s" prefix prefix prefix prefix country)))
   5158:	 (sql (format "SELECT emails_id, emails_subject, get_accounts_name(emails_mailinglist) FROM emails WHERE emails_subject ~* %s" query)))
   5165:	 (sql (format "SELECT emails_id, emails_subject, coalesce(get_accounts_name(emails_mailinglist),'UNKNOWN') FROM emails WHERE emails_body ~* %s" query)))
   5176:  (let* ((sql (format "SELECT domains_name || tlds_tld FROM domains, tlds WHERE domains_ownercontact = %s AND domains_tlds = tlds_id" id)))
   5187:    (let ((sql (format "SELECT accounts_id, accounts_name FROM accounts WHERE accounts_member1 = %s OR accounts_member2 = %s OR accounts_member3 = %s" id id id)))
   5199:		   (sql (format "SELECT people_id, get_full_contacts_name(people_id) FROM people WHERE people_id IN (%s)" list))
   5262:    (let ((sql (format "SELECT personaltransactions_id, personaltransactions_date,
   5385:  (let ((sql (format "SELECT id, text FROM %s_combo ORDER BY id" table)))
   5426:	 (sql (format "COMMENT ON TABLE %s IS %s" table comment)))
   5628:	 (sql (format "SELECT hyobjects_id FROM hyobjects WHERE %s = %s" column value)))
   5746:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id), coalesce(get_accounts_name(people_account1),get_accounts_name(people_account2),get_accounts_name(people_account3)) FROM people where people_tokens @@ to_tsquery('%s')" query)))
   5753:	     (sql (format "UPDATE people SET people_tokens = to_tsvector(concat_ws(' ', people_firstname, people_middlenames, people_lastname, people_email1, people_email2, people_email3, get_accounts_name(people_account1), get_accounts_name(people_account2), get_accounts_name(people_account3), people_city1, CASE WHEN people_country1 IS NOT NULL THEN country_name(people_country1) ELSE '' END, CASE WHEN people_country2 IS NOT NULL THEN country_name(people_country2) ELSE '' END, people_description, (select string_agg(sms_body,' ') from sms where sms_contacts = people_id))) %s" where)))
   5898:	    (sql (format "UPDATE %s SET %s = NULL WHERE %s_id = %s" table column table new-id)))
   6164:	 (sql (format "SELECT count(1)::text FROM people WHERE people_country1 = %s OR people_country2 = %s" country country)))
   6197:    (let ((sql (format "SELECT pages_id, pages_title, areas_name FROM pages, areas WHERE areas_id = pages_area AND pages_pagetype = %s" id)))
   6319:         (sql (format "SELECT pages_id FROM pages WHERE pages_title !~~ 'EMPTY PAGE' AND pages_area = %s %s %s ORDER BY pages_id %s %s" area cat-sql excluded order limit))
   6530:  (let* ((sql (format "SELECT pages_id, pages_title, 'page', pages_priority AS priority FROM pages WHERE pages_area = %s AND pages_categories IS NULL AND pages_notinmenu IS NOT TRUE UNION (SELECT categories_id, categories_name, 'category', categories_priority AS priority FROM categories WHERE categories_parent IS NULL and categories_area = %s AND categories_notinmenu IS NOT TRUE UNION SELECT pages_id, pages_title, 'page', pages_priority AS priority FROM pages WHERE pages_area = %s AND pages_categories IS NOT NULL AND pages_notinmenu IS NOT TRUE) ORDER BY priority" area area area))
   6571:  (let ((sql (format "DELETE FROM pages WHERE pages_id = %d" id)))
   6630:	   (sql (format "SELECT pages_id || ' ' ||  pages_title || ', ' || areas_name FROM pages, areas WHERE pages_area = areas_id AND (pages_title ~* '%s' OR pages_description ~* '%s')" query query))
   6767:         (sql (format "SELECT tlds_id FROM tlds WHERE tlds_tld = '%s'" tld))
   6775:           (sql (format "UPDATE pages SET pages_filename = '%s' WHERE pages_id = %s" slug page-id)))
   6789:	   (sql (format "SELECT pages_id, pages_title, areas_name FROM pages, areas WHERE areas_id = pages_area %s ORDER BY areas_name" where)))
   6795:    (let ((sql (format "SELECT pages_id, pages_title, coalesce(pages_filename,''), areas_name FROM pages, areas WHERE areas_id = pages_area AND pages_area = %s" id)))
   6810:  (let ((sql (format "SELECT pages_title FROM pages WHERE pages_id = %s" id)))
   6831:	(let ((sql (format "UPDATE pages SET pages_ogimage = '%s' WHERE pages_id = %s AND pages_ogimage !~ '/'" (public-html-rest image-1536) id)))
   6857:  (let ((sql (format "SELECT pages_id FROM pages WHERE pages_area = %s AND pages_ogimage !~ '//'" area)))
   6920:	 (sql (format "SELECT pages_id, pages_title, 'page', pages_priority AS priority, pages_notinmenu FROM pages WHERE pages_area = %s AND pages_categories IS NULL UNION ALL SELECT categories_id, categories_name, 'category', categories_priority, categories_notinmenu AS priority FROM categories WHERE categories_parent IS NULL and categories_area = %s ORDER BY priority DESC" area area))
   6932:	 (sql (format "SELECT pages_id, pages_title, 'page', pages_priority AS priority, pages_notinmenu FROM pages WHERE pages_area = %s AND pages_categories = %s UNION ALL SELECT categories_id, categories_name, 'category', categories_priority, categories_notinmenu AS priority FROM categories WHERE categories_parent = %s AND categories_area = %s ORDER BY priority DESC" area category category area))
   6951:	(let* ((sql (format "INSERT INTO categories (categories_area, categories_parent, categories_slug, categories_name, categories_menuname) VALUES (%s, %s, '%s', %s, %s) RETURNING categories_id" area parent slug (sql-escape-string name) (sql-escape-string menu)))
   6963:	 (sql (format "SELECT pages_id, pages_title || ', ' || pages_priority FROM pages WHERE pages_area = %s AND pages_categories %s" area category))
   7065:    (let ((sql (format "SELECT categories_id, categories_name FROM categories WHERE categories_area = %s" id)))
   7071:	 (sql (format "SELECT variables_id, variables_name FROM variables WHERE variables_area = %s" area)))
   7092:  (let ((sql (format "SELECT targets_id FROM targets WHERE targets_area = %d AND targets_active IS TRUE" area)))
   7186:  (let ((sql (format "SELECT pages_ogimage FROM pages WHERE pages_area = %d ORDER BY pages_id" area)))
   7208:      (let* ((sql (format "SELECT pages_priority FROM pages WHERE pages_categories = %s ORDER BY pages_priority DESC LIMIT 1" id)))
   7238:	     (sql (format "INSERT INTO pages (pages_area, pages_filename, pages_title, pages_description, pages_keywords, pages_priority, pages_categories, pages_content, pages_templates) VALUES (%s, '%s', %s, %s, '%s', %s, %s, %s, %s) RETURNING pages_id" area filename (sql-escape-string title) (sql-escape-string description) keywords new-priority category (sql-escape-string page) template))
   7309:  (let* ((sql (format "SELECT pages_id FROM pages WHERE pages_area = %s AND pages_content ~* %s" area (sql-escape-string query)))
   7351:	 (sql (format "SELECT categories_id, parent_category_name(categories_id) || '::' || categories_name FROM categories %s ORDER BY categories_parent, categories_priority" where)))
   7356:  (let ((sql (format "SELECT pages_id FROM pages WHERE pages_filename = '%s' AND pages_categories = %s" slug id)))
   7455:	 (sql (format "INSERT INTO pages (pages_area, pages_categories, pages_title, pages_description, pages_content, pages_keywords, pages_priority, pages_ogimage, pages_mediaurl, pages_mediatypes, pages_mediasize, pages_mainpage, pages_filename, pages_menuname, pages_active, pages_notinmenu, pages_content2, pages_templates) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING pages_id" area category (sql-escape-string title) (sql-escape-string description) (sql-escape-string body) (sql-escape-string keywords) new-priority (sql-escape-string ogimage) (sql-escape-string media) media-type media-size main (sql-escape-string slug) (sql-escape-string menu) active hidden (sql-escape-string body2) template))
   7462:	 (sql (format "SELECT pages_id FROM pages WHERE pages_content ~* %s" region))
   7557:	 (sql (format "INSERT INTO categories (categories_area, categories_parent, categories_slug, categories_name) VALUES (%s, %s, %s, %s) RETURNING categories_id;" area parent slug name))
   7576:         (sql (format "SELECT pages_id FROM pages WHERE pages_area = %s  
   7836:  (let* ((sql (format "SELECT pages_id FROM pages WHERE pages_area = %s AND pages_mediaurl = '%s'" area media))
   7846:  (let ((sql (format "SELECT pages_id FROM pages WHERE pages_mediaurl = '%s'" media)))
   7892:  (let ((sql (format "SELECT mediatypes_name FROM pages, mediatypes WHERE mediatypes_id = pages_mediatypes AND pages_id = %s" page-id)))
   7897:  (let ((sql (format "SELECT pages_mediaurl FROM pages WHERE pages_id = %s" page-id)))
   7996:	 (sql (format "SELECT pages_id FROM pages WHERE pages_area = %s AND pages_categories = %s AND pages_id != %s AND pages_filename = '%s' ORDER BY pages_id" area category checked-page slug))
   8019:  (let* ((sql (format "SELECT categories_id FROM categories WHERE categories_area = %s ORDER BY categories_id" area)))
   8025:	 (sql (format "SELECT pages_id FROM pages WHERE pages_area = %s AND pages_categories = %s %s ORDER BY pages_id" area category exclude-main)))
   8063:  (let ((sql (format "SELECT pages_id FROM pages WHERE pages_area = %s ORDER BY pages_id" area)))
   8116:    (let* ((sql (format "INSERT INTO hyobjects (hyobjects_language, hyobjects_name, hyobjects_link, hyobjects_description, hyobjects_text) SELECT pages_language, pages_title, '', pages_description, pages_content FROM pages WHERE pages_id = %s RETURNING hyobjects_id" id))
   8166:	 (sql (format "SELECT pages_id, pages_title, areas_name FROM pages, areas WHERE areas_id = pages_area AND pages_ogimage ~* %s ORDER BY pages_id" query)))
   8174:	 (sql (format "SELECT pages_id, pages_title, areas_name FROM pages, areas WHERE areas_id = pages_area AND pages_mediaurl ~* %s ORDER BY pages_id" query)))
   8255:         (sql (format "SELECT categories_id, categories_name FROM categories
   8280:         (sql (format "SELECT pages_id FROM pages WHERE pages_notinmenu IS NOT TRUE AND pages_title !~~ 'EMPTY' %s ORDER BY pages_priority" parent))
   8361:      (let ((sql (format "SELECT pages_id, pages_title, (select count(1) FROM relatedpages WHERE relatedpages_pages1 = pages_id OR relatedpages_pages2 = pages_id)::text AS related FROM pages WHERE pages_area = %d ORDER BY related DESC" area)))
   8370:	     (sql (format "SELECT pages_id, pages_title, (select count(1) FROM relatedpages WHERE relatedpages_pages1 = pages_id OR relatedpages_pages2 = pages_id)::text AS related FROM pages WHERE pages_area = %d AND (pages_content ~* %s OR pages_title ~* %s) ORDER BY related DESC" area query query)))
   8384:	 (sql (format "SELECT pages_id, pages_title FROM pages WHERE pages_content ~* %s" query)))
   8388:  (let* ((sql (format "SELECT categories_id, categories_name, count(pages_id)::text FROM categories, pages WHERE pages_categories = categories_id AND categories_area = %s GROUP BY categories_id ORDER BY categories_priority DESC" area)))
   8394:    (let ((sql (format "SELECT relatedpages_pages2, pages_title, areas_name FROM relatedpages, pages,areas WHERE relatedpages_pages1 = %d AND pages_id = relatedpages_pages2 AND pages_area = areas_id UNION SELECT relatedpages_pages1, pages_title, areas_name FROM relatedpages, pages, areas WHERE relatedpages_pages2 = %d AND pages_id = relatedpages_pages1 AND pages_area = areas_id" id id)))
   8444:  (let ((sql (format "SELECT targets_id FROM targets WHERE targets_area = %s AND targets_active IS TRUE" area)))
   8479:	 (sql (format "INSERT INTO personaltransactions (personaltransactions_name, personaltransactions_date, personaltransactions_amount, personaltransactions_currency, personaltransactions_fromperson, personaltransactions_fromaccount, personaltransactions_toperson, personaltransactions_toaccount, personaltransactions_description, personaltransactions_signature) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING personaltransactions_id" name date amount currency from-person from-account to-person to-account description signature)))
   8526:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id), accounts_name FROM people, accounts WHERE people_account1 = accounts_id AND people_country1 = 224 AND accounts_name ~* 'jiji' AND people_mobilephone ~ '25677' AND people_id NOT IN (%s)" sms)))
   8539:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id), accounts_name FROM people, accounts WHERE people_account1 = accounts_id AND 
   8556:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id), accounts_name FROM people, accounts WHERE people_account1 = accounts_id AND 
   8591:	 (sql (format "SELECT people_id, get_full_contacts_name(people_id) FROM people WHERE (substring(people_officephone, 2, 6) IN (%s) OR substring(people_mobilephone, 2, 6) IN (%s) OR substring(people_homephone, 2, 6) IN (%s) OR substring(people_otherphone, 2, 6) IN (%s) OR substring(people_fax, 2, 6) IN (%s)) ORDER BY people_id DESC LIMIT %s" prefixes prefixes prefixes prefixes prefixes how-many)))
   8611:    (let ((sql (format "SELECT locations_id, locations_name, locations_priority::text FROM locations WHERE locations_locationsets = %s ORDER BY locations_priority, locations_id DESC" id)))
   8649:	     (sql (format "INSERT INTO locations (locations_locationsets, locations_geocoordformats, locations_name, locations_description, locations_latitude, locations_longitude, locations_contacts) VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING locations_id" id geocoordformat name description latitude longitude person))
   8665:	 (sql (format "INSERT INTO peoplegroups (peoplegroups_name, peoplegroups_description) VALUES (%s, %s) RETURNING peoplegroups_id" name description))
   8684:	 (sql (format "SELECT peoplegroupmembers_id, get_full_contacts_name(peoplegroupmembers_person), coalesce(get_accounts_name(people_account1),get_accounts_name(people_account2),get_accounts_name(people_account2),'UNKNOWN') FROM peoplegroupmembers, people WHERE people_id = peoplegroupmembers_person %s" group)))
   8697:	 (sql (format "INSERT INTO peoplegroupmembers (peoplegroupmembers_person, peoplegroupmembers_peoplegroups, peoplegroupmembers_description) VALUES (%s, %s, %s) RETURNING peoplegroupmembers_id" person group description))
   8715:      ;; 	   (sql (format "INSERT INTO contactskills (contactskills_skills, contactskills_contacts) VALUES (107, %s)" person)))
   8805:  (let ((sql (format "SELECT (DATE_PART('day', '%s'::timestamp - '%s'::timestamp) * 24 
   8826:  (let ((sql (format "SELECT '%s'::timestamp + interval '%s'" timestamp interval)))
   8833:	 (sql (format "SELECT CASE WHEN current_timestamp::time < '%s'
   8961:	 (sql (format "INSERT INTO reminders (reminders_name, reminders_remindertypes) 

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1408 bytes --]

On Mon, Jun 21, 2021 at 11:36:25PM +0300, Jean Louis wrote:
> * tomas@tuxteam.de <tomas@tuxteam.de> [2021-06-21 17:12]:
> > But you still see extremely bad habits "out there" which wouldn't be
> > necessary these days -- because, well, they are "out there" (for
> > example: assebling SQL queries with sprintf [1]). They take a life
> > of their own :-)
> > 
> > Cheers
> > [1] https://xkcd.com/327/
> 
> Your small reference is definitely a possible danger if SQL input is
> anyhow exposed [...]

M< hint was rather a metaphor: using string operations on things that
aren't really strings (in the original case: file system paths) can
lead to surprising results.

> the danger mentioned on the funny comic is practically non-existent as
> it will never take place on my side [...]

But your side is not "the world", and therefore Eli's warning was
spot-on. Someone will browse the mail archives and copy your solution
without knowing the dangers.

> I am heavy user of the Emacs package: emacs-libpq @ Github
> https://github.com/anse1/emacs-libpq

No idea and no bandwidth to read it all. If you are tied to
PostgreSQL (a good choice, I'd say), consider using prepared
queries: they do what client-side template expansion (even the
careful kind, with unescaping and all), and I'd expect them to
do it much better, since PostgreSQL knows its own syntax best.

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Jean Louis]

* tomas@tuxteam.de <tomas@tuxteam.de> [2021-06-22 00:17]:
> > the danger mentioned on the funny comic is practically non-existent as
> > it will never take place on my side [...]
> 
> But your side is not "the world", and therefore Eli's warning was
> spot-on. Someone will browse the mail archives and copy your solution
> without knowing the dangers.

I agree on that. But we cannot possibly expect all possible dangers to
be known by all possible programmers at all times especially on this
mailing list, and then in so many external Emacs Packages. My
intention to improve is (should) be perceivable. The email you are
replying to is a proof that I did not claim it is "solution" at
all. Quite contrary, I have validated your point and found 400+
possible problems in the program. It should be clear it is not a
definite solution to every reader. Programs develop. They are never
perfect until they get perfect.

Without a single occurence of the incident with SQL it is exaggeration
to say there is practical danger, rather hypothetical danger.

Then when we speak of the PostgreSQL, users should anyway not be given
permissions to DROP tables as that should be left to
administrators. There is similar approach to updates of tables, there
is row level security and users can update whatever they are permitted
to, but not what they are not permitted to. All the dangers we speak
about are usually solved at the database level.

> > I am heavy user of the Emacs package: emacs-libpq @ Github
> > https://github.com/anse1/emacs-libpq
> 
> No idea and no bandwidth to read it all. If you are tied to
> PostgreSQL (a good choice, I'd say), consider using prepared
> queries: they do what client-side template expansion (even the
> careful kind, with unescaping and all), and I'd expect them to
> do it much better, since PostgreSQL knows its own syntax best.

I will do, thanks.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

> I agree on that. But we cannot possibly expect all possible
> dangers to be known by all possible programmers at all times
> especially on this mailing list

OK, so the SQL injection is a common attack vector, but what
should we call this issue?

Uhm, the well-known and feared Windows vulnerability, let's
see, argh, the Emacs Lisp string file system path
attack vector?

I don't know, it doesn't quite have the same ring to it...

-- 
underground experts united
https://dataswamp.org/~incal

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-06-22 03:32]:
> Jean Louis wrote:
> 
> > I agree on that. But we cannot possibly expect all possible
> > dangers to be known by all possible programmers at all times
> > especially on this mailing list
> 
> OK, so the SQL injection is a common attack vector, but what
> should we call this issue?

It is probably lack of database administration skills. It is nothing
related to Emacs really. There is nothing special to SQL then to any
other kind of user's input. In fact, PostgreSQL and MySQL or MariaDB
are rather safe databases. If user does not have permission to DROP
tables or do something malicious, then it does not have it, finished
there. One can try it, but it will not work.

On the other hand injecting simple malicious Emacs Lisp anywhere in
any file is as a possible option omni-present on Internet, and we
don't even speak about that.

Thousands of users are blindly accepting programs from MELPA or any
kind of ELPA without knowing what is going to happen with the
data. And then we worry about possible SQL injections in Emacs Lisp. I
just wonder how and where SQL injection. Is it maybe by malicious
staff members versed in PostgreSQL? Or maybe Emacs Lisp running some
sensitive data online (without backup) being exposed to online
crackers trying to cheat the code and DROP some tables or do other bad
stuff? It is so unlikely to take place.

Bounty is US $10 from my side if somebody succeeds to SQL inject in my
software a DROP of a table.


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>>> I agree on that. But we cannot possibly expect all
>>> possible dangers to be known by all possible programmers
>>> at all times especially on this mailing list
>> 
>> OK, so the SQL injection is a common attack vector, but
>> what should we call this issue?
>
> It is probably lack of database administration skills. It is
> nothing related to Emacs really.

It doesn't? :)

> There is nothing special to SQL then to any other kind of
> user's input. In fact, PostgreSQL and MySQL or MariaDB are
> rather safe databases.

I think that has turned into a schoolbook example because it
has a cool name and everyone will understand it instantly.
So it can serve the educational purpose to illuminate this in
all of computing to not execute or use user input, without
checking it out first. Indeed it would surprise me if you
could just do it out-of-the-box for modern database management
systems and expect it to be just wide open there for you to
do it.

> On the other hand injecting simple malicious Emacs Lisp
> anywhere in any file is as a possible option omni-present on
> Internet, and we don't even speak about that.

Well, it doesn't work like that, really.

> Thousands of users are blindly accepting programs from MELPA

Ha ha, thousands of users are doing that blindly! My, my.
How many people are doing it with ONE EYE open, you think?

M-x how-many RET

> And then we worry about possible SQL injections in Emacs
> Lisp.

No we aren't...

> Bounty is US $10 from my side if somebody succeeds to SQL
> inject in my software a DROP of a table.

But where are your tables?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Jean Louis]

* Emanuel Berg <moasenwood@zoho.eu> [2021-06-26 02:31]:
> > There is nothing special to SQL then to any other kind of
> > user's input. In fact, PostgreSQL and MySQL or MariaDB are
> > rather safe databases.
> 
> I think that has turned into a schoolbook example because it
> has a cool name and everyone will understand it instantly.
> So it can serve the educational purpose to illuminate this in
> all of computing to not execute or use user input, without
> checking it out first. Indeed it would surprise me if you
> could just do it out-of-the-box for modern database management
> systems and expect it to be just wide open there for you to
> do it.

There is way to go and that is by providing parameters that are automatically escaped.

pq:query is a module function.

(pq:query CONN COMMAND &rest PARAMETERS)

Execute COMMAND on CONN with optional PARAMETERS.

Run an SQL command on a connection obtained by ‘pq:connectdb’.
If PARAMETERS are used, you can reference them in COMMAND using
$1, $2, ..., $12.

Return the list of rows returned by the last statement in COMMAND.
Rows are returned as atomic values if the statement yields a single
column, or a vector of values if it yields more than one column.

I am anyway SQL escaping all of the input values to database. 

Since we talked I have moved to better workflow and now I am
first preparing one part of the SQL command without user's input
and then using the user's input as a parameter that prevents those problems.

I just guess it should be similar in MariaDB or MySQL. But I
don't use it due to lack of readline and command expansion.

> > On the other hand injecting simple malicious Emacs Lisp
> > anywhere in any file is as a possible option omni-present on
> > Internet, and we don't even speak about that.
> 
> Well, it doesn't work like that, really.

It is about the same danger. Practically there was never danger
on my side as it is me who is using the database which is motly
the case with many database applications. Problem comes with a
large users base where malicious people may start doing something
to the company or public or unknown crackers or script
kiddies. It is better not to classify possible problematic
sources and just do it more safe.

> > Thousands of users are blindly accepting programs from MELPA
> 
> Ha ha, thousands of users are doing that blindly! My, my.
> How many people are doing it with ONE EYE open, you think?

Well, without watching in the code. Click and install. There is
no real security involved and users don't watch what is being
installed, as Internet is large it is quite easy to create a game
and make backdoor in Emacs or do other malicious stuff. It is
also possible to make a module and do same. Until we know it, we
cannot know it.

> > Bounty is US $10 from my side if somebody succeeds to SQL
> > inject in my software a DROP of a table.
> 
> But where are your tables?

Find it for the bounty.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

> MySQL [...]

"My" is a Swedish girl name if you didn't know. (And maybe
a Danish, Finish, and Norwegian name as well.) So it isn't
a possessive pronoun, here, even!

Eheh, LISA, anyone?

No, this is more clever! And better.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-07-06 05:01]:
> Jean Louis wrote:
> 
> > MySQL [...]
> 
> "My" is a Swedish girl name if you didn't know. (And maybe
> a Danish, Finish, and Norwegian name as well.) So it isn't
> a possessive pronoun, here, even!

It makes sense now, maybe she felt the international pressure and
confusion, so she changed to Maria?

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>> "My" is a Swedish girl name if you didn't know. (And maybe
>> a Danish, Finish, and Norwegian name as well.) So it isn't
>> a possessive pronoun, here, even!
>
> It makes sense now, maybe she felt the international
> pressure and confusion, so she changed to Maria?

A sweet theory of yours no doubt, here is the actual story
tho:

  https://mariadb.com/kb/en/the-aria-name/

-- 
underground experts united
https://dataswamp.org/~incal

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Emanuel Berg via Users list for the GNU Emacs text editor]

tomas wrote:

> My hint was rather a metaphor: using string operations on
> things that aren't really strings (in the original case:
> file system paths)

Emacs disagrees:

  (setq file "~/.emacs")
  (file-exists-p file) ; t
  (stringp file) ; t

> But your side is not "the world", and therefore Eli's
> warning was spot-on

False alarm, as shown. It is safe to say, that warning can
be ignored.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Eli Zaretskii]

> Date: Tue, 22 Jun 2021 02:23:03 +0200
> From:  Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org>
> 
> tomas wrote:
> 
> > My hint was rather a metaphor: using string operations on
> > things that aren't really strings (in the original case:
> > file system paths)
> 
> Emacs disagrees:
> 
>   (setq file "~/.emacs")
>   (file-exists-p file) ; t
>   (stringp file) ; t
> 
> > But your side is not "the world", and therefore Eli's
> > warning was spot-on
> 
> False alarm, as shown. It is safe to say, that warning can
> be ignored.

Not a false alarm; ignore that warning at your own peril:

  (string-equal "~/foo" "/home/users/eliz/foo") => nil
  (file-equal-p "~/foo" "/home/users/eliz/foo") => t

and also:

  (string-equal "/home/users/eliz/foo" "/server/homes/users/eliz/foo") => nil
  (file-equal-p "/home/users/eliz/foo" "/server/homes/users/eliz/foo") => t

And what about the below, what's going on there?

  $ ls -l /usr/bin/emacs*

  -rwxr-xr-x 2 eliz None 81081674 2021-03-25 15:54 /usr/bin/emacs
  -rwxr-xr-x 2 eliz None 81081674 2021-03-25 15:54 /usr/bin/emacs-27.2

  (file-equal-p "/usr/bin/emacs" "/usr/bin/emacs-27.2") => t

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Jean Louis]

* Eli Zaretskii <eliz@gnu.org> [2021-06-22 15:14]:
> > Date: Tue, 22 Jun 2021 02:23:03 +0200
> > Emacs disagrees:
> > 
> >   (setq file "~/.emacs")
> >   (file-exists-p file) ; t
> >   (stringp file) ; t
> > 
> > > But your side is not "the world", and therefore Eli's
> > > warning was spot-on
> > 
> > False alarm, as shown. It is safe to say, that warning can
> > be ignored.
> 
> Not a false alarm; ignore that warning at your own peril:
> 
>   (string-equal "~/foo" "/home/users/eliz/foo") => nil
>   (file-equal-p "~/foo" "/home/users/eliz/foo") => t
> 
> and also:
> 
>   (string-equal "/home/users/eliz/foo" "/server/homes/users/eliz/foo") => nil
>   (file-equal-p "/home/users/eliz/foo"
>   "/server/homes/users/eliz/foo") => t

The above example is insightful. Though it does not change the fact
that even `file-equal-p' function uses strings as its parameters. Of
course it is handling files with their file names represented in
strings accordingly to the file system. 

The meaning of a string did not change its type.

(rcd-db-connect DATABASE &optional PORT HOST USERNAME PASSWORD)

The meaning of a string DATABASE will not change the type of a string
accepted. It would not be clear to say that DATABASE is not just a
string because if I provide wrong name of the database I would never
connect to it. There is diffference between the meaning and type.

> And what about the below, what's going on there?
> 
>   $ ls -l /usr/bin/emacs*
> 
>   -rwxr-xr-x 2 eliz None 81081674 2021-03-25 15:54 /usr/bin/emacs
>   -rwxr-xr-x 2 eliz None 81081674 2021-03-25 15:54 /usr/bin/emacs-27.2
> 
>   (file-equal-p "/usr/bin/emacs" "/usr/bin/emacs-27.2") => t

(type-of "/usr/bin/emacs") ⇒ string
(type-of "/usr/bin/emacs-27.2") ⇒ string

Repeat after me: "The meaning of a string is not equal to its type.."

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Eli Zaretskii]

> Date: Tue, 22 Jun 2021 15:37:46 +0300
> From: Jean Louis <bugs@gnu.support>
> Cc: help-gnu-emacs@gnu.org
> 
> >   (string-equal "~/foo" "/home/users/eliz/foo") => nil
> >   (file-equal-p "~/foo" "/home/users/eliz/foo") => t
> > 
> > and also:
> > 
> >   (string-equal "/home/users/eliz/foo" "/server/homes/users/eliz/foo") => nil
> >   (file-equal-p "/home/users/eliz/foo"
> >   "/server/homes/users/eliz/foo") => t
> 
> The above example is insightful. Though it does not change the fact
> that even `file-equal-p' function uses strings as its parameters.

We _represent_ file names as strings, but they are not normal strings.
Just like characters are represented as integers, but they are not
just simple integers, they have additional special attributes and
behaviors.

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Jean Louis]

* Eli Zaretskii <eliz@gnu.org> [2021-06-22 16:11]:
> We _represent_ file names as strings, but they are not normal strings.
> Just like characters are represented as integers, but they are not
> just simple integers, they have additional special attributes and
> behaviors.

I get your point, what I don't agree with is the Formulierung. It
is for same reason as you mentioned it, people are reading our
writings and our form or text should be possibly clear. If you
wish to say they are not normal strings, demonstrate it. By
demonstration below it shows to be of type string. What is else
there?

I have a file:

~/tmp/new.txt

This works fine:

(find-file "~/tmp/new.txt") ⇒ #<buffer new.txt>

Is it normal string?

(type-of "~/tmp/new.txt") ⇒ string

Of course I know there are functions that will get file attributes,
and those are fine. Of course that meanings of strings are so much
different.

But subject was that we deal with strings in those functions. 

It was not about the meaning of strings.

I have functions which chunk human names in 3 parts. Functions use
strings. It really does not matter that I want to parse the human
names and chunk it or fit it into some form, or do something with
it. It does not change the type of a string and fact that strings are
supplied to functions.

The function `file-attributes' gives some different type as a result:

(file-attributes "~/tmp/new.txt") ⇒ (nil 1 1001 1001 (24786 1053 990790 326000) (24786 1033 720790 188000) (24786 1033 720790 188000) 0 "-rw-r--r--" t 51912267 65024)

(type-of (file-attributes "~/tmp/new.txt")) ⇒ cons

Then we can work on that with:

(setq attributes (file-attributes "~/tmp/new.txt"))
(file-attribute-size attributes) ⇒ 0
(file-attribute-status-change-time attributes) ⇒ (24786 1033 720790 188000)

I do understand you wish to point out to functions which interpret and represent the meanings. It still did not change the fact that string was supplied to it: "~/tmp/new.txt"


Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Eli Zaretskii]

> Date: Tue, 22 Jun 2021 18:45:56 +0300
> From: Jean Louis <bugs@gnu.support>
> Cc: help-gnu-emacs@gnu.org
> 
> * Eli Zaretskii <eliz@gnu.org> [2021-06-22 16:11]:
> > We _represent_ file names as strings, but they are not normal strings.
> > Just like characters are represented as integers, but they are not
> > just simple integers, they have additional special attributes and
> > behaviors.
> 
> I get your point, what I don't agree with is the Formulierung. It
> is for same reason as you mentioned it, people are reading our
> writings and our form or text should be possibly clear. If you
> wish to say they are not normal strings, demonstrate it. By
> demonstration below it shows to be of type string. What is else
> there?

What is there is that not every operation, which is valid and has
trivially-expected results with strings, is valid and has the same
trivial results with file names.  See the examples I presented.  And
there's more, for example, (concat "/foo/" "/bar") could be equivalent
to "/bar" when those are file names.  Likewise with characters: if c
is a valid character, (* 2 c) may not be a valid character, even if it
is a valid number.

> But subject was that we deal with strings in those functions. 

And my point is that it is dangerous (a.k.a. "wrong") using string
functions on file names when there are specially-designed file-name
functions for those use cases.  Because those special-purpose
functions are there for a reason, and disregarding those reasons is
asking for trouble.  Like using string comparison for comparing file
names: that was actually a reason for quite a few bugs in our code.
My point was trying to prevent people from making those same mistakes.

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Jean Louis]

* Eli Zaretskii <eliz@gnu.org> [2021-06-22 19:14]:
> And my point is that it is dangerous (a.k.a. "wrong") using string
> functions on file names when there are specially-designed file-name
> functions for those use cases.  Because those special-purpose
> functions are there for a reason, and disregarding those reasons is
> asking for trouble.  Like using string comparison for comparing file
> names: that was actually a reason for quite a few bugs in our code.
> My point was trying to prevent people from making those same mistakes.

I agree that dedicated file functions should be used whenever
possible. Now I am looking in some of my functions to review that:

People have their ID number, so directory may exist for each
person. But I cannot see a possibility to work with file paths that
are structured without using string functions. I can maybe verify if
some possible string represents directory or file, if it exists or if
it is readable, but more than that, I would not know practically what
to do.

Maybe my scope of using such generated file paths is out of the
mentioned scope.

To summarize, it is better to use file related functions whenever
possible, checking if it is directory, using functions like
`file-name-as-directory' and so on. 

Avoiding string functions related to files seem to be now impossible.

Like here below I have to `concat' or concatenate directory with
the file base name, as when I do "filing" it is rather filed by
year/month/date and either `concat' or format' has to be used.

    (dolist (file files)
      (let* ((basename (file-name-nondirectory file))
	     (target (concat date-dir basename))
	     (target (rcd-unique-file-name target)))
	(if (file-exists-p file)
	    (progn
	      (message (format "cf/file-by-contact: moving \"%s\" to \"%s\"" file target))
	      (rename-file file target))
	  (error (format "File does not exist: %s" file)))))))

(defun cf-directory-by-id (id)
  "Opens the dired directory for ID"
  (let* ((dir (dir-id id)))
    (unless (file-directory-p dir)
      (make-dir-id id))
    (dired dir)))

(defun dir-id (id)
  "Returns directory for contact ID"
  (format "%s/%s/" (rcd-crm-directory-by-id) id))

(defun rcd-crm-directory-by-id ()
  (concat (rcd-crm-directory) "/" (cadr (rcd-crm-directory-data))))

(defun rcd-crm-directory-data ()
  "Returns default CRM related directories"
  (let* ((sql "SELECT defaults_crmdir, defaults_crmbyid, defaults_crmbyname, defaults_crmdiraccount FROM defaults")
	 (dirs (rcd-sql-first sql cf-db)) 
	 (crmdir (aref dirs 0))
	 (crmbyid (aref dirs 1))
	 (crmbyname (aref dirs 2))
	 (crmdiraccount (aref dirs 3)))
    (list crmdir crmbyid crmbyname crmdiraccount)))

(defun rcd-crm-directory ()
  (car (rcd-crm-directory-data)))

(defun rcd-crm-directory-by-id ()
  (concat (rcd-crm-directory) "/" (cadr (rcd-crm-directory-data))))


I can now think of safer functions something like: `file-concat'
that could or make sure that concatenated directories and file on
the end exist or not.

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Eli Zaretskii]

> Date: Tue, 22 Jun 2021 21:01:59 +0300
> From: Jean Louis <bugs@gnu.support>
> Cc: help-gnu-emacs@gnu.org
> 
> Avoiding string functions related to files seem to be now impossible.

I never said anything to the contrary.

> (defun rcd-crm-directory-by-id ()
>   (concat (rcd-crm-directory) "/" (cadr (rcd-crm-directory-data))))
> 
> 
> I can now think of safer functions something like: `file-concat'
> that could or make sure that concatenated directories and file on
> the end exist or not.

It depends on what you concatenate.  Whenever you concatenate a
directory and a file under that directory, expand-file-name is a
better choice.

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

> To summarize, it is better to use file related functions
> whenever possible, checking if it is directory, using
> functions like `file-name-as-directory' and so on.

Yes, of course.

> (format "%s/%s/" (rcd-crm-directory-by-id) id))
> (concat (rcd-crm-directory) "/" (cadr (rcd-crm-directory-data))))
> (concat (rcd-crm-directory) "/" (cadr (rcd-crm-directory-data))))

Is this what you are not allowed to do?

I ask because it looks similar to mine, only not as good,
perhaps. Ha.

OK, how is it suppose to look then?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Eli Zaretskii]

> Date: Sat, 26 Jun 2021 08:46:06 +0200
> From:  Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org>
> 
> Jean Louis wrote:
> 
> > (format "%s/%s/" (rcd-crm-directory-by-id) id))
> > (concat (rcd-crm-directory) "/" (cadr (rcd-crm-directory-data))))
> > (concat (rcd-crm-directory) "/" (cadr (rcd-crm-directory-data))))
> 
> Is this what you are not allowed to do?
> 
> I ask because it looks similar to mine, only not as good,
> perhaps. Ha.
> 
> OK, how is it suppose to look then?

Like this:

  (expand-file-name (cadr (rcd-crm-directory-data)) (rcd-crm-directory))

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Jean Louis]

* Eli Zaretskii <eliz@gnu.org> [2021-06-26 03:15]:
> > Date: Sat, 26 Jun 2021 08:46:06 +0200
> > From:  Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org>
> > 
> > Jean Louis wrote:
> > 
> > > (format "%s/%s/" (rcd-crm-directory-by-id) id))
> > > (concat (rcd-crm-directory) "/" (cadr (rcd-crm-directory-data))))
> > > (concat (rcd-crm-directory) "/" (cadr (rcd-crm-directory-data))))
> > 
> > Is this what you are not allowed to do?
> > 
> > I ask because it looks similar to mine, only not as good,
> > perhaps. Ha.
> > 
> > OK, how is it suppose to look then?
> 
> Like this:
> 
>   (expand-file-name (cadr (rcd-crm-directory-data)) (rcd-crm-directory))

I have to improve various functions based on our conversation:

This is for example not meant to be without slash on the end:

(expand-file-name (cadr (rcd-crm-directory-data)) (rcd-crm-directory))
⇒ "/home/data1/protected/hyperscope/3/5/5/8/2/By-ID"

So there is work to do by using the function `file-name-as-directory'
wherever directory is meant to be.


Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

> I have to improve various functions based on our
> conversation:

I still don't know how it should be done the right way, I also
have a lot of URLs and hyperlinks, not the least in my my
Emacs-w3m stuff

  https://dataswamp.org/~incal/emacs-init/w3m/

which I also don't know how they are supposed to look :)

> I have to improve [...]
>
> Jean
>
> Take action in Free Software Foundation campaigns:
> https://www.fsf.org/campaigns
>
> In support of Richard M. Stallman
> https://stallmansupport.org/

Please improve this as well,
RFC 3676, section 4.3 (Usenet Signature Convention)
https://www.ietf.org/rfc/rfc3676.txt

-- 
underground experts united
https://dataswamp.org/~incal

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-07-06 05:11]:

> Please improve this as well,
> RFC 3676, section 4.3 (Usenet Signature Convention)
> https://www.ietf.org/rfc/rfc3676.txt

ohhhh.... I am sorry, did your Emacs segfault?

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>> Please improve this as well,
>> RFC 3676, section 4.3 (Usenet Signature Convention)
>> https://www.ietf.org/rfc/rfc3676.txt
>
> ohhhh.... I am sorry, did your Emacs segfault?

Well, to begin with, why do it the wrong way when it is so
easy to do it the right way?

Also, if you do it the wrong way configuration such as, in my
case, `gnus-article-hide-signature' won't work. So it is not
respectful of the people who don't want to see it and don't
want to have to delete these lines every time they answer
your posts.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-07-06 23:26]:
> Jean Louis wrote:
> 
> >> Please improve this as well,
> >> RFC 3676, section 4.3 (Usenet Signature Convention)
> >> https://www.ietf.org/rfc/rfc3676.txt
> >
> > ohhhh.... I am sorry, did your Emacs segfault?
> 
> Well, to begin with, why do it the wrong way when it is so
> easy to do it the right way?

You are mistaken. My signatures are for majority of times the "right
way", but rarely without --

> Also, if you do it the wrong way configuration such as, in my
> case, `gnus-article-hide-signature' won't work. So it is not
> respectful of the people who don't want to see it and don't
> want to have to delete these lines every time they answer
> your posts.

I don't have a problem deleting whatever is remaining in the email, as
I use the function `message-kill-to-signature' and I have it on the
key binding. It is thus very easy to delete anything else not
otherwise commented.

It delets until my signature... so if there is your signature there
without -- it will also get deleted without me even knowing it.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Jean Louis]

* Emanuel Berg <moasenwood@zoho.eu> [2021-06-26 02:50]:
> Jean Louis wrote:
> 
> > To summarize, it is better to use file related functions
> > whenever possible, checking if it is directory, using
> > functions like `file-name-as-directory' and so on.
> 
> Yes, of course.
> 
> > (format "%s/%s/" (rcd-crm-directory-by-id) id))
> > (concat (rcd-crm-directory) "/" (cadr (rcd-crm-directory-data))))
> > (concat (rcd-crm-directory) "/" (cadr (rcd-crm-directory-data))))
> 
> Is this what you are not allowed to do?
> 
> I ask because it looks similar to mine, only not as good,
> perhaps. Ha.
> 
> OK, how is it suppose to look then?

I think by using file- related functions:

(defun doi-file-find (md5)
  "Return existing MD5 file for Double Opt-In.confirmation or NIL."
  (let ((file (concat (file-name-as-directory doi-directory)
		      (file-name-as-directory (substring md5 0 doi-substring-to))
		      md5)))
    (if (file-exists-p file) file nil)))

in that case `file-name-as-directory' is protecting the case of
vraiable `doi-directory' not to have slash on the and.

Before I have used my function:

(defun slash-add (path)
  "Adds slash `/` quickly on the end of string"
  (if (string= path "")
      "/"
    (let ((last (substring (reverse path) 0 1)))
      (if (string= last "/") path
	(concat path "/")))))

Or

(defun string-add (string add)
  "Returns string 'add' on the end of the string if there is none"
  (let* ((string (replace-regexp-in-string " *" "" string))
	 (added (string-match (format "%s$" add) string)))
    (if added string (concat string add))))

When concatenating directory names it is important to take care of slashes.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>> OK, how is it suppose to look then?
>
> I think by using file- related functions:

Well, yes, but I meant my stuff in particular:

  https://dataswamp.org/~incal/conf/.emacs

> (defun doi-file-find (md5)
>   "Return existing MD5 file for Double Opt-In.confirmation
>   or NIL."

Heh :)

> (if (file-exists-p file) file nil)))

(and (file-exists-p file) file)

-- 
underground experts united
https://dataswamp.org/~incal

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-07-06 05:06]:

> > (if (file-exists-p file) file nil)))
> 
> (and (file-exists-p file) file)

I wasn't use to that form. But yes!

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>>> (if (file-exists-p file) file nil)))
>> 
>> (and (file-exists-p file) file)
>
> I wasn't use to that form. But yes!

Everyone should have a pal like me...

-- 
underground experts united
https://dataswamp.org/~incal

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>> We _represent_ file names as strings, but they are not
>> normal strings. Just like characters are represented as
>> integers, but they are not just simple integers, they have
>> additional special attributes and behaviors.
>
> I get your point, what I don't agree with is the
> Formulierung. It is for same reason as you mentioned it,
> people are reading our writings and our form or text should
> be possibly clear. If you wish to say they are not normal
> strings, demonstrate it. By demonstration below it shows to
> be of type string. What is else there?

I don't know :)

-- 
underground experts united
https://dataswamp.org/~incal

Re: Printf and quoting in general, SQL injection in particular [was: Emacs Modular Configuration: the preferable way]

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

> (type-of "/usr/bin/emacs") ; string
> (type-of "/usr/bin/emacs-27.2") ; string
>
> Repeat after me: "The meaning of a string is not equal to
> its type.."

(require 'cl-lib)

(setq denominator 1) ; 1

(cl-decf denominator) ; 0

(/ 1 denominator) ; Arithmetic error - oh, no!

OK, lissen up guys - this does it! From now on, no more
subtraction from integer denominators! This example shows the
dangers of that. Please use `denominator-arithmetics' that
will make sure the denominator isn't accidentally set to 0
before division. /Administration

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

Philip Kaludercic wrote:

>> Why are you using 'format' where expand-file-name should
>> be used?
>
> Does it really make a difference in a personal
> configuration, especially when you don't use Windows or
> other non /-delimiting systems?

I knew it, this is some Windows thing, I can't say I care
about Windows.

BTW, are we even allowed to talk about Windows here or did
they acquire a free software licence which I'm unaware of?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Modular Configuration: the preferable way.

[Lars Ingebrigtsen]

Philip Kaludercic <philipk@posteo.net> writes:

> Does it really make a difference in a personal configuration, especially
> when you don't use Windows or other non /-delimiting systems?

No, the code here is fine, and I think chiding people for writing
perfectly find code is counterproductive.  It's not what I'd put in a
general library meant for redistribution, but nobody has any obligation
to write code on any particular form for their own usage.

Besides, `expand-file-name' has annoying corner cases, and Emacs should
really grow a variation that's actually sound and reliable instead of
the current DWIM-ish mess.  For instance, the following loop might
signal an error:

(dolist (file (directory-files "/tmp/"))
  (unless (string-match "^/tmp" (expand-file-name file "/tmp"))
    (error "boom")))

while

(dolist (file (directory-files "/tmp/"))
  (unless (string-match "^/tmp" (format "/tmp/%s" file))
    (error "boom")))

never will.  To see the difference, say

$ touch "/tmp/~$USER"

first.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

Lars Ingebrigtsen wrote:

>> Does it really make a difference in a personal
>> configuration, especially when you don't use Windows or
>> other non /-delimiting systems?
>
> No, the code here is fine, and I think chiding people for
> writing perfectly find code is counterproductive. It's not
> what I'd put in a general library meant for redistribution,
> but nobody has any obligation to write code on any
> particular form for their own usage.

Yeah, however you write that from a position not just of
highness but of vastness and interestingness, but to all the
underground little devils that only have the accursed
configuration file as an entry point to Lisp they might want
to put it the right way in their code as well...

Or the olympic athlete, it is more important that he does the
pushups than the prisoner, right, because if the athlete
doesn't do them, he'll loose the gold medal game, while if the
prisoner doesn't do them, he'll loose his sanity...

?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Modular Configuration: the preferable way.

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-06-21 05:57]:
>   (let ((emacs-init-files '(
>                             abc.el
>                             align-incal.el
>                             batch.el
>                             bibtex-incal.el
>                             bike.el
>                             bmi.el
>                             buc.el
>                             buffer-menu.el
>                             caps-back.el
>                             close.el
>                             console-keys.el
>                             count.el
>                             custom-vars.el
>                             day.el
>                             dired-incal.el
>                             ecat-incal.el
>                             echo-message.el
>                             edit.el
>                             elpa.el
>                             emacs-shell.el
>                             face.el
>                             file-write-to.el
>                             file.el
>                             fill-incal.el
>                             frame-size.el
>                             geh.el
>                             get-search-string.el
>                             help-incal.el
>                             info-incal.el
>                             isbn-verify.el
>                             issn-verify.el
>                             iterate-files.el
>                             jean.el
>                             keys.el
>                             kill-path.el
>                             kill.el
>                             latex.el
>                             lights.el
>                             linux-shell.el
>                             list-quoted-functions.el
>                             man-incal.el
>                             match-data-format.el
>                             math.el
>                             measure.el
>                             minor-modes.el
>                             misc.el
>                             mode-by-filename.el
>                             mode-line.el
>                             navigate-fs-keys.el
>                             negative-subtraction.el
>                             perm.el
>                             printer.el
>                             quit.el
>                             random.el
>                             re-make-list.el
>                             replace-list.el
>                             revert-buffer.el
>                             scale.el
>                             scroll.el
>                             search-regexp-in-files.el
>                             sequence-string.el
>                             shell-cli.el
>                             show-command.el
>                             signal.el
>                             sort-incal.el
>                             spell.el
>                             street.el
>                             string.el
>                             sudo-user-path.el
>                             super.el
>                             survivor.el
>                             switch-to-buffer-regexp.el
>                             switch-to-buffer.el
>                             tabs.el
>                             test-face.el
>                             test.el
>                             time-cmp.el
>                             time-incal.el
>                             time-insert.el
>                             todo-did.el
>                             tramp-incal.el
>                             variance.el
>                             vt.el
>                             window-incal.el
>                             wood.el
>                             wrap-search.el
>                             xsel.el
>                             yank.el
>                             )))
>     (dolist (f emacs-init-files)
>       (load-file (format "%s/%s" init-dir f) )))

You have many files to load and they are probably in different
directories together with other files which are not to be loaded. 

I would then create a new directory and use key command `Y' from
package `dired-x' to place symbolic link in the designated directory.

In that directory I would have symbolic links of files that have to be
loaded. If I don't need a file, I would remove symbolic link. The
original file would be wherever it is.

Then instead of having the long list there and updating it each time,
I would use this:

(mapc 'load-file (directory-files "~/.emacs.d/" t ".el$"))




-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Emacs Modular Configuration: the preferable way.

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

> You have many files to load and they are probably in
> different directories

That is evident from the code...

> together with other files which are not to be loaded.

That doesn't matter since every file that is to be loaded is
mentioned explicitly.

> Then instead of having the long list there and updating it
> each time, I would use this:
>
> (mapc 'load-file (directory-files "~/.emacs.d/" t ".el$"))

But then you can't do binary search on a file basis, like
I said, which is a huge advantage, especially if you have
a lot of source.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Modular Configuration: the preferable way.

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-06-22 03:16]:
> Jean Louis wrote:
> 
> > You have many files to load and they are probably in
> > different directories
> 
> That is evident from the code...
> 
> > together with other files which are not to be loaded.
> 
> That doesn't matter since every file that is to be loaded is
> mentioned explicitly.
> 
> > Then instead of having the long list there and updating it
> > each time, I would use this:
> >
> > (mapc 'load-file (directory-files "~/.emacs.d/" t ".el$"))
> 
> But then you can't do binary search on a file basis, like
> I said, which is a huge advantage, especially if you have
> a lot of source.

What is binary search on a file basis?


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Emacs Modular Configuration: the preferable way.

[Jean Louis]

* Hongyi Zhao <hongyi.zhao@gmail.com> [2021-06-21 04:51]:
> Dear all,
> 
> I noticed the following instructions/tools for Emacs modular configuration:
> 
> https://www.emacswiki.org/emacs/DotEmacsModular
> https://www.emacswiki.org/emacs/DotEmacsStructuring
> https://github.com/emacs-jp/init-loader
> 
> There are so many ways to do this. What's the preferable way?

Whatever you wish and want. You can feel if you need multiple files or
not. Solutions are very individuals.


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Emacs Modular Configuration: the preferable way.

[Hongyi Zhao]

On Mon, Jun 21, 2021 at 2:40 PM Jean Louis <bugs@gnu.support> wrote:
>
> * Hongyi Zhao <hongyi.zhao@gmail.com> [2021-06-21 04:51]:
> > Dear all,
> >
> > I noticed the following instructions/tools for Emacs modular configuration:
> >
> > https://www.emacswiki.org/emacs/DotEmacsModular
> > https://www.emacswiki.org/emacs/DotEmacsStructuring
> > https://github.com/emacs-jp/init-loader
> >
> > There are so many ways to do this. What's the preferable way?
>
> Whatever you wish and want. You can feel if you need multiple files or
> not. Solutions are very individuals.

For now, my demand is as follows:

1. Use straight and use-package as the package manager.
2. All packages configuration file should call 1., especially for a
testing scenario when debugging a package/several packages.
3. Better granular control based on different major mode, popular
programming languages.

Regards
-- 
Assoc. Prof. Hongyi Zhao <hongyi.zhao@gmail.com>
Theory and Simulation of Materials
Hebei Vocational University of Technology and Engineering
NO. 552 North Gangtie Road, Xingtai, China

Re: Emacs Modular Configuration: the preferable way.

[Arthur Miller]

Hongyi Zhao <hongyi.zhao@gmail.com> writes:

> On Mon, Jun 21, 2021 at 2:40 PM Jean Louis <bugs@gnu.support> wrote:
>>
>> * Hongyi Zhao <hongyi.zhao@gmail.com> [2021-06-21 04:51]:
>> > Dear all,
>> >
>> > I noticed the following instructions/tools for Emacs modular configuration:
>> >
>> > https://www.emacswiki.org/emacs/DotEmacsModular
>> > https://www.emacswiki.org/emacs/DotEmacsStructuring
>> > https://github.com/emacs-jp/init-loader
>> >
>> > There are so many ways to do this. What's the preferable way?
>>
>> Whatever you wish and want. You can feel if you need multiple files or
>> not. Solutions are very individuals.
>
> For now, my demand is as follows:
>
> 1. Use straight and use-package as the package manager.

Why? Some special reason?

> 2. All packages configuration file should call 1., especially for a
> testing scenario when debugging a package/several packages.

Why? Just for sake of calling it or you have some better reason?

> 3. Better granular control based on different major mode, popular
> programming languages.
>

You don't need straight, nor any special package for this, it is how
you structure your code. Check here, granularity is per package (mostly):

https://github.com/amno1/.emacs.d/blob/main/init.org 

Nothing external required, just built-in stuff that comes out of the
box. Package.el is used to fetch packages from elpa/melpa and few
simple macros to save some typing, not even use-package needed and I
think it is pretty structured init file. I can even configure
early-init.el as it was a hook, which lets me structure entire init
process in one place. 

Re: Emacs Modular Configuration: the preferable way.

[Hongyi Zhao]

On Mon, Jun 21, 2021 at 6:06 PM Arthur Miller <arthur.miller@live.com> wrote:
>
> Hongyi Zhao <hongyi.zhao@gmail.com> writes:
>
> > On Mon, Jun 21, 2021 at 2:40 PM Jean Louis <bugs@gnu.support> wrote:
> >>
> >> * Hongyi Zhao <hongyi.zhao@gmail.com> [2021-06-21 04:51]:
> >> > Dear all,
> >> >
> >> > I noticed the following instructions/tools for Emacs modular configuration:
> >> >
> >> > https://www.emacswiki.org/emacs/DotEmacsModular
> >> > https://www.emacswiki.org/emacs/DotEmacsStructuring
> >> > https://github.com/emacs-jp/init-loader
> >> >
> >> > There are so many ways to do this. What's the preferable way?
> >>
> >> Whatever you wish and want. You can feel if you need multiple files or
> >> not. Solutions are very individuals.
> >
> > For now, my demand is as follows:
> >
> > 1. Use straight and use-package as the package manager.
>
> Why? Some special reason?

For working/dealing with all packages, regardless that they are hosted
by Emacs package repos or not. Straight works with git perfectly, and
at the same time, it is also seamlessly integrates with use-package.

>
> > 2. All packages configuration file should call 1., especially for a
> > testing scenario when debugging a package/several packages.
>
> Why? Just for sake of calling it or you have some better reason?

Same as above.

>
> > 3. Better granular control based on different major mode, popular
> > programming languages.
> >
>
> You don't need straight, nor any special package for this, it is how
> you structure your code. Check here, granularity is per package (mostly):
>
> https://github.com/amno1/.emacs.d/blob/main/init.org
>
> Nothing external required, just built-in stuff that comes out of the
> box. Package.el is used to fetch packages from elpa/melpa and few
> simple macros to save some typing, not even use-package needed and I
> think it is pretty structured init file. I can even configure
> early-init.el as it was a hook, which lets me structure entire init
> process in one place.

From my experience, package.el's package fetch mechanism/speed is
fragile and very poor by comparison
with straight.

Regards
-- 
Assoc. Prof. Hongyi Zhao <hongyi.zhao@gmail.com>
Theory and Simulation of Materials
Hebei Vocational University of Technology and Engineering
NO. 552 North Gangtie Road, Xingtai, China

Re: Emacs Modular Configuration: the preferable way.

[Arthur Miller]

Hongyi Zhao <hongyi.zhao@gmail.com> writes:

> On Mon, Jun 21, 2021 at 6:06 PM Arthur Miller <arthur.miller@live.com> wrote:
>>
>> Hongyi Zhao <hongyi.zhao@gmail.com> writes:
>>
>> > On Mon, Jun 21, 2021 at 2:40 PM Jean Louis <bugs@gnu.support> wrote:
>> >>
>> >> * Hongyi Zhao <hongyi.zhao@gmail.com> [2021-06-21 04:51]:
>> >> > Dear all,
>> >> >
>> >> > I noticed the following instructions/tools for Emacs modular configuration:
>> >> >
>> >> > https://www.emacswiki.org/emacs/DotEmacsModular
>> >> > https://www.emacswiki.org/emacs/DotEmacsStructuring
>> >> > https://github.com/emacs-jp/init-loader
>> >> >
>> >> > There are so many ways to do this. What's the preferable way?
>> >>
>> >> Whatever you wish and want. You can feel if you need multiple files or
>> >> not. Solutions are very individuals.
>> >
>> > For now, my demand is as follows:
>> >
>> > 1. Use straight and use-package as the package manager.
>>
>> Why? Some special reason?
>
> For working/dealing with all packages, regardless that they are hosted
> by Emacs package repos or not. Straight works with git perfectly, and

Which opens for a vulnerability which was discussed on emacs-help some
few months ago. Of course it is  your personal choice what you will
download and use, but I would be very careful to just download stuff
from git without looking at it first.

>> Why? Just for sake of calling it or you have some better reason?
>
> Same as above.

I get that "granularity" you speak above without need for 3k+ lines of
code :).

>>
>> > 3. Better granular control based on different major mode, popular
>> > programming languages.
>> >
>>
>> You don't need straight, nor any special package for this, it is how
>> you structure your code. Check here, granularity is per package (mostly):
>>
>> https://github.com/amno1/.emacs.d/blob/main/init.org
>>
>> Nothing external required, just built-in stuff that comes out of the
>> box. Package.el is used to fetch packages from elpa/melpa and few
>> simple macros to save some typing, not even use-package needed and I
>> think it is pretty structured init file. I can even configure
>> early-init.el as it was a hook, which lets me structure entire init
>> process in one place.
>
> From my experience, package.el's package fetch mechanism/speed is
> fragile and very poor by comparison
> with straight.

Really? Does straight.el implemnt it's own network stack? I thought it
uses Emacs facilities, same as package.el. Do you have any numbers to
back up your opinion?

Since you are so experienced and already have made your mind I don't see
the point of even asking. Especially since author of package.el have
already done all the "thinking" for you, at least as he puts it on the
project page:

"straight.el frees you from needing to think about package management,
since I already did all the thinking to figure how best to design
everything."

:D I just have no comments on that one.

Re: Emacs Modular Configuration: the preferable way.

[Hongyi Zhao]

On Mon, Jun 21, 2021 at 7:11 PM Arthur Miller <arthur.miller@live.com> wrote:
>
> Hongyi Zhao <hongyi.zhao@gmail.com> writes:
>
> > On Mon, Jun 21, 2021 at 6:06 PM Arthur Miller <arthur.miller@live.com> wrote:
> >>
> >> Hongyi Zhao <hongyi.zhao@gmail.com> writes:
> >>
> >> > On Mon, Jun 21, 2021 at 2:40 PM Jean Louis <bugs@gnu.support> wrote:
> >> >>
> >> >> * Hongyi Zhao <hongyi.zhao@gmail.com> [2021-06-21 04:51]:
> >> >> > Dear all,
> >> >> >
> >> >> > I noticed the following instructions/tools for Emacs modular configuration:
> >> >> >
> >> >> > https://www.emacswiki.org/emacs/DotEmacsModular
> >> >> > https://www.emacswiki.org/emacs/DotEmacsStructuring
> >> >> > https://github.com/emacs-jp/init-loader
> >> >> >
> >> >> > There are so many ways to do this. What's the preferable way?
> >> >>
> >> >> Whatever you wish and want. You can feel if you need multiple files or
> >> >> not. Solutions are very individuals.
> >> >
> >> > For now, my demand is as follows:
> >> >
> >> > 1. Use straight and use-package as the package manager.
> >>
> >> Why? Some special reason?
> >
> > For working/dealing with all packages, regardless that they are hosted
> > by Emacs package repos or not. Straight works with git perfectly, and
>
> Which opens for a vulnerability which was discussed on emacs-help some
> few months ago. Of course it is  your personal choice what you will
> download and use, but I would be very careful to just download stuff
> from git without looking at it first.
>
> >> Why? Just for sake of calling it or you have some better reason?
> >
> > Same as above.
>
> I get that "granularity" you speak above without need for 3k+ lines of
> code :).
>
> >>
> >> > 3. Better granular control based on different major mode, popular
> >> > programming languages.
> >> >
> >>
> >> You don't need straight, nor any special package for this, it is how
> >> you structure your code. Check here, granularity is per package (mostly):
> >>
> >> https://github.com/amno1/.emacs.d/blob/main/init.org
> >>
> >> Nothing external required, just built-in stuff that comes out of the
> >> box. Package.el is used to fetch packages from elpa/melpa and few
> >> simple macros to save some typing, not even use-package needed and I
> >> think it is pretty structured init file. I can even configure
> >> early-init.el as it was a hook, which lets me structure entire init
> >> process in one place.
> >
> > From my experience, package.el's package fetch mechanism/speed is
> > fragile and very poor by comparison
> > with straight.
>
> Really? Does straight.el implemnt it's own network stack? I thought it
> uses Emacs facilities, same as package.el. Do you have any numbers to
> back up your opinion?

TBF, no. Just out of limited intuition. To a large extent, I could be
wrong. As you've told, it only relies on git for network operations.

HY
>
> Since you are so experienced and already have made your mind I don't see
> the point of even asking. Especially since author of package.el have
> already done all the "thinking" for you, at least as he puts it on the
> project page:
>
> "straight.el frees you from needing to think about package management,
> since I already did all the thinking to figure how best to design
> everything."
>
> :D I just have no comments on that one.
>


-- 
Assoc. Prof. Hongyi Zhao <hongyi.zhao@gmail.com>
Theory and Simulation of Materials
Hebei Vocational University of Technology and Engineering
NO. 552 North Gangtie Road, Xingtai, China