Surprising behaviour of 'append' with strings

Surprising behaviour of 'append' with strings

[R. Diez]

Hi all:

I want to build a list of command-line arguments to run a program. Each element of the list must be a string.

The commands are complex shell invocations with nested shell commands. I have been shell-quoting and concatenating plain strings for a long time, but it is error prone (in my experience).

So now I am testing a different approach: I am building a list of strings, joining them with (string-join my-cmd-args " "), and passing the result to 'compile'.

For nested commands, I use instead (shell-quote-argument (string-join my-sub-cmd-args " ")), and then append the resulting string to the parent command's list of arguments. And so on.

When building a single command, there are optional, complex lists of arguments that get built somewhere else. So the code is building and concatenating many lists. This is the idea (very simplified):

(setq cmd-args (list "arg 1" "arg 2"))

(setq whole-cmd (append "cmd-exe" cmd-args))

However, I got into trouble with 'append'. Its documentation states:

"Each argument may be a list, vector or string."

I tested it like this:

(append "1")      -> "1"
(append "1" "2")  -> (49 . "2")

That behaviour is surprising and it does not fit the bill.

I need a routine like this:

(my-append
   "1"
   (list "2" "3")
   "4"
)

The result should be:
  ("1" "2" "3" "4")

As a bonus, 'my-append' should check that all elements are strings, or list of strings (no nested lists, and no other data types).

The trouble is, my Lisp skills are rather limited. Can somebody help?

Thanks in advance,
   rdiez

Re: Surprising behaviour of 'append' with strings

[Stephen Berman]

On Sun, 6 Nov 2022 12:05:07 +0100 "R. Diez" <rdiezmail-emacs@yahoo.de> wrote:

> Hi all:
>
> I want to build a list of command-line arguments to run a
> program. Each element of the list must be a string.
[...]
> However, I got into trouble with 'append'. Its documentation states:
>
> "Each argument may be a list, vector or string."
>
> I tested it like this:
>
> (append "1")      -> "1"
> (append "1" "2")  -> (49 . "2")
>
> That behaviour is surprising and it does not fit the bill.

It appears you didn't appreciate the preceding line of the doc string:
"The result is a list whose elements are the elements of all the
arguments."  So the result contains the elements of the string "1",
which is just the single character ?1, whose value is 49.

> I need a routine like this:
>
> (my-append
>   "1"
>   (list "2" "3")
>   "4"
> )
>
> The result should be:
>  ("1" "2" "3" "4")
>
> As a bonus, 'my-append' should check that all elements are strings, or
> list of strings (no nested lists, and no other data types).

If you are willing to make the argument have this kind of form:

'("1" ("2" "3") "4")

then flatten-tree should do the job.  To also check that the elements
are either a string or a list of strings, you could use this:

(defun my-append (l)
  (dolist (e l)
    (cond ((proper-list-p e)
	   (dolist (s e)
	     (unless (stringp s)
	       (user-error "Invalid list"))))
	  (t (unless (stringp e)
	       (user-error "Invalid list")))))
  (flatten-tree l))

Steve Berman

Re: Surprising behaviour of 'append' with strings

[Emanuel Berg]

R. Diez wrote:

> I want to build a list of command-line arguments to run
> a program. Each element of the list must be a string.
>
> The commands are complex shell invocations with nested shell
> commands.

Isn't it better to use a shell script language then?

In Elisp maybe you can use `let' and then
`shell-command-to-string' but even that is an uphill
battle IMO ...

-- 
underground experts united
https://dataswamp.org/~incal

Re: Surprising behaviour of 'append' with strings

[Emanuel Berg]

R. Diez wrote:

> However, I got into trouble with 'append'. Its documentation
> states:
>
> "Each argument may be a list, vector or string."
>
> I tested it like this:
>
> (append "1")      -> "1"
> (append "1" "2")  -> (49 . "2")

`append' is like that with strings (well, obviously), try it
with lists instead.

Also see `cl-concatenate', `push', and many other ways to
create and manipulate lists.

But I think you are on the wrong track here, you insert
a layer of complexity with this Elisp to semi-manually hammer
out shell commands, this will be very time-consuming and
error-prone, better to have all the shell stuff in shell
functions and scripts, those OTOH you can absolutely execute
from Emacs/Elisp and use their results - and that's a fine way
to do it.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Surprising behaviour of 'append' with strings

[Emanuel Berg]

Stephen Berman wrote:

>> I tested it like this:
>>
>> (append "1")      -> "1"
>> (append "1" "2")  -> (49 . "2")
>>
>> That behaviour is surprising and it does not fit the bill.
>
> It appears you didn't appreciate the preceding line of the
> doc string: "The result is a list whose elements are the
> elements of all the arguments." So the result contains the
> elements of the string "1", which is just the single
> character ?1, whose value is 49.

Strings are not really lists of characters. Well, conceptually
they are! And `append' seems to agree ...

-- 
underground experts united
https://dataswamp.org/~incal

Re: Surprising behaviour of 'append' with strings

[Eli Zaretskii]

> Date: Sun, 6 Nov 2022 12:05:07 +0100
> From: "R. Diez" <rdiezmail-emacs@yahoo.de>
> 
> I need a routine like this:
> 
> (my-append
>    "1"
>    (list "2" "3")
>    "4"
> )
> 
> The result should be:
>   ("1" "2" "3" "4")

You want 'mapconcat', I think.  'append' doesn't "append strings", it
does something very different.

Re: Surprising behaviour of 'append' with strings

[Jean Louis]

Hello,

* R. Diez <rdiezmail-emacs@yahoo.de> [2022-11-06 14:07]:
> I want to build a list of command-line arguments to run a
> program. Each element of the list must be a string.

First I was using function `shell-command' which is pretty liberate,
and it invokes shell and string as command, but that one is in reality
error prone and difficult, especially when there is quoting required.

> The commands are complex shell invocations with nested shell
> commands. I have been shell-quoting and concatenating plain strings
> for a long time, but it is error prone (in my experience).

You should use `call-process'

(call-process PROGRAM &optional INFILE DESTINATION DISPLAY &rest ARGS)
  Probably introduced at or before Emacs version 18.

Then look ARGS, that is where you place the list.

> So now I am testing a different approach: I am building a list of
> strings, joining them with (string-join my-cmd-args " "), and
> passing the result to 'compile'.

I guess that could be all thinking in wrong direction, look at
`call-process' first and tell us if you tried that one out. It
minimizes all errors.

> (append "1") ⇒ "1"
> (append "1" "2") ⇒ (49 . "2")

Right

> (my-append
>   "1"
>   (list "2" "3")
>   "4"
> )
> 
> The result should be:
>  ("1" "2" "3" "4")

`append' is to append for example lists:

(append (list 1) (list 2 3 4)) ⇒ (1 2 3 4)

(append (list "1") (list "2" "3" "4")) ⇒ ("1" "2" "3" "4")

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Surprising behaviour of 'append' with strings

[Michael Heerdegen]

Emanuel Berg <incal@dataswamp.org> writes:

> Strings are not really lists of characters. Well, conceptually
> they are! And `append' seems to agree ...

Lists and strings are sequences.  The elements of the sequences are
appended.

Michael.

Re: Surprising behaviour of 'append' with strings

[Michael Heerdegen]

"R. Diez" <rdiezmail-emacs@yahoo.de> writes:

> I need a routine like this:
>
> (my-append
>   "1"
>   (list "2" "3")
>   "4"
> )
>
> The result should be:
>  ("1" "2" "3" "4")

(defun my-append (&rest args)
  (cl-mapcan (lambda (x) (if (consp x) x (list x)))
             args))

> As a bonus, 'my-append' should check that all elements are strings, or
> list of strings (no nested lists, and no other data types).

Use `cl-every' on the result of `my-append' with a `stringp' predicate.

Michael.

Re: Surprising behaviour of 'append' with strings

[Emanuel Berg]

Michael Heerdegen wrote:

>> Strings are not really lists of characters. Well,
>> conceptually they are! And `append' seems to agree ...
>
> Lists and strings are sequences.

Hm, I wonder what those are called according to
CS Type theory? Metatypes?

https://en.wikipedia.org/wiki/Type_theory

-- 
underground experts united
https://dataswamp.org/~incal

RE: [External] : Re: Surprising behaviour of 'append' with strings

[Drew Adams]

> > As a bonus, 'my-append' should check that all elements are strings, or
> > list of strings (no nested lists, and no other data types).
> 
> Use `cl-every' on the result of `my-append' with a `stringp' predicate.

Maybe.  Depending on what OP means by "should check".

Maybe he means silently filter out anything that doesn't check OK.  Or maybe he means raise an error in that case.  Or maybe he wants a report (e.g. a list) of all such "bad" elements.

Maybe he doesn't mind creating multiple copies of lists, to do the checking.  Or maybe he does care.

If he wants to know all of the "bad" elements then he needs to always traverse the whole list.  If he wants to raise an error as soon as the first "bad" element is detected, then he may not need to traverse it past the first element.