Re: [elpa] externals/org 26ef5e3e5b: org-src: Use `sh-mode' for all the shells it can handle

Re: [elpa] externals/org 26ef5e3e5b: org-src: Use `sh-mode' for all the shells it can handle

[Stefan Monnier]

>  (defcustom org-src-lang-modes
> -  '(("C" . c)
> +  `(("C" . c)
>      ("C++" . c++)
>      ("asymptote" . asy)
> -    ("bash" . sh)
>      ("beamer" . latex)
>      ("calc" . fundamental)
>      ("cpp" . c++)
> @@ -208,9 +215,10 @@ but which mess up the display of a snippet in Org exported files.")
>      ("elisp" . emacs-lisp)
>      ("ocaml" . tuareg)
>      ("screen" . shell-script)
> -    ("shell" . sh)
>      ("sqlite" . sql)
> -    ("toml" . conf-toml))
> +    ("toml" . conf-toml)
> +    ("shell" . sh)
> +    ,@(org-src--get-known-shells))
>    "Alist mapping languages to their major mode.

Side note: while it really doesn't matter here for such trivial
top-level code, I prefer to put such ,@ at the beginning rather than the
end of lists, when it's an option.  Basically because it's more
efficient to add to the beginning rather than to the end of a list:

    ELISP> (macroexpand '`(,@(list 1 2) a b c d))
    (append (list 1 2) '(a b c d))
    ELISP> (macroexpand '`(a b c d ,@(list 1 2)))
    (cons 'a (cons 'b (cons 'c (cons 'd (list 1 2)))))
    ELISP>


-- Stefan

Re: [elpa] externals/org 26ef5e3e5b: org-src: Use `sh-mode' for all the shells it can handle

[Lynn Winebarger]

[-- Attachment #1: Type: text/plain, Size: 1447 bytes --]

On Mon, Apr 17, 2023, 12:10 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> >  (defcustom org-src-lang-modes
> > -  '(("C" . c)
> > +  `(("C" . c)
> >      ("C++" . c++)
> >      ("asymptote" . asy)
> > -    ("bash" . sh)
> >      ("beamer" . latex)
> >      ("calc" . fundamental)
> >      ("cpp" . c++)
> > @@ -208,9 +215,10 @@ but which mess up the display of a snippet in Org
> exported files.")
> >      ("elisp" . emacs-lisp)
> >      ("ocaml" . tuareg)
> >      ("screen" . shell-script)
> > -    ("shell" . sh)
> >      ("sqlite" . sql)
> > -    ("toml" . conf-toml))
> > +    ("toml" . conf-toml)
> > +    ("shell" . sh)
> > +    ,@(org-src--get-known-shells))
> >    "Alist mapping languages to their major mode.
>
> Side note: while it really doesn't matter here for such trivial
> top-level code, I prefer to put such ,@ at the beginning rather than the
> end of lists, when it's an option.  Basically because it's more
> efficient to add to the beginning rather than to the end of a list:
>
>     ELISP> (macroexpand '`(,@(list 1 2) a b c d))
>     (append (list 1 2) '(a b c d))
>     ELISP> (macroexpand '`(a b c d ,@(list 1 2)))
>     (cons 'a (cons 'b (cons 'c (cons 'd (list 1 2)))))
>     ELISP>
>


Why is the former more efficient than the latter?  It looks like the former
would have to construct the '(1 2) list twice, and the latter only once.
And the '(a b c d) cons cells are only allocated once either way.

Lynn

[-- Attachment #2: Type: text/html, Size: 2473 bytes --]

Re: [elpa] externals/org 26ef5e3e5b: org-src: Use `sh-mode' for all the shells it can handle

[Philip Kaludercic]

Lynn Winebarger <owinebar@gmail.com> writes:

> On Mon, Apr 17, 2023, 12:10 PM Stefan Monnier <monnier@iro.umontreal.ca>
> wrote:
>
>> >  (defcustom org-src-lang-modes
>> > -  '(("C" . c)
>> > +  `(("C" . c)
>> >      ("C++" . c++)
>> >      ("asymptote" . asy)
>> > -    ("bash" . sh)
>> >      ("beamer" . latex)
>> >      ("calc" . fundamental)
>> >      ("cpp" . c++)
>> > @@ -208,9 +215,10 @@ but which mess up the display of a snippet in Org
>> exported files.")
>> >      ("elisp" . emacs-lisp)
>> >      ("ocaml" . tuareg)
>> >      ("screen" . shell-script)
>> > -    ("shell" . sh)
>> >      ("sqlite" . sql)
>> > -    ("toml" . conf-toml))
>> > +    ("toml" . conf-toml)
>> > +    ("shell" . sh)
>> > +    ,@(org-src--get-known-shells))
>> >    "Alist mapping languages to their major mode.
>>
>> Side note: while it really doesn't matter here for such trivial
>> top-level code, I prefer to put such ,@ at the beginning rather than the
>> end of lists, when it's an option.  Basically because it's more
>> efficient to add to the beginning rather than to the end of a list:
>>
>>     ELISP> (macroexpand '`(,@(list 1 2) a b c d))
>>     (append (list 1 2) '(a b c d))
>>     ELISP> (macroexpand '`(a b c d ,@(list 1 2)))
>>     (cons 'a (cons 'b (cons 'c (cons 'd (list 1 2)))))
>>     ELISP>

Is this just an implementation detail, or is there a reason that this
could not expand to 

  (append '(a b c d) (list 1 2))

> Why is the former more efficient than the latter?  It looks like the former
> would have to construct the '(1 2) list twice, and the latter only once.
> And the '(a b c d) cons cells are only allocated once either way.

(I believe) It is more efficient, since there are fewer function calls.
I have had issues in the past with ,@ expanding to code that would have
exceeded the maximal evaluation depth, until I split it up and manually
used `append' that has the advantage of being implemented in C.

> Lynn

Re: [elpa] externals/org 26ef5e3e5b: org-src: Use `sh-mode' for all the shells it can handle

[Andreas Schwab]

On Apr 17 2023, Philip Kaludercic wrote:

>> Why is the former more efficient than the latter?  It looks like the former
>> would have to construct the '(1 2) list twice, and the latter only once.
>> And the '(a b c d) cons cells are only allocated once either way.
>
> (I believe) It is more efficient, since there are fewer function calls.
> I have had issues in the past with ,@ expanding to code that would have
> exceeded the maximal evaluation depth, until I split it up and manually
> used `append' that has the advantage of being implemented in C.

cons is a bytecode intrinsic, so it shouldn't make that much of a
difference.
(cons 'a (cons 'b (cons 'c (cons 'd (list 1 2 3))))) is translated into

0	constant  a
1	constant  b
2	constant  c
3	constant  d
4	constant  1
5	constant  2
6	constant  3
7	listN	  7

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: [elpa] externals/org 26ef5e3e5b: org-src: Use `sh-mode' for all the shells it can handle

[Mattias Engdegård]

17 apr. 2023 kl. 18.27 skrev Philip Kaludercic <philipk@posteo.net>:

> Is this just an implementation detail, or is there a reason that this
> could not expand to 
> 
>  (append '(a b c d) (list 1 2))

We could actually turn

   (cons X1 ... (cons Xn Y)...)

into

   (nconc (list X1 ... Xn) Y)

in the byte-compiler: the latter is smaller for n≥2, and a quick benchmark suggests it's faster for n≥7. I may give it a go. Backquote would clearly benefit.

Re: [elpa] externals/org 26ef5e3e5b: org-src: Use `sh-mode' for all the shells it can handle

[Philip Kaludercic]

Andreas Schwab <schwab@linux-m68k.org> writes:

> On Apr 17 2023, Philip Kaludercic wrote:
>
>>> Why is the former more efficient than the latter?  It looks like the former
>>> would have to construct the '(1 2) list twice, and the latter only once.
>>> And the '(a b c d) cons cells are only allocated once either way.
>>
>> (I believe) It is more efficient, since there are fewer function calls.
>> I have had issues in the past with ,@ expanding to code that would have
>> exceeded the maximal evaluation depth, until I split it up and manually
>> used `append' that has the advantage of being implemented in C.
>
> cons is a bytecode intrinsic, so it shouldn't make that much of a
> difference.
> (cons 'a (cons 'b (cons 'c (cons 'd (list 1 2 3))))) is translated into
>
> 0	constant  a
> 1	constant  b
> 2	constant  c
> 3	constant  d
> 4	constant  1
> 5	constant  2
> 6	constant  3
> 7	listN	  7

Would this also affect code in a `eval-when-compile' block (which was my
anecdotal case)?  From what I see, that shouldn't be byte-compiled.

Re: [elpa] externals/org 26ef5e3e5b: org-src: Use `sh-mode' for all the shells it can handle

[Mattias Engdegård]

17 apr. 2023 kl. 18.39 skrev Mattias Engdegård <mattiase@acm.org>:
>   (nconc (list X1 ... Xn) Y)
> 
> in the byte-compiler: the latter is smaller for n≥2

Correction: n≥3

Re: [elpa] externals/org 26ef5e3e5b: org-src: Use `sh-mode' for all the shells it can handle

[Stefan Monnier]

>>     ELISP> (macroexpand '`(,@(list 1 2) a b c d))
>>     (append (list 1 2) '(a b c d))
>>     ELISP> (macroexpand '`(a b c d ,@(list 1 2)))
>>     (cons 'a (cons 'b (cons 'c (cons 'd (list 1 2)))))
>>     ELISP>
> Why is the former more efficient than the latter?

One allocates N₁+N₂ cons cells, then other only allocates N₁ cons-cells.
This is true regardless of how we macroexpand and byte-compile the code.
This only makes a diference for code that's executed several times
(i.e. not at toplevel).


        Stefan

Re: [elpa] externals/org 26ef5e3e5b: org-src: Use `sh-mode' for all the shells it can handle

[Lynn Winebarger]

[-- Attachment #1: Type: text/plain, Size: 844 bytes --]

On Mon, Apr 17, 2023, 1:11 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> >>     ELISP> (macroexpand '`(,@(list 1 2) a b c d))
> >>     (append (list 1 2) '(a b c d))
> >>     ELISP> (macroexpand '`(a b c d ,@(list 1 2)))
> >>     (cons 'a (cons 'b (cons 'c (cons 'd (list 1 2)))))
> >>     ELISP>
> > Why is the former more efficient than the latter?
>
> One allocates N₁+N₂ cons cells, then other only allocates N₁ cons-cells.
> This is true regardless of how we macroexpand and byte-compile the code.
> This only makes a diference for code that's executed several times
> (i.e. not at toplevel).
>

I see - what you meant by "such ,@" was when the preceding elements are
constant.

I've seen a veteran Lisp programmer get bitten by side-effecting a list
constructed with that feature of backquote.

Lynn

[-- Attachment #2: Type: text/html, Size: 1604 bytes --]

Re: [elpa] externals/org 26ef5e3e5b: org-src: Use `sh-mode' for all the shells it can handle

[Stefan Monnier]

> We could actually turn
>
>    (cons X1 ... (cons Xn Y)...)
>
> into
>
>    (nconc (list X1 ... Xn) Y)

Hmm.., turning a pure expression into a (internally) side-effecting one?
How 'bout adding&using `list*` instead?


        Stefan

Re: [elpa] externals/org 26ef5e3e5b: org-src: Use `sh-mode' for all the shells it can handle

[Mattias Engdegård]

17 apr. 2023 kl. 19.37 skrev Stefan Monnier <monnier@iro.umontreal.ca>:

>>   (cons X1 ... (cons Xn Y)...)
>> 
>> into
>> 
>>   (nconc (list X1 ... Xn) Y)
> 
> Hmm.., turning a pure expression into a (internally) side-effecting one?

The transform would have to be made at a low enough level where purity no longer matters, perhaps in codegen. (Until we get a GC that penalises mutation.)

> How 'bout adding&using `list*` instead?

I wouldn't mind adding `list*` at all. (It would eventually compile to list + nconc, of course.)

We could make backquote emit `list*` and avoid stack overflow problems of the sort that was mentioned earlier (or maybe it was something else).

Re: [elpa] externals/org 26ef5e3e5b: org-src: Use `sh-mode' for all the shells it can handle

[Stefan Monnier]

>> How 'bout adding&using `list*` instead?
> I wouldn't mind adding `list*` at all. (It would eventually compile to list + nconc, of course.)

I was thinking of adding it as a C-implemented function.
It might still lose to list+nconc unless we also give it a bytecode, tho.


        Stefan

Re: [elpa] externals/org 26ef5e3e5b: org-src: Use `sh-mode' for all the shells it can handle

[Mattias Engdegård]

17 apr. 2023 kl. 22.08 skrev Stefan Monnier <monnier@iro.umontreal.ca>:

>> I wouldn't mind adding `list*` at all. (It would eventually compile to list + nconc, of course.)
> 
> I was thinking of adding it as a C-implemented function.
> It might still lose to list+nconc unless we also give it a bytecode, tho.

I think it would, but it's easy enough to find out, and we can change our minds later. I would go for a Lisp implementation, with special attention in codegen.

One thing that matters is what normal form to use during compilation, to make transforms manageable. I'm leaning towards representing `list*` as chained `cons` calls, since the other way around (represent `cons` as `list*`) would be more work for little benefit.

On the other hand it's good to avoid bad complexity and stack overflows for long list* calls (or long backquote lists). Philip, you wrote earlier:

> I have had issues in the past with ,@ expanding to code that would have
> exceeded the maximal evaluation depth, until I split it up and manually
> used `append' that has the advantage of being implemented in C.

Could you give us a representative example of the code that you had to split up? Did the max-eval-lisp-depth limit hit interpretation or compilation?