Re: [Pre-PATCH v2] Add the capability to specify lexical scope in tangled files (was: Add new :lexical header argument)

Re: [Pre-PATCH v2] Add the capability to specify lexical scope in tangled files (was: Add new :lexical header argument)

[No Wayman]

> From: Ihor Radchenko <yantar92@posteo.net>
> I am not sure if I like this approach.
> I have 2 problems with the patch:
 
> 1. Previous users of :lexical header arg might be surprised.
>    Though it is an OK breaking change since people who used 
>    :lexical
>    argument and expected it to be ignored in the tangled file 
>    probably
>    did something wrong.

Previous related discussion:

https://list.orgmode.org/87o89184h1.fsf@gmail.com/

I, and others, have been surprised to find that the tangled file 
does not have lexical binding enabled when :lexical blocks are 
tangled.

Re: [Pre-PATCH v2] Add the capability to specify lexical scope in tangled files (was: Add new :lexical header argument)

[Evgenii Klimov]

No Wayman <iarchivedmywholelife@gmail.com> writes:

[...]
> I, and others, have been surprised to find that the tangled file does
> not have lexical binding enabled when :lexical blocks are tangled.

Am I correct that language-specific header arguments are not yet covered
in the manual?  I can't find any reference of "lexical" there.

Re: [Pre-PATCH v2] Add the capability to specify lexical scope in tangled files (was: Add new :lexical header argument)

[Thomas S. Dye]

Evgenii Klimov <eugene.dev@lipklim.org> writes:

> No Wayman <iarchivedmywholelife@gmail.com> writes:
>
> [...]
>> I, and others, have been surprised to find that the tangled 
>> file does
>> not have lexical binding enabled when :lexical blocks are 
>> tangled.
>
> Am I correct that language-specific header arguments are not yet 
> covered
> in the manual?  I can't find any reference of "lexical" there.

Yes, I believe so.  Language-specific header arguments for many 
languages are documented at 
https://orgmode.org/worg/org-contrib/babel/languages/index.html. 
Several languages are not documented there; in these cases the 
documentation is typically in the source code.

All the best,
Tom

-- 
Thomas S. Dye
https://tsdye.online/tsdye

Re: [Pre-PATCH v2] Add the capability to specify lexical scope in tangled files (was: Add new :lexical header argument)

[Ihor Radchenko]

"Thomas S. Dye" <tsd@tsdye.online> writes:

>> Am I correct that language-specific header arguments are not yet 
>> covered
>> in the manual?  I can't find any reference of "lexical" there.
>
> Yes, I believe so.  Language-specific header arguments for many 
> languages are documented at 
> https://orgmode.org/worg/org-contrib/babel/languages/index.html. 
> Several languages are not documented there; in these cases the 
> documentation is typically in the source code.

It is actually slightly awkward that built-in babel backends are not
documented in the manual, while, for example, export backends are.

However, unless my memory deceives me, Bastien or Nicolas previously
voiced against adding babel documentation. Though I cannot find the
relevant discussion now and cannot recall the arguments (if there was
such discussion at all).

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [Pre-PATCH v2] Add the capability to specify lexical scope in tangled files (was: Add new :lexical header argument)

[Thomas S. Dye]

Ihor Radchenko <yantar92@posteo.net> writes:

> "Thomas S. Dye" <tsd@tsdye.online> writes:
>
>>> Am I correct that language-specific header arguments are not 
>>> yet 
>>> covered
>>> in the manual?  I can't find any reference of "lexical" there.
>>
>> Yes, I believe so.  Language-specific header arguments for many 
>> languages are documented at 
>> https://orgmode.org/worg/org-contrib/babel/languages/index.html. 
>> Several languages are not documented there; in these cases the 
>> documentation is typically in the source code.
>
> It is actually slightly awkward that built-in babel backends are 
> not
> documented in the manual, while, for example, export backends 
> are.
>
Agreed, but the two differ.  With export, the goal is to produce 
code recognized as correct by the target software.  With babel, 
the general goal is to provide language agnostic literate 
programming, which often produces something that meets standards 
of correctness not implemented in software.  The idea of built-in 
babel backends, as distinct from contributed backends, or backends 
distributed by package archive is a function of maintenance burden 
and distribution channel, rather than something intrinsic to the 
backend itself and how it contributes to literate programming.  

> However, unless my memory deceives me, Bastien or Nicolas 
> previously
> voiced against adding babel documentation. Though I cannot find 
> the
> relevant discussion now and cannot recall the arguments (if 
> there was
> such discussion at all).

IIRC, there wasn't much discussion.   The current situation 
doesn't seem ripe for documentation in the manual.

Here are some potential hurdles:
 - there are likely too many built-in backends;
 - the built-in backends are a mixed bag--ob-lua seems 
 half-finished to me, though I don't program in lua and struggled 
 to set up the language to write the documentation stub on Worg;
 - nearly a dozen of the built-in babel backends lack 
 documentation outside the source code (see 
 https://orgmode.org/worg/org-contrib/babel/languages/index.html#orgbc466c5); 
 and
 - language support is inconsistent (see 
 https://orgmode.org/worg/org-contrib/babel/languages/lang-compat.html), 
 which introduces complications for language agnostic literate 
 programming.

One way forward might distinguish between babel backends for GNU 
software and babel backends for non-GNU software, with the former 
built in, guaranteed to be consistent to some standard (which 
needs to be formulated), and documented in the manual and the 
latter moved to contrib or a package archive, with documentation 
(if any) on Worg.

Others might have better ideas.

All the best,
Tom
-- 
Thomas S. Dye
https://tsdye.online/tsdye

Re: [Pre-PATCH v2] Add the capability to specify lexical scope in tangled files (was: Add new :lexical header argument)

[Berry, Charles]

> On Jul 15, 2023, at 9:11 AM, Thomas S. Dye <tsd@tsdye.online> wrote:
> 
> 
> Ihor Radchenko <yantar92@posteo.net> writes:
> 
>> "Thomas S. Dye" <tsd@tsdye.online> writes:
>> 
>>>> Am I correct that language-specific header arguments are not yet covered
>>>> in the manual?  I can't find any reference of "lexical" there.
>>> 
>>> Yes, I believe so.  Language-specific header arguments for many languages are documented at https://orgmode.org/worg/org-contrib/babel/languages/index.html. Several languages are not documented there; in these cases the documentation is typically in the source code.
>> 
>> It is actually slightly awkward that built-in babel backends are not
>> documented in the manual, while, for example, export backends are.
>> 
> Agreed, but the two differ.  With export, the goal is to produce code recognized as correct by the target software.  With babel, the general goal is to provide language agnostic literate programming, which often produces something that meets standards of correctness not implemented in software.  The idea of built-in babel backends, as distinct from contributed backends, or backends distributed by package archive is a function of maintenance burden and distribution channel, rather than something intrinsic to the backend itself and how it contributes to literate programming.  
>> However, unless my memory deceives me, Bastien or Nicolas previously
>> voiced against adding babel documentation. Though I cannot find the
>> relevant discussion now and cannot recall the arguments (if there was
>> such discussion at all).
> 
> IIRC, there wasn't much discussion.   The current situation doesn't seem ripe for documentation in the manual.
> 


Agreed. Maybe it would suffice to use enhanced docstrings for the org-babel-execute:<lang> functions and point to them in the manual.


The params argument could list the language specific keys and the language specific handling of key common to all languages (e.g. `:results graphics' in R jumps through a number of hoops that might deserve mention)

Chuck 


> Here are some potential hurdles:
> - there are likely too many built-in backends;
> - the built-in backends are a mixed bag--ob-lua seems half-finished to me, though I don't program in lua and struggled to set up the language to write the documentation stub on Worg;
> - nearly a dozen of the built-in babel backends lack documentation outside the source code (see https://orgmode.org/worg/org-contrib/babel/languages/index.html#orgbc466c5); and
> - language support is inconsistent (see https://orgmode.org/worg/org-contrib/babel/languages/lang-compat.html), which introduces complications for language agnostic literate programming.
> 
> One way forward might distinguish between babel backends for GNU software and babel backends for non-GNU software, with the former built in, guaranteed to be consistent to some standard (which needs to be formulated), and documented in the manual and the latter moved to contrib or a package archive, with documentation (if any) on Worg.
> 
> Others might have better ideas.
> 
> All the best,
> Tom
> -- 
> Thomas S. Dye
> https://tsdye.online/tsdye
> 
> 

Re: [Pre-PATCH v2] Add the capability to specify lexical scope in tangled files (was: Add new :lexical header argument)

[Ihor Radchenko]

"Berry, Charles" <ccberry@health.ucsd.edu> writes:

>> IIRC, there wasn't much discussion.   The current situation doesn't seem ripe for documentation in the manual.
>
> Agreed. Maybe it would suffice to use enhanced docstrings for the org-babel-execute:<lang> functions and point to them in the manual.

IMHO, this is worse than pointing to WORG pages, as we currently do.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

[TASK] Ensure built-in babel backend consistency (standard header support) and ad documentation to the manual (was: [Pre-PATCH v2] Add the capability to specify lexical scope in tangled files (was: Add new :lexical header argument))

[Ihor Radchenko]

"Thomas S. Dye" <tsd@tsdye.online> writes:

> IIRC, there wasn't much discussion.   The current situation 
> doesn't seem ripe for documentation in the manual.
>
> Here are some potential hurdles:
>  - there are likely too many built-in backends;
> ...
> One way forward might distinguish between babel backends for GNU 
> software and babel backends for non-GNU software, with the former 
> built in, guaranteed to be consistent to some standard (which 
> needs to be formulated), and documented in the manual and the 
> latter moved to contrib or a package archive, with documentation 
> (if any) on Worg.

We have recently reduced the number of built-in backends:
https://list.orgmode.org/orgmode/87bl9rq29m.fsf@gnu.org/
Presumably, all that's left is useful is worth maintaining upstream.

>  - the built-in backends are a mixed bag--ob-lua seems 
>  half-finished to me, though I don't program in lua and struggled 
>  to set up the language to write the documentation stub on Worg;

AFAIK, most people assume that built-in backends are stable.
If they are not, it is a bug anyway. Or we should declare that we do not
maintain them.

>  - nearly a dozen of the built-in babel backends lack 
>  documentation outside the source code (see 
>  https://orgmode.org/worg/org-contrib/babel/languages/index.html#orgbc466c5); 
>  and

>  - language support is inconsistent (see 
>  https://orgmode.org/worg/org-contrib/babel/languages/lang-compat.html), 
>  which introduces complications for language agnostic literate 
>  programming.

That's what we should work on.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [TASK] Ensure built-in babel backend consistency (standard header support) and ad documentation to the manual (was: [Pre-PATCH v2] Add the capability to specify lexical scope in tangled files (was: Add new :lexical header argument))

[Thomas S. Dye]

Ihor Radchenko <yantar92@posteo.net> writes:

> "Thomas S. Dye" <tsd@tsdye.online> writes:
>
>> IIRC, there wasn't much discussion.   The current situation 
>> doesn't seem ripe for documentation in the manual.
>>
>> Here are some potential hurdles:
>>  - there are likely too many built-in backends;
>> ...
>> One way forward might distinguish between babel backends for 
>> GNU 
>> software and babel backends for non-GNU software, with the 
>> former 
>> built in, guaranteed to be consistent to some standard (which 
>> needs to be formulated), and documented in the manual and the 
>> latter moved to contrib or a package archive, with 
>> documentation 
>> (if any) on Worg.
>
> We have recently reduced the number of built-in backends:
> https://list.orgmode.org/orgmode/87bl9rq29m.fsf@gnu.org/
> Presumably, all that's left is useful is worth maintaining 
> upstream.
>
Yes, thanks for the link.  I read this thread as a first step 
toward enlisting maintainers for built-in backends. This effort 
had some success--20 packages have a designated maintainer--but 
still leaves 23 packages for the Org mode programmers, which seems 
like a lot to me.

>>  - the built-in backends are a mixed bag--ob-lua seems 
>>  half-finished to me, though I don't program in lua and 
>>  struggled 
>>  to set up the language to write the documentation stub on 
>>  Worg;
>
> AFAIK, most people assume that built-in backends are stable.
> If they are not, it is a bug anyway. Or we should declare that 
> we do not
> maintain them.
>
I thought "declare that we do not maintain them" meant "move them 
to org-contrib".  I'm not sure what you have in mind here.

>>  - nearly a dozen of the built-in babel backends lack 
>>  documentation outside the source code (see 
>>  https://orgmode.org/worg/org-contrib/babel/languages/index.html#orgbc466c5); 
>>  and
>
>>  - language support is inconsistent (see 
>>  https://orgmode.org/worg/org-contrib/babel/languages/lang-compat.html), 
>>  which introduces complications for language agnostic literate 
>>  programming.
>
> That's what we should work on.

Wonderful!

All the best,
Tom

-- 
Thomas S. Dye
https://tsdye.online/tsdye

Re: [TASK] Ensure built-in babel backend consistency (standard header support) and ad documentation to the manual (was: [Pre-PATCH v2] Add the capability to specify lexical scope in tangled files (was: Add new :lexical header argument))

[Ihor Radchenko]

"Thomas S. Dye" <tsd@tsdye.online> writes:

>> AFAIK, most people assume that built-in backends are stable.
>> If they are not, it is a bug anyway. Or we should declare that 
>> we do not
>> maintain them.
>>
> I thought "declare that we do not maintain them" meant "move them 
> to org-contrib".  I'm not sure what you have in mind here.

I mean that we should work towards developing all the built-in backends
up to standards.

Basically, if a backend does not support a standard behaviour described
in the manual, I call it a legitimate bug we should fix (sooner or
later). 

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [TASK] Ensure built-in babel backend consistency (standard header support) and ad documentation to the manual (was: [Pre-PATCH v2] Add the capability to specify lexical scope in tangled files (was: Add new :lexical header argument))

[Thomas S. Dye]

Ihor Radchenko <yantar92@posteo.net> writes:

> "Thomas S. Dye" <tsd@tsdye.online> writes:
>
>>> AFAIK, most people assume that built-in backends are stable.
>>> If they are not, it is a bug anyway. Or we should declare that 
>>> we do not
>>> maintain them.
>>>
>> I thought "declare that we do not maintain them" meant "move 
>> them 
>> to org-contrib".  I'm not sure what you have in mind here.
>
> I mean that we should work towards developing all the built-in 
> backends
> up to standards.
>
> Basically, if a backend does not support a standard behaviour 
> described
> in the manual, I call it a legitimate bug we should fix (sooner 
> or
> later). 

Great!

All the best,
Tom

-- 
Thomas S. Dye
https://tsdye.online/tsdye