* A requires/provides approach to linking source code blocks @ 2021-07-08 16:24 Marko Schuetz-Schmuck 2021-07-08 19:32 ` Tim Cross 2021-07-09 7:29 ` Stefan Nobis 0 siblings, 2 replies; 12+ messages in thread From: Marko Schuetz-Schmuck @ 2021-07-08 16:24 UTC (permalink / raw) To: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 843 bytes --] Dear All, AFAIU in the current support for literate programming I can establish sequence between blocks by either tangling the entire file whereby the blocks are written to the source code file in the sequence in which they appear in the org-mode file or I can name the blocks and use noweb linking to explicitly state the precise sequence in which the blocks appear in the source code file. I would find it useful to have a more declarative way for specifying sequence. I imagine e.g. using "#+REQUIRES:" and "#+PROVIDES:" to capture dependency and then have the exporter compute a sequence satisfying these. I would think that this makes the maintenance of the dependencies more convenient. I'd be interested in feedback on the idea. Please include my email address in the reply as I am not subscribed to the list. Best regards, Marko [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 223 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: A requires/provides approach to linking source code blocks 2021-07-08 16:24 A requires/provides approach to linking source code blocks Marko Schuetz-Schmuck @ 2021-07-08 19:32 ` Tim Cross 2021-07-09 8:22 ` Slowing down new features Timothy ` (2 more replies) 2021-07-09 7:29 ` Stefan Nobis 1 sibling, 3 replies; 12+ messages in thread From: Tim Cross @ 2021-07-08 19:32 UTC (permalink / raw) To: Marko Schuetz-Schmuck; +Cc: emacs-orgmode Marko Schuetz-Schmuck <MarkoSchuetz@web.de> writes: > [[PGP Signed Part:Undecided]] > Dear All, > > AFAIU in the current support for literate programming I can establish > sequence between blocks by either tangling the entire file whereby the > blocks are written to the source code file in the sequence in which they > appear in the org-mode file or I can name the blocks and use noweb > linking to explicitly state the precise sequence in which the blocks > appear in the source code file. > > I would find it useful to have a more declarative way for specifying > sequence. I imagine e.g. using "#+REQUIRES:" and "#+PROVIDES:" to > capture dependency and then have the exporter compute a sequence > satisfying these. I would think that this makes the maintenance of the > dependencies more convenient. > > I'd be interested in feedback on the idea. > > Please include my email address in the reply as I am not subscribed to > the list. > My concern here is with the additional complexity. This is already a somewhat complex aspect of org mode and the behaviour you describe can effectively be done using noweb, although as you say, not as declarative in style. I'm not sure the added complexity is worth the additional maintenance overhead and I would be concerned regarding how much confusion this could cause i.e. someone uses both noweb and declarative provide/require. These additions are also likely to hamper any efforts to improve performance - a topic which comes up on a fairly regular basis. This could just be me, but recently, I'm becoming very concerned about the growth of additional features and options in org mode. Many of the new things being requested are for relatively narrow use cases and represent functionality which can largely be achieved with existing features. At the same time, there also seems to be a growing frequency in patches which introduce regression bugs and patches to fix patches. This is all beginning to feel like we are running very close to the tipping point at which time we will have something that is so complex that only a very few people are able to maintain the code base and keep the system stable. New maintainers are discouraged because of the code complexity. We could end up in a position where really important issues cannot be addressed or addressed in a timely manner because of the overall complexity of the code base and time it takes to fix and test and dependence on a very few number of maintainers who are already swamped with work. At some point, I think we may want to consider a temporary freeze on new functionality and spend a few months just focusing on bug fixes and code base improvements or re-factoring. It might also be worthwhile providing some guidelines or criteria/procedures for assessing proposed new features to avoid a perception of new features being accepted/rejected based on personality, loudness of voice or some other real or perceived and irrelevant basis. -- Tim Cross ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Slowing down new features 2021-07-08 19:32 ` Tim Cross @ 2021-07-09 8:22 ` Timothy 2021-07-09 13:38 ` A requires/provides approach to linking source code blocks Maxim Nikulin 2021-07-09 20:27 ` Berry, Charles via General discussions about Org-mode. 2 siblings, 0 replies; 12+ messages in thread From: Timothy @ 2021-07-09 8:22 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-orgmode, Marko Schuetz-Schmuck [-- Attachment #1: Type: text/plain, Size: 2302 bytes --] Hi Tim, > This is all beginning to feel like we are running very close to the > tipping point at which time we will have something that is so complex > that only a very few people are able to maintain the code base and keep > the system stable. New maintainers are discouraged because of the code > complexity. We could end up in a position where really important issues > cannot be addressed or addressed in a timely manner because of the > overall complexity of the code base and time it takes to fix and test > and dependence on a very few number of maintainers who are already > swamped with work. > > At some point, I think we may want to consider a temporary freeze on new > functionality and spend a few months just focusing on bug fixes and code > base improvements or re-factoring. It might also be worthwhile providing > some guidelines or criteria/procedures for assessing proposed new > features to avoid a perception of new features being accepted/rejected > based on personality, loudness of voice or some other real or perceived > and irrelevant basis. I’ve got some changes that improve existing features that I know I’d like to push (and would be happy to maintain), but I get the same feeling as you that overall that we’ve been working on a lot of new things and it would be worth taking some time to refine what’s here instead of adding more. That said, I’m not sure a hard freeze would be the best way to approach this, I’d be more of a fan of just a shift in focus and maybe deferring some ideas to be considered later. ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― FWIW, these are the new things I’d like to push: • ob-julia • inline src block fortification • smart generated export preamble • a new LaTeX src block syntax highlighting option (this actually isn’t that much code) These are all currently maturing in my config, and in particular the “smart generated export preamble” and new LaTeX syntax highlighting option have been massaged into a state I’m very happy with over a number of months. All the best, *Timothy* ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: A requires/provides approach to linking source code blocks 2021-07-08 19:32 ` Tim Cross 2021-07-09 8:22 ` Slowing down new features Timothy @ 2021-07-09 13:38 ` Maxim Nikulin 2021-07-09 20:27 ` Berry, Charles via General discussions about Org-mode. 2 siblings, 0 replies; 12+ messages in thread From: Maxim Nikulin @ 2021-07-09 13:38 UTC (permalink / raw) To: emacs-orgmode On 09/07/2021 02:32, Tim Cross wrote: > Marko Schuetz-Schmuck writes: > >> I would find it useful to have a more declarative way for specifying >> sequence. I imagine e.g. using "#+REQUIRES:" and "#+PROVIDES:" to >> capture dependency and then have the exporter compute a sequence >> satisfying these. I would think that this makes the maintenance of the >> dependencies more convenient. > > My concern here is with the additional complexity. This is already a > somewhat complex aspect of org mode and the behaviour you describe can > effectively be done using noweb, although as you say, not as > declarative in style. There was a feature request for dependencies between code blocks to run them in proper order a half of a year ago. I think, it is better to try such feature as an extension of org, as a separate package. I suspect that some non-obvious issue may arise. Likely, to be convenient, it will be desired to autofill requires/provides attributes using some tools related to particular language. The question is whether org code is organized in such way, so extensions of this kind are possible without code duplication. Maybe API could be adjusted. On the other hand, even maintaining of stable semi-internal API sometimes is significant burden. P.S. I am reading the mail list through NNTP news.gmain.io. I have noticed that personal copies sometimes arrives without mail list address. Sorry if you will have to manually add it to reply. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: A requires/provides approach to linking source code blocks 2021-07-08 19:32 ` Tim Cross 2021-07-09 8:22 ` Slowing down new features Timothy 2021-07-09 13:38 ` A requires/provides approach to linking source code blocks Maxim Nikulin @ 2021-07-09 20:27 ` Berry, Charles via General discussions about Org-mode. 2 siblings, 0 replies; 12+ messages in thread From: Berry, Charles via General discussions about Org-mode. @ 2021-07-09 20:27 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-orgmode@gnu.org, Marko Schuetz-Schmuck Tim, > On Jul 8, 2021, at 12:32 PM, Tim Cross <theophilusx@gmail.com> wrote: > > My concern here is with the additional complexity. This is already a > somewhat complex aspect of org mode and the behaviour you describe can > effectively be done using noweb, although as you say, not as > declarative in style. This (and the rest of what you said) is very well taken. I share your concern that adding features to an already rich feature suite will make babel very difficult to penetrate. Best, Chuck p.s. Extracting code from src blocks need not depend on `org-babel-tangle' and friends. Custom exporters have to potential to render code using attribute keys and other export features. And such exporters have the virtue of not adding complexity to the org code base. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: A requires/provides approach to linking source code blocks 2021-07-08 16:24 A requires/provides approach to linking source code blocks Marko Schuetz-Schmuck 2021-07-08 19:32 ` Tim Cross @ 2021-07-09 7:29 ` Stefan Nobis 1 sibling, 0 replies; 12+ messages in thread From: Stefan Nobis @ 2021-07-09 7:29 UTC (permalink / raw) To: Marko Schuetz-Schmuck; +Cc: emacs-orgmode Marko Schuetz-Schmuck <MarkoSchuetz@web.de> writes: > I would find it useful to have a more declarative way for specifying > sequence. I imagine e.g. using "#+REQUIRES:" and "#+PROVIDES:" to > capture dependency and then have the exporter compute a sequence > satisfying these. I would say that declaring an explicit ordering of blocks via noweb is quite declarative. The tensions seems more to be between implicit and explicit ordering. I'm quite a fan of letting the computer compute things, but in the context of literate programming I tend to prefer explicit over implicit. IMHO that's the main point of literate programming: Be more explicit and document all the details. Try not to hide important aspects. If you have so many blocks that maintaining the order of the concatenation seems like a burden and tiresome I would argue that you are doing it wrong. Because in the end you need quite a good understanding of the final order of the blocks or else maintaining the implicit order via require/provide will also get out of hand. I also second Tims concerns about the additional complexity and feature creep. -- Until the next mail..., Stefan. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: A requires/provides approach to linking source code blocks
@ 2021-07-09 17:00 autofrettage
2021-07-12 9:55 ` Alexander Adolf
0 siblings, 1 reply; 12+ messages in thread
From: autofrettage @ 2021-07-09 17:00 UTC (permalink / raw)
To: emacs-orgmode@gnu.org
Tim wrote:
> This could just be me, but recently, I'm becoming very concerned
> about the growth of additional features and options in org mode.
Count me in. I have been mostly been hanging around in the shadows, but this is serious enough for me to wave a flag on the right side.
I would go as far as saying that several suggestions have been so niche, as to be labeled feature bloats. They can be made available as user-added extensions through melpa, but should stay outside org itself.
just my ¢2
Rasmus
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: A requires/provides approach to linking source code blocks 2021-07-09 17:00 autofrettage @ 2021-07-12 9:55 ` Alexander Adolf 2021-07-13 19:18 ` Tom Gillespie 0 siblings, 1 reply; 12+ messages in thread From: Alexander Adolf @ 2021-07-12 9:55 UTC (permalink / raw) To: autofrettage; +Cc: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 426 bytes --] > On 9. Jul 2021, at 19:01, autofrettage <autofrettage@protonmail.ch> wrote: > > Tim wrote: >> This could just be me, but recently, I'm becoming very concerned >> about the growth of additional features and options in org mode. > > Count me in. I have been mostly been hanging around in the shadows, but this is serious enough for me to wave a flag on the right side. > […] Same concern here. — Alex [-- Attachment #2: smime.p7s --] [-- Type: application/pkcs7-signature, Size: 1944 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: A requires/provides approach to linking source code blocks 2021-07-12 9:55 ` Alexander Adolf @ 2021-07-13 19:18 ` Tom Gillespie 2021-07-13 19:42 ` George Mauer 2021-07-14 0:44 ` Tim Cross 0 siblings, 2 replies; 12+ messages in thread From: Tom Gillespie @ 2021-07-13 19:18 UTC (permalink / raw) To: emacs-orgmode; +Cc: Tim Cross We have been receiving many new feature suggestions and requests coming in for org babel. I think that Tim's suggestion is the right one. Nearly all of these need to be implemented as an extension first and tested independently. Further, even if this is done, it should be clear that there is zero expectation that such extensions will be incorporated. Once I wrap up the formal grammar for org, one of the next things I plan to work on is a clear specification for org babel. This is critical because so many of the suggestions that come in deal with individuals' specific problems and thus fail to account for how such features interact with existing features and how the newly proposed feature would block some other features in the future, confuse users, etc. Such suggestions also often fail to account for increased complexity, nor have they been exposed to a sufficient number of examples to reveal fundamental ambiguities in how they could be interpreted. The issues with variable behavior between ob langs for :pre :post :prologue :epilogue etc. are already enough to keep us busy for quite some time. With regard to this thread in particular, it is of some interest, but there are fundamental issues, including the fact that certain languages (e.g. racket) expect module code to exist somewhere on the file system. There are ways around many of these issues, in fact there are likely many ways around any individual issue, so org babel needs to systematically consider the issues and provide a clear specification, or at least a guide for how such cases should be handled. To give an example from one of my continual pain points: I start writing python or racket in an org src block and then I want it to be a library so that it can be reused by other code both inside and outside the org file without having to resort to noweb. What is the best way to handle this? I don't know. Right now I tangle things and resort to awful hacks for the reuse-in-this-org-file case, but I'm guessing there is a better generic solution which would allow _any_ org block to be exported as a library instead of nowebbed in. Before jumping for any particular suggestion for how to handle this we need to explore the diversity of cases that various ob langs present, so that we can find a solution that will work for all of them. After all, packaging code to a library for reuse is an extremely common pattern that org babel should be able to abstract, but it is a major undertaking, not just the addition of a keyword here and there. In short I suggest that we issue a general moratorium on new org babel feature suggestions until we can stabilize what we already have and provide a clear specification for correct behavior. Until we have that spec we could encourage users to create extensions that implement those features. Best, Tom PS The other next thing that I am working on might be another way out for this particular feature request. Namely, it is simplifying and extending org keyword syntax so that new keywords (with options) and associated keywords can be specified using keyword syntax within a single org file. This would make it possible to get useful high level keyword behavior in a single file without burdening the core implementation with more special cases for associated keywords, and it would allow users to write small elisp functions that could do some of what is suggested here, all without need to add anything to the core org implementation. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: A requires/provides approach to linking source code blocks 2021-07-13 19:18 ` Tom Gillespie @ 2021-07-13 19:42 ` George Mauer 2021-07-14 0:44 ` Tim Cross 1 sibling, 0 replies; 12+ messages in thread From: George Mauer @ 2021-07-13 19:42 UTC (permalink / raw) To: emacs-orgmode; +Cc: Tom Gillespie [-- Attachment #1: Type: text/plain, Size: 4848 bytes --] In implementing an extension you might consider doing it as a generalized form of what I did with ob-racket (https://github.com/togakangaroo/ob-racket ). I think it is best to rely on the import/require/include mechanism of the language you're using. Pretty much all of them support adjacent files. The problem with code block execution is that it expands the block and writes it to a temporary directory in a non-predictable location. If you want to reference other blocks, you need them to *also* write into files in the same folder. What I do in ob-racket is support a header :adjacent-file which allows you to pass in the name of another block and then writes it into the same folder as the file to be executed with a predictable name (the name of the block) So something like this #+begin_src racket :adjacent-file stacker-reader-expander.rkt #lang reader "./stacker-reader-expander.rkt" Will make sure the block named `stacker-reader-expander.rkt` is written into that same temporary directory with that same name. There's a few rough edges but for the most part it works well. Would love to see someone take this and generalize it into its own extension. On Tue, Jul 13, 2021 at 2:18 PM Tom Gillespie <tgbugs@gmail.com> wrote: > We have been receiving many new feature suggestions and requests > coming in for org babel. I think that Tim's suggestion is the right > one. Nearly all of these need to be implemented as an extension first > and tested independently. Further, even if this is done, it should be > clear that there is zero expectation that such extensions will be > incorporated. > > Once I wrap up the formal grammar for org, one of the next things I > plan to work on is a clear specification for org babel. This is > critical because so many of the suggestions that come in deal with > individuals' specific problems and thus fail to account for how such > features interact with existing features and how the newly proposed > feature would block some other features in the future, confuse users, > etc. Such suggestions also often fail to account for increased > complexity, nor have they been exposed to a sufficient number of > examples to reveal fundamental ambiguities in how they could be > interpreted. The issues with variable behavior between ob langs for > :pre :post :prologue :epilogue etc. are already enough to keep us busy > for quite some time. > > With regard to this thread in particular, it is of some interest, but > there are fundamental issues, including the fact that certain > languages (e.g. racket) expect module code to exist somewhere on the > file system. There are ways around many of these issues, in fact there > are likely many ways around any individual issue, so org babel needs > to systematically consider the issues and provide a clear > specification, or at least a guide for how such cases should be > handled. > > To give an example from one of my continual pain points: I start > writing python or racket in an org src block and then I want it to be > a library so that it can be reused by other code both inside and > outside the org file without having to resort to noweb. > > What is the best way to handle this? I don't know. Right now I tangle > things and resort to awful hacks for the reuse-in-this-org-file case, but > I'm guessing there is a better generic solution which would allow _any_ > org block to be exported as a library instead of nowebbed in. > > Before jumping for any particular suggestion for how to handle this > we need to explore the diversity of cases that various ob langs > present, so that we can find a solution that will work for all of > them. After all, packaging code to a library for reuse is an > extremely common pattern that org babel should be able to > abstract, but it is a major undertaking, not just the addition of a > keyword here and there. > > In short I suggest that we issue a general moratorium on new org babel > feature suggestions until we can stabilize what we already have and > provide a clear specification for correct behavior. Until we have that spec > we could encourage users to create extensions that implement those > features. > > Best, > Tom > > > PS The other next thing that I am working on might be another way out > for this particular feature request. Namely, it is simplifying and > extending org keyword syntax so that new keywords (with options) and > associated keywords can be specified using keyword syntax within a > single org file. This would make it possible to get useful high level > keyword behavior in a single file without burdening the core > implementation with more special cases for associated keywords, and it > would allow users to write small elisp functions that could do some of > what is suggested here, all without need to add anything to the core > org implementation. > > [-- Attachment #2: Type: text/html, Size: 5553 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: A requires/provides approach to linking source code blocks 2021-07-13 19:18 ` Tom Gillespie 2021-07-13 19:42 ` George Mauer @ 2021-07-14 0:44 ` Tim Cross 2021-07-14 1:53 ` Samuel Wales 1 sibling, 1 reply; 12+ messages in thread From: Tim Cross @ 2021-07-14 0:44 UTC (permalink / raw) To: Tom Gillespie; +Cc: emacs-orgmode Tom Gillespie <tgbugs@gmail.com> writes: [snip] > > Once I wrap up the formal grammar for org, one of the next things I > plan to work on is a clear specification for org babel. This is > critical because so many of the suggestions that come in deal with > individuals' specific problems and thus fail to account for how such > features interact with existing features and how the newly proposed > feature would block some other features in the future, confuse users, > etc. Such suggestions also often fail to account for increased > complexity, nor have they been exposed to a sufficient number of > examples to reveal fundamental ambiguities in how they could be > interpreted. The issues with variable behavior between ob langs for > :pre :post :prologue :epilogue etc. are already enough to keep us busy > for quite some time. > Yes, that clearly summs up my concerns as well. Often, a 'fix' for some problem can seem straight-forward, possibly even trivial, when considered only within one person's use case/workflow. Part of what makes org so powerful is the level of flexibility it supports. However, this flexibility makes it extremely difficult to consider, or even know about, all the existing use cases. I suspect that once we have some formal specifications for babel, the next step will be to develop some good unit tests to verify these specifications. This would at least highlight/alert developers to unforeseen impact from changes and alert them to things they may not have considered. > With regard to this thread in particular, it is of some interest, but > there are fundamental issues, including the fact that certain > languages (e.g. racket) expect module code to exist somewhere on the > file system. There are ways around many of these issues, in fact there > are likely many ways around any individual issue, so org babel needs > to systematically consider the issues and provide a clear > specification, or at least a guide for how such cases should be > handled. > > To give an example from one of my continual pain points: I start > writing python or racket in an org src block and then I want it to be > a library so that it can be reused by other code both inside and > outside the org file without having to resort to noweb. > This is an interesting point. I think a number of languages have challenges here. I run into very similar issues with Clojure. To some extent, I think this is a grey area within the literate programming paradigm. The original literate programming model was developed at a time when most languages were compiled rather then interpreted. You generated source code, compiled it and then ran it. These days, many 'modern' languages are based around an interpreter and concepts like 'just in time' compilation. In org mode, things become even more complex because in addition to generation (tangling) of code, we also want to have evaluation of code blocks, plus we have added the concept of 'sessions'. > What is the best way to handle this? I don't know. Right now I tangle > things and resort to awful hacks for the reuse-in-this-org-file case, but > I'm guessing there is a better generic solution which would allow _any_ > org block to be exported as a library instead of nowebbed in. > > Before jumping for any particular suggestion for how to handle this > we need to explore the diversity of cases that various ob langs > present, so that we can find a solution that will work for all of > them. After all, packaging code to a library for reuse is an > extremely common pattern that org babel should be able to > abstract, but it is a major undertaking, not just the addition of a > keyword here and there. > Agree. I'm not convinced we really understand the requirements here and more exploration and specification is required. The more we add 'simple' extensions, options, keywords etc the more likely it is we will make this a much harder process and will likely result in even larger 'breakage' once we do define a clearer specification. > In short I suggest that we issue a general moratorium on new org babel > feature suggestions until we can stabilize what we already have and > provide a clear specification for correct behavior. Until we have that spec > we could encourage users to create extensions that implement those > features. > Yes. While it might sound harsh or overly limiting, I do think such a moratorium may be required. We may be able to lift it once we have some core specifications in place. We would still accept bug fixes (though we may need to provide some clarity on what is a bug fix and what is a feature enhancement/change - I regularly see posts flagged as bug fixes which are actually feature enhancements or extensions). > > PS The other next thing that I am working on might be another way out > for this particular feature request. Namely, it is simplifying and > extending org keyword syntax so that new keywords (with options) and > associated keywords can be specified using keyword syntax within a > single org file. This would make it possible to get useful high level > keyword behavior in a single file without burdening the core > implementation with more special cases for associated keywords, and it > would allow users to write small elisp functions that could do some of > what is suggested here, all without need to add anything to the core > org implementation. Sounds interesting. To some extent, recognising there may be at least two types of code blocks (library code, executable/driver code) and at least two types of languages (compiled/interpreted) may help identify base code block permutation and required options, keywords etc. However, we are unlikely to find a definitive set which supports all use cases/workflows and the ability to easily extend/customise behaviours would be very useful. -- Tim Cross ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: A requires/provides approach to linking source code blocks 2021-07-14 0:44 ` Tim Cross @ 2021-07-14 1:53 ` Samuel Wales 0 siblings, 0 replies; 12+ messages in thread From: Samuel Wales @ 2021-07-14 1:53 UTC (permalink / raw) To: Tim Cross; +Cc: Tom Gillespie, emacs-orgmode one thing about org that i think has been making it complex, in addition to number of features, is non-orthogonality. On 7/13/21, Tim Cross <theophilusx@gmail.com> wrote: > > Tom Gillespie <tgbugs@gmail.com> writes: > > [snip] > >> >> Once I wrap up the formal grammar for org, one of the next things I >> plan to work on is a clear specification for org babel. This is >> critical because so many of the suggestions that come in deal with >> individuals' specific problems and thus fail to account for how such >> features interact with existing features and how the newly proposed >> feature would block some other features in the future, confuse users, >> etc. Such suggestions also often fail to account for increased >> complexity, nor have they been exposed to a sufficient number of >> examples to reveal fundamental ambiguities in how they could be >> interpreted. The issues with variable behavior between ob langs for >> :pre :post :prologue :epilogue etc. are already enough to keep us busy >> for quite some time. >> > > Yes, that clearly summs up my concerns as well. Often, a 'fix' for some > problem can seem straight-forward, possibly even trivial, when > considered only within one person's use case/workflow. Part of what > makes org so powerful is the level of flexibility it supports. However, > this flexibility makes it extremely difficult to consider, or even know > about, all the existing use cases. > > I suspect that once we have some formal specifications for babel, the > next step will be to develop some good unit tests to verify these > specifications. This would at least highlight/alert developers to > unforeseen impact from changes and alert them to things they may > not have considered. > >> With regard to this thread in particular, it is of some interest, but >> there are fundamental issues, including the fact that certain >> languages (e.g. racket) expect module code to exist somewhere on the >> file system. There are ways around many of these issues, in fact there >> are likely many ways around any individual issue, so org babel needs >> to systematically consider the issues and provide a clear >> specification, or at least a guide for how such cases should be >> handled. >> >> To give an example from one of my continual pain points: I start >> writing python or racket in an org src block and then I want it to be >> a library so that it can be reused by other code both inside and >> outside the org file without having to resort to noweb. >> > > This is an interesting point. I think a number of languages have > challenges here. I run into very similar issues with Clojure. > > To some extent, I think this is a grey area within the literate > programming paradigm. The original literate programming model was > developed at a time when most languages were compiled rather then > interpreted. You generated source code, compiled it and then ran it. > > These days, many 'modern' languages are based around an interpreter and > concepts like 'just in time' compilation. In org mode, things become > even more complex because in addition to generation (tangling) of code, > we also want to have evaluation of code blocks, plus we have added the > concept of 'sessions'. > >> What is the best way to handle this? I don't know. Right now I tangle >> things and resort to awful hacks for the reuse-in-this-org-file case, but >> I'm guessing there is a better generic solution which would allow _any_ >> org block to be exported as a library instead of nowebbed in. >> >> Before jumping for any particular suggestion for how to handle this >> we need to explore the diversity of cases that various ob langs >> present, so that we can find a solution that will work for all of >> them. After all, packaging code to a library for reuse is an >> extremely common pattern that org babel should be able to >> abstract, but it is a major undertaking, not just the addition of a >> keyword here and there. >> > > Agree. I'm not convinced we really understand the requirements here and > more exploration and specification is required. The more we add 'simple' > extensions, options, keywords etc the more likely it is we will make > this a much harder process and will likely result in even larger > 'breakage' once we do define a clearer specification. > >> In short I suggest that we issue a general moratorium on new org babel >> feature suggestions until we can stabilize what we already have and >> provide a clear specification for correct behavior. Until we have that >> spec >> we could encourage users to create extensions that implement those >> features. >> > > Yes. While it might sound harsh or overly limiting, I do think such a > moratorium may be required. We may be able to lift it once we have some > core specifications in place. We would still accept bug fixes (though we > may need to provide some clarity on what is a bug fix and what is a > feature enhancement/change - I regularly see posts flagged as bug fixes > which are actually feature enhancements or extensions). >> >> PS The other next thing that I am working on might be another way out >> for this particular feature request. Namely, it is simplifying and >> extending org keyword syntax so that new keywords (with options) and >> associated keywords can be specified using keyword syntax within a >> single org file. This would make it possible to get useful high level >> keyword behavior in a single file without burdening the core >> implementation with more special cases for associated keywords, and it >> would allow users to write small elisp functions that could do some of >> what is suggested here, all without need to add anything to the core >> org implementation. > > Sounds interesting. To some extent, recognising there may be at least > two types of code blocks (library code, executable/driver code) and at > least two types of languages (compiled/interpreted) may help identify > base code block permutation and required options, keywords etc. However, > we are unlikely to find a definitive set which supports all use > cases/workflows and the ability to easily extend/customise behaviours > would be very useful. > -- > Tim Cross > > -- The Kafka Pandemic Please learn what misopathy is. https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2021-07-14 1:53 UTC | newest] Thread overview: 12+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-07-08 16:24 A requires/provides approach to linking source code blocks Marko Schuetz-Schmuck 2021-07-08 19:32 ` Tim Cross 2021-07-09 8:22 ` Slowing down new features Timothy 2021-07-09 13:38 ` A requires/provides approach to linking source code blocks Maxim Nikulin 2021-07-09 20:27 ` Berry, Charles via General discussions about Org-mode. 2021-07-09 7:29 ` Stefan Nobis -- strict thread matches above, loose matches on Subject: below -- 2021-07-09 17:00 autofrettage 2021-07-12 9:55 ` Alexander Adolf 2021-07-13 19:18 ` Tom Gillespie 2021-07-13 19:42 ` George Mauer 2021-07-14 0:44 ` Tim Cross 2021-07-14 1:53 ` Samuel Wales
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.