[BUG] Noweb reference eval syntax does not work

[BUG] Noweb reference eval syntax does not work

[fm4d]

The noweb reference syntax that should insert results of
evaluation of src block itself instead of src block as
described in `org-babel-expand-noweb-reference` does not
seems to work.


Code for replication:

* Assign 

First we assign abc:

#+begin_src python :noweb-ref assign_abc
abc = "abc" + "def"
#+end_src

* Use

Then we use it in a function:

#+begin_src python :noweb tangle :tangle noweb-test.py
def x():
  <<assign_abc>>
  return abc

print(x())
#+end_src


<<assign_abc>> does work, <assign_abc()> fails with this error:

(error "Reference ‘assign_abc’ not found in this buffer")
  signal(error ("Reference ‘assign_abc’ not found in this buffer"))
  error("Reference `%s' not found in this buffer" "assign_abc")
  org-babel-ref-resolve("assign_abc()")
  ...


I am not sure if this is a bug or I am doing something wrong,
the documentation on this feature is not very comprehensive.

Re: [BUG] Noweb reference eval syntax does not work

[Nicolas Goaziou]

Hello,

fm4d <m@fm4d.net> writes:

> The noweb reference syntax that should insert results of
> evaluation of src block itself instead of src block as
> described in `org-babel-expand-noweb-reference` does not
> seems to work.
>
>
> Code for replication:
>
> * Assign 
>
> First we assign abc:
>
> #+begin_src python :noweb-ref assign_abc
> abc = "abc" + "def"
> #+end_src
>
> * Use
>
> Then we use it in a function:
>
> #+begin_src python :noweb tangle :tangle noweb-test.py
> def x():
>   <<assign_abc>>
>   return abc
>
> print(x())
> #+end_src
>
>
> <<assign_abc>> does work, <assign_abc()> fails with this error:
>
> (error "Reference ‘assign_abc’ not found in this buffer")
>   signal(error ("Reference ‘assign_abc’ not found in this buffer"))
>   error("Reference `%s' not found in this buffer" "assign_abc")
>   org-babel-ref-resolve("assign_abc()")
>   ...
>
>
> I am not sure if this is a bug or I am doing something wrong,
> the documentation on this feature is not very comprehensive.

I just discovered the :noweb-ref parameter.

It is redundant with #+NAME: keyword and slightly broken. Also it
induces hacks like `org-babel-use-quick-and-dirty-noweb-expansion' to
work-around its shortcomings. 

Besides, it doesn't make much sense to add the same parameters to
a bunch of blocks, so I find the syntax dubious.

I understand it can be a handy shortcut for inserting multiple blocks,
but, all in all, I tend to think it would be simpler to just remove the
feature, along with `:noweb-sep' and
`org-babel-use-quick-and-dirty-noweb-expansion'.

What do you, and others, think? Is NAME enough for noweb syntax, or is
there a real need fo :noweb-ref?


Regards,

-- 
Nicolas Goaziou

Re: [BUG] Noweb reference eval syntax does not work

[Rasmus]

Hi,

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> fm4d <m@fm4d.net> writes:
>
>> The noweb reference syntax that should insert results of
>> evaluation of src block itself instead of src block as
>> described in `org-babel-expand-noweb-reference` does not
>> seems to work.
>>
>>
>> Code for replication:
>>
>> * Assign 
>>
>> First we assign abc:
>>
>> #+begin_src python :noweb-ref assign_abc
>> abc = "abc" + "def"
>> #+end_src
>>
>> * Use
>>
>> Then we use it in a function:
>>
>> #+begin_src python :noweb tangle :tangle noweb-test.py
>> def x():
>>   <<assign_abc>>
>>   return abc
>>
>> print(x())
>> #+end_src
>>
>>
>> <<assign_abc>> does work, <assign_abc()> fails with this error:
>>
>> (error "Reference ‘assign_abc’ not found in this buffer")
>>   signal(error ("Reference ‘assign_abc’ not found in this buffer"))
>>   error("Reference `%s' not found in this buffer" "assign_abc")
>>   org-babel-ref-resolve("assign_abc()")
>>   ...
>>
>>
>> I am not sure if this is a bug or I am doing something wrong,
>> the documentation on this feature is not very comprehensive.
>
> I just discovered the :noweb-ref parameter.

I have used :noweb yes in the past, but it seems :noweb-ref can be
replaced by name in at least the single blog case:

    #+name: data
    #+BEGIN_SRC R
      set.seed(42)
      df1 <- data.frame(x=rnorm(10), y=rnorm(10))
    #+END_SRC

    #+BEGIN_SRC R :noweb yes
    <<data>>
    colMeans(df1)
    #+END_SRC

As you mention, we’d loose the ability to chain together multiple blocks.
I reckon they are meaningfully the same language, so I don’t see a loss.
The example shown in the manual also does not convince me of the
usefullness of this.

> It is redundant with #+NAME: keyword and slightly broken. Also it
> induces hacks like `org-babel-use-quick-and-dirty-noweb-expansion' to
> work-around its shortcomings. 
>
> Besides, it doesn't make much sense to add the same parameters to
> a bunch of blocks, so I find the syntax dubious.
>
> I understand it can be a handy shortcut for inserting multiple blocks,
> but, all in all, I tend to think it would be simpler to just remove the
> feature, along with `:noweb-sep' and
> `org-babel-use-quick-and-dirty-noweb-expansion'.

I’m happy to kill it off in Org-9.  I don’t know how widely the chaining
of blocks is used, though, and whether the fix is always as simple as
uniting the blocks.

> What do you, and others, think? Is NAME enough for noweb syntax, or is
> there a real need fo :noweb-ref?

I've put Aaron in carbon copy as he’s quite familiar with ob.

Rasmus

-- 
Governments should be afraid of their people

Re: [BUG] Noweb reference eval syntax does not work

[Thomas S. Dye]

Aloha all,

Rasmus writes:
>
>> What do you, and others, think? Is NAME enough for noweb syntax, or is
>> there a real need fo :noweb-ref?
>
> I've put Aaron in carbon copy as he’s quite familiar with ob.

The motivation for noweb-ref is discussed in this thread from about 5
years ago:
http://thread.gmane.org/gmane.emacs.orgmode/42636/focus=42639

FWIW, I haven't used noweb-ref.

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: [BUG] Noweb reference eval syntax does not work

[Nicolas Goaziou]

Hello,

"Thomas S. Dye" <tsd@tsdye.com> writes:

> The motivation for noweb-ref is discussed in this thread from about 5
> years ago:
> http://thread.gmane.org/gmane.emacs.orgmode/42636/focus=42639
>
> FWIW, I haven't used noweb-ref.

Thank you for the pointers. It seems that :noweb-ref is here for
a task, that NAME alone cannot fulfill. I guess we have to keep it,
then. 

Although I understand that <<noweb-refs>> can refer to a concatenation
of source block contents, I cannot find any meaning in <<noweb-refs()>>,
which would be the result of evaluation of a block that doesn't exist.

Maybe we should simply error out on this one.

WDYT?


Regards,

-- 
Nicolas Goaziou

Re: [BUG] Noweb reference eval syntax does not work

[Samuel Wales]

i use noweb-ref to concatenate different blocks.

i use quick and dirty for speed.


On 5/11/16, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
> Hello,
>
> "Thomas S. Dye" <tsd@tsdye.com> writes:
>
>> The motivation for noweb-ref is discussed in this thread from about 5
>> years ago:
>> http://thread.gmane.org/gmane.emacs.orgmode/42636/focus=42639
>>
>> FWIW, I haven't used noweb-ref.
>
> Thank you for the pointers. It seems that :noweb-ref is here for
> a task, that NAME alone cannot fulfill. I guess we have to keep it,
> then.
>
> Although I understand that <<noweb-refs>> can refer to a concatenation
> of source block contents, I cannot find any meaning in <<noweb-refs()>>,
> which would be the result of evaluation of a block that doesn't exist.
>
> Maybe we should simply error out on this one.
>
> WDYT?
>
>
> Regards,
>
> --
> Nicolas Goaziou
>
>


-- 
The Kafka Pandemic: http://thekafkapandemic.blogspot.com

The disease DOES progress.  MANY people have died from it.  And
ANYBODY can get it.

Denmark: free Karina Hansen NOW.

Re: [BUG] Noweb reference eval syntax does not work

[Charles C. Berry]

On Wed, 11 May 2016, Nicolas Goaziou wrote:

> Hello,
>
> "Thomas S. Dye" <tsd@tsdye.com> writes:
>
>> The motivation for noweb-ref is discussed in this thread from about 5
>> years ago:
>> http://thread.gmane.org/gmane.emacs.orgmode/42636/focus=42639
>>
>> FWIW, I haven't used noweb-ref.
>
> Thank you for the pointers. It seems that :noweb-ref is here for
> a task, that NAME alone cannot fulfill. I guess we have to keep it,
> then.
>
> Although I understand that <<noweb-refs>> can refer to a concatenation
> of source block contents, I cannot find any meaning in <<noweb-refs()>>,
> which would be the result of evaluation of a block that doesn't exist.
>
> Maybe we should simply error out on this one.
>
> WDYT?

Error out sounds good.

If one wants to do this

#+name: lotsa-blocks
#+begin_src my-lang :noweb yes :var a="this"
   <<noweb-refs>>
#+end_src

#+begin_src my-lang :noweb yes
<<lotsa-blocks(a="that")>>
#+end_src

it seems concise enough already.

Chuck

Re: [BUG] Noweb reference eval syntax does not work

[Aaron Ecay]

Hi Rasmus, hi all,

2016ko maiatzak 8an, Rasmus-ek idatzi zuen:

[...]

> As you mention, we’d loose the ability to chain together multiple blocks.
> I reckon they are meaningfully the same language, so I don’t see a loss.
> The example shown in the manual also does not convince me of the
> usefullness of this.

Ditto.

> 
>> It is redundant with #+NAME: keyword and slightly broken. Also it
>> induces hacks like `org-babel-use-quick-and-dirty-noweb-expansion' to
>> work-around its shortcomings. 
>> 
>> Besides, it doesn't make much sense to add the same parameters to
>> a bunch of blocks, so I find the syntax dubious.
>> 
>> I understand it can be a handy shortcut for inserting multiple blocks,
>> but, all in all, I tend to think it would be simpler to just remove the
>> feature, along with `:noweb-sep' and
>> `org-babel-use-quick-and-dirty-noweb-expansion'.
> 
> I’m happy to kill it off in Org-9.  I don’t know how widely the chaining
> of blocks is used, though, and whether the fix is always as simple as
> uniting the blocks.

I think that we can provide a replacement to noweb-ref as follows:

* Code blocks
:PROPERTIES:
:header-args: :noweb-ref foo
:END:

#+begin_src python
block 1
#+end_src

#+begin_src python
block 2
#+end_src

* Concat

The old way

#+begin_src python
<<foo>>
#+end_src

The new way:

#+begin_src python
<<concat-blocks-of-lang-in-headline("python","Code blocks")>>
#+end_src

concat-blocks-of-lang-in-headline would have to be an elisp source block
implementing the appropriate behavior which is present in the document
or in the library of babel.

(And I think it would be good if the LoB contained some useful
predefined blocks like this one, in addition to those added by the
user.  A “standard library of babel” as it were.)

> 
>> What do you, and others, think? Is NAME enough for noweb syntax, or is
>> there a real need fo :noweb-ref?

To put it another way: it seems to me that the functionality of
:noweb-ref can be reimplemented in terms of other primitives.  And given
Nicolas’s comments about the complications and bugs it introduces, I’d
be in favor of deprecating and eventually removing it.

-- 
Aaron Ecay

Re: [BUG] Noweb reference eval syntax does not work

[Nicolas Goaziou]

Hello,

Aaron Ecay <aaronecay@gmail.com> writes:

> I think that we can provide a replacement to noweb-ref as follows:
>
> * Code blocks
> :PROPERTIES:
> :header-args: :noweb-ref foo
> :END:
>
> #+begin_src python
> block 1
> #+end_src
> #+begin_src python
> block 2
> #+end_src
>
> * Concat
>
> The old way
>
> #+begin_src python
> <<foo>>
> #+end_src
>
> The new way:
>
> #+begin_src python
> <<concat-blocks-of-lang-in-headline("python","Code blocks")>>
> #+end_src
>
> concat-blocks-of-lang-in-headline would have to be an elisp source block
> implementing the appropriate behavior which is present in the document
> or in the library of babel.

[...]

> To put it another way: it seems to me that the functionality of
> :noweb-ref can be reimplemented in terms of other primitives.  And given
> Nicolas’s comments about the complications and bugs it introduces, I’d
> be in favor of deprecating and eventually removing it.

I like the idea of separating block chaining and basic block syntax.
That way, you don't need to pay the full price of the former if you only
use the latter.

However, filtering per language and headline header may be a burden. For
example, this ignores blocks before first heading, and forces user to
store them under the same headline. We could also keep :noweb-ref
references and chain all blocks with a given :noweb-ref, with a special
command. What about <<chain(..ref..)>> ?

Eventually we could add into the manual how a chained block can be
evaluated, since <<chain()>> is not meaningful.

WDYT?

Regards,

-- 
Nicolas Goaziou

Re: [BUG] Noweb reference eval syntax does not work

[Karsten Schmidt]

Hi all,

sorry for coming in super late to this thread, but I'd like to make a
plea to *not* deprecate the :noweb-ref header arg.

On 12 May 2016 at 19:35:15, Aaron Ecay wrote:

> To put it another way: it seems to me that the functionality of
> :noweb-ref can be reimplemented in terms of other primitives.  And given
> Nicolas’s comments about the complications and bugs it introduces, I’d
> be in favor of deprecating and eventually removing it.

I've been investing thousands of hours over the past years to author
several largish opensource projects (~40,000 SLOC total) exclusively
in org-mode, using hundreds of code blocks with :noweb-ref headers,
all of which would have to be replaced, re-tangled and the resulting
code re-tested, often manually... Maybe some of this could be
automated, but in any way it'll be a pretty large undertaking, which
I'd like to avoid, if possible...

I also think, having a feature deprecation like this should require a
major version change (depending how one interprets semantic
versioning) of org-mode (e.g v9.0.0). Already, I often struggle
convincing users of my libs that authoring libraries in org-mode is
better than in plain vanilla source code, but I think it'd be wrong to
not just force a non-standard tool, but then also specific versions of
that tool in order for people to help contributing.

Finally, and in summary, I never ever had a single issue w/
:noweb-refs and they're by far in my top 3 favourite org-mode
features... Please don't kill 'em!

Just my two pence...

K.
-- 
Karsten Schmidt
http://thi.ng | http://postspectacular.com | http://toxiclibs.org

Re: [BUG] Noweb reference eval syntax does not work

[Charles C. Berry]

On Thu, 4 Aug 2016, Karsten Schmidt wrote:

> Hi all,
>
> sorry for coming in super late to this thread, but I'd like to make a
> plea to *not* deprecate the :noweb-ref header arg.
>
> On 12 May 2016 at 19:35:15, Aaron Ecay wrote:
>
>> To put it another way: it seems to me that the functionality of
>> :noweb-ref can be reimplemented in terms of other primitives.


Maybe you should respond to Nicolas' last posting in this thread:

https://lists.gnu.org/archive/html/emacs-orgmode/2016-05/msg00263.html

However, it doesn't look like :noweb-ref functionality is going away, but 
it may be shifted to require a noweb reference *evaluation* of a src block 
or lob block or maybe an idiom that looks like such (with the reference 
name as an argument) to invoke it. When that change occurs, I expect that 
a replace-regexp would be enough to update a file that uses :noweb-ref.

If you want to get ahead of that change, you can probably assume that 
:noweb-ref headers will continue to exist and roll your own src block to 
handle the invocation. `(org-babel-map-src-blocks nil BODY)' with a 
suitably tooled BODY seems like it would be enough. See the docstring.

If you require chained *evaluation* (including the _results_ of chained 
src blocks rather than just the src block bodies), you might show an 
example. So far, there hasn't been much concern that chained *evaluation* 
is useful. But it can be supported via the lob/src block gambit and you 
might get some tips on how to implement that or maybe convince someone to 
create a lob block that handles your case.

HTH,

Chuck