How to handle an inline multiline noweb reference

How to handle an inline multiline noweb reference

[Marcin Borkowski]

Hi folks,

I have a multiline source block (a complex sql query).  I'd like it to
be inserted into another (shell) source block, so that I can run psql on
a remote server (via ssh) and execute this query (I cannot access the
database from my machine).

I came up with this:

--8<---------------cut here---------------start------------->8---
#+name: sql_join
#+begin_src sql
  SELECT ... -- complicated, multiline query;
#+end_src

#+name: sql_oneline
#+begin_src elisp :noweb yes
  (replace-regexp-in-string "\n" " " "
<<sql_join>>" t t)
#+end_src

#+begin_src sh :noweb yes :results output verbatim
ssh user@host "psql connector -c '<<sql_oneline()>>'"
#+end_src
--8<---------------cut here---------------end--------------->8---

but it's fairly complicated.  Any better ideas?

TIA,

--
Marcin Borkowski
http://mbork.pl

Re: How to handle an inline multiline noweb reference

[Grant Rettke]

On Tue, Nov 27, 2018 at 3:57 AM Marcin Borkowski <mbork@mbork.pl> wrote:
> but it's fairly complicated.  Any better ideas?

What is the part that you think is too complicated?

Re: How to handle an inline multiline noweb reference

[Samuel Wales]

this might be the most ridiculous idea you have ever heard, but does
babel have a concept similar to stdio pipes?  i've often wanted to
just pipe stuff.  dunno if this would help marcin.

the only inter-block communication i am aware of are variables and
noweb inclusion.

[and, unless i am mistaken, one or the other inclusion mechanism
always outputs intermediate lines of text as messages in the
minibuffer, dunno if that can be turned off.]

Re: How to handle an inline multiline noweb reference

[Eric S Fraga]

On Tuesday, 27 Nov 2018 at 19:33, Samuel Wales wrote:
> this might be the most ridiculous idea you have ever heard, but does
> babel have a concept similar to stdio pipes?  i've often wanted to
> just pipe stuff.  dunno if this would help marcin.
>
> the only inter-block communication i am aware of are variables and
> noweb inclusion.

Check out the :post option for src blocks.  May be exactly what you
want.
-- 
Eric S Fraga via Emacs 27.0.50, Org release_9.1.14-1035-gfeb442

Re: How to handle an inline multiline noweb reference

[Grant Rettke]

On Wed, Nov 28, 2018 at 1:08 AM Eric S Fraga <esflists@gmail.com> wrote:
> Check out the :post option for src blocks.  May be exactly what you
> want.

Thanks for sharing that!

Re: How to handle an inline multiline noweb reference

[Samuel Wales]

fyi only: i don't understand the examples or the relevance to pipes.

On 11/28/18, Eric S Fraga <esflists@gmail.com> wrote:
> On Tuesday, 27 Nov 2018 at 19:33, Samuel Wales wrote:
>> this might be the most ridiculous idea you have ever heard, but does
>> babel have a concept similar to stdio pipes?  i've often wanted to
>> just pipe stuff.  dunno if this would help marcin.
>>
>> the only inter-block communication i am aware of are variables and
>> noweb inclusion.
>
> Check out the :post option for src blocks.  May be exactly what you
> want.
> --
> Eric S Fraga via Emacs 27.0.50, Org release_9.1.14-1035-gfeb442
>


-- 
The Kafka Pandemic: <http://thekafkapandemic.blogspot.com>

The disease DOES progress. MANY people have died from it. And ANYBODY
can get it at any time.

"You’ve really gotta quit this and get moving, because this is murder
by neglect." ---
<http://www.meaction.net/2017/02/03/pwme-people-with-me-are-being-murdered-by-neglect>.

Re: How to handle an inline multiline noweb reference

[Eric S Fraga]

On Wednesday, 28 Nov 2018 at 15:35, Samuel Wales wrote:
> fyi only: i don't understand the examples or the relevance to pipes.

The output (or more generally the results) of one src block is passed directly as input to another using the :post argument.  

See, for instance:

#+begin_src org
  ,#+name: doubleit
  ,#+begin_src octave :results output :var x=[0.1]
  disp(2*x)
  ,#+end_src

  ,#+name: test
  ,#+begin_src octave :post doubleit(x=*this*)
    2*3.14
  ,#+end_src

  ,#+results: test
  :  12.560
#+end_src

HTH,
eric

-- 
Eric S Fraga via Emacs 27.0.50, Org release_9.1.14-1035-gfeb442

Re: How to handle an inline multiline noweb reference

[Samuel Wales]

On 11/28/18, Eric S Fraga <esflists@gmail.com> wrote:
> The output (or more generally the results) of one src block is passed
> directly as input to another using the :post argument.

that helps.

i'm not familiar with many languages.  shell and lisp work best for
me, but in this case saying that clarifies.

not sure where *this* comes in.  perhaps it is a variable in the
second block being sent to the first block?

> #+begin_src org
>   ,#+name: doubleit
>   ,#+begin_src octave :results output :var x=[0.1]
>   disp(2*x)
>   ,#+end_src
>
>   ,#+name: test
>   ,#+begin_src octave :post doubleit(x=*this*)
>     2*3.14
>   ,#+end_src
>
>   ,#+results: test
>   :  12.560
> #+end_src
>
> HTH,
> eric
>
> --
> Eric S Fraga via Emacs 27.0.50, Org release_9.1.14-1035-gfeb442
>


-- 
The Kafka Pandemic: <http://thekafkapandemic.blogspot.com>

The disease DOES progress. MANY people have died from it. And ANYBODY
can get it at any time.

"You’ve really gotta quit this and get moving, because this is murder
by neglect." ---
<http://www.meaction.net/2017/02/03/pwme-people-with-me-are-being-murdered-by-neglect>.

Re: How to handle an inline multiline noweb reference

[Samuel Wales]

it still feels like variables to me.  if you did a shell example with
1000 lines, would those be piped in?  or would the variable contain
1000 lines?  i don't understandt he difference between post and just a
variable.

onte: i am only telling you this to improve documentation if you feel
like it.  i don't need this now.  just want to let you know that it is
not speakign to me.

Re: How to handle an inline multiline noweb reference

[Eric S Fraga]

On Thursday, 29 Nov 2018 at 15:18, Samuel Wales wrote:
> it still feels like variables to me.  

I agree.  It's not the *same* as a pipe but does meet some of the requirements for some use cases...  I think expecting behaviour like pipes is possibly too much given the single threaded nature of much of emacs?
-- 
Eric S Fraga via Emacs 27.0.50, Org release_9.1.14-1035-gfeb442

Re: How to handle an inline multiline noweb reference

[Eric S Fraga]

On Thursday, 29 Nov 2018 at 15:16, Samuel Wales wrote:
> On 11/28/18, Eric S Fraga <esflists@gmail.com> wrote:
>> The output (or more generally the results) of one src block is passed
>> directly as input to another using the :post argument.
>
> that helps.
>
> i'm not familiar with many languages.  shell and lisp work best for
> me, but in this case saying that clarifies.
>
> not sure where *this* comes in.  perhaps it is a variable in the
> second block being sent to the first block?

*this* is replaced by the results of the block that has the :post argument so, in this example, the results (i.e. the output) of the test block are placed in the x variable for the doubleit block.

-- 
Eric S Fraga via Emacs 27.0.50, Org release_9.1.14-1035-gfeb442

Re: How to handle an inline multiline noweb reference

[Marcin Borkowski]

On 2018-11-28, at 08:08, Eric S Fraga <esflists@gmail.com> wrote:

> On Tuesday, 27 Nov 2018 at 19:33, Samuel Wales wrote:
>> this might be the most ridiculous idea you have ever heard, but does
>> babel have a concept similar to stdio pipes?  i've often wanted to
>> just pipe stuff.  dunno if this would help marcin.
>>
>> the only inter-block communication i am aware of are variables and
>> noweb inclusion.
>
> Check out the :post option for src blocks.  May be exactly what you
> want.

Thanks, this looks *very* promising!

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: How to handle an inline multiline noweb reference

[Marcin Borkowski]

On 2018-11-28, at 03:08, Grant Rettke <gcr@wisdomandwonder.com> wrote:

> On Tue, Nov 27, 2018 at 3:57 AM Marcin Borkowski <mbork@mbork.pl> wrote:
>> but it's fairly complicated.  Any better ideas?
>
> What is the part that you think is too complicated?

I hoped that I could somehow skip the "manually replace newlines with
spaces" step.

Best,

--
Marcin Borkowski
http://mbork.pl

Re: How to handle an inline multiline noweb reference

[Grant Rettke]

On Fri, Nov 30, 2018 at 11:25 AM Marcin Borkowski <mbork@mbork.pl> wrote:
> On 2018-11-28, at 03:08, Grant Rettke <gcr@wisdomandwonder.com> wrote:
> > On Tue, Nov 27, 2018 at 3:57 AM Marcin Borkowski <mbork@mbork.pl> wrote:
> >> but it's fairly complicated.  Any better ideas?
> >
> > What is the part that you think is too complicated?
>
> I hoped that I could somehow skip the "manually replace newlines with
> spaces" step.

Thank you for explaining!