proposal for a tool to translate orgmode outlines into programs

proposal for a tool to translate orgmode outlines into programs

[Isaac]

proposal for a tool to translate orgmode outlines into programs

Idea: 
    a program to translate orgmode note into script/programs which can be 
used as a template/starting point for a real program after debugging, 
refactoring. 

Background and reasons: 
    As I am accumulating more orgmode notes for problem solving, task 
planning, project coordination,  I think it would be nice if a tool can 
translate the orgmode (especially those problem solving ones) into programs 
(as much as it can)- at least a program skeleton, where after 
ideas(outlines) are done, we can generate a corresponding program which can 
be quickly edited/debugged/re factored/tightened up to be useful running 
programs - to facilitate problem solving and task repetition in the future.


Questions and Discussions:
    1. whether this would be a worthwhile idea, or such idea has been tried 
before?

    2. do we have alternative solutions?

    3. if indeed this idea is interesting, what programming language it 
would e worthwhile to translate to?  
        currently I am thinking more along of the line of python, as its 
indenting structure more or less resemble outlines. (though my personal take 
for writing script is in ruby, lua is another interesting choice). this 3rd 
question is what I don't know for sure, I found for me ruby is more 
productive for scripting, and python has better supports ... 
        ruby could be another attractive choice - level 1 headlines maybe 
translated to a class, while other levels translate to methods ...
        

    Welcome your comments and ideas

    thanks, Isaac

    (while I am doing some search, found 
    1. the reverse direction of this is: https://github.com/bjonnh/PyOrgMode 
- python reading and writing orgmode, but would I would prefer is orgmode => 
python/ruby instead.
    2. another candidate is tangle - but idea here is different, not to 
export codes written in orgmode, but to translate/turn orgmode text directly 
into codes.
    3. an interesting node.js parser for orgmode: 
https://github.com/daitangio/org-mode-parser

    Some further elaboration:
        1. say if a headline is an action:
            translate to def ... a function
        2. say if a headline is some description:
            translate to class ...
        3. some long lines can be simply turned to comments/docstring
        4. ... maybe ... some NLP can be used to decide what actions to 
take?

Re: proposal for a tool to translate orgmode outlines into programs

[Thomas S. Dye]

Aloha Isaac,

This sounds to me a lot like literate programming, which can be
accomplished in Org with very many languages, including ruby and python
(but not lua, yet).  See
http://orgmode.org/worg/org-contrib/babel/languages.html. An advantage
of literate programming is that it generates documentation in addition
to the program code.

hth,
Tom

Isaac <isaacpei@gmail.com> writes:

> proposal for a tool to translate orgmode outlines into programs
>
> Idea: 
>     a program to translate orgmode note into script/programs which can be 
> used as a template/starting point for a real program after debugging, 
> refactoring. 
>
> Background and reasons: 
>     As I am accumulating more orgmode notes for problem solving, task 
> planning, project coordination,  I think it would be nice if a tool can 
> translate the orgmode (especially those problem solving ones) into programs 
> (as much as it can)- at least a program skeleton, where after 
> ideas(outlines) are done, we can generate a corresponding program which can 
> be quickly edited/debugged/re factored/tightened up to be useful running 
> programs - to facilitate problem solving and task repetition in the future.
>
>
> Questions and Discussions:
>     1. whether this would be a worthwhile idea, or such idea has been tried 
> before?
>
>     2. do we have alternative solutions?
>
>     3. if indeed this idea is interesting, what programming language it 
> would e worthwhile to translate to?  
>         currently I am thinking more along of the line of python, as its 
> indenting structure more or less resemble outlines. (though my personal take 
> for writing script is in ruby, lua is another interesting choice). this 3rd 
> question is what I don't know for sure, I found for me ruby is more 
> productive for scripting, and python has better supports ... 
>         ruby could be another attractive choice - level 1 headlines maybe 
> translated to a class, while other levels translate to methods ...
>         
>
>     Welcome your comments and ideas
>
>     thanks, Isaac
>
>     (while I am doing some search, found 
>     1. the reverse direction of this is: https://github.com/bjonnh/PyOrgMode 
> - python reading and writing orgmode, but would I would prefer is orgmode => 
> python/ruby instead.
>     2. another candidate is tangle - but idea here is different, not to 
> export codes written in orgmode, but to translate/turn orgmode text directly 
> into codes.
>     3. an interesting node.js parser for orgmode: 
> https://github.com/daitangio/org-mode-parser
>
>     Some further elaboration:
>         1. say if a headline is an action:
>             translate to def ... a function
>         2. say if a headline is some description:
>             translate to class ...
>         3. some long lines can be simply turned to comments/docstring
>         4. ... maybe ... some NLP can be used to decide what actions to 
> take?
>
>
>
>

-- 
Thomas S. Dye
http://www.tsdye.com

Re: proposal for a tool to translate orgmode outlines into programs

[Isaac]

Thomas S. Dye <tsd <at> tsdye.com> writes:

> 
> Aloha Isaac,
> 
> This sounds to me a lot like literate programming, which can be
> accomplished in Org with very many languages, including ruby and python
> (but not lua, yet).  See
> http://orgmode.org/worg/org-contrib/babel/languages.html. An advantage
> of literate programming is that it generates documentation in addition
> to the program code.
> 
> hth,
> Tom
> 

yes and no - it's literate programming, but rather than programming in orgmode 
- it's translating orgmode (thoughts) to build a skeleton of codes for code 
generation ... not sure if someone has done something similar before? it would 
be good to know. 

Thanks,
Isaac

Re: proposal for a tool to translate orgmode outlines into programs

[Thorsten Jolitz]

Isaac <isaacpei@gmail.com> writes:

Hi Isaac,

> Thomas S. Dye <tsd <at> tsdye.com> writes:
>
>> 
>> Aloha Isaac,
>> 
>> This sounds to me a lot like literate programming, which can be
>> accomplished in Org with very many languages, including ruby and python
>> (but not lua, yet).  See
>> http://orgmode.org/worg/org-contrib/babel/languages.html. An advantage
>> of literate programming is that it generates documentation in addition
>> to the program code.
>> 
>> hth,
>> Tom
>> 
>
> yes and no - it's literate programming, but rather than programming in
> orgmode - it's translating orgmode (thoughts) to build a skeleton of
> codes for code generation ... not sure if someone has done something
> similar before? it would be good to know.

Assume this is file tmp.org:
#+begin_src org
* A*B*/C/
*Fett* and /Kursiv/ text and some Programmtext:  ~var = x + y~ 
#+end_src

eval this in the *scratch* buffer:
#+begin_src emacs-lisp
  (with-current-buffer 
      (find-file "/path/to/tmp.org")
    (let ((print-circle t)) ; or nil
      (print
       (org-element-parse-buffer))))
#+end_src

and you get the org-file as a nested list (parse-tree) [fn:1]:

#+begin_src emacs-lisp
  #2=(org-data nil #1=(headline (:raw-value "A*B*/C/" :begin 1 :end
   75 :pre-blank 0 :hiddenp nil :contents-begin 11 :contents-end
   75 :level 1 :priority nil :tags nil :todo-keyword nil :todo-type
   nil :post-blank 0 :footnote-section-p nil :archivedp
   nil :commentedp nil :quotedp nil :CATEGORY
   nil :title (#("A*B*/C/" 0 7 (:parent #1#))) :parent #2#)
   #3=(section (:begin 11 :end 75 :contents-begin 11 :contents-end
   75 :post-blank 0 :parent #1#) #4=(paragraph (:begin 11 :end
   75 :contents-begin 11 :contents-end 75 :post-blank
   0 :post-affiliated 11 :parent #3#) #5=(bold (:begin 11 :end
   18 :contents-begin 12 :contents-end 16 :post-blank 1 :parent
   #4#) #("Fett" 0 4 (:parent #5#))) #("and " 0 4 (:parent #4#))
   #6=(italic (:begin 22 :end 31 :contents-begin 23 :contents-end
   29 :post-blank 1 :parent #4#) #("Kursiv" 0 6 (:parent #6#)))
   #("text and some Programmtext: " 0 29 (:parent
   #4#)) (code (:value "var = x + y" :begin 60 :end 74 :post-blank
   1 :parent #4#)) #("
    " 0 1 (:parent #4#))))))
#+end_src

Checkout Org elements:

#+begin_src sh
  org-element-all-elements is a variable defined in `org-element.el'.
  Its value is
  
  (babel-call center-block clock comment comment-block diary-sexp drawer
  dynamic-block example-block export-block fixed-width footnote-definition
  headline horizontal-rule inlinetask item keyword latex-environment
  node-property paragraph plain-list planning property-drawer quote-block
  quote-section section special-block src-block table table-row verse-block)

  Complete list of element types.
  
#+end_src

and Org objects:

#+begin_src sh
  org-element-all-objects is a variable defined in `org-element.el'.
  Its value is
  
  (bold code entity export-snippet footnote-reference inline-babel-call
  inline-src-block italic line-break latex-fragment link macro radio-target
  statistics-cookie strike-through subscript superscript table-cell target
  timestamp underline verbatim)
  
  Complete list of object types.
#+end_src

Now in lisp there is the equivalence of code and data, thus this is data

#+begin_src emacs-lisp
  '(paragraph (:begin 11 :end 75 :contents-begin 11 :contents-end
       75 :post-blank 0 :post-affiliated 11 :parent nil)
#+end_src

but the following would be code, if a there would be a

#+begin_src emacs-lisp
  (defun paragraph (plst)
    (message "I print my argument: %s" plst))
#+end_src

#+results:
: paragraph

defined somewhere =>

#+begin_src emacs-lisp
  (eval (paragraph '(:begin 11 :end 75 :contents-begin 11 :contents-end
       75 :post-blank 0 :post-affiliated 11 :parent nil)))
#+end_src

gives

#+results:
: I print my argument: (:begin 11 :end 75 :contents-begin 11 :contents-end
: 75 :post-blank 0 :post-affiliated 11 :parent nil)

so in a certain sense, the new org-parser already transforms the plain text of
an org document into lisp (code = data), only that nobody has written
functions for the elements and objects yet, the parse-tree is just used
as data for the exporter.

* Footnotes

[fn:1] Those #1= and #1# are only print-syntax, otherwise they are references
to already defined elisp objects. 

-- 
cheers,
Thorsten

Re: proposal for a tool to translate orgmode outlines into programs

[Eric Schulte]

Isaac <isaacpei@gmail.com> writes:

> proposal for a tool to translate orgmode outlines into programs
>

Hi Isaac,

This is a hard problem and an area of active research.  Search for
"program synthesis" [1] to see a selection of work in this field.
Generally programs are built from descriptions in formal languages which
are much more structured than Org-mode, although some have also tried to
automatically write programs from man pages [2].

Armando Solar-Lezama's work [3] is related if we are including code
snippets in Org-mode code blocks, namely "program sketching" [4] which
attempts to build programs from incomplete program-language sketches and
test cases.

Best,

Footnotes: 
[1]  http://scholar.google.com/scholar?q=program%20synthesis

[2]  https://www.usenix.org/conference/hotosxiii/macho-programming-man-pages

[3]  http://people.csail.mit.edu/asolar/

[4]  http://people.csail.mit.edu/asolar/papers/thesis.pdf

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: proposal for a tool to translate orgmode outlines into programs

[Isaac]

Eric Schulte <schulte.eric <at> gmail.com> writes:

> 
> Isaac <isaacpei <at> gmail.com> writes:
> 
> > proposal for a tool to translate orgmode outlines into programs
> >
> 
> Hi Isaac,
> 
> This is a hard problem and an area of active research.  Search for
> "program synthesis" [1] to see a selection of work in this field.
> Generally programs are built from descriptions in formal languages which
> are much more structured than Org-mode, although some have also tried to
> automatically write programs from man pages [2].
> 
> Armando Solar-Lezama's work [3] is related if we are including code
> snippets in Org-mode code blocks, namely "program sketching" [4] which
> attempts to build programs from incomplete program-language sketches and
> test cases.
> 
> Best,
> 
> Footnotes: 
> [1]  http://scholar.google.com/scholar?q=program%20synthesis
> 
> [2]  https://www.usenix.org/conference/hotosxiii/macho-programming-man-
pages
> 
> [3]  http://people.csail.mit.edu/asolar/
> 
> [4]  http://people.csail.mit.edu/asolar/papers/thesis.pdf
> 


Thanks everybody for the references and knowledge of the parser!

related, I happen to see this today from coffeescript site, which is 
somewhat related (orgmode export to markdown format) - and coffeescript 
literate is can use markdown as codes -maybe it's a direction  ... (but 
javascript!)

=====================
Literate CoffeeScript

Besides being used as an ordinary programming language, CoffeeScript may 
also be written in "literate" mode. If you name your file with a  .litcoffee 
extension, you can write it as a Markdown document — a document that also 
happens to be executable CoffeeScript code. The compiler will treat any 
indented blocks (Markdown's way of indicating source code) as code, and 
ignore the rest as comments.

Just for kicks, a little bit of the compiler is currently implemented in 
this fashion: See it as a document, raw, and properly highlighted in a text 
editor.

I'm fairly excited about this direction for the language, and am looking 
forward to writing (and more importantly, reading) more programs in this 
style. More information about Literate CoffeeScript, including an example 
program, are available in this blog post.
http://coffeescript.org/


best, Isaac