Re: [ANN] Org-babel integrated into Org-mode

[Dan Davison <dandavison7@gmail.com>]

"Eric Schulte" <schulte.eric@gmail.com> writes:

> Hi Carsten, Matt, Scott,
>
> Carsten Dominik <carsten.dominik@gmail.com> writes:
>
>> Hi Matt, hi Eric,
>>
>> Matt, thanks a lot for bringing this up.  This is indeed a very
>> important and serious issue.  We need to address it.  We need to
>> step back and reconsider this carefully.
>>
>> Don't get me wrong, I absolutely think that Org Babel should give
>> you enough rope to hang yourself.  But we have to make sure that
>> this will not happen to a happy and unsuspecting Org mode, or even
>> an unsuspecting Emacs user who by chance opens a file with extension
>> .org.
>>
>> I remember very well when  first realized that shell links could
>> really affect you badly.  It scared me.
>>
>> You main proposal was to make Org Babel an optional module.
>> This will not solve the problem fully, I think, because we also
>> don't want that people who turn it on automatically commit
>> to potentially dangerous operations.  There is a lot of good stuff
>> in Babel which has nothing to do with code evaluation.
>>
>> Here is what I propose (several items are similar to what Eric proposes)
>>
>> 1. A new variable org-turn-on-babel.  We can discuss the default.
>>    If it is nil, org-babel should not be loaded.
>>    A default of t would be fine with me if we implement other
>>    measures listed below.
>>
>
> This sounds like a good idea to me, and it should address Matt's desire
> for enabling minimal Org-mode installs.  I would like this to default to
> t, so that new users can try out Org-babel without overmuch effort.

I'm not clear yet what the point of this is. Unless it is the load time
which is the issue, what else is gained by this variable? In principle
I'm also all for minimalism and modularity, but what does it actually
mean here?

If the effect of this variable is to not load org-babel code at all,
then this needs to be thought about carefully, as it is tantamount to a
statement that all org-babel code is orthogonal to the rest of
org-mode. I.e. core org-mode will not be able to make use of any
org-babel code, because there will always be the risk that the user has
set this variable to nil. Are we sure that we might not want some
org-babel code (e.g. block export or tangling or something) to be used
in core Org functionality?

As an example of an area of overlap of core org-mode and babel, I'd been
considering extending the [[shell:]] and [[elisp:]] links that are
present in core Org-mode to support other languages. If that happens it
might make sense to let babel code handle the shell and elisp
evaluation, rather than having that functionality duplicated in the code
base.

Essentially I had been envisioning the incorporation of org-babel into
org-mode as the beginning of a phase in which the code bases can
interact and evolve together, rather than as a promise of eternal
orthogonality.

>
>>
>> 2. As Eric proposes, a variable similar to org-confirm-shell-link-
>>    function
>>    This should by default query for confirmation on any org-babel
>>    code execution, and can be configured to shut up by people who know
>>    what they are doing.
>>
>
> Sounds good, I think this is a reasonable safety measure.

Yes, I think this is the key. Other than the doubts expressed above I
agree with what Eric wrote.

Dan

>
>>
>> 3. Not loading emacs lisp evaluation by default.
>>
>
> I would push back on this point.  Largely because we have now crossed
> the like to where it is impossible to play with a code block w/o first
> dropping down to your configuration files, and evaluating require
> statements.
>
>>
>> 4. A new key in the babel keymap for org-babel-execute-code-block,
>>    for example `C-c C-v e'.  This should be documented as the default
>>    key for this operation.
>>
>
> Hmm, I'm less enthusiastic about this point and point 5.  I really like
> how 'C-c C-c' naturally does whatever-I-want given the context in which
> it's called, and I wouldn't want to lose that intuitiveness.  Similarly
> 'C-c C-o' currently opens the results of a code block, I also find this
> very appealing as it allows for a uniform top-level interface across an
> Org-mode document, be it a code block or a link.
>
> Here are my reasons why I think leaving this keybinding is safe.
>
> 1) Unlike with shell/elisp links, the contents of code blocks is almost
>    always visible right under the user's point.  So it is less likely to
>    evaluate something w/o having any idea what you are evaluating.
>
> 2) Adding a protection variable (e.g. org-confirm-babel-eval) means that
>    the only users who could potentially evaluate a code block with a
>    slip of the fingers would be users who have explicitly said that they
>    want to be able to easily run code blocks without confirmation.
>
> 3) Emacs exposes a number of entry points into code evaluation.  M-!
>    allows users to run shell commands, C-M-x evaluate the elisp at
>    point, and these have not caused problems in the past.
>
>>
>> 5. Removing org-babel-execute-code-block from `C-c C-c'.  Inclusion
>>    should be optional.
>>
>> 6. A section in the manual on code execution and associated security
>>    risks in Org mode.  This is not only about babel, but also about
>>    org-eval, org-eval-light, shell links and elisp links.  I have meant
>>    to write this section for a long time and would be willing to
>>    draft it. We could then refer to this section from a couple of
>>    places in the docs, without cluttering the docs with disclaimers.
>>
>
> This sounds like a very good idea.  I'd be happy to help write such a
> section.
>
>>
>> The reason for 4 and 5 is that I believe Org-mode users are trained
>> to blindly press `C-c C-c' whenever they want to update something at
>> point.  Matt's example of a blog post about `rm -rf' is a very
>> realistic example for bad code being evaluated by mistake, not even
>> due to malicious cations.  I belive that a special key for this
>> action would gove a good measure of protection.
>>
>
> As I mentioned, I personally feel that an org-confirm-babel-eval
> variable is sufficient protection.  I think it's safe to assume that if
> a user has explicitly customized that variable, then they know what
> they're doing and trust themselves to execute code responsibly.  I think
> it's likely that the casual Org-babel user would never customize this
> variable, which seems to me entirely appropriate.
>
>>
>> This is what I think - please let me know if you think I am overdoing
>> it.
>>
>
> So to summarize, I think that the combination of (1), (2) and (6),
> should be sufficient to protect users from accidental code evaluation.
> Please let me know what you think, I am of course looking to compromise
> and I fully understand that the general consensus may be that we need
> more layers of protection.
>
> Best -- Eric
>
>>
>> - Carsten
>>
>>
>> On Jun 29, 2010, at 8:23 PM, Matt Lundin wrote:
>>
>>> Hi Eric,
>>>
>>> Thanks again for all the work that you, Dan, and Tom have put into
>>> org-babel. I'm glad to see it become part of org-mode!
>>>
>>> "Eric Schulte" <schulte.eric@gmail.com> writes:
>>>
>>>> 2) Babel will now be loaded by default along with the rest of Org-
>>>> mode.
>>>>   This means that *everyone* currently using babel will need to
>>>> change
>>>>   their Emacs config and remove the (require 'org-babel-int) and/or
>>>>   (require 'org-babel) lines.
>>>
>>> I would like to request that org-babel be made an optional module. I
>>> ask
>>> this as someone who uses org-babel regularly. Here are my reasons:
>>>
>>>  - Org-babel adds rather specific and complex functionality to org-
>>> mode
>>>    that those who use it as a simple outliner and todo manager do not
>>>    require. (In other words, an option to turn it off might be nice
>>> for
>>>    those who are worried about "feature creep.")
>>>
>>>  - Org-babel increases the risk of accidentally executing malicious or
>>>    dangerous code when typing C-c C-c on a src block or exporting a
>>>    file. Perhaps users should activate it only after they understand
>>>    the risks.
>>>
>>>    + For instance, I might write a blog post warning about the dangers
>>>      of typing "rm -rf ~/". If I put this between #+begin_src sh
>>>      and #+end_src and unthinkingly hit C-c C-c, I would be in
>>> trouble.
>>>      I believe this is the reason for the variables
>>>      org-confirm-shell-link-function and
>>>      org-confirm-elisp-link-function.
>>>
>>>    + This is admitted a bit far-fetched as an example, as it would
>>>      require one to have loaded ob-sh.el. But since elisp execution is
>>>      activated by default, there remain opportunities for unwittingly
>>>      executing code that is meant for other purposes (e.g., warnings,
>>>      examples, etc.).
>>>
>>>>   Support for evaluating emacs-lisp code blocks is loaded by default.
>>>>   All other languages will need to be required explicitly.  To
>>>> conform
>>>>   to Emacs filename specifications all language require lines have
>>>> been
>>>>   shortened from e.g.
>>>>
>>>>   (require 'org-babel-sh)
>>>>
>>>>   to
>>>>
>>>>   (require 'ob-sh)
>>>
>>> When I run make clean && make && make install I find that the language
>>> directory is not installed. Does the langs directory require a manual
>>> installation?
>>>
>>> Also, with make install, the ob-* files are installed on the same
>>> level
>>> as the org-files, yet lines 108-114 in org.el indicate that they
>>> should
>>> be installed in a babel subdirectory.
>>>
>>> Thanks!
>>> Matt
>>>
>>> _______________________________________________
>>> Emacs-orgmode mailing list
>>> Please use `Reply All' to send replies to the list.
>>> Emacs-orgmode@gnu.org
>>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>>
>> - Carsten
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode