From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christian Moe Subject: Re: Re: [ANN] Org-babel integrated into Org-mode Date: Fri, 02 Jul 2010 00:13:16 +0200 Message-ID: <4C2D12FC.3020900@christianmoe.com> References: <87wrtp78rg.fsf@gmail.com> <874oglofzs.fsf@fastmail.fm> <87sk4432t8.fsf@gmail.com> <874ogjrzg4.fsf@gmail.com> <871vbnj5rj.fsf@gmail.com> Reply-To: Christian Moe Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Return-path: Received: from [140.186.70.92] (port=42485 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OUS09-00078o-3M for emacs-orgmode@gnu.org; Thu, 01 Jul 2010 18:13:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OUS03-0005WF-HF for emacs-orgmode@gnu.org; Thu, 01 Jul 2010 18:12:56 -0400 Received: from mars.hitrost.net ([91.185.193.39]:55709) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OUS02-0005W3-U0 for emacs-orgmode@gnu.org; Thu, 01 Jul 2010 18:12:51 -0400 In-Reply-To: <871vbnj5rj.fsf@gmail.com> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Eric Schulte Cc: Org Mode Hi, Thanks for getting the safety catch on fast. And thanks to Matt Lundin and Carsten Dominik for raising the concerns that were mounting in my mind as I caught up with the powers Org-Babel have placed at my fingertips. I love knowing it's there, but until I learn to use it, I just want to know I won't be causing any mischief with the SRC blocks in my humble HOWTO note files. Yours, Christian Moe Eric Schulte wrote: > Hi, > > Pursuant to the below, I've created a new "babel-safety" branch of the > repository. It includes two new commits, the first of which implements > confirmation before *any* code block evaluation, adds the keybinds for > code block evaluation, and adds a documentation for disassociating code > block evaluation from C-c C-c. > > ,----[1d0b2f48c7f4b12b07dcfff0b5c1d29cc2c863bb] > | babel: evaluation of code blocks now requires confirmation > | > | * lisp/babel/ob.el (org-confirm-babel-evaluate): variable used to > | control evaluation of code blocks, default value it t, meaning all > | code block evaluation requires confirmation > | > | (org-babel-confirm-evaluate): function used to request confirmation > | of code block evaluation from the user > | > | (org-babel-execute-src-block): this function is the single point of > | entry for evaluation of code blocks (whether initiated through lob > | call, through direct code block evaluation, or as part of file > | exportation). Every time this function is called it will now > | request confirmation from the user. The newly added > | `org-confirm-babel-evaluate' variable can be used to configure this > | behavior. > | > | * lisp/org.el (org-ctrl-c-ctrl-c): added documentation of code block > | evaluation behavior > | > | * lisp/babel/ob-keys.el (org-babel-key-bindings): adding keybindings > | for executing code blocks and for opening their results > `---- > > the second commit creates org-babel-load-languages, which can be used to > enable or disable any babel language. > > ,----[5f5f41a206ccef10b8ff49af8c689995d05da37c] > | babel: `org-babel-load-languages' activates code blocks by language > | > | * lisp/org.el (org-babel-load-languages): this variable controls which > | languages will be loaded by org-babel. It is customizable through > | the customize interface. > | > | (org-babel-do-load-languages): load those languages in > | org-babel-load-languages and disable those with nil cdr's > `---- > > While this variable works, and has a very nice customization widget > interface, it is awkward to customize from a user's configuration file, > this is because it relies on the defcustom :set function with which is > it associated, and as far as I can tell, the only way to ensure that the > set function is called, is to set this variable with something along the > lines of the following in your configuration. > > --8<---------------cut here---------------start------------->8--- > (org-babel-do-load-languages > 'org-babel-load-languages > '((emacs-lisp . nil) > (ruby . t) > (python . nil) > (R . t))) > --8<---------------cut here---------------end--------------->8--- > > As of right now I can't think of a more natural way to implement such a > variable -- suggestions welcom :). > > As I mentioned I'm keeping this is the babel-safety branch for now, as > it *will* disrupt the configuration and experience of Babel users, and > I've been hard on them already these past few weeks. Maybe this is best > folded into main along with the version 7.0 release so that it will be > accompanied with web-accessible documentation how to handle the > incomparable changes. > > Thanks -- Eric > > "Eric Schulte" writes: > >> Hi, >> >> Thanks for finding such a good compromises solution. This new plan >> looks great to me, specifics below >> >> Carsten Dominik writes: >> >>> Hi everyone, >>> >>> first of all, I think it is clear that I may have overreacted >>> with the "6 point plan". But it is good that we are having >>> this discussion. >>> >>> On Jun 30, 2010, at 6:25 PM, Eric Schulte wrote: >>> >>>> Hi Carsten, Matt, Scott, >>>> >>>> Carsten Dominik writes: >>>> >>>>> Hi Matt, hi Eric, >>>>> >>>>> Matt, thanks a lot for bringing this up. This is indeed a very >>>>> important and serious issue. We need to address it. We need to >>>>> step back and reconsider this carefully. >>>>> >>>>> Don't get me wrong, I absolutely think that Org Babel should give >>>>> you enough rope to hang yourself. But we have to make sure that >>>>> this will not happen to a happy and unsuspecting Org mode, or even >>>>> an unsuspecting Emacs user who by chance opens a file with extension >>>>> .org. >>>>> >>>>> I remember very well when first realized that shell links could >>>>> really affect you badly. It scared me. >>>>> >>>>> You main proposal was to make Org Babel an optional module. >>>>> This will not solve the problem fully, I think, because we also >>>>> don't want that people who turn it on automatically commit >>>>> to potentially dangerous operations. There is a lot of good stuff >>>>> in Babel which has nothing to do with code evaluation. >>>>> >>>>> Here is what I propose (several items are similar to what Eric >>>>> proposes) >>>>> >>>>> 1. A new variable org-turn-on-babel. We can discuss the default. >>>>> If it is nil, org-babel should not be loaded. >>>>> A default of t would be fine with me if we implement other >>>>> measures listed below. >>>>> >>>> This sounds like a good idea to me, and it should address Matt's >>>> desire >>>> for enabling minimal Org-mode installs. I would like this to >>>> default to >>>> t, so that new users can try out Org-babel without overmuch effort. >>> Actually, following Dan's argument, I am paddling back on this one. >>> Lets just keep it on. >>> >>> Instead of having a function to unload emacs-lisp, maybe a good way >>> would be a customize option org-babel-load-languages with a checkbox >>> for each language, emacs-lisp on by default. This would make it easy fo >>> people to select the languages they want, without having to add >>> several require statements to .emacs. >>> >> This sounds like a good idea, such a variable would also play the >> important role of advertising what languages currently support execution >> in Babel. One issue with this setup is that I think languages which do >> not have FSF attribution (currently only oz) would still require an >> explicit `require' statement, however that shouldn't be too surprising >> as those statements live in the contrib directory, and users should be >> used to having to explicitly require functionality located in contrib. >> >> One nice thing about this setup is that it shouldn't break user's >> current config (existing `require' statements will still work). >> >>>>> 2. As Eric proposes, a variable similar to org-confirm-shell-link- >>>>> function >>>>> This should by default query for confirmation on any org-babel >>>>> code execution, and can be configured to shut up by people who know >>>>> what they are doing. >>>>> >>>> Sounds good, I think this is a reasonable safety measure. >>>> >>>>> 3. Not loading emacs lisp evaluation by default. >>>>> >>>> I would push back on this point. Largely because we have now crossed >>>> the like to where it is impossible to play with a code block w/o first >>>> dropping down to your configuration files, and evaluating require >>>> statements. >>>> >>>>> 4. A new key in the babel keymap for org-babel-execute-code-block, >>>>> for example `C-c C-v e'. This should be documented as the default >>>>> key for this operation. >>>>> >>>> Hmm, I'm less enthusiastic about this point and point 5. I really >>>> like >>>> how 'C-c C-c' naturally does whatever-I-want given the context in >>>> which >>>> it's called, and I wouldn't want to lose that intuitiveness. >>>> Similarly >>>> 'C-c C-o' currently opens the results of a code block, I also find >>>> this >>>> very appealing as it allows for a uniform top-level interface across >>>> an >>>> Org-mode document, be it a code block or a link. >>>> >>>> Here are my reasons why I think leaving this keybinding is safe. >>>> >>>> 1) Unlike with shell/elisp links, the contents of code blocks is >>>> almost >>>> always visible right under the user's point. So it is less likely to >>>> evaluate something w/o having any idea what you are evaluating. >>>> >>>> 2) Adding a protection variable (e.g. org-confirm-babel-eval) means >>>> that >>>> the only users who could potentially evaluate a code block with a >>>> slip of the fingers would be users who have explicitly said that they >>>> want to be able to easily run code blocks without confirmation. >>>> >>>> 3) Emacs exposes a number of entry points into code evaluation. M-! >>>> allows users to run shell commands, C-M-x evaluate the elisp at >>>> point, and these have not caused problems in the past. >>> These are all very well taken points. And I agree that a somewhat >>> regular Org-mode user should be protected by this well enough. >>> >>> There are actually two kinds of users and two levels where >>> we need to think about this. >>> >>> 1. I am worried about is this: Org mode (including Babel) >>> will soon be part of Emacs an be shipped to a very large number of >>> people who have nothing to do with Org mode and might pick a file >>> of the web to try playing with it. I want to protect these users and >>> also us, as the Org mode community, from a stupid accident happening >>> like that. But, in fact, a yes-or-no-p confirmation would probably >>> cover this well enough. OK for this part. BTW, Eric, >>> I think this confirmation variable should also be allowed to take >>> a function with a two arguments, the language of the snippet >>> and the snippet. Users could then write a function which would >>> get confirmation for some snippets, but not for others. >>> >> This sounds like a good idea. I'll update the confirmation function as >> you've described. One possible issue here is that with complicated >> setups, a single action could require multiple confirmations -- as >> executing one code block can call on other code blocks as arguments, but >> I think that behavior is required to ensure that the user is fully aware >> of the full extent of the processes taking place. >> >>> 2. The other thing is that I am afraid of myself in this context. >>> I envision myself turning off the check eval confirmation check sooner >>> rather than later because I don't like to press the confirmation key >>> all the time. Repetitive things like this annoy me and I turn them off. >>> So I am happily working with code in a document fine. >>> >>> Later, I see myself accidentally pressing C-c C-c in a place where I did >>> not mean to press it. Like in Matt's example, this could be a blog post >>> or any other document where I have some source code examples. >>> I press key combinations with C-c *so* many times >>> a day that a couple of `C-c C-c' come up by accident every day. >>> In fact, in this context I am more worried about `C-c C-c' than `C-c >>> C- >>> o' >>> This is why I was proposing to not have this in C-c C-c (and, now >>> you mention it, in C-c C-o) by default. I definitely think >>> that it would be good to give users a variable to not include >>> these into `C-c C-c' and `C-c C-o'. Having additional bindings >>> for these two commands in the `C-c C-v' map would not hurt in >>> any case. >>> >>> On the other hand, I totally see how C-c C-c is a great and >>> natural binding if you wan to work with source code, of cause, >>> and I do understand why you defend it and want to have it in by >>> default. >>> >> Thanks for your understanding on this point. >> >>> So in summary, I think I could be fine with a situation >>> where the variable I just described exists and is set >>> so that C-c C-c and C-c C-o do the evaluation, and where >>> the issues are clearly documented. >>> >> I see your point, and I think your proposed solution is a good >> compromise. I'll add C-c C-v keybindings for both evaluation and >> opening of code blocks. Also, I'll add a variable which can be used to >> disable the binding of code block execution to C-c C-c. We can also >> mention this variable in the documentation for the confirmation >> variable, so that as users disable block confirmation they will >> (hopefully) read the documentation of the variable they are disabling, >> and will stop and think "maybe in a world w/o confirmation, I'd feel >> safer using a higher friction keybinding". >> >> BTW: any suggestions for C-c C-v key bindings, I'm thinking of the >> following, but there may be options with better ergonomics. >> | C-c C-v e | for code block execution, mnemonic "execute" | >> | C-c C-v o | for opening of results, mnemonic "open" | >> >> Thanks -- Eric >> >>> - Carsten >>> >>> >>> >>>>> 5. Removing org-babel-execute-code-block from `C-c C-c'. Inclusion >>>>> should be optional. >>>>> >>>>> 6. A section in the manual on code execution and associated security >>>>> risks in Org mode. This is not only about babel, but also about >>>>> org-eval, org-eval-light, shell links and elisp links. I have meant >>>>> to write this section for a long time and would be willing to >>>>> draft it. We could then refer to this section from a couple of >>>>> places in the docs, without cluttering the docs with disclaimers. >>>>> >>>> This sounds like a very good idea. I'd be happy to help write such a >>>> section. >>>> >>>>> The reason for 4 and 5 is that I believe Org-mode users are trained >>>>> to blindly press `C-c C-c' whenever they want to update something at >>>>> point. Matt's example of a blog post about `rm -rf' is a very >>>>> realistic example for bad code being evaluated by mistake, not even >>>>> due to malicious cations. I belive that a special key for this >>>>> action would gove a good measure of protection. >>>>> >>>> As I mentioned, I personally feel that an org-confirm-babel-eval >>>> variable is sufficient protection. I think it's safe to assume that >>>> if >>>> a user has explicitly customized that variable, then they know what >>>> they're doing and trust themselves to execute code responsibly. I >>>> think >>>> it's likely that the casual Org-babel user would never customize this >>>> variable, which seems to me entirely appropriate. >>>> >>>>> This is what I think - please let me know if you think I am overdoing >>>>> it. >>>>> >>>> So to summarize, I think that the combination of (1), (2) and (6), >>>> should be sufficient to protect users from accidental code evaluation. >>>> Please let me know what you think, I am of course looking to >>>> compromise >>>> and I fully understand that the general consensus may be that we need >>>> more layers of protection. >>>> >>>> Best -- Eric >>>> >>>>> - Carsten >>>>> >>>>> >>>>> On Jun 29, 2010, at 8:23 PM, Matt Lundin wrote: >>>>> >>>>>> Hi Eric, >>>>>> >>>>>> Thanks again for all the work that you, Dan, and Tom have put into >>>>>> org-babel. I'm glad to see it become part of org-mode! >>>>>> >>>>>> "Eric Schulte" writes: >>>>>> >>>>>>> 2) Babel will now be loaded by default along with the rest of Org- >>>>>>> mode. >>>>>>> This means that *everyone* currently using babel will need to >>>>>>> change >>>>>>> their Emacs config and remove the (require 'org-babel-int) and/or >>>>>>> (require 'org-babel) lines. >>>>>> I would like to request that org-babel be made an optional module. I >>>>>> ask >>>>>> this as someone who uses org-babel regularly. Here are my reasons: >>>>>> >>>>>> - Org-babel adds rather specific and complex functionality to org- >>>>>> mode >>>>>> that those who use it as a simple outliner and todo manager do not >>>>>> require. (In other words, an option to turn it off might be nice >>>>>> for >>>>>> those who are worried about "feature creep.") >>>>>> >>>>>> - Org-babel increases the risk of accidentally executing malicious >>>>>> or >>>>>> dangerous code when typing C-c C-c on a src block or exporting a >>>>>> file. Perhaps users should activate it only after they understand >>>>>> the risks. >>>>>> >>>>>> + For instance, I might write a blog post warning about the dangers >>>>>> of typing "rm -rf ~/". If I put this between #+begin_src sh >>>>>> and #+end_src and unthinkingly hit C-c C-c, I would be in >>>>>> trouble. >>>>>> I believe this is the reason for the variables >>>>>> org-confirm-shell-link-function and >>>>>> org-confirm-elisp-link-function. >>>>>> >>>>>> + This is admitted a bit far-fetched as an example, as it would >>>>>> require one to have loaded ob-sh.el. But since elisp execution is >>>>>> activated by default, there remain opportunities for unwittingly >>>>>> executing code that is meant for other purposes (e.g., warnings, >>>>>> examples, etc.). >>>>>> >>>>>>> Support for evaluating emacs-lisp code blocks is loaded by default. >>>>>>> All other languages will need to be required explicitly. To >>>>>>> conform >>>>>>> to Emacs filename specifications all language require lines have >>>>>>> been >>>>>>> shortened from e.g. >>>>>>> >>>>>>> (require 'org-babel-sh) >>>>>>> >>>>>>> to >>>>>>> >>>>>>> (require 'ob-sh) >>>>>> When I run make clean && make && make install I find that the >>>>>> language >>>>>> directory is not installed. Does the langs directory require a >>>>>> manual >>>>>> installation? >>>>>> >>>>>> Also, with make install, the ob-* files are installed on the same >>>>>> level >>>>>> as the org-files, yet lines 108-114 in org.el indicate that they >>>>>> should >>>>>> be installed in a babel subdirectory. >>>>>> >>>>>> Thanks! >>>>>> Matt >>>>>> >>>>>> _______________________________________________ >>>>>> Emacs-orgmode mailing list >>>>>> Please use `Reply All' to send replies to the list. >>>>>> Emacs-orgmode@gnu.org >>>>>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode >>>>> - Carsten >>>> _______________________________________________ >>>> Emacs-orgmode mailing list >>>> Please use `Reply All' to send replies to the list. >>>> Emacs-orgmode@gnu.org >>>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode >>> - Carsten > > _______________________________________________ > Emacs-orgmode mailing list > Please use `Reply All' to send replies to the list. > Emacs-orgmode@gnu.org > http://lists.gnu.org/mailman/listinfo/emacs-orgmode > -- Christian Moe E-mail: mail@christianmoe.com Website: http://christianmoe.com