From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:1008:1e59::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id AJPRIlmwimXMgwEAkFu2QA (envelope-from ) for ; Tue, 26 Dec 2023 11:52:09 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id IKv4HVmwimX0SQAA62LTzQ (envelope-from ) for ; Tue, 26 Dec 2023 11:52:09 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=excalamus.com header.s=zmail header.b=HN2w2BYV; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; arc=pass ("zohomail.com:s=zohoarc:i=1"); dmarc=none ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1703587929; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=Yl7mVQpc0kfPyr+QPclherljwFFWcy211LV9BDSxn/4=; b=tFewZDrMdloTYMgZ0SJC0/C3AJQrsz1vWGpZzbVi4Mp0xG+UfiVXwaKmwQqRBJ1+0wd1G3 58s7l3m27BurDAx/IJJ4YxZjL1a6bRZk8bU1q0FrjS+Fmdm7i9+jt80F/eW0ixeC1tMx+G nJmap8D6CyCr29wE6g0xfdTJzuatns8rT0Cz+Bj+4gJ6W2lfdHGopS7oeaIixoE22qC51g dXq5Tlpd54lX7C2YjbkjpH6qRkpeDMvyr6f8UxMHUTbc6PPSjELX80WPxZEXDMBC7smZxp 2ZF8L+dvx8Wae7ueiATNuAh8BK8iwayBQ46/KlshAsOeuAJp2nYoYZqZ3jGjRQ== ARC-Seal: i=2; s=key1; d=yhetil.org; t=1703587929; a=rsa-sha256; cv=pass; b=VlkdA9c5e4nbcWBCJ5YZPTvShmuYMRRt/KHzcUnXPbG04QEylNMzn5/anbzndgXCwZQ1Tn /owdnpm4U6z5H7q/c+lEMxpyq/V1QSm+ZSISJj+GspR0/Epghm2b5gKkuXLk0lH7NPByYO xImtsSo5PSjBO8/URjb8kNavBeC+pw0ggc75j396y02QFdTeoCOV1nNIXj86pHoEQ1kD16 Kx72qIKfjFbadetBAffUlJKy/UiyymPlPJpRg2zQvfMzieAkmTu/ds5zkmX1fRGXq6UdJl CDt9FimDOE/I3ulLrMRxH6RiCFkSNM70qieovaS205Rv+NDg4KFjyE8L6zwUCg== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=pass header.d=excalamus.com header.s=zmail header.b=HN2w2BYV; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; arc=pass ("zohomail.com:s=zohoarc:i=1"); dmarc=none Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 151E711851 for ; Tue, 26 Dec 2023 11:52:09 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI51f-00088Y-5n; Tue, 26 Dec 2023 05:51:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI51d-00088L-JG for emacs-orgmode@gnu.org; Tue, 26 Dec 2023 05:51:05 -0500 Received: from sender3-op-o17.zoho.com ([136.143.184.17]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI51Z-0004Ku-FV for emacs-orgmode@gnu.org; Tue, 26 Dec 2023 05:51:05 -0500 ARC-Seal: i=1; a=rsa-sha256; t=1703587854; cv=none; d=zohomail.com; s=zohoarc; b=MwnLsMW8e0xJ4TCxkgvcZesux/f5HagRpgwpXHEa8kKQKvZV/NMNH0iLtVGszfV+ROo74UDGcISLjvHCdAtsnDfO9Dtl+TD/wXQvlYu9irX/NA3izioP68tHHFXg5XrF9c4hXus5JvIWsYYAtrE0hWg0W51iyR/EyKOMICPLJ+0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1703587854; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=Yl7mVQpc0kfPyr+QPclherljwFFWcy211LV9BDSxn/4=; b=Lw4K0PPC3Sex7f3wBaBOiOTbC6x97wCUD9B9kFM62fdsGNeda9GnSF2n31HMsQkv73p+iWZHpDcua/sMABrTsTRYbPkhZDj+6XTSjCxu/TsZAyupVdOdoz6FOCEujPcvrxB5awvt7ySuvJ1LcpYZ2oWLT3bYsgOMgK6LHqW/wGw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=excalamus.com; spf=pass smtp.mailfrom=matt@excalamus.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1703587854; s=zmail; d=excalamus.com; i=matt@excalamus.com; h=Date:Date:From:From:To:To:Cc:Cc:Message-ID:In-Reply-To:References:Subject:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To; bh=Yl7mVQpc0kfPyr+QPclherljwFFWcy211LV9BDSxn/4=; b=HN2w2BYVrZDMpZVoT23nVEL5jN7SoFpLgACqK1ZAEiovo7nSmSQhZuzpNe+madCs a9DBnloIrJt7s6oEee+zYWrABOoEkKsoHT3De43AoN2Fd97nsYY+13E0g+r/1EiG1au SqVPUKKzoG8qYJXDwWjvKRb37UustSYT3T4gip7M= Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1703587853351146.8031632524105; Tue, 26 Dec 2023 02:50:53 -0800 (PST) Date: Tue, 26 Dec 2023 11:50:53 +0100 From: Matt To: "Tim Landscheidt" Cc: "emacs-orgmode" Message-ID: <18ca5bfb3dd.11759408f111931.3142909769107086842@excalamus.com> In-Reply-To: <87o7ee5jca.fsf@vagabond.tim-landscheidt.de> References: <87o7ee5jca.fsf@vagabond.tim-landscheidt.de> Subject: Documentation Results of Evaluation (was Re: Documentation of hline symbol in source blocks results) MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Importance: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail Received-SPF: pass client-ip=136.143.184.17; envelope-from=matt@excalamus.com; helo=sender3-op-o17.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -7.91 X-Spam-Score: -7.91 X-Migadu-Queue-Id: 151E711851 X-Migadu-Scanner: mx11.migadu.com X-TUID: CvipkkxBPZ0X Thanks for your message, Tim! You make many good points and I can only respond to a few of them. I'm also pleased to see you've got FSF Assignment: https://orgmode.org/worg/contributors.html Do you have time to work together on this? I have four goals: 1. Understand your perspective 2. Remove confusion 3. Detail specific problems to solve 4. Suggest, or commit, specific changes to the manual which resolve the specific problems This email focuses on the first three. I'll first respond to you, try to understand your perspective, remove confusion, and highlight specific problems to solve. I'll then give some comments on the specific problems. My hope is that after several exchanges, we'll have a list of specific problems that the mailing list can then discuss for solutions. ---- On Mon, 25 Dec 2023 17:37:41 +0100 Tim Landscheidt wrote --- > the documentation reads like users having had an issue...and then > someone feeling the need to document the solution right then and > there. What you say likely happened. > IMNSHO this leads to documentation that is not very usable > for the general audience. Agreed! > Just for further inspiration how the documentation could be > rewritten, consider the current wording of "Results of Eval- > uation/Collection/value": > > | Default for most Babel libraries(1). Functional mode. Org gets > | the value by wrapping the code in a function definition in the > | language of the source block. That is why when using =E2=80=98:r= esults > | value=E2=80=99, code should execute like a function and return a = value. > | For languages like Python, an explicit =E2=80=98return=E2=80=99 s= tatement is > | mandatory when using =E2=80=98:results value=E2=80=99. Result is= the value > | returned by the last statement in the code block. > > | When evaluating the code block in a session (see *note Environmen= t > | of a Code Block::), Org passes the code to an interpreter running > | as an interactive Emacs inferior process. Org gets the value fro= m > | the source code interpreter=E2=80=99s last statement output. Org= has to > | use language-specific methods to obtain the value. For example, > | from the variable =E2=80=98_=E2=80=99 in Ruby, and the value of = =E2=80=98.Last.value=E2=80=99 in R. > > Wrapping the code? When you write #+begin_src C printf("hello, world!\n"); #+end_src The code that's executed is actually: int main() { printf("hello, world!\n"); return 0; } When you write #+begin_src python print("hello, world!") #+end_src The code that's executed is actually (brace for it...): def __org_babel_python_format_value(result, result_file, result_params): with open(result_file, 'w') as f: if 'graphics' in result_params: result.savefig(result_file) elif 'pp' in result_params: import pprint f.write(pprint.pformat(result)) elif 'list' in result_params and isinstance(result, dict): f.write(str(['{} :: {}'.format(k, v) for k, v in result.items()= ])) else: if not set(result_params).intersection(['scalar', 'verbatim', '= raw']): def dict2table(res): if isinstance(res, dict): return [(k, dict2table(v)) for k, v in res.items()] elif isinstance(res, list) or isinstance(res, tuple): return [dict2table(x) for x in res] else: return res if 'table' in result_params: result =3D dict2table(result) try: import pandas except ImportError: pass else: if isinstance(result, pandas.DataFrame) and 'table' in = result_params: result =3D [[result.index.name or ''] + list(result= .columns)] + [None] + [[i] + list(row) for i, row in result.iterrows()] elif isinstance(result, pandas.Series) and 'table' in r= esult_params: result =3D list(result.items()) try: import numpy except ImportError: pass else: if isinstance(result, numpy.ndarray): if 'table' in result_params: result =3D result.tolist() else: result =3D repr(result) f.write(str(result)) def main(): print("hello, world!") __org_babel_python_format_value(main(), '/tmp/babel-7Zq1c7/python-e7IyhX', = ["replace"]) However, not every language is wrapped. For bash, when you write #+begin_src bash echo "hello, world!" #+end_src Something like this is passed to a bash process: echo "hello, world!" PROBLEM: "wrapping" is inaccurate > "Code /should/"? I agree, this can improve. "Should" in this context reads as "probably" and is non-committal. Documentation has a duty to be authoritative. PROBLEM: "code should execute like a function" is non-committal > "Like a function"? I agree that this is unclear. I believe what it's trying to address is that it's possible to have multiple return values from a source block. For example, Unix shell commands return an exit code indicating success, failure, and failure type. They may also write text to stdout and stderr. The header arguments ":results value" and ":results output" distinguish these two types of return values. PROBLEM: terms are not well-defined > Why is the Python requirement stated here? I see several reasons. The simplest is the piecemeal development style already mentioned. PROBLEM: non-specific requirement reference. What other languages are "lik= e Python"? > Why "using =E2=80=98:results value=E2=80=99" when the paragraph should (= only) > document this? Do I understand correctly that you're saying "using ':results value'" is redundant? PROBLEM: redundant words > "Result is the value"? What's the problem you see with this? PROBLEM: change of voice PROBLEM: singular form used to reference a plural (should we say "result" o= r "results") > What kind of value? What do you mean by "kind"? > Why are there references to Ruby and R here? At risk of being pedantic, because of the previous sentence: "Org has to use language-specific methods to obtain the value." I'm having difficulty seeing your perspective. Can you please share more of your thoughts about the confusion for this sentence? > All this confuses me and does not provide the information I > searched for (emphasis on me). I'm sorry for your confusion. I've been there. Few things make my blood boil quite as much as bad documentation. My understanding is that the information you searched for is the answer to these questions: - "How to make a named column table result for a source block?" - "How to insert a horizontal line between two rows of a table result for a source block?" Is that correct? If so, I think Ihor addressed this already by saying that, unfortunately, this functionality isn't currently available (although contributions are welcome). > I would probably prefer a clean-slate approach that starts > with something along the lines of: "Source blocks produce > results that can be integrated into an Org document and used > as input for other source blocks. This is covered in the "Feature Overview": https://www.gnu.org/software/emacs/manual/html_node/org/Features-Overview.h= tml That's not to say that the manual is clear. Maybe including it at the very start (in the "Working with Source Code" section) would have helped? https://www.gnu.org/software/emacs/manual/html_node/org/Working-wit= h-Source-Code.html ----- A specific section of manual was referenced, "Results of Evaluation": - https://www.gnu.org/software/emacs/manual/html_node/org/Results-of-Evalua= tion.html - https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/doc/org-manual.= org#n18460 The specific problems I've identified: PROBLEM: "wrapping" is inaccurate PROBLEM: "code should execute like a function" is non-committal PROBLEM: terms are not well-defined Different language has been used over the past 10+ years to describe behavior of source blocks. The "Results of Evaluation" section mentions "functional mode" and "scripting mode". Collaboration notes (org-babel.org) were kept in the early days of Babel. Eric (the original author of Org Babel) made a distinction between (what he calls) "functional" and "imperative." It's not clear to me what's meant by these terms. Some snapshots of org-babel.org: - https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/contrib/babel/o= rg-babel.org?id=3Dc0554c775344207e6adaf191258312fb8d1b6a15 - https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/org-babel.org?i= d=3Db1c103890c1523f99e380d88ed684454d902414e Here's how I think of it: "Session" means an environment is "persistent." Each call is executed in the same environment. State exists between calls. In the early-history of Babel, this was called the "imperative" style. "Non-session" means an environment is "temporary." Each call is executed in an independent environment. State does not exist between calls. In the early-history of Babel, this was called the "functional" style. All of this relates to the ":results value" and ":results output" header arguments. This is not the first time this has been discussed: https://list.orgmode.org/orgmode/CA+A2iZaziAfMeGpBqL6qGrzrWEVvLvC0DUw++T4gC= F3NGuW-DQ@mail.gmail.com/ Before discussing too much, maybe it would help to get a list of related terms that need clarification? - "imperative" - "functional" - "session" - "scripting" PROBLEM: non-specific requirement reference. What other languages are "lik= e Python"? PROBLEM: redundant words PROBLEM: change of voice PROBLEM: singular form used to reference a plural (should we say "result" o= r "results") -- Matt Trzcinski Emacs Org contributor (ob-shell) Learn more about Org mode at https://orgmode.org Support Org development at=C2=A0https://liberapay.com/org-mode