From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id cPSWDk6TVl9UMwAA0tVLHw (envelope-from ) for ; Mon, 07 Sep 2020 20:08:46 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id uD9iCk6TVl+NLwAAB5/wlQ (envelope-from ) for ; Mon, 07 Sep 2020 20:08:46 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 7CF7B9404C5 for ; Mon, 7 Sep 2020 20:08:45 +0000 (UTC) Received: from localhost ([::1]:52214 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kFNRU-0002U4-F2 for larch@yhetil.org; Mon, 07 Sep 2020 16:08:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34088) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kFNQO-0001R3-S3 for emacs-orgmode@gnu.org; Mon, 07 Sep 2020 16:07:36 -0400 Received: from mail-pj1-x102d.google.com ([2607:f8b0:4864:20::102d]:33741) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kFNQM-0000Hv-7S for emacs-orgmode@gnu.org; Mon, 07 Sep 2020 16:07:36 -0400 Received: by mail-pj1-x102d.google.com with SMTP id np15so7736178pjb.0 for ; Mon, 07 Sep 2020 13:07:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=4zDpbO5NK+JOxcr5lJPuJSRWbscDvKHOA6SYU1nWzmo=; b=rNqpjcHmYLMYHhkGcmiQwgE8qlGDnR9Tla4BRpbSu6SB6IgT0CxRNLmIzHghIkl/fs ZLSSyotRFwALmOw3ojbvs6mntpb/axfR+yU7aUr65Ln7YIsHYOEcKEax25bzcD5/9WL0 MObdNoVtH650aQ86/aEr49EzrYoiQH5r4ju6M3Ycr6uIAPVMeAxotirDDpxr7r1P74hr bPTzeIJ6eoq0avBQLYML82u2UdPecAJfxcQzfyLIuVV5I9zUaipFStfFIvapCsT80JjQ gPgzUbKHqTXDN62J4O8VIXYaGEjr9OQ7/8oZH6wcDLPPGLMpcXU5SAhlz03dpXE9dlpq JjLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=4zDpbO5NK+JOxcr5lJPuJSRWbscDvKHOA6SYU1nWzmo=; b=dZxFHtYv+YqZDZfP1fxMq/vzZ2PiSjhmarXfnq2pN3ZaAHOaBGtykqkJjRBSjQuhxd jCewmxF9VrbJKschpWCYN7ZQC8Rik+lmcdTHWYDifVcdpRQDfxVbzT37DjFWqs9UuKaB eqBQXxOHKs2lHJxAQjriQirhDi/U+s5ijsDxONs8m03cK+vtVJRsfvwgedI1ek9jBc+x dYFAgjumeKcZt38u2Kqk0U97fp06cMLjMjKxxyusi6ec+K6iEBLrtSzroMmJpixPTjJ4 HaGzbT8HVuonY2UHEj4hjcub+3J5240l6L1LLT6h1O/khqwS1XriZ2xiLTQZrh9cbddk 4wJA== X-Gm-Message-State: AOAM530pYWYCKJjydAVCe+t1U1Xmd6Ssl0iotDFHF2OMbzsnOlZr6tF3 j0RUPb8LQz94wDorD6hhiww= X-Google-Smtp-Source: ABdhPJyztN1gtHUlRAfi8a9w9fCOMMzPxsY03FRUit4MpM/RRZetNIObcZXjg/0BP691YG+qrx2qOQ== X-Received: by 2002:a17:90b:ecc:: with SMTP id gz12mr743245pjb.219.1599509252822; Mon, 07 Sep 2020 13:07:32 -0700 (PDT) Received: from localhost (199-83-220-90.PUBLIC.monkeybrains.net. [199.83.220.90]) by smtp.gmail.com with ESMTPSA id j35sm12638449pgi.91.2020.09.07.13.07.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Sep 2020 13:07:31 -0700 (PDT) From: Jack Kamm To: "Berry, Charles" Subject: Re: Help debugging R source code block output problem with :session In-Reply-To: <48AD2294-E831-44D7-B564-0F7B4019C61B@health.ucsd.edu> References: <87r1rqled0.fsf@havana> <87k0xifa9h.fsf@gmail.com> <87d03art6a.fsf@havana> <87h7slgbi5.fsf@gmail.com> <352C7149-743F-4944-ACA5-7A1242B5A3AA@health.ucsd.edu> <87tuwa9ez4.fsf@gmail.com> <87pn6y9dqy.fsf@gmail.com> <48AD2294-E831-44D7-B564-0F7B4019C61B@health.ucsd.edu> Date: Mon, 07 Sep 2020 13:07:31 -0700 Message-ID: <87ft7t9wqk.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=2607:f8b0:4864:20::102d; envelope-from=jackkamm@gmail.com; helo=mail-pj1-x102d.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "emacs-orgmode@gnu.org" , Dylan Schwilk Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=gmail.com header.s=20161025 header.b=rNqpjcHm; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Spam-Score: 0.09 X-TUID: lTAnM/PqAiLQ --=-=-= Content-Type: text/plain Hi Chuck, > This does not work for my remote session. Thanks for testing this. The remote tests I originally tried did not cover this case -- I only tested the case where both org-file and session were remote, but didn't test the case where org-file was local and session was remote. I've now tested this case as well, and added some fixes for it. I'm reattaching the patch in full. However, this implementation does require the R session to have the correct default-directory -- see details below. > The problem is that tempfiles on the remote host are like "/tmp/RtmpeFHudh/file23a66d2fc1f9", but emacs tries to use '/var/folders/kb/2hchpbyj7lb6z76l0q73w_fh0000gn/T/babel-OSXKNd/R-oNOVVB' > > `org-babel-temp-file' doesn't honor remote connections AFAICS. org-babel-temp-file does honor remote connections, but relies on default-directory to do so. I've moved the call to org-babel-temp-file so it happens when the session buffer is current -- that way, the tempfile will be correctly handled, assuming that the session's default-directory is on the remote location. If the session's default-directory isn't at the remote location, then this will break. But, in this case some other things break as well, for example inserting links to remote plots. > Maybe there is some comint or tramp idiom that would solve this, but I do not know what it is. Here are some ways to start a remote R session with correct default-directory: 1. Do "M-x R", then when it prompts for a directory, use "/scp:hostname:/some/path". Optionally, first visit that location with "C-x C-f /scp:hostname:/some/path", so that the default value is already there. 2. Alternatively, start the R session by evaluating a source block with header argument ":dir /scp:hostname:/some/path". 3. If you prefer to use "M-x shell" with "ess-remote", first visit the remote location with "C-x C-f /scp:hostname:/some/path", before calling "M-x shell". 4. If you prefer to start "M-x shell" locally and then ssh in, it's still possible to have default-directory set, but it requires some configuration [1]. Or you could use "M-x cd" to set it as well. [1] https://emacs.stackexchange.com/questions/5589/automatically-update-default-directory-when-pwd-changes-in-shell-mode-and-term-m/5592#5592 Best, Jack --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-ob-R-Fix-session-output-with-substrings-matching-pro.patch >From e7f1a59167de88fb9a5b96a0e1ac3199f105f600 Mon Sep 17 00:00:00 2001 From: Jack Kamm Date: Mon, 7 Sep 2020 00:41:52 -0700 Subject: [PATCH] ob-R: Fix session output with substrings matching prompts * lisp/ob-R.el (ess-send-string): Declare external function. (org-babel-R-evaluate-session): New implementation for session output results, that replaces calls to org-babel-comint-with-output with custom code. * testing/lisp/test-ob-R.el (test-ob-R/prompt-output): New test for output results containing angle brackets. (test-ob-R/output-nonprinted): New test for output results that aren't explicitly printed. Fixes issue reported in https://orgmode.org/list/875zgjh8wn.fsf@gmail.com/, https://orgmode.org/list/87r1rqled0.fsf@havana/ --- lisp/ob-R.el | 38 ++++++++++++++++++++------------------ testing/lisp/test-ob-R.el | 13 +++++++++++++ 2 files changed, 33 insertions(+), 18 deletions(-) diff --git a/lisp/ob-R.el b/lisp/ob-R.el index 5e9d35f58..dffbbe112 100644 --- a/lisp/ob-R.el +++ b/lisp/ob-R.el @@ -38,6 +38,8 @@ (declare-function ess-make-buffer-current "ext:ess-inf" ()) (declare-function ess-eval-buffer "ext:ess-inf" (vis)) (declare-function ess-wait-for-process "ext:ess-inf" (&optional proc sec-prompt wait force-redisplay)) +(declare-function ess-send-string "ext:ess-inf" + (process string &optional visibly message _type)) (defconst org-babel-header-args:R '((width . :any) @@ -437,24 +439,24 @@ (defun org-babel-R-evaluate-session (org-babel-import-elisp-from-file tmp-file '(16))) column-names-p))) (output - (mapconcat - 'org-babel-chomp - (butlast - (delq nil - (mapcar - (lambda (line) (when (> (length line) 0) line)) - (mapcar - (lambda (line) ;; cleanup extra prompts left in output - (if (string-match - "^\\([>+.]\\([ ][>.+]\\)*[ ]\\)" - (car (split-string line "\n"))) - (substring line (match-end 1)) - line)) - (org-babel-comint-with-output (session org-babel-R-eoe-output) - (insert (mapconcat 'org-babel-chomp - (list body org-babel-R-eoe-indicator) - "\n")) - (inferior-ess-send-input)))))) "\n")))) + (with-current-buffer session + (let* ((tmp-file (org-babel-temp-file "R-")) + (process (get-buffer-process (current-buffer))) + (string-buffer "") + (comint-output-filter-functions + (cons (lambda (text) (setq string-buffer + (concat string-buffer text))) + comint-output-filter-functions))) + (with-temp-file tmp-file + (insert body)) + (ess-send-string + process (format "tryCatch(source('%s', print.eval=TRUE), finally=print(%s))" + (org-babel-process-file-name tmp-file 'noquote) + org-babel-R-eoe-indicator)) + (while (not (string-match (regexp-quote org-babel-R-eoe-output) + string-buffer)) + (accept-process-output process)) + (substring string-buffer 0 (match-beginning 0))))))) (defun org-babel-R-process-value-result (result column-names-p) "R-specific processing of return value. diff --git a/testing/lisp/test-ob-R.el b/testing/lisp/test-ob-R.el index 7ce340ba4..ff7ea19d5 100644 --- a/testing/lisp/test-ob-R.el +++ b/testing/lisp/test-ob-R.el @@ -97,6 +97,19 @@ (ert-deftest test-ob-R/results-file () (org-babel-goto-named-result "TESTSRC") (forward-line 1) (should (string= "[[file:junk/test.org]]" (buffer-substring-no-properties (point-at-bol) (point-at-eol))))))) + +(ert-deftest test-ob-R/prompt-output () + (let (ess-ask-for-ess-directory ess-history-file) + (org-test-with-temp-text + "#+begin_src R :results output :session\nprint(\" \")\nprint(\"one three\")\nprint(\"end\")\n#+end_src\n" + (should (string= "[1] \" \"\n[1] \"one three\"\n[1] \"end\"\n" (org-babel-execute-src-block)))))) + +(ert-deftest test-ob-R/output-nonprinted () + (let (ess-ask-for-ess-directory ess-history-file) + (org-test-with-temp-text + "#+begin_src R :results output :session\n4.0 * 3.5\nlog(10)\nlog10(10)\n(3 + 1) * 5\n3^-1\n1/0\n#+end_src\n" + (should (string= "[1] 14\n[1] 2.302585\n[1] 1\n[1] 20\n[1] 0.3333333\n[1] Inf\n" (org-babel-execute-src-block)))))) + (provide 'test-ob-R) ;;; test-ob-R.el ends here -- 2.28.0 --=-=-=--