From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id eKVlLFEZCmR/BgAASxT56A (envelope-from ) for ; Thu, 09 Mar 2023 18:37:21 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id mNtpLFEZCmSIlwAAauVa8A (envelope-from ) for ; Thu, 09 Mar 2023 18:37:21 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 414F92F6E6 for ; Thu, 9 Mar 2023 18:37:21 +0100 (CET) Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=excalamus.com header.s=zmail header.b=FNIz62T8; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; arc=pass ("zohomail.com:s=zohoarc:i=1"); dmarc=none ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1678383441; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=tFrPm4bHeafbwrDd7hUOb8dkVUoqeKMcDUbZouoBClo=; b=gQw9dWTZwFRVT6eBIqot3aIdHWf9IPu+/PQmVPFRxFPE3oAsoP7gOI0jNThQVwqoGloE3q +XZWrKHWLxN4H8ru75/0vzr92M73VbojyGqbrzVD5M0LHOhgxcvDCqxXNWJKnJr39I+QWR +32rCPpRmhSa3pEmw06BesNdR3kX7ZfzXDeS/MhjbJz9I0nwaXTGUFOiYb9HUXt8ALgoMY jBP6UkCDeXjvTBI0JHK6xmxn0bnBGlPRCUzq63Uphm+pBvQfisczTSXvO2v43jazisvX6o A9eMXglbgmequMGEND/mTfadMBPzkYd8PEek6A7Zx9kZZA8QopvYnkb8fwyEkA== ARC-Seal: i=2; s=key1; d=yhetil.org; t=1678383441; a=rsa-sha256; cv=pass; b=JGfpi7a8jVTNDUgDs8ciDE+tuQDZKtYyco9nXsWfKIFfTJcGWg+xKEZZvwbPtC7EC0bMvq Ps8GlallwfRoLfl0sC/BtsGVYcNl3uroyPjL5lxBL8QJGS5VOwPxdHD/7Fm8wo5pvTdPJR z9kKY66plGbkKH/ulWOya9DnBgo5fdpG9t3qXBVKPz7VZnYesVOreklu7K442ORdJno6Zf +Y3yedqPVEg8yOfdOAVtxBB5WDCz0Cu5X9PrrhjabFvApqPrI/8E2qPvBzcYDa9rh5TZVx Hps52mJKzscJuFWKnPH/p7HB3WRGJfOsHIlfMD1LVBAYXtXY/FohSiUTnjtewQ== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=pass header.d=excalamus.com header.s=zmail header.b=FNIz62T8; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; arc=pass ("zohomail.com:s=zohoarc:i=1"); dmarc=none Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1paKBo-0004Dh-1L; Thu, 09 Mar 2023 12:36:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1paKBm-0004DU-Kr for emacs-orgmode@gnu.org; Thu, 09 Mar 2023 12:36:26 -0500 Received: from sender4-op-o15.zoho.com ([136.143.188.15]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1paKBk-0008Ss-PI for emacs-orgmode@gnu.org; Thu, 09 Mar 2023 12:36:26 -0500 ARC-Seal: i=1; a=rsa-sha256; t=1678383380; cv=none; d=zohomail.com; s=zohoarc; b=h3cKBfW+ZZZGKdRbXkX5jSXntSHM/2jNAhsSKUXA1G7CJVRDHkb3THWLJ2AO4M3h4mUqjGnEIe6VMmassNQzTN3kdQ6WnbOH6v8mHINN2F3a59DaUitFqD5ei+eaNFKzSUR5y5fx2ssGJohoRtGVkYalZ9TsmbTF4Q7lrWnCTAY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1678383380; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; bh=tFrPm4bHeafbwrDd7hUOb8dkVUoqeKMcDUbZouoBClo=; b=n8CZmHd/QzfOtsdglAl0HROTh2DmHqPyUwRJBAMrrOUvwYWls0O+uNPp2WK8vCgEFi81b5/uCwSX6/3M7x47yWZ1pCHoLq2kSkxgUNZYNiztH4ZYF3MN2Wwih53BIlgl6G4Vqbjp16pjhzOwcpbGm4+DoK7k+K39476lwPl6+/o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=excalamus.com; spf=pass smtp.mailfrom=matt@excalamus.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1678383380; s=zmail; d=excalamus.com; i=matt@excalamus.com; h=Date:Date:From:From:To:To:Cc:Cc:Message-ID:In-Reply-To:References:Subject:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To; bh=tFrPm4bHeafbwrDd7hUOb8dkVUoqeKMcDUbZouoBClo=; b=FNIz62T8NyOibHgyQGW4cYDUHlR/UxJTHuEJRc9P6yVDAxybhM3UWGDeI40LkevO jgrBdWkSakrpCZ/MwjFHFbRaqeSY2HH5683KoBKmhKKVlSiqd4r4N8cEn/aIlxJwXrv DuB2ahf3D41dDtNVULxxhmpu3ioPfIWOEOVMkWhs= Received: from mail.zoho.com by mx.zohomail.com with SMTP id 167838337738821.979064911478872; Thu, 9 Mar 2023 09:36:17 -0800 (PST) Date: Thu, 09 Mar 2023 12:36:17 -0500 From: Matt To: "Ihor Radchenko" Cc: "emacs-orgmode" , "jeremiejuste" , "jackkamm" Message-ID: <186c771ebdb.fadb25d63466542.8590370552765842508@excalamus.com> In-Reply-To: <87ttywg26t.fsf@localhost> References: <186283d230a.129f5feb61660123.3289004102603503414@excalamus.com> <87v8kd8zzw.fsf@localhost> <1863472efe9.10fdd5ba4258906.5972264927968042941@excalamus.com> <87y1p7axpe.fsf@localhost> <1863d69188f.11e078bd1228881.451172865246440158@excalamus.com> <87fsbcbf47.fsf@localhost> <186471d8980.c36aecd0374444.2548279362371712281@excalamus.com> <87sff73r0g.fsf@localhost> <18658639a30.dec640641420614.2364646537016397698@excalamus.com> <87lekwd10z.fsf@localhost> <1866bf50874.bc4ef1a92164564.1238825567948423531@excalamus.com> <87r0ukefzv.fsf@localhost> <1866fda933d.cf33745c2396994.7118762013608820270@excalamus.com> <87sfeyc7qr.fsf@localhost> <1869ff63e3e.f8c4ed671354538.4585567073216319262@excalamus.com> <87bkl96g6e.fsf@localhost> <186a89b5a24.cc4913fe1885630.21323979015166045@excalamus.com> <87ilffz95g.fsf@localhost> <186b5aa9d56.1238923db2169250.3173443400140454866@excalamus.com> <87ttywg26t.fsf@localhost> Subject: Re: [PATCH] Async evaluation in ob-shell MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Importance: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail Received-SPF: pass client-ip=136.143.188.15; envelope-from=matt@excalamus.com; helo=sender4-op-o15.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: X-Migadu-Scanner: scn0.migadu.com X-Migadu-Queue-Id: 414F92F6E6 X-Spam-Score: -5.48 X-Migadu-Spam-Score: -5.48 List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-TUID: mlWGlo01pqbW Hi Jack and Jeremie! I'm curious your thoughts about what Ihor and I are discussing at the end of this message about `md5'. ---- On Tue, 07 Mar 2023 07:45:02 -0500 Ihor Radchenko wrote --- > Matt matt@excalamus.com> writes: > > > > The actual prompt is "org_babel_sh_prompt> ". > > > > Agreed. > > > > > And we want to skip leading spaces in addition. > > > > What do you mean by this? > > I was following the pattern described in the docstring of > `comint-prompt-regexp', where it is suggested that prompts should follow > the pattern "^ *". > > In the case of ob-shell.el, `org-babel-sh-prompt' is a string to be used > as a prompt and the corresponding regexp patter will be > "^ *". Hence > > (concat "^" (regexp-quote org-babel-sh-prompt) " *") > > > > Adding " *" does not make the prompt match 2 spaces, but 1+. > > > > Agreed. > > > > It's not clear to me what pattern you're looking to match. > > I hope the above clarified things. I'm confused because when I run the Org from HEAD, I get: (concat "^" (regexp-quote org-babel-sh-prompt) " *") -> "^org_babel_sh_prompt> *" That's *two* spaces between '>' and '*', not one. The second space comes from either 1) the definition of `org-babel-sh-prompt', which is "org_babel_sh_prompt> " (with a single space) or 2) the " *" in the (concat "^" (regexp-quote org-babel-sh-prompt) " *") expression. Currently, the two combine via the concat to give *two* spaces in the regexp. If I understand you correctly, you're trying to match the pattern given in the `comint-prompt-regexp' which is *one* space. That's what I'm trying to do, too. Way back in https://list.orgmode.org/87sfeyc7qr.fsf@localhost/, we had this exchange: ---- On Mon, 20 Feb 2023 11:24:52 +0000 Ihor Radchenko wrote --- > Matt writes: > > > +(defun ob-shell-async-chunk-callback (string) > > + "Filter applied to results before insertion. > > +See `org-babel-comint-async-chunk-callback'." > > + (replace-regexp-in-string (concat org-babel-sh-prompt "*") "" string)) > > Why not using `comint-prompt-regexp'? I switched out `org-babel-sh-prompt' with `comint-prompt-regexp' so that the expression looks like: +(defun ob-shell-async-chunk-callback (string) + "Filter applied to results before insertion. +See `org-babel-comint-async-chunk-callback'." + (replace-regexp-in-string comint-prompt-regexp "" string)) This causes the new test `test-ob-shell/session-async-evaluation' to fail, as you pointed out: https://list.orgmode.org/87bkl96g6e.fsf@localhost/ The test fails when we switch out the prompt in the callback because `comint-prompt-regexp' has two spaces in it. The second space causes a prompt to not be filtered (by the callback). The output becomes ": 1\n: 2\n: org_babel_sh_prompt>\n" instead of ": 1\n: 2\n" . This looks like a bug in the `comint-prompt-regexp''. It could be that `test-ob-shell/session-async-evaluation' doesn't test correctly, but it looks right to me (I could certainly be mistaken). Therefore, I see only two options to fix it: remove a space from the concat expression (which I did in my latest patch) or remove a space from `org-babel-sh-prompt'. Am I missing something? > > > `md5' will be slightly faster compared to `org-id-uuid'. But it should > > > not matter. > > > > > > If we want use `org-id-uuid', lets move it to org-macs.el. Requiring the > > > whole org-id.el must not be done. It has side effects of defining id: > > > links. > > > > In the next revision (once we figure out the regex), I can create a separate commit moving `org-id-uuid' to org-macs.el and updating ob-R and ob-python from `md5' to `org-id-uuid' (assuming that's not an issue for the maintainers of those). If you think speed is a concern, however, I can switch ob-shell.el to use plain `md5'. > > I am in favour of using `org-id-uuid'. It might also be a useful generic > function for other purposes. > > A slight concern is that some third-party code might depend on the > current pattern used for UUID string in ob-python. But we made no > promises here. > > To be a bit safer, we can also refactor `org-uuidgen-p' exposing the > regexp used to match UUID. Also, it will make sense to move > `org-uuidgen-p' to org-macs.el. I'm okay with all that. I wonder, do Jack Kamm (of ob-python fame) and Jeremie Juste (of ob-R fame) have any thoughts on the matter. I ask out of courtesy since they're the maintainers of those packages and I don't want to cross any boundaries by changing their implementations beneath them.