Re: [BUG] Unexpected result when evaluating python src block asynchronously [9.7-pre (release_9.6.17-1131-gc9ed03.dirty @ /home/yantar92/.emacs.d/straight/build/org/)]

[Ihor Radchenko <yantar92@posteo.net>]

Jack Kamm <jackkamm@gmail.com> writes:

>> So, we have to insert some kind of indicator for async result.
>
> I meant that we could return something like "async:uuid-abcd-1234" or
> "async:/path/to/tmpfile", so that `org-babel-comint-async-filter' could
> still find the result.

Of course, we could. But that would not solve all the possible problems.
In particular, when header arguments tell Org babel to write result to a
file, returning UUID will still not work.

>> Of course, the existing scheme of coordination between
>> `org-babel-insert-result' and `org-babel-comint-async-filter' is
>> erroneous:
>>
>> 1. We have the problem with :results file value discussed here
>> 2. We have a worse problem with :results file :file foo when the result
>>    may not be unique
>> 3. We have :results append/prepend completely broken because
>>    `org-babel-comint-async-filter' simply calls
>>    `org-babel-insert-result' implicitly assuming that the existing
>>    indicator is replaced.
>>
>> The whole thing should be re-designed.
> ...
> A bit of a tangent, but if you are thinking about re-designing this,
> then it may be worth considering ob-jupyter's implementation of async
> sessions [1]. In particular, I believe it leaves a marker [2] where it
> needs to insert the future result. I don't remember the details,
> e.g. how it keeps track of which marker is for which result. But it
> seems neat, and might work better for some cases such as
> appending/prepending results.

Markers are not as reliable as you think. If text around marker gets
deleted, the marker will still exist potentially causing the async
result to be inserted in the middle of unexpected place.

Having an actual text indicator is more reliable - if user removes it
before the async evaluation is completed, we will not write anything
unexpected.

Also,
https://github.com/emacs-jupyter/jupyter/blob/master/ob-jupyter.el#L540

        ;; KLUDGE: Remove the file result-parameter so that
        ;; `org-babel-insert-result' doesn't attempt to handle it while
        ;; async results are pending.  Do the same in the synchronous
        ;; case, but not if link or graphics are also result-parameters,
        ;; only in Org >= 9.2, since those in combination with file mean
        ;; to interpret the result as a file link, a useful meaning that
        ;; doesn't interfere with Jupyter style result insertion.

They also had to work around the same problem.

> I agree that it would be good to redesign it, but am not sure where to
> start.

For example,

1. Change `org-babel-comint-async-register' to return UUID and to store
   PARAMS as passed by the backend (current approach with PARAMS being
   derived from src blocks prevents backends to transform src block
   PARAMS dynamically).
2. Change `org-babel-insert-result' to handle :async t specially,
   inserting something reliable, like #+async: <UUID> in place of result
   without performing extra transformations.
3. Change `org-babel-insert-result' to accept an internal parameter
   that will make it replace #+async: <UUID> keyword rather than perform
   normal result insertion.
4. Change `org-babel-comint-async-filter' to use the previously passed
   PARAMS, remove :async t from them, and arrange the call to
   `org-babel-insert-result' to replace the #+async: <UUID> keyword.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>