Re: [Emacs-diffs] trunk r117592: Grab all Python process output before inferior-python-mode hooks.

Re: [Emacs-diffs] trunk r117592: Grab all Python process output before inferior-python-mode hooks.

[Stefan Monnier]

> +  ;; Ensure all the output is accepted before running any hooks.
> +  (accept-process-output (get-buffer-process (current-buffer)))
> +  (sit-for 0.1 t))
 
Why do you need sit-for here?


        Stefan

Re: [Emacs-diffs] trunk r117592: Grab all Python process output before inferior-python-mode hooks.

[Fabián Ezequiel Gallina]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> +  ;; Ensure all the output is accepted before running any hooks.
>> +  (accept-process-output (get-buffer-process (current-buffer)))
>> +  (sit-for 0.1 t))
>  
> Why do you need sit-for here?
>

I just added a comment about it in revno 117605.

  ;; At this point, all process output should have been received, but
  ;; on GNU/Linux, calling `python-shell-internal-send-string' without
  ;; a running internal shell fails to grab output properly unless
  ;; this `sit-for' is in place.

Sorry about that, I should have placed that comment before!




Cheers,
Fabián

Re: [Emacs-diffs] trunk r117592: Grab all Python process output before inferior-python-mode hooks.

[Stefan Monnier]

>>> +  ;; Ensure all the output is accepted before running any hooks.
>>> +  (accept-process-output (get-buffer-process (current-buffer)))
>>> +  (sit-for 0.1 t))
>> Why do you need sit-for here?
> I just added a comment about it in revno 117605.
>   ;; At this point, all process output should have been received, but
>   ;; on GNU/Linux, calling `python-shell-internal-send-string' without
>   ;; a running internal shell fails to grab output properly unless
>   ;; this `sit-for' is in place.
> Sorry about that, I should have placed that comment before!

Sounds like a workaround for a problem somewhere in Emacs.
Could you make a bug-report for it, ideally providing a simple
test case?


        Stefan

Re: [Emacs-diffs] trunk r117592: Grab all Python process output before inferior-python-mode hooks.

[Fabián Ezequiel Gallina]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>>> +  ;; Ensure all the output is accepted before running any hooks.
>>>> +  (accept-process-output (get-buffer-process (current-buffer)))
>>>> +  (sit-for 0.1 t))
>>> Why do you need sit-for here?
>> I just added a comment about it in revno 117605.
>>   ;; At this point, all process output should have been received, but
>>   ;; on GNU/Linux, calling `python-shell-internal-send-string' without
>>   ;; a running internal shell fails to grab output properly unless
>>   ;; this `sit-for' is in place.
>> Sorry about that, I should have placed that comment before!
>
> Sounds like a workaround for a problem somewhere in Emacs.
> Could you make a bug-report for it, ideally providing a simple
> test case?

So I think I finally understand what's going on:

When Python starts up, it actually sends two things to stdout in
separate calls.

The first one is the banner and the second is the initial prompt.

What it seems to be happening is that on the first accept-process-output
call, the output that is grabbed is just the banner, leaving the prompt
outside, and that's what the sit-for was actually "fixing".  The truth
is (I think) that sit-for should be another accept-process-output call.

To exercise this, I wrote the following example.

    (let ((buffer (get-buffer-create "*accept-bug*")))
      (with-current-buffer "*accept-bug*"
        (start-process "python" (current-buffer) "python" "-i")
        (let ((process (get-buffer-process (current-buffer))))
          (set-process-query-on-exit-flag process nil)
          (message "Output (0)\n====================\n%s\n\n"
                   (buffer-substring-no-properties (point-min) (point-max)))
          (accept-process-output process)  ; (a)
          (message "Output (1)\n====================\n%s\n\n"
                   (buffer-substring-no-properties (point-min) (point-max)))
          (accept-process-output process)  ; (b)
          (message "Output (2)\n====================\n%s\n\n"
                   (buffer-substring-no-properties (point-min) (point-max)))
          (process-send-string 
           process (format "print ('a' * 16)\n"))
          (accept-process-output process)  ; (c)
          (prog1
              (let ((output
                     (buffer-substring-no-properties
                      (point-min)
                      (point-max))))
                (format "Output (3)\n====================\n%s\n\n" output))
            (kill-buffer (current-buffer))))))

Evaluating that should let you see in the *Messages* buffer the behavior
I'm describing.

Commenting any of the accept-process-output calls, ends up returning
just the banner and the prompt sans the output from the print statement.

My guess then is that this is not really an Emacs bug but rather a
singularity of how the Python process sends its output, but since I'm no
expert to tell if accept-process-output should be smart enough to handle
that case, I would let you to decide.



Saludos,
Fabián

Re: [Emacs-diffs] trunk r117592: Grab all Python process output before inferior-python-mode hooks.

[Stefan Monnier]

> What it seems to be happening is that on the first accept-process-output
> call, the output that is grabbed is just the banner, leaving the prompt
> outside, and that's what the sit-for was actually "fixing".  The truth
> is (I think) that sit-for should be another accept-process-output call.

Actually, there's no guarantee that the two chunks that are sent by
Python will be received as 2 chunks by Emacs.  It all depends on the OS
and timing and buffering.  It could be received as a single chunk
(typically if Emacs gets to run "late" enough that both chunks have
already been sent), or it could be further divided by the OS into
smaller chunks (typically if the chunks are large).

So you should use a loop, which checks the actual text received, looking
for a tell-tale sign (e.g. a prompt) that "we've received all we need so
far".


        Stefan