bug#25753: 25.2; Python mode shell interaction not working 100%

[charles@aurox.ch (Charles A. Roelli)]

On Thu, Feb 23 2017 at 09:20:01 am, npostavs@users.sourceforge.net wrote:

> charles@aurox.ch (Charles A. Roelli) writes:
>>
>> Tested, and it has the same problem.  Here is what *Python* normally
>> looks like at the start, run from M-x run-python in emacs -Q:
>>
>> Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 26 2016, 12:10:39) 
>> [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import codecs, os;__pyfile =
>>>>> codecs.open('''/var/folders/WP/WPe0Q1iAGc0J7iI6J50jcU+++TI/-Tmp-/py16611qgD''',
>>>>> encoding='''utf-8''');__code =
>>>>> __pyfile.read().encode('''utf-8''');__pyfile.close();os.remove('''/var/folders/WP/WPe0Q1iAGc0J7iI6J50jcU+++TI/-Tmp-/py16611qgD''');exec(compile(__code,
>>>>> '''/var/folders/WP/WPe0Q1iAGc0J7iI6J50jcU+++TI/-Tmp-/py16611qgD''',
>>>>> 'exec'));
>> python.el: native completion setup loaded
>>
>> It seems like the line starting with ">>> " should not be printed, if
>> <nyc4bos@aol.com>'s test output is deemed to be running more or less
>> correctly.  Maybe that can point us in the right direction?
>
> Hmm, it's odd, though since you're still getting the "setup loaded"
> message, the code *is* getting evaluated anyway, so it's likely that
> this problem is not directly related.  Let's see if we can track it down
> anyway, try the loading the attached as
>
>     emacs -Q -l py-trace-bad-output.el
>
> and see if anything shows up in *Messages*.
>
> (require 'python)
>
> (advice-add
>  'python-shell-make-comint :filter-return
>  (lambda (proc-buffer-name)
>    (with-current-buffer proc-buffer-name
>      (add-hook 'after-change-functions
>                (lambda (beg end len)
>                  (ignore-errors
>                    (save-excursion
>                      (save-match-data
>                        (when (string-prefix-p "import codecs" (buffer-substring beg end))
>                          (backtrace))))))
>                nil t))
>    proc-buffer-name))
>
> (run-python)
>
> (switch-to-buffer "*Python*")
> (display-buffer "*Messages*")
>

Here is what I got:

Loading ~/Downloads/py-trace-bad-output.el (source)...done
  backtrace()
  (progn (backtrace))
  (if (string-prefix-p "import codecs" (buffer-substring beg end)) (progn (backtrace)))
  (progn (if (string-prefix-p "import codecs" (buffer-substring beg end)) (progn (backtrace))))
  (unwind-protect (progn (if (string-prefix-p "import codecs" (buffer-substring beg end)) (progn (backtrace)))) (set-match-data save-match-data-internal (quote evaporate)))
  (let ((save-match-data-internal (match-data))) (unwind-protect (progn (if (string-prefix-p "import codecs" (buffer-substring beg end)) (progn (backtrace)))) (set-match-data save-match-data-internal (quote evaporate))))
  (save-excursion (let ((save-match-data-internal (match-data))) (unwind-protect (progn (if (string-prefix-p "import codecs" (buffer-substring beg end)) (progn (backtrace)))) (set-match-data save-match-data-internal (quote evaporate)))))
  (progn (save-excursion (let ((save-match-data-internal (match-data))) (unwind-protect (progn (if (string-prefix-p "import codecs" (buffer-substring beg end)) (progn (backtrace)))) (set-match-data save-match-data-internal (quote evaporate))))))
  (condition-case nil (progn (save-excursion (let ((save-match-data-internal (match-data))) (unwind-protect (progn (if (string-prefix-p "import codecs" ...) (progn ...))) (set-match-data save-match-data-internal (quote evaporate)))))) (error nil))
  (lambda (beg end len) (condition-case nil (progn (save-excursion (let ((save-match-data-internal (match-data))) (unwind-protect (progn (if ... ...)) (set-match-data save-match-data-internal (quote evaporate)))))) (error nil)))(191 556 0)
  comint-output-filter(#<process Python> "import codecs, os;__pyfile = codecs.open('''/var/folders/WP/WPe0Q1iAGc0J7iI6J50jcU+++TI/-Tmp-/py683UGH''', encoding='''utf-8''');__code = __pyfile.read().encode('''utf-8''');__pyfile.close();os.remove('''/var/folders/WP/WPe0Q1iAGc0J7iI6J50jcU+++TI/-Tmp-/py683UGH''');exec(compile(__code, '''/var/folders/WP/WPe0Q1iAGc0J7iI6J50jcU+++TI/-Tmp-/py683UGH''', 'exec'));
")
  accept-process-output(#<process Python> 1.0)
  python-shell-accept-process-output(#<process Python> 1.0)
  python-shell-completion-native-setup()
  python-shell-completion-native-turn-on-maybe(t)
  python-shell-completion-native-turn-on-maybe-with-msg()
  run-hooks(python-shell-first-prompt-hook)
  python-shell-comint-watch-for-first-prompt-output-filter(">>> ")
  run-hook-with-args(python-shell-comint-watch-for-first-prompt-output-filter ">>> ")
  comint-output-filter(#<process Python> ">>> ")
  read-event(nil t 2)
  sit-for(2)
  execute-extended-command(nil "load-file" "load-file")
  funcall-interactively(execute-extended-command nil "load-file" "load-file")
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)

Shell native completion is disabled, using fallback

>>> Can you test the python code from python-shell-completion-native-setup
>>> outside of Emacs, e.g., save it to a file called 'native-completion.py'
>>> and then run 'python -i native-completion.py' and then type an
>>> underscore and hit <tab>.
>>
>> I get the following:
>>
>>>>> ___package__
>> __PYTHON_EL_native_completion_setup
>> __name__
>> __doc__
>> __import__
>> __debug__
>>
>> Interestingly, none of the dummy completions pop up.
>
> This seems to be the core of the problem.  I gather that macOS uses
> libedit instead of readline by default, perhaps that is the source of
> incompatibility.  Can you figure out how to change the python code so
> that the dummy completions do show up?

Yes, I will look into it.