bug#12340: 24.2.50; python-shell-switch-to-shell in TRAMP buffer gives "No such file or directory: '/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333GTT'"

bug#12340: 24.2.50; python-shell-switch-to-shell in TRAMP buffer gives "No such file or directory: '/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333GTT'"

[Matt McClure]

This bug report will be sent to the Bug-GNU-Emacs mailing list
and the GNU bug tracker at debbugs.gnu.org.  Please check that
the From: line contains a valid email address.  After a delay of up
to one day, you should receive an acknowledgment at that address.

Please write in English if possible, as the Emacs maintainers
usually do not have translators for other languages.

Please describe exactly what actions triggered the bug, and
the precise symptoms of the bug.  If you can, give a recipe
starting from `emacs -Q':

***

To reproduce:

    $ open -n -a Emacs --args -Q --no-site-file --eval '(progn
(find-file "/<user>@<host>:~/<foo>.py")
(python-shell-switch-to-shell))'

    RET n (or y)

The *Python* buffer contains:

    Python 2.4.3 (#1, Jun 18 2012, 08:55:23)
    [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    IOError: [Errno 2] No such file or directory:
'/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333GTT'
    >>> Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    IOError: [Errno 2] No such file or directory:
'/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333TdZ'
    >>> Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    IOError: [Errno 2] No such file or directory:
'/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333gnf'
    >>>

Looks to me as though python.el and TRAMP need to use the temporary
directory on the remote machine rather than the local machine.

I'm able to reproduce the defect in
Emacs-2012-07-14-109087-universal-10.6.8 at least through
Emacs-2012-08-28-109798-universal-10.6.8 but not in
Emacs-2012-06-08-108519-universal-10.6.8. In
Emacs-2012-06-08-108519-universal-10.6.8 the equivalent recipe to get
to a Python shell is:

    open -n -a Emacs --args -Q --no-site-file --eval '(progn
(find-file "/vagrant@192.168.33.2:~/hello.py")
(python-switch-to-python nil))'

***

If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
For information about debugging Emacs, please read the file
/Applications/Emacs.app/Contents/Resources/etc/DEBUG.


In GNU Emacs 24.2.50.1 (x86_64-apple-darwin, NS apple-appkit-1038.36)
 of 2012-08-28 on bob.porkrind.org
Bzr revision: 109798 dmantipov@yandex.ru-20120828062008-t2jfhu5niq9upru9
Windowing system distributor `Apple', version 10.3.1138
Configured using:
 `configure '--host=x86_64-apple-darwin' '--build=i686-apple-darwin'
 '--with-ns' 'build_alias=i686-apple-darwin'
 'host_alias=x86_64-apple-darwin' 'CC=gcc -mmacosx-version-min=10.5''

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Inferior Python

Minor modes in effect:
  compilation-shell-minor-mode: t
  shell-dirtrack-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<return> n C-x h <escape> w <escape> x r e p o r t
<tab> <return>

Recent messages:

Tramp: Waiting for prompts from remote shell
Tramp: Sending command `exec ssh -q -l vagrant  -o
ControlPath=/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/tramp.623335IN.%r@%h:%p
-o ControlMaster=yes -e none 192.168.33.2'
Tramp: Found remote shell prompt on `192.168.33.2'
Tramp: Opening connection for vagrant@192.168.33.2 using scpc...done
Sent python-shell-completion-setup-code
Sent python-ffap-setup-code
Sent python-eldoc-setup-code
Mark set [2 times]
delete-backward-char: Text is read-only

Load-path shadows:
None found.

Features:
(shadow sort mail-extr misearch multi-isearch emacsbug message rfc822
mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mail-utils
compile python rx easymenu tramp-cache tramp-sh tramp tramp-compat
auth-source eieio byte-opt bytecomp byte-compile cconv macroexp
gnus-util mm-util mail-prsvr password-cache tramp-loaddefs shell
pcomplete comint ansi-color ring format-spec advice help-fns
advice-preload time-date tooltip ediff-hook vc-hooks lisp-float-type
mwheel ns-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment lisp-mode register page menu-bar rfn-eshadow
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core frame cham georgian utf-8-lang misc-lang vietnamese tibetan
thai tai-viet lao korean japanese hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
cus-face files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote
make-network-process ns multi-tty emacs)


-- 
Matt McClure
http://www.matthewlmcclure.com
http://www.mapmyfitness.com/profile/matthewlmcclure

bug#12340: 24.2.50; python-shell-switch-to-shell in TRAMP buffer gives "No such file or directory: '/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333GTT'"

[Matt McClure]

Here's a patch that I think might resolve the issue. Only lightly tested.

--- python.el.orig	2012-09-03 19:30:56.000000000 -0400
+++ python.el	2012-09-03 19:31:56.000000000 -0400
@@ -1824,7 +1824,13 @@
         (lines (split-string string "\n" t)))
     (and msg (message "Sent: %s..." (nth 0 lines)))
     (if (> (length lines) 1)
-        (let* ((temp-file-name (make-temp-file "py"))
+
+        (let* ((temporary-file-directory
+                (if (file-remote-p default-directory)
+                    (concat (file-remote-p default-directory) "/tmp")
+                  temporary-file-directory))
+               (f (make-temp-file "py"))
+               (temp-file-name (or (file-remote-p f 'localname) f))
                (file-name (or (buffer-file-name) temp-file-name)))
           (with-temp-file temp-file-name
             (insert string)


-- 
Matt McClure
http://www.matthewlmcclure.com
http://www.mapmyfitness.com/profile/matthewlmcclure

bug#12340: 24.2.50; python-shell-switch-to-shell in TRAMP buffer gives "No such file or directory: '/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333GTT'"

[Matt McClure]

If you prefer to pull the change from a Git repo, here's the same
change on GitHub:
<https://github.com/matthewlmcclure/emacs/commit/8afa8fa5569488a8831cbe86a1cc2971cf0b7280>

bug#12340: 24.2.50; python-shell-switch-to-shell in TRAMP buffer gives "No such file or directory: '/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333GTT'"

[Michael Albinus]

Matt McClure <matthewlmcclure@gmail.com> writes:

> Here's a patch that I think might resolve the issue. Only lightly tested.

It looks good, you have derived it from the patch we have created some
weeks ago :-)

However:

> +        (let* ((temporary-file-directory
> +                (if (file-remote-p default-directory)
> +                    (concat (file-remote-p default-directory) "/tmp")
> +                  temporary-file-directory))
> +               (f (make-temp-file "py"))
> +               (temp-file-name (or (file-remote-p f 'localname) f))
>                 (file-name (or (buffer-file-name) temp-file-name)))

Shouldn't this be

                 (file-name (or (buffer-file-name) f)))

Best regards, Michael.

bug#12340: 24.2.50; python-shell-switch-to-shell in TRAMP buffer gives "No such file or directory: '/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333GTT'"

[Matt McClure]

On Tue, Sep 4, 2012 at 9:51 AM, Michael Albinus <michael.albinus@gmx.de> wrote:
> However:
>
>> +        (let* ((temporary-file-directory
>> +                (if (file-remote-p default-directory)
>> +                    (concat (file-remote-p default-directory) "/tmp")
>> +                  temporary-file-directory))
>> +               (f (make-temp-file "py"))
>> +               (temp-file-name (or (file-remote-p f 'localname) f))
>>                 (file-name (or (buffer-file-name) temp-file-name)))
>
> Shouldn't this be
>
>                  (file-name (or (buffer-file-name) f)))

Actually, the first attempt didn't really work. Here's an updated
patch: <https://github.com/matthewlmcclure/emacs/compare/master...debbug-12340>.
I've tested this one on local and remote interpreters, using
python-shell-send-{buffer,region,file}.


-- 
Matt McClure
http://www.matthewlmcclure.com
http://www.mapmyfitness.com/profile/matthewlmcclure

bug#12340: 24.2.50; python-shell-switch-to-shell in TRAMP buffer gives "No such file or directory: '/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333GTT'"

[Michael Albinus]

Matt McClure <matthewlmcclure@gmail.com> writes:

> Actually, the first attempt didn't really work. Here's an updated
> patch: <https://github.com/matthewlmcclure/emacs/compare/master...debbug-12340>.
> I've tested this one on local and remote interpreters, using
> python-shell-send-{buffer,region,file}.

This are more than 15 changed lines, it doesn't count as tiny change. If
you want to get it pushed to Emacs, you would need to sign FSF legal
papers. Are you interested in?

I believe, the patch is too complicate. When you call
python-shell-send-file, you pass file-name in remote syntax, but
temp-file-name already stripped to the local file name part on the
remote side. This confuses, I believe. I wouldn't use
host-rel-temp-file-name here, and just pass temp-file-name as it is. All
the remote name strippings could be performed in python-shell-send-file.
Something like this (completely untested, but you get the idea):

--8<---------------cut here---------------start------------->8---
*** ~/src/emacs/lisp/progmodes/python.el.~109871~	2012-09-05 16:01:08.029147300 +0200
--- ~/src/emacs/lisp/progmodes/python.el	2012-09-05 16:35:03.756949973 +0200
***************
*** 1824,1830 ****
          (lines (split-string string "\n" t)))
      (and msg (message "Sent: %s..." (nth 0 lines)))
      (if (> (length lines) 1)
!         (let* ((temp-file-name (make-temp-file "py"))
                 (file-name (or (buffer-file-name) temp-file-name)))
            (with-temp-file temp-file-name
              (insert string)
--- 1824,1834 ----
          (lines (split-string string "\n" t)))
      (and msg (message "Sent: %s..." (nth 0 lines)))
      (if (> (length lines) 1)
!         (let* ((temporary-file-directory
!                 (if (file-remote-p default-directory)
!                     (concat (file-remote-p default-directory) "/tmp")
!                   temporary-file-directory))
!                (temp-file-name (make-temp-file "py"))
                 (file-name (or (buffer-file-name) temp-file-name)))
            (with-temp-file temp-file-name
              (insert string)
***************
*** 1940,1946 ****
        (concat "__pyfile = open('''%s''');"
                "exec(compile(__pyfile.read(), '''%s''', 'exec'));"
                "__pyfile.close()")
!       (or temp-file-name file-name) file-name)
       process)))

  (defun python-shell-switch-to-shell ()
--- 1944,1951 ----
        (concat "__pyfile = open('''%s''');"
                "exec(compile(__pyfile.read(), '''%s''', 'exec'));"
                "__pyfile.close()")
!       (or (file-remote-p temp-file-name 'localname) temp-file-name)
!       (or (file-remote-p file-name 'localname) file-name))
       process)))

  (defun python-shell-switch-to-shell ()
--8<---------------cut here---------------end--------------->8---

Best regards, Michael.

bug#12340: 24.2.50; python-shell-switch-to-shell in TRAMP buffer gives "No such file or directory: '/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333GTT'"

[Matt McClure]

On Sep 5, 2012, at 11:03 AM, Michael Albinus <michael.albinus@gmx.de> wrote:

> This are more than 15 changed lines, it doesn't count as tiny change. If
> you want to get it pushed to Emacs, you would need to sign FSF legal
> papers. Are you interested in?

Sure. Where do I get and send the required paperwork?

I'll refactor the change to simplify it.

Matt McClure
http://matthewlmcclure.com
http://www.mapmyfitness.com/profile/matthewlmcclure

bug#12340: 24.2.50; python-shell-switch-to-shell in TRAMP buffer gives "No such file or directory: '/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333GTT'"

[Chong Yidong]

Matt McClure <matthewlmcclure@gmail.com> writes:

> Sure. Where do I get and send the required paperwork?

Please email the following information to fsf-records@gnu.org, and they
will send you the assignment form for your past and future changes.
Please use your full name as the subject line of the message.

[What is your full legal name?]

[What is the name of the program or package you're contributing to?]

GNU Emacs

[Did you copy any files or text written by someone else in these changes?
Even if that material is free software, we need to know about it.]

[Do you have an employer who might have a basis to claim to own
your changes?  Do you attend a school which might make such a claim?]

[For the copyright registration, what country are you a citizen of?]

[What year were you born?]

[Please write your email address here.]

[Please write your snail address here.]

[Which files have you changed so far, and which new files have you written
so far?]

bug#12340: 24.2.50; python-shell-switch-to-shell in TRAMP buffer gives "No such file or directory: '/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333GTT'"

[Matt McClure]

On Thu, Sep 6, 2012 at 12:15 AM, Chong Yidong <cyd@gnu.org> wrote:
> Matt McClure <matthewlmcclure@gmail.com> writes:
>
>> Sure. Where do I get and send the required paperwork?
>
> Please email the following information to fsf-records@gnu.org, and they
> will send you the assignment form for your past and future changes.

Thanks, Chong. I sent the info to fsf-records.

-- 
Matt McClure
http://www.matthewlmcclure.com
http://www.mapmyfitness.com/profile/matthewlmcclure

bug#12340: 24.2.50; python-shell-switch-to-shell in TRAMP buffer gives "No such file or directory: '/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333GTT'"

[Matt McClure]

On Wed, Sep 5, 2012 at 11:03 AM, Michael Albinus <michael.albinus@gmx.de> wrote:
> I believe, the patch is too complicate.

Michael,

Here's a much simpler patch with the same behavior.

https://github.com/matthewlmcclure/emacs/compare/master...debbug-12340

(same link, updated content)

Matt

bug#12340: 24.2.50; python-shell-switch-to-shell in TRAMP buffer gives "No such file or directory: '/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333GTT'"

[Michael Albinus]

Matt McClure <matthewlmcclure@gmail.com> writes:

> Michael,

Hi Matt,

> Here's a much simpler patch with the same behavior.
>
> https://github.com/matthewlmcclure/emacs/compare/master...debbug-12340

Looks OK to me.

But there is still one weak point (not introduced by your patch). In
`python-shell-send-file', there is the form

(or (expand-file-name
     (or (file-remote-p file-name 'localname)
         file-name))
    temp-file-name)

The intention is obvious: if `file-name' is nil, use `temp-file-name'
instead. But in this case, we would eval (expand-file-name nil), which
raises an error. Could you fix this as well?

I would commit this patch in your name, if this is fixed. Ah, a
ChangeLog entry would be great.

> Matt

Best regards, Michael.

bug#12340: 24.2.50; python-shell-switch-to-shell in TRAMP buffer gives "No such file or directory: '/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333GTT'"

[Matt McClure]

On Fri, Sep 7, 2012 at 7:45 AM, Michael Albinus <michael.albinus@gmx.de> wrote:
> The intention is obvious: if `file-name' is nil, use `temp-file-name'
> instead. But in this case, we would eval (expand-file-name nil), which
> raises an error. Could you fix this as well?

Fixed. And added some line breaks to eliminate the long lines I had introduced.

> I would commit this patch in your name, if this is fixed. Ah, a
> ChangeLog entry would be great.

Done.

Same link as above for the new patch.

-- 
Matt McClure
http://www.matthewlmcclure.com
http://www.mapmyfitness.com/profile/matthewlmcclure

bug#12340: 24.2.50; python-shell-switch-to-shell in TRAMP buffer gives "No such file or directory: '/var/folders/vk/p95wm1qx0vdf0087p0z9qzc40000gn/T/py62333GTT'"

[Michael Albinus]

Matt McClure <matthewlmcclure@gmail.com> writes:

> Fixed. And added some line breaks to eliminate the long lines I had introduced.

Yep, that's it. (And with 13 lines it counts as tiny patch :-)

Committed to the trunk, closing the bug.

Thanks, and best regards, Michael.