Support for PEP492 async quasi-keyword in python.el

Support for PEP492 async quasi-keyword in python.el

[Lele Gaifax]

[-- Attachment #1: Type: text/plain, Size: 855 bytes --]

Hi all,

I spent the afternoon teaching python.el to properly indent the new “async”
constructs introduced in Python 3.5, and I'm attaching the result of my
attempt. 

Of course it's far from being either correct or complete, but maybe some of
you could give me some feedback on the following points:

a) is there a better way to introduce the new "async" variants of
   "def|with|for" in the `block-start' regexp?

b) how should the "await" quasi-keyword be handled? I mean, it's not a real
   keyword (that is, the assignment “await = 1” is valid code for example),
   but in an expression such as “return await foo()” it should probably be
   highlighted as such...

c) the diff below breaks one existing test, namely `python-indent-region-5'; I
   will try to understand what's going on...

Thank you in advance for any hint,
ciao, lele.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: PEP492 async support for python.el --]
[-- Type: text/x-diff, Size: 4450 bytes --]

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 6ff12b5..9f5c087 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -384,7 +384,10 @@ python-mode-map
   (defconst python-rx-constituents
     `((block-start          . ,(rx symbol-start
                                    (or "def" "class" "if" "elif" "else" "try"
-                                       "except" "finally" "for" "while" "with")
+                                       "except" "finally" "for" "while" "with"
+                                       ;; Python 3.5+ PEP492
+                                       (and "async" (+ space)
+                                            (or "def" "for" "with")))
                                    symbol-end))
       (dedenter            . ,(rx symbol-start
                                    (or "elif" "else" "except" "finally")
@@ -395,7 +398,11 @@ python-mode-map
                                   symbol-end))
       (decorator            . ,(rx line-start (* space) ?@ (any letter ?_)
                                    (* (any word ?_))))
-      (defun                . ,(rx symbol-start (or "def" "class") symbol-end))
+      (defun                . ,(rx symbol-start
+                                   (or "def" "class"
+                                       ;; Python 3.5+ PEP492
+                                       (and "async" (+ space) "def"))
+                                   symbol-end))
       (if-name-main         . ,(rx line-start "if" (+ space) "__name__"
                                    (+ space) "==" (+ space)
                                    (any ?' ?\") "__main__" (any ?' ?\")
@@ -527,6 +534,8 @@ python-font-lock-keywords
           ;; fontified like that in order to keep font-lock consistent between
           ;; Python versions.
           "nonlocal"
+          ;; Python 3.5+ PEP492
+          (and "async" (+ space) (or "def" "for" "with"))
           ;; Extra:
           "self")
          symbol-end)
diff --git a/test/automated/python-tests.el b/test/automated/python-tests.el
index 44b05e2..23d799a 100644
--- a/test/automated/python-tests.el
+++ b/test/automated/python-tests.el
@@ -604,6 +604,42 @@ python-tests-visible-string
    (should (eq (car (python-indent-context)) :after-line))
    (should (= (python-indent-calculate-indentation) 0))))
 
+(ert-deftest python-indent-after-async-block-1 ()
+  "Test PEP492 async def."
+  (python-tests-with-temp-buffer
+   "
+async def foo(a, b, c=True):
+"
+   (should (eq (car (python-indent-context)) :no-indent))
+   (should (= (python-indent-calculate-indentation) 0))
+   (goto-char (point-max))
+   (should (eq (car (python-indent-context)) :after-block-start))
+   (should (= (python-indent-calculate-indentation) 4))))
+
+(ert-deftest python-indent-after-async-block-2 ()
+  "Test PEP492 async with."
+  (python-tests-with-temp-buffer
+   "
+async with foo(a) as mgr:
+"
+   (should (eq (car (python-indent-context)) :no-indent))
+   (should (= (python-indent-calculate-indentation) 0))
+   (goto-char (point-max))
+   (should (eq (car (python-indent-context)) :after-block-start))
+   (should (= (python-indent-calculate-indentation) 4))))
+
+(ert-deftest python-indent-after-async-block-3 ()
+  "Test PEP492 async for."
+  (python-tests-with-temp-buffer
+   "
+async for a in sequencer():
+"
+   (should (eq (car (python-indent-context)) :no-indent))
+   (should (= (python-indent-calculate-indentation) 0))
+   (goto-char (point-max))
+   (should (eq (car (python-indent-context)) :after-block-start))
+   (should (= (python-indent-calculate-indentation) 4))))
+
 (ert-deftest python-indent-after-backslash-1 ()
   "The most common case."
   (python-tests-with-temp-buffer
@@ -1483,6 +1519,26 @@ python-tests-visible-string
                 (beginning-of-line)
                 (point))))))
 
+(ert-deftest python-nav-beginning-of-defun-3 ()
+  (python-tests-with-temp-buffer
+   "
+class C(object):
+
+    async def m(self):
+        return await self.c()
+
+    async def c(self):
+        pass
+"
+   (python-tests-look-at "self.c()")
+   (should (= (save-excursion
+                (python-nav-beginning-of-defun)
+                (point))
+              (save-excursion
+                (python-tests-look-at "async def m" -1)
+                (beginning-of-line)
+                (point))))))
+
 (ert-deftest python-nav-end-of-defun-1 ()
   (python-tests-with-temp-buffer
    "

[-- Attachment #3: Type: text/plain, Size: 206 bytes --]


-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it  |                 -- Fortunato Depero, 1929.

Re: Support for PEP492 async quasi-keyword in python.el

[Lele Gaifax]

Hi,

I wonder if this is the wrong forum for the topic, since none has answered
yet. Would a bugreport containing the patch be a better approach maybe?

FWIW, the following statement was a false alarm

> c) the diff below breaks one existing test, namely `python-indent-region-5'; I
>    will try to understand what's going on...

further checks showed no regressions introduced by my patch, which I'm using
with satisfaction.

thanks&bye, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it  |                 -- Fortunato Depero, 1929.

Re: Support for PEP492 async quasi-keyword in python.el

[Dmitry Gutov]

On 10/07/2015 11:36 AM, Lele Gaifax wrote:

> I wonder if this is the wrong forum for the topic, since none has answered
> yet. Would a bugreport containing the patch be a better approach maybe?

Submitting the patch as a bug report would be better, but either way 
we're waiting until Fabián has the time to review it.

Re: Support for PEP492 async quasi-keyword in python.el

[Lele Gaifax]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Submitting the patch as a bug report would be better, but either way we're
> waiting until Fabián has the time to review it.

Thank you Dmitry, will do that.

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it  |                 -- Fortunato Depero, 1929.