bug#46817: update python grammar

bug#46817: update python grammar

[Martin Puttke]

[-- Attachment #1: Type: text/plain, Size: 258 bytes --]

Hi out there,

I made some fixes to enhance the python grammar file to enable it 
to understand python type hints and have non-trivial base classes. 
Please feel free to include my changes into emacs.

please find my patches attached.

All the best

Martin


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-add-python-grammar-for-type-hints-in-variable-defini.patch --]
[-- Type: text/x-diff, Size: 960 bytes --]

From b5eaf8174267ac466ad3d8be63bae4448f476d16 Mon Sep 17 00:00:00 2001
From: Kapuze Martin <m.s.p@posteo.de>
Date: Wed, 18 Nov 2020 17:54:22 +0100
Subject: [PATCH 1/3] add python grammar for type hints in variable definitions

---
 admin/grammars/python.wy | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/admin/grammars/python.wy b/admin/grammars/python.wy
index aaa25ced20..b7254e2533 100644
--- a/admin/grammars/python.wy
+++ b/admin/grammars/python.wy
@@ -1140,7 +1140,7 @@ fpdef_opt_test
 
 ;; fpdef: NAME | '(' fplist ')'
 fpdef
-  : NAME
+  : NAME type_hint
     (VARIABLE-TAG $1 nil nil)
  ;; Below breaks the parser.  Don't know why, but my guess is that
  ;; LPAREN/RPAREN clashes with the ones in function_parameters.
@@ -1160,6 +1160,15 @@ fpdef
 ;;   | fpdef_list COMMA fpdef
 ;;   ;
 
+type_hint
+  : ;;EMPTY
+  | COLON type
+  ;
+
+type
+  : dotted_name
+  ;
+
 ;; ['=' test]
 eq_test_opt
   : ;;EMPTY
-- 
2.25.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-add-python-grammar-for-type-hints-for-function-retur.patch --]
[-- Type: text/x-diff, Size: 1405 bytes --]

From 3d8c711acc63bb2e61dfeccb73223ff3bf29b67e Mon Sep 17 00:00:00 2001
From: Kapuze Martin <m.s.p@posteo.de>
Date: Wed, 18 Nov 2020 18:11:36 +0100
Subject: [PATCH 2/3] add python grammar for type hints for function return
 value

---
 admin/grammars/python.wy | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/admin/grammars/python.wy b/admin/grammars/python.wy
index b7254e2533..cc5093b0b7 100644
--- a/admin/grammars/python.wy
+++ b/admin/grammars/python.wy
@@ -184,6 +184,7 @@
 %token <punctuation> ASSIGN 	"="
 %token <punctuation> BACKQUOTE	"`"
 %token <punctuation> AT         "@"
+%token <punctuation> FOLLOWS    "->"
 
 
 ;; -----------------
@@ -808,12 +809,17 @@ decorators
 
 ;; funcdef: [decorators] 'def' NAME parameters ':' suite
 funcdef
-  : DEF NAME function_parameter_list COLON suite
+  : DEF NAME function_parameter_list return_type_hint COLON suite
     (wisent-python-reconstitute-function-tag
-     (FUNCTION-TAG $2 nil $3) $5)
-  | decorators DEF NAME function_parameter_list COLON suite
+     (FUNCTION-TAG $2 nil $3) $6)
+  | decorators DEF NAME function_parameter_list return_type_hint COLON suite
     (wisent-python-reconstitute-function-tag
-     (FUNCTION-TAG $3 nil $4 :decorators $1) $6)
+     (FUNCTION-TAG $3 nil $4 :decorators $1) $7)
+  ;
+
+return_type_hint
+  : ;;EMPTY
+  | FOLLOWS type
   ;
 
 function_parameter_list
-- 
2.25.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0003-generalize-type-definition-and-apply-it-to-parent-cl.patch --]
[-- Type: text/x-diff, Size: 877 bytes --]

From cf4cd6d4ca1debc6f0b82aa5c9a03a78730980b1 Mon Sep 17 00:00:00 2001
From: Kapuze Martin <m.s.p@posteo.de>
Date: Mon, 23 Nov 2020 21:56:09 +0100
Subject: [PATCH 3/3] generalize type definition and apply it to parent classes

---
 admin/grammars/python.wy | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/admin/grammars/python.wy b/admin/grammars/python.wy
index cc5093b0b7..bb3d96d703 100644
--- a/admin/grammars/python.wy
+++ b/admin/grammars/python.wy
@@ -893,7 +893,7 @@ paren_classes
 ;; parser can parse general expressions, I don't see much benefit in
 ;; generating a string of expression as base class "name".
 paren_class
-  : dotted_name
+  : type
   ;
 
 ;;;****************************************************************************
@@ -1172,7 +1172,7 @@ type_hint
   ;
 
 type
-  : dotted_name
+  : test
   ;
 
 ;; ['=' test]
-- 
2.25.1


[-- Attachment #5: Type: text/plain, Size: 49 bytes --]


-- 
Composed with Spacemacs (www.spacemacs.org)

bug#46817: update python grammar

[Lars Ingebrigtsen]

Martin Puttke <martinstefanputtke@posteo.de> writes:

> I made some fixes to enhance the python grammar file to enable it to
> understand python type hints and have non-trivial base classes. Please
> feel free to include my changes into emacs.

Thanks; I'm not a Python or Wisent expert, but if I read the patch
correctly, it looks good to me, and I've applied your patch to Emacs 28.

This change was small enough to apply without assigning copyright to the
FSF, but for future patches you want to submit, it might make sense to
get the paperwork started now, so that subsequent patches can be applied
speedily. Would you be willing to sign such paperwork?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#46817: update python grammar

[Martin Puttke]

Hi Lars,

thank you for your offer to do the FSF paperwork. But at the 
moment I have no plans to submit another patch to emacs. It may 
come, but I prefer to do it when needed.

All the best
Martin

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Martin Puttke <martinstefanputtke@posteo.de> writes:
>
>> I made some fixes to enhance the python grammar file to enable 
>> it to
>> understand python type hints and have non-trivial base classes. 
>> Please
>> feel free to include my changes into emacs.
>
> Thanks; I'm not a Python or Wisent expert, but if I read the 
> patch
> correctly, it looks good to me, and I've applied your patch to 
> Emacs 28.
>
> This change was small enough to apply without assigning 
> copyright to the
> FSF, but for future patches you want to submit, it might make 
> sense to
> get the paperwork started now, so that subsequent patches can be 
> applied
> speedily. Would you be willing to sign such paperwork?


-- 
Composed with Spacemacs (www.spacemacs.org)

bug#46817: update python grammar

[Lars Ingebrigtsen]

Martin Puttke <martinstefanputtke@posteo.de> writes:

> thank you for your offer to do the FSF paperwork. But at the moment I
> have no plans to submit another patch to emacs. It may come, but I
> prefer to do it when needed.

Sure; that's fine.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no