bug#16198: 24.3.50; [PATCH 1/2] eww: Does not support file upload.

bug#16198: 24.3.50; [PATCH 1/2] eww: Does not support file upload.

[Kenjiro NAKAYAMA]

This report includes a patch to eww file upload. Please, review and
install it to the official tree if appreciated.

* Issue
Since Eww does not support file upload(input with type="file"), following
HTML can not display and handle properly.

example:
  <form method="post" action="example.cgi" enctype="multipart/form-data">
    <input name="example" type="file">
    <input type="submit" value="send">
  </form>

Kenjiro NAKAYAMA
  
Signed-off-by: Kenjiro NAKAYAMA <nakayamakenjiro@gmail.com>

        * gnus/mm-url.el (mm-url-encode-multipart-form-data):
          Restore to handle "maltipart/form-data" by eww.
        * net/eww.el(eww-form-file(defface)): New defface of file upload form.
        (eww-submit-file): New key map of file upload.
        (eww-form-file): New file upload button and file name context.
        (eww-select-file): Select file and display selected file name.
        (eww-tag-input): Handle input tag of file type. 
        (eww-update-field): Add point offset.
        (eww-submit): Add submit with multipart/form-data.

---
 lisp/gnus/mm-url.el |  44 +++++++++++++++++++++
 lisp/net/eww.el     | 108 +++++++++++++++++++++++++++++++++++++++++++++-------
 2 files changed, 138 insertions(+), 14 deletions(-)

diff --git a/lisp/gnus/mm-url.el b/lisp/gnus/mm-url.el
index 6e83b18..b8a4d46 100644
--- a/lisp/gnus/mm-url.el
+++ b/lisp/gnus/mm-url.el
@@ -416,6 +416,50 @@ spaces.  Die Die Die."
 
 (autoload 'mml-compute-boundary "mml")
 
+(defun mm-url-encode-multipart-form-data (pairs &optional boundary)
+  "Return PAIRS encoded in multipart/form-data."
+  ;; RFC1867
+  ;; Get a good boundary
+  (unless boundary
+    (setq boundary (mml-compute-boundary '())))
+  (concat
+   ;; Start with the boundary
+   "--" boundary "\r\n"
+   ;; Create name value pairs
+   (mapconcat
+    'identity
+    ;; Delete any returned items that are empty
+    (delq nil
+          (mapcar (lambda (data)
+                    (cond ((equal (car data) "file")
+                           ;; For each pair
+                           (concat
+                            ;; Encode the name
+                            "Content-Disposition: form-data; name=\"" (cdr (assoc "name" (cdr data))) "\"; filename=\"" (cdr (assoc "filename" (cdr data))) "\"\r\n"
+                            "Content-Type: text/plain; charset=utf-8\r\n"
+                            "Content-Transfer-Encoding: binary\r\n\r\n"
+                            (cond ((stringp (cdr (assoc "filedata" (cdr data))))
+                                   (cdr (assoc "filedata" (cdr data))))
+                                  ((integerp (cdr (assoc "filedata" (cdr data))))
+                                   (int-to-string (cdr (assoc "filedata" (cdr data))))))
+                            "\r\n"))
+                          ((equal (car data) "submit")
+                           (concat
+                            "Content-Disposition: form-data; name=\"submit\"\r\n\r\n"
+                            "Submit"
+                            ))
+                          (t
+                           (concat
+                            "Content-Disposition: form-data;name=" (car data) "\r\n\r\n"
+                            (concat (mm-url-form-encode-xwfu (cdr data))
+                                    )))
+                          ))
+                  pairs))
+    ;; use the boundary as a separator
+    (concat "\r\n--" boundary "\r\n"))
+   ;; put a boundary at the end.
+   "--" boundary "--\r\n"))
+
 (defun mm-url-remove-markup ()
   "Remove all HTML markup, leaving just plain text."
   (goto-char (point-min))

bug#16198: 24.3.50; [PATCH 1/2] eww: Does not support file upload.

[Ted Zlatanov]

On Fri, 20 Dec 2013 18:29:37 +0900 Kenjiro NAKAYAMA <nakayamakenjiro@gmail.com> wrote: 

KN> This report includes a patch to eww file upload. Please, review and
KN> install it to the official tree if appreciated.

KN> * Issue
KN> Since Eww does not support file upload(input with type="file"), following
KN> HTML can not display and handle properly.

KN> example:
KN>   <form method="post" action="example.cgi" enctype="multipart/form-data">
KN>     <input name="example" type="file">
KN>     <input type="submit" value="send">
KN>   </form>

This looks like a good change but I'm not comfortable installing it,
it's too deep and affects Gnus as well, and I don't know the MIME pieces
in Gnus.  I'll let Lars review it.

Ted

bug#16198: 24.3.50; [PATCH 1/2] eww: Does not support file upload.

[Lars Ingebrigtsen]

Kenjiro NAKAYAMA <nakayamakenjiro@gmail.com> writes:

> This report includes a patch to eww file upload. Please, review and
> install it to the official tree if appreciated.

It sounds like useful functionality, but Emacs went into feature freeze
yesterday, so it'll have to wait until Emacs thaws again.

Comment about the code:

> +                           (concat
> +                            "Content-Disposition: form-data;name=" (car data) "\r\n\r\n"
> +                            (concat (mm-url-form-encode-xwfu (cdr data))
> +                                    )))
> +                          ))

This is not the Emacs parenthesis style.  All the closing parentheses
should be on the same line.  And there's a superfluous `concat' there...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#16198: 24.3.50; [PATCH 1/2] eww: Does not support file upload.

[Kenjiro NAKAYAMA]

> This is not the Emacs parenthesis style.  All the closing parentheses
> should be on the same line.  And there's a superfluous `concat' there...

Thank you Lars and Ted,
Since I fixed the patches, I send again.

Signed-off-by: Kenjiro NAKAYAMA <nakayamakenjiro@gmail.com>

        * gnus/mm-url.el (mm-url-encode-multipart-form-data):
          Restore to handle "maltipart/form-data" by eww.
        * net/eww.el(eww-form-file(defface)): New defface of file upload form.
        (eww-submit-file): New key map of file upload.
        (eww-form-file): New file upload button and file name context.
        (eww-select-file): Select file and display selected file name.
        (eww-tag-input): Handle input tag of file type.
        (eww-update-field): Add point offset.
        (eww-submit): Add submit with multipart/form-data.

---
 lisp/gnus/mm-url.el | 45 +++++++++++++++++++++++--
 lisp/net/eww.el     | 97 +++++++++++++++++++++++++++++++++++++++++++++--------
 2 files changed, 126 insertions(+), 16 deletions(-)

diff --git a/lisp/gnus/mm-url.el b/lisp/gnus/mm-url.el
index 6e83b18..9deb160 100644
--- a/lisp/gnus/mm-url.el
+++ b/lisp/gnus/mm-url.el
@@ -416,13 +416,54 @@ spaces.  Die Die Die."

 (autoload 'mml-compute-boundary "mml")

+(defun mm-url-encode-multipart-form-data (pairs &optional boundary)
+  "Return PAIRS encoded in multipart/form-data."
+  ;; RFC1867
+  ;; Get a good boundary
+  (unless boundary
+    (setq boundary (mml-compute-boundary '())))
+  (concat
+   ;; Start with the boundary
+   "--" boundary "\r\n"
+   ;; Create name value pairs
+   (mapconcat
+    'identity
+    ;; Delete any returned items that are empty
+    (delq nil
+          (mapcar (lambda (data)
+                    (cond ((equal (car data) "file")
+                           ;; For each pair
+                           (concat
+                            ;; Encode the name
+                            "Content-Disposition: form-data; name=\"" (cdr (assoc "name" (cdr data))) "\"; filename=\"" (cdr (assoc "filename" (cdr data))) "\"\r\n"
+                            "Content-Type: text/plain; charset=utf-8\r\n"
+                            "Content-Transfer-Encoding: binary\r\n\r\n"
+                            (cond ((stringp (cdr (assoc "filedata" (cdr data))))
+                                   (cdr (assoc "filedata" (cdr data))))
+                                  ((integerp (cdr (assoc "filedata" (cdr data))))
+                                   (int-to-string (cdr (assoc "filedata" (cdr data))))))
+                            "\r\n"))
+                          ((equal (car data) "submit")
+                           (concat
+                            "Content-Disposition: form-data; name=\"submit\"\r\n\r\n"
+                            "Submit"))
+                          (t
+                           (concat
+                            "Content-Disposition: form-data;name=" (car data) "\r\n\r\n"
+							(mm-url-form-encode-xwfu (cdr data))))))
+                  pairs))
+    ;; use the boundary as a separator
+    (concat "\r\n--" boundary "\r\n"))
+   ;; put a boundary at the end.
+   "--" boundary "--\r\n"))
+
 (defun mm-url-remove-markup ()
   "Remove all HTML markup, leaving just plain text."
   (goto-char (point-min))
   (while (search-forward "<!--" nil t)
     (delete-region (match-beginning 0)
-		   (or (search-forward "-->" nil t)
-		       (point-max))))
+				   (or (search-forward "-->" nil t)
+                       (point-max))))
   (goto-char (point-min))
   (while (re-search-forward "<[^>]+>" nil t)
     (replace-match "" t t)))


     --- [1/2] ---

larsi@gnus.org writes:

> Kenjiro NAKAYAMA <nakayamakenjiro@gmail.com> writes:
>
>> This report includes a patch to eww file upload. Please, review and
>> install it to the official tree if appreciated.
>
> It sounds like useful functionality, but Emacs went into feature freeze
> yesterday, so it'll have to wait until Emacs thaws again.
>
> Comment about the code:
>
>> +                           (concat
>> +                            "Content-Disposition: form-data;name=" (car data) "\r\n\r\n"
>> +                            (concat (mm-url-form-encode-xwfu (cdr data))
>> +                                    )))
>> +                          ))
>
> This is not the Emacs parenthesis style.  All the closing parentheses
> should be on the same line.  And there's a superfluous `concat' there...

bug#16198: 24.3.50; [PATCH 1/2] eww: Does not support file upload.

[Lars Ingebrigtsen]

Kenjiro NAKAYAMA <nakayamakenjiro@gmail.com> writes:

Looks good.  One bit that could perhaps be changed is this:

> +                            "Content-Disposition: form-data; name=\"" (cdr (assoc "name" (cdr data))) "\"; filename=\"" (cdr (assoc "filename" (cdr data))) "\"\r\n"

Lines shouldn't be longer than 80 characters, and these file names may
perhaps contain the " character, which would make these specs invalid?

It's usually best to use `format' with %S in these cases:

(setq file "foo\"bar")

(insert (concat "name=\"" file "\""))
name="foo"bar"

(insert (format "name=%S" file))
name="foo\"bar"

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#16198: 24.3.50; [PATCH 1/2] eww: Does not support file upload.

[Kenjiro NAKAYAMA]

Thank you Lars,

> Lines shouldn't be longer than 80 characters, and these file names may

But If I include linefeed and indent in following line, the data has
some "\t"s due to the indent. So I think one line is better.

   (format "Content-Disposition: form-data; name=%S; filename=%S\r\nContent-Type: text/plain; ....)

-> With New Line and indent.(Many \t\t\t... are included in the data.)
  Content-Disposition: form-data; name="test"; filename="~/example.txt"\r\n\t\t\t\t\t\t\tContent-Type: text/plain; charset=utf-8\r\n\t\t\t\t\t\t\tContent-Transfer-Encoding: binary\r\n\r\n

-> Non-breaking (The linefeed looks good)
  Content-Disposition: form-data; name="test"; filename="~/example.txt"\r\n
  Content-Type: text/plain; charset=utf-8\r\n
  Content-Transfer-Encoding: binary\r\n\r\n

If I misunderstood your explanation, I am sorry.

Since I made the patch again, I resend.
The patch for the eww.el ([PATCH 2/2]) has no change.

Signed-off-by: Kenjiro NAKAYAMA <nakayamakenjiro@gmail.com>

        * gnus/mm-url.el (mm-url-encode-multipart-form-data):
          Restore to handle "maltipart/form-data" by eww.
        * net/eww.el(eww-form-file(defface)): New defface of file upload form.
        (eww-submit-file): New key map of file upload.
        (eww-form-file): New file upload button and file name context.
        (eww-select-file): Select file and display selected file name.
        (eww-tag-input): Handle input tag of file type.
        (eww-update-field): Add point offset.
        (eww-submit): Add submit with multipart/form-data.
        
---
 lisp/gnus/mm-url.el | 42 ++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 40 insertions(+), 2 deletions(-)

diff --git a/lisp/gnus/mm-url.el b/lisp/gnus/mm-url.el
index 6e83b18..4b5fedb 100644
--- a/lisp/gnus/mm-url.el
+++ b/lisp/gnus/mm-url.el
@@ -416,13 +416,51 @@ spaces.  Die Die Die."
 
 (autoload 'mml-compute-boundary "mml")
 
+(defun mm-url-encode-multipart-form-data (pairs &optional boundary)
+  "Return PAIRS encoded in multipart/form-data."
+  ;; RFC1867
+  ;; Get a good boundary
+  (unless boundary
+    (setq boundary (mml-compute-boundary '())))
+  (concat
+   ;; Start with the boundary
+   "--" boundary "\r\n"
+   ;; Create name value pairs
+   (mapconcat
+    'identity
+    ;; Delete any returned items that are empty
+    (delq nil
+          (mapcar (lambda (data)
+                    (cond ((equal (car data) "file")
+                           ;; For each pair
+                           (format
+                            ;; Encode the name
+                            "Content-Disposition: form-data; name=%S; filename=%S\r\nContent-Type: text/plain; charset=utf-8\r\nContent-Transfer-Encoding: binary\r\n\r\n%s"
+                            (cdr (assoc "name" (cdr data))) (cdr (assoc "filename" (cdr data)))
+                            (cond ((stringp (cdr (assoc "filedata" (cdr data))))
+                                   (cdr (assoc "filedata" (cdr data))))
+                                  ((integerp (cdr (assoc "filedata" (cdr data))))
+                                   (number-to-string (cdr (assoc "filedata" (cdr data))))))))
+                          ((equal (car data) "submit")
+                           "Content-Disposition: form-data; name=\"submit\"\r\n\r\nSubmit\r\n")
+                          (t
+                           (format
+                            "Content-Disposition: form-data;name=%S\r\n\r\n%s\r\n"
+                            (car data) (concat (mm-url-form-encode-xwfu (cdr data)))
+                            ))))
+                  pairs))
+    ;; use the boundary as a separator
+    (concat "\r\n--" boundary "\r\n"))
+   ;; put a boundary at the end.
+   "--" boundary "--\r\n"))
+
 (defun mm-url-remove-markup ()
   "Remove all HTML markup, leaving just plain text."
   (goto-char (point-min))
   (while (search-forward "<!--" nil t)
     (delete-region (match-beginning 0)
-		   (or (search-forward "-->" nil t)
-		       (point-max))))
+                   (or (search-forward "-->" nil t)
+                       (point-max))))
   (goto-char (point-min))
   (while (re-search-forward "<[^>]+>" nil t)
     (replace-match "" t t)))
-- 
1.8.3.1

Regards,

Kenjiro


larsi@gnus.org writes:

> Kenjiro NAKAYAMA <nakayamakenjiro@gmail.com> writes:
>
> Looks good.  One bit that could perhaps be changed is this:
>
>> +                            "Content-Disposition: form-data; name=\"" (cdr (assoc "name" (cdr data))) "\"; filename=\"" (cdr (assoc "filename" (cdr data))) "\"\r\n"
>
> Lines shouldn't be longer than 80 characters, and these file names may
> perhaps contain the " character, which would make these specs invalid?
>
> It's usually best to use `format' with %S in these cases:
>
> (setq file "foo\"bar")
>
> (insert (concat "name=\"" file "\""))
> name="foo"bar"
>
> (insert (format "name=%S" file))
> name="foo\"bar"

bug#16198: 24.3.50; [PATCH 1/2] eww: Does not support file upload.

[Lars Magne Ingebrigtsen]

Kenjiro NAKAYAMA <nakayamakenjiro@gmail.com> writes:

> Since I made the patch again, I resend.
> The patch for the eww.el ([PATCH 2/2]) has no change.

Thanks; applied.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no