Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Richard Stallman]

    I traced down the bug to `open-tls-stream', which returns immediately
    after having seen the regexp `tls-success' in the buffer.  However,
    both `openssl s_client' and `gnutls' print more information after that
    line, thus `open-tls-stream' may occasionally return the buffer to the
    caller when point is still amidst openssl/gnutls output.  

    A solution is to have `open-tls-stream' wait a little more to accept
    output; after applying the patch below I cannot trigger the error
    anymore.

+        (if (not done)
+            (delete-process process)
+          ;; both `openssl s_client' and `gnutls' print some more info
+          ;; after the line matched by regexp `tls-success'; wait here
+          ;; a bit more, so we can be sure that point is at the end of
+          ;; it all, before handing the buffer back to caller
+          (accept-process-output process 1)
+          (setq done process))))

This is not totally reliable, because it is possible for
accept-process-output to get just part of the output.
That may be unlikely, which is why you have not seen it happen,
but it is possible.

So here is another suggestion.  Can you change the `tls-success'
regexp so that it matches all the text that openssl or gnutls
would produce?  (You might need to use \(...\|...\) to match
each on separately.)

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Riccardo Murri]

On 10/28/07, Richard Stallman <rms@gnu.org> wrote:
>     I traced down the bug to `open-tls-stream', which returns immediately
>     after having seen the regexp `tls-success' in the buffer.  However,
>     both `openssl s_client' and `gnutls' print more information after that
>     line, thus `open-tls-stream' may occasionally return the buffer to the
>     caller when point is still amidst openssl/gnutls output.
>
>     A solution is to have `open-tls-stream' wait a little more to accept
>     output;
>     [deletia]
>
> This is not totally reliable, because it is possible for
> accept-process-output to get just part of the output.
> That may be unlikely, which is why you have not seen it happen,
> but it is possible.
>
> So here is another suggestion.  Can you change the `tls-success'
> regexp so that it matches all the text that openssl or gnutls
> would produce?  (You might need to use \(...\|...\) to match
> each on separately.)
>

No, I don't think that is feasible: `openssl s_client' is *very*
verbose and its output is meant for humans to read rather than for
programs to parse, so any regexp that matches the informational
messages has a good chance of eating up some of the data too.

I had a look at the sources, and wrote a regexp that tries to match
only the *last* informational message; the appended patch looks for
that *after* `tls-success' has been matched.

Unless I got it all wrong, this is much more complicated than the
original three-liner; I wonder if it is worth going this way instead of
a very simple patch that could still be buggy on rare occasions.


-- 
Riccardo Murri, via Galeazzo Alessi 61, 00176 Roma


--- src/emacs22/lisp/net/tls.el 2007-08-05 21:06:12.000000000 +0200
+++ emacs/lisp/tls.el   2007-10-28 13:12:11.000000000 +0100
@@ -51,6 +51,10 @@
   (autoload 'format-spec "format-spec")
   (autoload 'format-spec-make "format-spec"))

+(eval-when-compile
+  (require 'rx)  ; for writing readable regexps
+  (require 'cl)) ; for `decf'
+
 (defgroup tls nil
   "Transport Layer Security (TLS) parameters."
   :group 'comm)
@@ -89,6 +93,50 @@
   :type 'string
   :group 'tls)

+(defcustom tls-end-of-info
+ (rx
+  (or
+   ;; `openssl s_client` regexp
+   (sequence
+    ;; see ssl/ssl_txt.c lines 219--220
+    line-start
+    "    Verify return code: "
+    (one-or-more not-newline)
+    "\n"
+    ;; according apps/s_client.c line 1515 this is always the last line
+    ;; that is printed by s_client before the real data
+    "---\n")
+
+   ;; `gnutls` regexp
+   (sequence
+    ;; see src/cli.c lines 721--
+    (sequence line-start "- Simple Client Mode:\n")
+    (zero-or-more
+     (or
+      "\n" ; ignore blank lines
+      ;; XXX: we have no way of knowing if the STARTTLS handshake
+      ;; sequence has completed successfully, because `gnutls` will
+      ;; only report failure.
+      (sequence line-start "\*\*\* Starting TLS handshake\n"))))))
+ "Regexp matching end of TLS session information; client data begins
after this.
+The default matches `openssl s_client' (version 0.9.8c) and
+`gnutls-cli' (version 2.0.1) output."
+  :version "22.1"
+  :type 'regexp
+  :group 'tls)
+
+(defcustom tls-end-of-info-match-attempts 10
+  "How many attempts to make at matching `tls-end-of-info'.
+This is intended as a safety measure to avoid a potentially
+endless loop: after this many attempts, presume `tls-end-of-info'
+regexp is broken and will never be matched, so consider that
+stream is already at the start of client data and hope for the best.
+
+Use a negative value to retry indefinitely."
+  :version "22.1"
+  :type 'integer
+  :group 'tls)
+
 (defun tls-certificate-information (der)
   "Parse X.509 certificate in DER format into an assoc list."
   (let ((certificate (concat "-----BEGIN CERTIFICATE-----\n"
@@ -130,6 +178,8 @@
        process cmd done)
     (if use-temp-buffer
        (setq buffer (generate-new-buffer " TLS")))
+    (save-excursion
+      (set-buffer buffer)
     (message "Opening TLS connection to `%s'..." host)
     (while (and (not done) (setq cmd (pop cmds)))
       (message "Opening TLS connection with `%s'..." cmd)
@@ -146,19 +196,39 @@
                              port)))))
        (while (and process
                    (memq (process-status process) '(open run))
-                   (save-excursion
-                     (set-buffer buffer) ;; XXX "blue moon" nntp.el bug
+                    (progn
                      (goto-char (point-min))
                      (not (setq done (re-search-forward tls-success nil t)))))
          (unless (accept-process-output process 1)
             (sit-for 1)))
        (message "Opening TLS connection with `%s'...%s" cmd
                 (if done "done" "failed"))
-       (if done
-           (setq done process)
-         (delete-process process))))
+        (if (not done)
+            (delete-process process)
+          ;; advance point to after all informational messages that
+          ;; `openssl s_client' and `gnutls' print
+          (lexical-let ((attempts tls-end-of-info-match-attempts)
+                        (start-of-data nil))
+            (while  (and
+                    (not (= 0 (if (> attempts 0) (decf attempts) -1)))
+                    ;; the string matching `tls-end-of-info' might come
+                    ;; in separate chunks from `accept-process-output',
+                    ;; so start the search always where `tls-success' ended
+                    (not (setq start-of-data
+                               (save-excursion
+                                 (if (re-search-forward tls-end-of-info nil t)
+                                     (match-end 0))))))
+              (unless (accept-process-output process 1)
+                (sit-for 1)))
+            (if start-of-data
+                ;; move point to start of client data
+                (goto-char start-of-data)
+              (warn
+               "`open-tls-stream': Could not match `tls-end-of-info'
regexp in %d attempts."
+               tls-end-of-info-match-attempts)))
+          (setq done process))))
     (message "Opening TLS connection to `%s'...%s"
-            host (if done "done" "failed"))
+             host (if done "done" "failed")))
     (when use-temp-buffer
       (if done (set-process-buffer process nil))
       (kill-buffer buffer))

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Richard Stallman]

    Unless I got it all wrong, this is much more complicated than the
    original three-liner; I wonder if it is worth going this way instead of
    a very simple patch that could still be buggy on rare occasions.

It is a bad thing for software to be unreliable.  We should write it
so that it works every time, not just 99.9% of the time.

It is worth paying this small complexity to make it work right.
Besides, the code can be simpler.

    +          ;; advance point to after all informational messages that
    +          ;; `openssl s_client' and `gnutls' print
    +          (lexical-let ((attempts tls-end-of-info-match-attempts)
    +                        (start-of-data nil))

Why not ordinary let?

    +                    (not (= 0 (if (> attempts 0) (decf attempts) -1)))

I think there should not be a limit -- it should just keep looping
until it finds what it is looking for.

    +              (unless (accept-process-output process 1)
    +                (sit-for 1)))

I think there is no need for sit-for.  accept-process-output will wait.

    +              (warn
    +               "`open-tls-stream': Could not match `tls-end-of-info'
    regexp in %d attempts."
    +               tls-end-of-info-match-attempts)))

If you get rid of the counter, you won't need this either.

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Riccardo Murri]

[-- Attachment #1: Type: text/plain, Size: 1189 bytes --]

> Besides, the code can be simpler.
>

Thanks for your suggestions; simplified patch to tls.el attached.


A few questions below:

>     +          ;; advance point to after all informational messages that
>     +          ;; `openssl s_client' and `gnutls' print
>     +          (lexical-let ((attempts tls-end-of-info-match-attempts)
>     +                        (start-of-data nil))
>
> Why not ordinary let?

Out of curiosity: why would ordinary `let' be better here?


>     +                    (not (= 0 (if (> attempts 0) (decf attempts) -1)))
>
> I think there should not be a limit -- it should just keep looping
> until it finds what it is looking for.
>

Yes, but this will result in a endless loop if the regex does not
match the program output for some reason (for instance, a new version
of `gnutls' changes the wording of the messages).

I understand that this problem may occur in many places, and that
tls.el would already loop forever if `tls-success' does not match;
but I would rather add more "watchdog" counters instead...

Anyway, I removed the `attempts' counter and related code, as per your
suggestion.

-- 
Riccardo Murri, via Galeazzo Alessi 61, 00176 Roma

[-- Attachment #2: tls.el.diff --]
[-- Type: text/plain, Size: 3750 bytes --]

--- src/emacs22/lisp/net/tls.el	2007-08-05 21:06:12.000000000 +0200
+++ emacs/lisp/tls.el	2007-10-29 19:17:33.000000000 +0100
@@ -51,6 +51,9 @@
   (autoload 'format-spec "format-spec")
   (autoload 'format-spec-make "format-spec"))
 
+(eval-when-compile
+  (require 'rx))  ; for writing readable regexps
+
 (defgroup tls nil
   "Transport Layer Security (TLS) parameters."
   :group 'comm)
@@ -89,6 +92,40 @@
   :type 'string
   :group 'tls)
 
+(defcustom tls-end-of-info
+ (rx 
+  (or
+   ;; `openssl s_client` regexp
+   (sequence
+    ;; see ssl/ssl_txt.c lines 219--220
+    line-start
+    "    Verify return code: "
+    (one-or-more not-newline)
+    "\n"
+    ;; according to apps/s_client.c line 1515 this is always the last
+    ;; line that is printed by s_client before the real data
+    "---\n")
+
+   ;; `gnutls` regexp
+   (sequence
+    ;; see src/cli.c lines 721--
+    (sequence line-start "- Simple Client Mode:\n")
+    (zero-or-more
+     (or 
+      "\n" ; ignore blank lines
+      ;; XXX: we have no way of knowing if the STARTTLS handshake
+      ;; sequence has completed successfully, because `gnutls` will
+      ;; only report failure.
+      (sequence line-start "\*\*\* Starting TLS handshake\n"))))))
+ "Regexp matching end of TLS client informational messages.
+Client data stream begins after the last character matched by this.
+
+The default matches `openssl s_client' (version 0.9.8c) and
+`gnutls-cli' (version 2.0.1) output."
+  :version "22.1"
+  :type 'regexp
+  :group 'tls)
+
 (defun tls-certificate-information (der)
   "Parse X.509 certificate in DER format into an assoc list."
   (let ((certificate (concat "-----BEGIN CERTIFICATE-----\n"
@@ -130,6 +167,8 @@
 	process	cmd done)
     (if use-temp-buffer
 	(setq buffer (generate-new-buffer " TLS")))
+    (save-excursion
+      (set-buffer buffer)
     (message "Opening TLS connection to `%s'..." host)
     (while (and (not done) (setq cmd (pop cmds)))
       (message "Opening TLS connection with `%s'..." cmd)
@@ -146,19 +185,34 @@
 			      port)))))
 	(while (and process
 		    (memq (process-status process) '(open run))
-		    (save-excursion
-		      (set-buffer buffer) ;; XXX "blue moon" nntp.el bug
+                    (progn
 		      (goto-char (point-min))
 		      (not (setq done (re-search-forward tls-success nil t)))))
 	  (unless (accept-process-output process 1)
             (sit-for 1)))
 	(message "Opening TLS connection with `%s'...%s" cmd
 		 (if done "done" "failed"))
-	(if done
-	    (setq done process)
-	  (delete-process process))))
+        (if (not done)
+            (delete-process process)
+          ;; advance point to after all informational messages that
+          ;; `openssl s_client' and `gnutls' print
+          (let ((start-of-data nil))
+            (while 
+                (not (setq start-of-data 
+                           ;; the string matching `tls-end-of-info'
+                           ;; might come in separate chunks from
+                           ;; `accept-process-output', so start the
+                           ;; search where `tls-success' ended
+                           (save-excursion
+                             (if (re-search-forward tls-end-of-info nil t)
+                                 (match-end 0)))))
+              (accept-process-output process 1))
+            (if start-of-data
+                ;; move point to start of client data
+                (goto-char start-of-data)))
+          (setq done process))))
     (message "Opening TLS connection to `%s'...%s"
-	     host (if done "done" "failed"))
+             host (if done "done" "failed")))
     (when use-temp-buffer
       (if done (set-process-buffer process nil))
       (kill-buffer buffer))

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Richard Stallman]

    Out of curiosity: why would ordinary `let' be better here?

It is better in general, because it is the usual thing.
Also it doesn't need CL.

    Yes, but this will result in a endless loop if the regex does not
    match the program output for some reason (for instance, a new version
    of `gnutls' changes the wording of the messages).

Right, but I think that is better than the chance of a failure
due to randomness.

Would you please post a change log for the patch?
Then it will be installed.

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Riccardo Murri]

On 10/30/07, Richard Stallman <rms@gnu.org> wrote:
> Would you please post a change log for the patch?
> Then it will be installed.
>

Draft changelog for the appended patch:

* (tls-end-of-info): New customization option (regexp).
* (open-tls-stream): Accept input until `tls-end-of-info' is matched.


-- 
Riccardo Murri, via Galeazzo Alessi 61, 00176 Roma


--- src/emacs22/lisp/net/tls.el 2007-08-05 21:06:12.000000000 +0200
+++ emacs/lisp/tls.el   2007-10-29 19:17:33.000000000 +0100
@@ -51,6 +51,9 @@
   (autoload 'format-spec "format-spec")
   (autoload 'format-spec-make "format-spec"))

+(eval-when-compile
+  (require 'rx))  ; for writing readable regexps
+
 (defgroup tls nil
   "Transport Layer Security (TLS) parameters."
   :group 'comm)
@@ -89,6 +92,40 @@
   :type 'string
   :group 'tls)

+(defcustom tls-end-of-info
+ (rx
+  (or
+   ;; `openssl s_client` regexp
+   (sequence
+    ;; see ssl/ssl_txt.c lines 219--220
+    line-start
+    "    Verify return code: "
+    (one-or-more not-newline)
+    "\n"
+    ;; according to apps/s_client.c line 1515 this is always the last
+    ;; line that is printed by s_client before the real data
+    "---\n")
+
+   ;; `gnutls` regexp
+   (sequence
+    ;; see src/cli.c lines 721--
+    (sequence line-start "- Simple Client Mode:\n")
+    (zero-or-more
+     (or
+      "\n" ; ignore blank lines
+      ;; XXX: we have no way of knowing if the STARTTLS handshake
+      ;; sequence has completed successfully, because `gnutls` will
+      ;; only report failure.
+      (sequence line-start "\*\*\* Starting TLS handshake\n"))))))
+ "Regexp matching end of TLS client informational messages.
+Client data stream begins after the last character matched by this.
+
+The default matches `openssl s_client' (version 0.9.8c) and
+`gnutls-cli' (version 2.0.1) output."
+  :version "22.1"
+  :type 'regexp
+  :group 'tls)
+
 (defun tls-certificate-information (der)
   "Parse X.509 certificate in DER format into an assoc list."
   (let ((certificate (concat "-----BEGIN CERTIFICATE-----\n"
@@ -130,6 +167,8 @@
        process cmd done)
     (if use-temp-buffer
        (setq buffer (generate-new-buffer " TLS")))
+    (save-excursion
+      (set-buffer buffer)
     (message "Opening TLS connection to `%s'..." host)
     (while (and (not done) (setq cmd (pop cmds)))
       (message "Opening TLS connection with `%s'..." cmd)
@@ -146,19 +185,34 @@
                              port)))))
        (while (and process
                    (memq (process-status process) '(open run))
-                   (save-excursion
-                     (set-buffer buffer) ;; XXX "blue moon" nntp.el bug
+                    (progn
                      (goto-char (point-min))
                      (not (setq done (re-search-forward tls-success nil t)))))
          (unless (accept-process-output process 1)
             (sit-for 1)))
        (message "Opening TLS connection with `%s'...%s" cmd
                 (if done "done" "failed"))
-       (if done
-           (setq done process)
-         (delete-process process))))
+        (if (not done)
+            (delete-process process)
+          ;; advance point to after all informational messages that
+          ;; `openssl s_client' and `gnutls' print
+          (let ((start-of-data nil))
+            (while
+                (not (setq start-of-data
+                           ;; the string matching `tls-end-of-info'
+                           ;; might come in separate chunks from
+                           ;; `accept-process-output', so start the
+                           ;; search where `tls-success' ended
+                           (save-excursion
+                             (if (re-search-forward tls-end-of-info nil t)
+                                 (match-end 0)))))
+              (accept-process-output process 1))
+            (if start-of-data
+                ;; move point to start of client data
+                (goto-char start-of-data)))
+          (setq done process))))
     (message "Opening TLS connection to `%s'...%s"
-            host (if done "done" "failed"))
+             host (if done "done" "failed")))
     (when use-temp-buffer
       (if done (set-process-buffer process nil))
       (kill-buffer buffer))

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Richard Stallman]

Would someone please install this patch by Riccardo Murri
<riccardo.murri@gmail.com> into Emacs 22?  And then ack?

Simon, would you please add comments near the code in GNUtls that
outputs these messages, telling people to watch out for the need for
Emacs to detect the last part of the messages?



* (tls-end-of-info): New variable.
* (open-tls-stream): Keep reading input until `tls-end-of-info' is matched.


-- 
Riccardo Murri, via Galeazzo Alessi 61, 00176 Roma


--- src/emacs22/lisp/net/tls.el 2007-08-05 21:06:12.000000000 +0200
+++ emacs/lisp/tls.el   2007-10-29 19:17:33.000000000 +0100
@@ -51,6 +51,9 @@
   (autoload 'format-spec "format-spec")
   (autoload 'format-spec-make "format-spec"))

+(eval-when-compile
+  (require 'rx))  ; for writing readable regexps
+
 (defgroup tls nil
   "Transport Layer Security (TLS) parameters."
   :group 'comm)
@@ -89,6 +92,40 @@
   :type 'string
   :group 'tls)

+(defcustom tls-end-of-info
+ (rx
+  (or
+   ;; `openssl s_client` regexp
+   (sequence
+    ;; see ssl/ssl_txt.c lines 219--220
+    line-start
+    "    Verify return code: "
+    (one-or-more not-newline)
+    "\n"
+    ;; according to apps/s_client.c line 1515 this is always the last
+    ;; line that is printed by s_client before the real data
+    "---\n")
+
+   ;; `gnutls` regexp
+   (sequence
+    ;; see src/cli.c lines 721--
+    (sequence line-start "- Simple Client Mode:\n")
+    (zero-or-more
+     (or
+      "\n" ; ignore blank lines
+      ;; XXX: we have no way of knowing if the STARTTLS handshake
+      ;; sequence has completed successfully, because `gnutls` will
+      ;; only report failure.
+      (sequence line-start "\*\*\* Starting TLS handshake\n"))))))
+ "Regexp matching end of TLS client informational messages.
+Client data stream begins after the last character matched by this.
+
+The default matches `openssl s_client' (version 0.9.8c) and
+`gnutls-cli' (version 2.0.1) output."
+  :version "22.1"
+  :type 'regexp
+  :group 'tls)
+
 (defun tls-certificate-information (der)
   "Parse X.509 certificate in DER format into an assoc list."
   (let ((certificate (concat "-----BEGIN CERTIFICATE-----\n"
@@ -130,6 +167,8 @@
        process cmd done)
     (if use-temp-buffer
        (setq buffer (generate-new-buffer " TLS")))
+    (save-excursion
+      (set-buffer buffer)
     (message "Opening TLS connection to `%s'..." host)
     (while (and (not done) (setq cmd (pop cmds)))
       (message "Opening TLS connection with `%s'..." cmd)
@@ -146,19 +185,34 @@
                              port)))))
        (while (and process
                    (memq (process-status process) '(open run))
-                   (save-excursion
-                     (set-buffer buffer) ;; XXX "blue moon" nntp.el bug
+                    (progn
                      (goto-char (point-min))
                      (not (setq done (re-search-forward tls-success nil t)))))
          (unless (accept-process-output process 1)
             (sit-for 1)))
        (message "Opening TLS connection with `%s'...%s" cmd
                 (if done "done" "failed"))
-       (if done
-           (setq done process)
-         (delete-process process))))
+        (if (not done)
+            (delete-process process)
+          ;; advance point to after all informational messages that
+          ;; `openssl s_client' and `gnutls' print
+          (let ((start-of-data nil))
+            (while
+                (not (setq start-of-data
+                           ;; the string matching `tls-end-of-info'
+                           ;; might come in separate chunks from
+                           ;; `accept-process-output', so start the
+                           ;; search where `tls-success' ended
+                           (save-excursion
+                             (if (re-search-forward tls-end-of-info nil t)
+                                 (match-end 0)))))
+              (accept-process-output process 1))
+            (if start-of-data
+                ;; move point to start of client data
+                (goto-char start-of-data)))
+          (setq done process))))
     (message "Opening TLS connection to `%s'...%s"
-            host (if done "done" "failed"))
+             host (if done "done" "failed")))
     (when use-temp-buffer
       (if done (set-process-buffer process nil))
       (kill-buffer buffer))

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Reiner Steib]

On Fri, Nov 02 2007, Richard Stallman wrote:

> +(eval-when-compile
> +  (require 'rx))  ; for writing readable regexps
[...]
> +(defcustom tls-end-of-info
> + (rx
> +  (or
> +   ;; `openssl s_client` regexp
> +   (sequence
> +    ;; see ssl/ssl_txt.c lines 219--220
> +    line-start
> +    "    Verify return code: "
> +    (one-or-more not-newline)
> +    "\n"

Well, at least for me using rx.el doesn't make the regexp more
readable.  YMMV.

> +    (save-excursion
> +      (set-buffer buffer)

`with-current-buffer'?

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Miles Bader]

Reiner Steib <reinersteib+gmane@imap.cc> writes:
> Well, at least for me using rx.el doesn't make the regexp more
> readable.  YMMV.

I'm not really sure I find the syntax of rx more readable per se (it
definitely makes the regexp less concise!), but it is kind of
interesting to see it, and the ability to insert comments into the
middle of the regexp actually does seem potentially quite useful!

-Miles

-- 
The car has become... an article of dress without which we feel uncertain,
unclad, and incomplete.  [Marshall McLuhan, Understanding Media, 1964]

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Lennart Borgman (gmail)]

Miles Bader wrote:
> Reiner Steib <reinersteib+gmane@imap.cc> writes:
>> Well, at least for me using rx.el doesn't make the regexp more
>> readable.  YMMV.
> 
> I'm not really sure I find the syntax of rx more readable per se (it
> definitely makes the regexp less concise!), but it is kind of
> interesting to see it, and the ability to insert comments into the
> middle of the regexp actually does seem potentially quite useful!

rx style regexp looks like lisp. Maybe a new more readable language 
looking like regexps?

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, Nov 02, 2007 at 11:50:28PM +0100, Lennart Borgman (gmail) wrote:
> Miles Bader wrote:
> >Reiner Steib <reinersteib+gmane@imap.cc> writes:
> >>Well, at least for me using rx.el doesn't make the regexp more
> >>readable.  YMMV.
> >
> >I'm not really sure I find the syntax of rx more readable per se (it
> >definitely makes the regexp less concise!), but it is kind of
> >interesting to see it, and the ability to insert comments into the
> >middle of the regexp actually does seem potentially quite useful!
> 
> rx style regexp looks like lisp. Maybe a new more readable language 
> looking like regexps?

Oh, no :)

The cool thing about rx is that it not only *looks* like lisp, but that
it *behaves* like lisp. Macro expansion, just to name one example.

Regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFHLAutBcgs9XrR2kYRAnh2AJ9uEYhwxoLs7IsUH9pvxwupcanEDwCdGLdv
9Hh8Le3wio2JijRA+jZPpDc=
=xnPD
-----END PGP SIGNATURE-----

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Andreas Schwab]

tomas@tuxteam.de writes:

> The cool thing about rx is that it not only *looks* like lisp, but that
> it *behaves* like lisp. Macro expansion, just to name one example.

You can get all the same effect with string concatenation, without any
need of weird syntax.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Glenn Morris]

Richard Stallman wrote:

> Would someone please install this patch by Riccardo Murri
> <riccardo.murri@gmail.com> into Emacs 22?  And then ack?

ack

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Simon Josefsson]

Richard Stallman <rms@gnu.org> writes:

> Simon, would you please add comments near the code in GNUtls that
> outputs these messages, telling people to watch out for the need for
> Emacs to detect the last part of the messages?

Done, the GnuTLS code now contains:

  /* Warning!  Do not touch this text string, it is used by external
     programs to search for when gnutls-cli has reached this point. */
  printf ("\n- Simple Client Mode:\n\n");

etc

> +   ;; `gnutls` regexp
> +   (sequence
> +    ;; see src/cli.c lines 721--
> +    (sequence line-start "- Simple Client Mode:\n")
> +    (zero-or-more
> +     (or
> +      "\n" ; ignore blank lines
> +      ;; XXX: we have no way of knowing if the STARTTLS handshake
> +      ;; sequence has completed successfully, because `gnutls` will
> +      ;; only report failure.
> +      (sequence line-start "\*\*\* Starting TLS handshake\n"))))))


I also installed the following patch, because I think the above comment
was incorrect.

/Simon

Index: lisp/ChangeLog
===================================================================
RCS file: /sources/emacs/emacs/lisp/ChangeLog,v
retrieving revision 1.12157
diff -u -p -r1.12157 ChangeLog
--- lisp/ChangeLog	5 Nov 2007 09:06:25 -0000	1.12157
+++ lisp/ChangeLog	5 Nov 2007 10:25:21 -0000
@@ -1,3 +1,7 @@
+2007-11-05  Simon Josefsson  <simon@josefsson.org>
+
+	* net/tls.el (tls-end-of-info): Doc fix.
+
 2007-11-05  Kenichi Handa  <handa@ni.aist.go.jp>
 
 	* international/utf-7.el (utf-7-imap): New coding system.
Index: lisp/net/tls.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/net/tls.el,v
retrieving revision 1.21
diff -u -p -r1.21 tls.el
--- lisp/net/tls.el	4 Nov 2007 01:25:49 -0000	1.21
+++ lisp/net/tls.el	5 Nov 2007 10:25:21 -0000
@@ -65,9 +65,9 @@
    ;; `gnutls' regexp. See src/cli.c lines 721-.
    "^- Simple Client Mode:\n"
    "\\(\n\\|"                           ; ignore blank lines
-   ;; XXX: We have no way of knowing if the STARTTLS handshake sequence
-   ;; has completed successfully, because `gnutls' will only report
-   ;; failure.
+   ;; According to src/cli.c lines 640-650 and 705-715 the handshake
+   ;; will start after this message.  If the handshake fails, the
+   ;; programs will abort.
    "^\\*\\*\\* Starting TLS handshake\n\\)*"
    "\\)")
   "Regexp matching end of TLS client informational messages.

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Stefan Monnier]

Thanks Simon,

just a nitpick:

> +   ;; According to src/cli.c lines 640-650 and 705-715 the handshake
> +   ;; will start after this message.  If the handshake fails, the
> +   ;; programs will abort.

Those line numbers will probably not last eternally, so please add some
revision information (e.g. the date).


        Stefan

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Simon Josefsson]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Thanks Simon,
>
> just a nitpick:
>
>> +   ;; According to src/cli.c lines 640-650 and 705-715 the handshake
>> +   ;; will start after this message.  If the handshake fails, the
>> +   ;; programs will abort.
>
> Those line numbers will probably not last eternally, so please add some
> revision information (e.g. the date).

Good idea, done.

/Simon

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Riccardo Murri]

Il 2007-11-05, Simon Josefsson <simon@josefsson.org> ha scritto:
>> +      ;; XXX: we have no way of knowing if the STARTTLS handshake
>> +      ;; sequence has completed successfully, because `gnutls` will
>> +      ;; only report failure.
>> +      (sequence line-start "\*\*\* Starting TLS handshake\n"))))))
>
> I also installed the following patch, because I think the above comment
> was incorrect.
>

That comment's purpose was to point out that there might be cases in
which `open-tls-stream' will exit successfully, returning the buffer
to the caller (as all of `tls-success' and `tls-end-of-info' matched),
*still* the TLS connection failed.


-- 
Riccardo Murri, via Galeazzo Alessi 61, 00176 Roma

Re: url-retrieve-synchronously randomly fails on https URLs (patch included)

[Simon Josefsson]

Richard Stallman <rms@gnu.org> writes:

> It might be better to say the function name instead of line numbers.
...
>     +   ;; According to src/cli.c lines 640-650 and 705-715 the handshake
>     +   ;; will start after this message.  If the handshake fails, the
>     +   ;; programs will abort.
>
> That is a good idea.  But is src/cli.c a program in GNUtls?  If so,
> please say that explicitly.  Otherwise people will think it refers to
> Emacs, and when they see there is no src/cli.c in Emacs, they will
> think it is erroneous.

I've changed it to:

   ;; According to GnuTLS v2.1.5 src/cli.c lines 640-650 and 705-715
   ;; in main() the handshake will start after this message.  If the
   ;; handshake fails, the programs will abort.

/Simon