bug#19583: nntp-open-server: wrong buffer for nnoo-change-server; breaks url-news

bug#19583: nntp-open-server: wrong buffer for nnoo-change-server; breaks url-news

[Ivan Shmakov]

[-- Attachment #1: Type: text/plain, Size: 1301 bytes --]

Package: emacs
Tags: patch

	As of 88bc8332eb14 (2015-01-07 13:51:41 +0000), nntp-open-server
	appears to call nnoo-change-server without switching to
	nntp-server-buffer first, thus installing the local variables in
	the whatever buffer current at the time it’s called.  One of
	these variables is nntp-port-number, which means that when
	open-protocol-stream is called in the proper buffer later, it
	gets 0 for the port number to connect, and thus fails.

	This affects url-retrieve-synchronously; consider, e. g.:

(let ((url-news-server "news.gmane.org"))
  (url-retrieve-synchronously "news:87y4p9y2cq.fsf@violet.siamics.net"))

	Please thus consider the patch MIMEd.

	* lisp/gnus/nntp.el (nntp-open-server): Switch to
	nntp-server-buffer temporarily while calling nnoo-change-server.

	To note is that to get url-retrieve-synchronously working in
	this case, I had to comment out the :starttls-function argument
	to open-protocol-stream in nntp-open-connection (an issue I
	haven’t investigated much as of yet, but which may be related to
	the use of a self-signed certificate), /and/ define the default
	port for the ‘news’ URI scheme, like the (untested) patch MIMEd.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/diff, Size: 558 bytes --]

--- a/lisp/gnus/nntp.el	2015-01-01 22:26:41 +0000
+++ b/lisp/gnus/nntp.el
@@ -1066,7 +1066,8 @@ deffoo nntp-open-server (server &optional defs connectionless)
       (setq defs (cons (list 'nntp-port-number (car defs)) (cdr defs))))
     (unless (assq 'nntp-address defs)
       (setq defs (append defs (list (list 'nntp-address server)))))
-    (nnoo-change-server 'nntp server defs)
+    (with-current-buffer nntp-server-buffer
+      (nnoo-change-server 'nntp server defs))
     (if connectionless
 	t
       (or (nntp-find-connection nntp-server-buffer)

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: Type: text/diff, Size: 689 bytes --]

--- a/lisp/gnus/nntp.el
+++ b/lisp/gnus/nntp.el	2015-01-13 15:48:37 +0000
@@ -1282,11 +1283,12 @@ defun nntp-open-connection (buffer)
 		     :end-of-command "^\\([2345]\\|[.]\\).*\n"
 		     :capability-command "HELP\r\n"
 		     :success "^3"
-		     :starttls-function
-		     (lambda (capabilities)
-		       (if (not (string-match "STARTTLS" capabilities))
-			   nil
-			 "STARTTLS\r\n")))
+		     ; :starttls-function
+		     ; (lambda (capabilities)
+		     ;   (if (not (string-match "STARTTLS" capabilities))
+		     ; 	   nil
+		     ; 	 "STARTTLS\r\n"))
+		     )
 		  (funcall nntp-open-connection-function pbuffer)))
 	    (error
 	     (nnheader-report 'nntp ">>> %s" err))

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: Type: text/diff, Size: 405 bytes --]

--- a/lisp/url/url-news.el	2015-01-01 22:26:41 +0000
+++ b/lisp/url/url-news.el	2015-01-13 16:05:25 +0000
@@ -28,6 +28,9 @@
 (autoload 'url-warn "url")
 (autoload 'gnus-group-read-ephemeral-group "gnus-group")
 
+(defconst url-news-default-port 119 "Default NNTP port.")
+(defconst url-snews-default-port 563 "Default NNTPS port.")
+
 ;; Unused.
 ;;; (defgroup url-news nil
 ;;;   "News related options."

bug#19583: nntp-open-server: wrong buffer for nnoo-change-server; breaks url-news

[Lars Magne Ingebrigtsen]

Ivan Shmakov <ivan@siamics.net> writes:

> 	As of 88bc8332eb14 (2015-01-07 13:51:41 +0000), nntp-open-server
> 	appears to call nnoo-change-server without switching to
> 	nntp-server-buffer first, thus installing the local variables in
> 	the whatever buffer current at the time it’s called.  One of
> 	these variables is nntp-port-number, which means that when
> 	open-protocol-stream is called in the proper buffer later, it
> 	gets 0 for the port number to connect, and thus fails.
>
> 	This affects url-retrieve-synchronously; consider, e. g.:

Since this hasn't worked in a year, this suggests that `url-news' isn't
used any more.  I suggest removing it.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#19583: nntp-open-server: wrong buffer for nnoo-change-server; breaks url-news

[Ivan Shmakov]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>>>>> Ivan Shmakov <ivan@siamics.net> writes:

 >> As of 88bc8332eb14 (2015-01-07 13:51:41 +0000), nntp-open-server
 >> appears to call nnoo-change-server without switching to
 >> nntp-server-buffer first, thus installing the local variables in the
 >> whatever buffer current at the time it’s called.

[…]

 >> This affects url-retrieve-synchronously; consider, e. g.:

 > Since this hasn't worked in a year, this suggests that `url-news'
 > isn't used any more.  I suggest removing it.

	Does the above mean that nntp-open-server is only used by
	‘url-news’ (and thus could be removed once it’s gone), /or/ does
	that mean that ‘url-news’ fails to call ‘nntp-open-server’
	properly (in which case, what’s the correct way to call it)?

	(My understanding of the nntp-open-server code is that it /does/
	indeed have a bug, which is somehow /not/ triggered by Gnus, but
	/may/ very well be triggered by just whatever other code that’d
	happen to call it.)

	Incidentally, there’s a reason for me to support the removal of
	url-news.el from Emacs proper, as (I guess) it would make it a
	bit easier for me to offer it as a separate package.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

bug#19583: nntp-open-server: wrong buffer for nnoo-change-server; breaks url-news

[Lars Ingebrigtsen]

Ivan Shmakov <ivan@siamics.net> writes:

> 	* lisp/gnus/nntp.el (nntp-open-server): Switch to
> 	nntp-server-buffer temporarily while calling nnoo-change-server.

Thanks; applied.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#19583: nntp-open-server: wrong buffer for nnoo-change-server; breaks url-news

[Lars Ingebrigtsen]

Ivan Shmakov <ivan@siamics.net> writes:

> 	This affects url-retrieve-synchronously; consider, e. g.:
>
> (let ((url-news-server "news.gmane.org"))
>   (url-retrieve-synchronously "news:87y4p9y2cq.fsf@violet.siamics.net"))
>
> 	Please thus consider the patch MIMEd.

I reverted the patch.

With it or without it, the result was the same: URL wasn't able to
retrieve anything but just returned the HTML below.

So there's a bigger problem in this area than just this.  The TLS stuff
you point to may be an issue, but I'm not sure how.

Content-type: text/html

<html>
 <head>
  <title>Error</title>
 </head>
 <body>
  <div>
   <h1>Error requesting article...</h1>
   <p>
    The status message returned by the NNTP server was:<br><hr>
    <xmp>
Couldn't open connection to news.gmane.org.    </xmp>
   </p>
   <p>
    If you feel this is an error, M-x report-emacs-bug RET.
   </p>
  </div>
 </body>
</html>
<!-- Automatically generated by URL vEmacs -->

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#19583: nntp-open-server: wrong buffer for nnoo-change-server; breaks url-news

[Ivan Shmakov]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:
>>>>> Ivan Shmakov <ivan@siamics.net> writes:

 >> This affects url-retrieve-synchronously; consider, e. g.:

 >> (let ((url-news-server "news.gmane.org"))
 >>   (url-retrieve-synchronously "news:87y4p9y2cq.fsf@violet.siamics.net"))

 >> Please thus consider the patch MIMEd.

 > I reverted the patch.

 > With it or without it, the result was the same: URL wasn't able to
 > retrieve anything but just returned the HTML below.

 > So there's a bigger problem in this area than just this.  The TLS
 > stuff you point to may be an issue, but I'm not sure how.

	That’s simple: the TLS stuff there is ought to reject
	self-signed certificates outright, and news.gmane.org just
	happens to use one.

	My guess was that setting nntp-open-connection-function to
	'nntp-open-plain-stream should overcome this issue, but it
	doesn’t.  Apparently, the variable gets reset to its “Use TLS if
	supported” default somewhere in the code (when the server is
	opened?), but I have little clue as to where that may happen,
	or where the default is stored, or how do I find it out.

	This same issue affects Gnus, but is easy to overcome by
	providing a suitable value to nntp-open-connection-function via
	the respective select method.

[…]

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

bug#19583: nntp-open-server: wrong buffer for nnoo-change-server; breaks url-news

[Lars Ingebrigtsen]

Ivan Shmakov <ivan@siamics.net> writes:

> 	That’s simple: the TLS stuff there is ought to reject
> 	self-signed certificates outright, and news.gmane.org just
> 	happens to use one.

No certificates are rejected; they're normally all prompted for by the
network security manager.  So the question is why that isn't happening
here, if indeed that's the problem.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#19583: nntp-open-server: wrong buffer for nnoo-change-server; breaks url-news

[Ivan Shmakov]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:
>>>>> Ivan Shmakov <ivan@siamics.net> writes:

 >> That’s simple: the TLS stuff there is ought to reject self-signed
 >> certificates outright, and news.gmane.org just happens to use one.

 > No certificates are rejected; they're normally all prompted for by
 > the network security manager.  So the question is why that isn't
 > happening here, if indeed that's the problem.

	It seems so; cf.:

(with-temp-buffer
  (kill-buffer " *nntpd*")
  (setq nntp-open-connection-function 'nntp-open-plain-stream)
  (nntp-open-server "news.gmane.org"
                    '((nntp-port-number 119))))
#<process nntpd>

(with-temp-buffer
  (kill-buffer " *nntpd*")
  (setq nntp-open-connection-function 'nntp-open-network-stream)
  (nntp-open-server "news.gmane.org"
                    '((nntp-port-number 119))))
nil

-- 
FSF associate member #7257  np. The Talisman — Iron Maiden   … B6A0 230E 334A

bug#19583: nntp-open-server: wrong buffer for nnoo-change-server; breaks url-news

[Lars Ingebrigtsen]

Ivan Shmakov <ivan@siamics.net> writes:

> (with-temp-buffer
>   (kill-buffer " *nntpd*")
>   (setq nntp-open-connection-function 'nntp-open-network-stream)
>   (nntp-open-server "news.gmane.org"
>                     '((nntp-port-number 119))))
> nil

Works for me in emacs -Q:

(with-temp-buffer
  (require 'nntp)
  (nntp-open-server "news.gmane.org"))

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#19583: nntp-open-server: wrong buffer for nnoo-change-server; breaks url-news

[Ivan Shmakov]

[-- Attachment #1: Type: text/plain, Size: 1372 bytes --]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:
>>>>> Ivan Shmakov <ivan@siamics.net> writes:
>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:

 >>> No certificates are rejected; they're normally all prompted for by
 >>> the network security manager.  So the question is why that isn't
 >>> happening here, if indeed that's the problem.

 >> It seems so; cf.:

 >> (with-temp-buffer (kill-buffer " *nntpd*")
 >>   (setq nntp-open-connection-function 'nntp-open-network-stream)
 >>   (nntp-open-server "news.gmane.org" '((nntp-port-number 119))))
 >> nil

 > Works for me in emacs -Q:

 > (with-temp-buffer
 >   (require 'nntp)
 >   (nntp-open-server "news.gmane.org"))

	Yet still url-retrieve fails for news: URIs referencing this
	same server?

	Also, AIUI, $ emacs -Q doesn’t prevent the use of
	nsm-settings-file.  Could you please try the above with that
	variable pointing to some non-existent file?

	BTW, I don’t seem to understand which nntp- functions do expect
	to be called from the server buffer.  For one thing,
	nntp-open-connection uses the nntp- variables intended (unless I
	be mistaken) to be local to the server buffer, yet I fail to see
	where the buffer gets switched.

	Perhaps this revised patch MIMEd makes more sense.

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/diff, Size: 809 bytes --]

--- a/lisp/gnus/nntp.el
+++ b/lisp/gnus/nntp.el
@@ -1066,11 +1066,12 @@ deffoo nntp-open-server (server &optional defs connectionless)
       (setq defs (cons (list 'nntp-port-number (car defs)) (cdr defs))))
     (unless (assq 'nntp-address defs)
       (setq defs (append defs (list (list 'nntp-address server)))))
-    (nnoo-change-server 'nntp server defs)
-    (if connectionless
-	t
-      (or (nntp-find-connection nntp-server-buffer)
-	  (nntp-open-connection nntp-server-buffer)))))
+    (with-current-buffer nntp-server-buffer
+      (nnoo-change-server 'nntp server defs)
+      (if connectionless
+	  t
+	(or (nntp-find-connection nntp-server-buffer)
+	    (nntp-open-connection nntp-server-buffer))))))
 
 (deffoo nntp-close-server (&optional server)
   (nntp-possibly-change-group nil server t)

bug#19583: nntp-open-server: wrong buffer for nnoo-change-server; breaks url-news

[Lars Ingebrigtsen]

Ivan Shmakov <ivan@siamics.net> writes:

> 	BTW, I don’t seem to understand which nntp- functions do expect
> 	to be called from the server buffer.  For one thing,
> 	nntp-open-connection uses the nntp- variables intended (unless I
> 	be mistaken) to be local to the server buffer, yet I fail to see
> 	where the buffer gets switched.

I can't really see any local variables in the " *nntpd*" buffer.  There
are plenty of them in the " *server ...*" buffer, though.

So I can't really see how this would make any difference:

-    (nnoo-change-server 'nntp server defs)
-    (if connectionless
-	t
-      (or (nntp-find-connection nntp-server-buffer)
-	  (nntp-open-connection nntp-server-buffer)))))
+    (with-current-buffer nntp-server-buffer
+      (nnoo-change-server 'nntp server defs)
+      (if connectionless
+	  t
+	(or (nntp-find-connection nntp-server-buffer)
+	    (nntp-open-connection nntp-server-buffer))))))

nntp-server-buffer => #<buffer  *nntpd*>

Are you still seeing this problem?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#19583: nntp-open-server: wrong buffer for nnoo-change-server; breaks url-news

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> nntp-server-buffer => #<buffer  *nntpd*>
>
> Are you still seeing this problem?

Apparently no response in almost a year, so I'm closing this bug
report.  If it's still a problem, please reopen.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no