unofficial mirror of bug-gnu-emacs@gnu.org 
 help / color / mirror / code / Atom feed
* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
@ 2013-05-10 12:49               ` João Távora
  2013-05-10 14:00                 ` Eli Zaretskii
  2013-05-24 22:20                 ` Ted Zlatanov
  0 siblings, 2 replies; 28+ messages in thread
From: João Távora @ 2013-05-10 12:49 UTC (permalink / raw)
  To: 14380


Here's a emacs -Q recipe:

    $ emacs -Q
    M-x eval-expression RET (setq gnus-select-method '(nnimap "imaps.mycompany.com")) RET
    M-x gnus

An error pops up:

    nnimap (imaps.mycompany.com) open error: ''gnutls-cli' is not recognized as an internal or external command,

In the *Message* buffer, we can see at at least one tls command succeeded

    Opening connection to imaps.mycompany.com via tls...
    gnutls.c: [1] (Emacs) GnuTLS library not found
    Opening TLS connection to `imaps.mycompany.com'...
    Opening TLS connection with `gnutls-cli --insecure -p 993 imaps.mycompany.com'...failed
    Opening TLS connection with `gnutls-cli --insecure -p 993 imaps.mycompany.com --protocols ssl3'...failed
    Opening TLS connection with `openssl s_client -connect imaps.mycompany.com:993 -no_ssl2 -ign_eof'...done
    Opening TLS connection to `imaps.mycompany.com'...done
    nnimap (imaps.mycompany.com) open error: ''gnutls-cli' is not recognized as an internal or external command,
    '.  Continue? (y or n)  n

This works fine, on the same server with linux.

My analysis and possible fix is as follows: The problem is that
`network-stream-open-tls' returns invalid information in its final form:

    (list stream
          (network-stream-get-response stream start eoc)
          (network-stream-command stream capability-command eoc)
          'tls)

This is turn is because code immediately preceding it

    ;; If we're using tls.el, we have to delete the output from
    ;; openssl/gnutls-cli.
    (when (and (null use-builtin-gnutls)
               eoc)
      (network-stream-get-response stream start eoc)
      (goto-char (point-min))
      (when (re-search-forward eoc nil t)
        (goto-char (match-beginning 0))
        (delete-region (point-min) (line-beginning-position))))

is not really doing what it says it's doing: it's not deleting enough
input, and the search for capabilities actually returns the IMAP
greeting.

This in turn is related to the CRLF lineendings that the function
`open-tls-stream' running under w32 only (in linux it doesn't happen)
inserts in the process buffer before the end of the tls handshake. The
code in `network-stream-open-tls' for getting rid of useless starting
output deletes up to the first CRLF (and apparently it should do so just
once, since other CRLF's are legitimate). But if fails and catches a
very early CRLF which is still part of the now useless TLS
handshake. The search performed by

    (network-stream-command stream capability-command eoc)

doesn't work. That said, a reasonable way to fix this might be adding a
line

    (delete-region (point-min) (point))

as the first form in the body of the `when'. This is because
`open-tls-stream' apparently guarantees that point will be left where
the tls handshake ended, as per its code snippet and comment:

    (if start-of-data
        ;; move point to start of client data
        (goto-char start-of-data))

Alternalively, make `open-tls-stream' do that itself in the end. I'm
doing just that with.

    (defadvice open-tls-stream (after dont-forget-to-erase-buffer-up-to-point activate)
      (delete-region (point-min) (point)))

Or maybe improve searches by `network-stream-get-response' or be smarter
when deleting output.

Here's the contents of the nnimap process buffer. Notice the CRLFs.


'gnutls-cli' is not recognized as an internal or external command,
operable program or batch file.

Process *nnimap* exited abnormally with code 1
'gnutls-cli' is not recognized as an internal or external command,
operable program or batch file.

Process *nnimap* exited abnormally with code 1
depth=0 C = PT, L = Lisboa, O = SISCOG.COM, OU = IMAP server, CN = *.siscog.com, emailAddress = suporte@siscog.pt
verify error:num=18:self signed certificate
verify return:1
depth=0 C = PT, L = Lisboa, O = SISCOG.COM, OU = IMAP server, CN = *.siscog.com, emailAddress = suporte@siscog.pt
verify return:1
CONNECTED(00000003)
---
Certificate chain
 0 s:/C=PT/L=Lisboa/O=SISCOG.COM/OU=IMAP server/CN=*.siscog.com/emailAddress=suporte@siscog.pt
   i:/C=PT/L=Lisboa/O=SISCOG.COM/OU=IMAP server/CN=*.siscog.com/emailAddress=suporte@siscog.pt
---
Server certificate
-----BEGIN CERTIFICATE-----
[elided]
-----END CERTIFICATE-----
subject=/C=PT/L=Lisboa/O=SISCOG.COM/OU=IMAP server/CN=*.siscog.com/emailAddress=suporte@siscog.pt
issuer=/C=PT/L=Lisboa/O=SISCOG.COM/OU=IMAP server/CN=*.siscog.com/emailAddress=suporte@siscog.pt
---
No client certificate CA names sent
---
SSL handshake has read 1240 bytes and written 518 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 1024 bit
Secure Renegotiation IS supported
Compression: zlib compression
Expansion: zlib compression
SSL-Session:
    Protocol  : TLSv1
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID: 9334E31E9FD9A90C9D301227A2799EB8210AAC9FFD583B6597251744CB6EBFB3
    Session-ID-ctx:
    Master-Key: 7A5A40FEFC3A3493C9AFC4B80BCB044FAC48C94E0B4DE6E35035866F90EC78FFDE18D1B40E93B125F4F3F73468A9D5F0
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Compression: 1 (zlib compression)
    Start Time: 1368189548
    Timeout   : 300 (sec)
    Verify return code: 18 (self signed certificate)
---
* OK Dovecot ready.

In GNU Emacs 24.3.1 (i386-mingw-nt5.1.2600)
 of 2013-03-17 on MARVIN
Windowing system distributor `Microsoft Corp.', version 5.1.2600
Configured using:
 `configure --with-gcc (4.7) --cflags
 -ID:/devel/emacs/libs/libXpm-3.5.8/include
 -ID:/devel/emacs/libs/libXpm-3.5.8/src
 -ID:/devel/emacs/libs/libpng-dev_1.4.3-1/include
 -ID:/devel/emacs/libs/zlib-dev_1.2.5-2/include
 -ID:/devel/emacs/libs/giflib-4.1.4-1/include
 -ID:/devel/emacs/libs/jpeg-6b-4/include
 -ID:/devel/emacs/libs/tiff-3.8.2-1/include
 -ID:/devel/emacs/libs/gnutls-3.0.9/include
 -ID:/devel/emacs/libs/libiconv-1.13.1-1-dev/include
 -ID:/devel/emacs/libs/libxml2-2.7.8/include/libxml2'

Important settings:
  value of $LANG: C.UTF-8
  locale-coding-system: cp1252
  default enable-multibyte-characters: t





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-05-10 12:49               ` João Távora
@ 2013-05-10 14:00                 ` Eli Zaretskii
  2013-05-10 16:00                   ` João Távora
  2013-05-24 22:20                 ` Ted Zlatanov
  1 sibling, 1 reply; 28+ messages in thread
From: Eli Zaretskii @ 2013-05-10 14:00 UTC (permalink / raw)
  To: João Távora; +Cc: 14380

> From: joaotavora@gmail.com (João Távora)
> Date: Fri, 10 May 2013 13:49:20 +0100
> 
> 
>     (list stream
>           (network-stream-get-response stream start eoc)
>           (network-stream-command stream capability-command eoc)
>           'tls)
> 
> This is turn is because code immediately preceding it
> 
>     ;; If we're using tls.el, we have to delete the output from
>     ;; openssl/gnutls-cli.
>     (when (and (null use-builtin-gnutls)
>                eoc)
>       (network-stream-get-response stream start eoc)
>       (goto-char (point-min))
>       (when (re-search-forward eoc nil t)
>         (goto-char (match-beginning 0))
>         (delete-region (point-min) (line-beginning-position))))
> 
> is not really doing what it says it's doing: it's not deleting enough
> input, and the search for capabilities actually returns the IMAP
> greeting.

Why don't you use the built-in gnutls?  This is the recommended way of
using GnuTLS on Windows.

Or did I misunderstand your analysis?






^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-05-10 14:00                 ` Eli Zaretskii
@ 2013-05-10 16:00                   ` João Távora
  2013-05-10 17:17                     ` Eli Zaretskii
  0 siblings, 1 reply; 28+ messages in thread
From: João Távora @ 2013-05-10 16:00 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 14380

More or less, you didn't misunderstand my analysis, but I didn't use
built-in tls because its not the default (I did see some
references to it, and the function seems to be fboundp (whereas in linux
its not) but the code path didn't lead that way for some reason).

With emacs -Q, setting gnus-select-method that way and then doing M-x gnus
fails.

If the builtin tls fixes this and its the "recommended way" it should
be the default no?

Maybe it *is* the default but emacs -Q possibly configured itself to use
some external gnutls and configure itself after it. I'm using cygwin, so
that might be the case. In that case, this might be seen as a cygwin bug,
in which case I apologize for the noise.

btw: how to I configure emacs to use builtin gnutls?

J

On Fri, May 10, 2013 at 3:00 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: joaotavora@gmail.com (João Távora)
> > Date: Fri, 10 May 2013 13:49:20 +0100
> >
> >
> >     (list stream
> >           (network-stream-get-response stream start eoc)
> >           (network-stream-command stream capability-command eoc)
> >           'tls)
> >
> > This is turn is because code immediately preceding it
> >
> >     ;; If we're using tls.el, we have to delete the output from
> >     ;; openssl/gnutls-cli.
> >     (when (and (null use-builtin-gnutls)
> >                eoc)
> >       (network-stream-get-response stream start eoc)
> >       (goto-char (point-min))
> >       (when (re-search-forward eoc nil t)
> >         (goto-char (match-beginning 0))
> >         (delete-region (point-min) (line-beginning-position))))
> >
> > is not really doing what it says it's doing: it's not deleting enough
> > input, and the search for capabilities actually returns the IMAP
> > greeting.
>
> Why don't you use the built-in gnutls?  This is the recommended way of
> using GnuTLS on Windows.
>
> Or did I misunderstand your analysis?




--
João Távora





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-05-10 16:00                   ` João Távora
@ 2013-05-10 17:17                     ` Eli Zaretskii
  2013-05-10 20:44                       ` João Távora
  0 siblings, 1 reply; 28+ messages in thread
From: Eli Zaretskii @ 2013-05-10 17:17 UTC (permalink / raw)
  To: João Távora; +Cc: 14380

> From: João Távora <joaotavora@gmail.com>
> Date: Fri, 10 May 2013 17:00:32 +0100
> Cc: 14380@debbugs.gnu.org
> 
> If the builtin tls fixes this and its the "recommended way" it should
> be the default no?

It _is_ the default, AFAIK, but you need to install the GnuTLS
libraries (DLLs) for it to work.  Try installing this one:

  http://sourceforge.net/projects/ezwinports/files/gnutls-3.0.9-w32-bin.zip/download

> btw: how to I configure emacs to use builtin gnutls?

I think you just need to install the above (and restart Emacs, if it's
already running).






^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-05-10 17:17                     ` Eli Zaretskii
@ 2013-05-10 20:44                       ` João Távora
  2013-05-11  7:06                         ` Eli Zaretskii
  2013-05-17 13:12                         ` Ted Zlatanov
  0 siblings, 2 replies; 28+ messages in thread
From: João Távora @ 2013-05-10 20:44 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 14380

OK, I see.

Honestly I thought builtin tls was some kind of tls library in elisp.
Needing external libraries doesn't make it very builtin :-), but
that's just my opinion.

In any case, my description shows how it "almost worked" with whatever
tls client it found (it was probably cygwin's, but I didn't check).
You might want to avoid these bug reports by fixing it the way I
suggested or some other way. Or not. All is OK from my part.

Thanks for the support,
João

On Fri, May 10, 2013 at 6:17 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: João Távora <joaotavora@gmail.com>
>> Date: Fri, 10 May 2013 17:00:32 +0100
>> Cc: 14380@debbugs.gnu.org
>>
>> If the builtin tls fixes this and its the "recommended way" it should
>> be the default no?
>
> It _is_ the default, AFAIK, but you need to install the GnuTLS
> libraries (DLLs) for it to work.  Try installing this one:
>
>   http://sourceforge.net/projects/ezwinports/files/gnutls-3.0.9-w32-bin.zip/download
>
>> btw: how to I configure emacs to use builtin gnutls?
>
> I think you just need to install the above (and restart Emacs, if it's
> already running).



-- 
João Távora





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-05-10 20:44                       ` João Távora
@ 2013-05-11  7:06                         ` Eli Zaretskii
  2013-05-17 13:12                         ` Ted Zlatanov
  1 sibling, 0 replies; 28+ messages in thread
From: Eli Zaretskii @ 2013-05-11  7:06 UTC (permalink / raw)
  To: João Távora; +Cc: 14380

> From: João Távora <joaotavora@gmail.com>
> Date: Fri, 10 May 2013 21:44:12 +0100
> Cc: 14380@debbugs.gnu.org
> 
> OK, I see.
> 
> Honestly I thought builtin tls was some kind of tls library in elisp.
> Needing external libraries doesn't make it very builtin :-), but
> that's just my opinion.

It's builtin in the sense that it does not need external programs.

> In any case, my description shows how it "almost worked" with whatever
> tls client it found (it was probably cygwin's, but I didn't check).
> You might want to avoid these bug reports by fixing it the way I
> suggested or some other way. Or not. All is OK from my part.

The native Windows build of Emacs doesn't work very well with an
external tls client, because their interaction is through signals,
which are not supported by Windows.






^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-05-10 20:44                       ` João Távora
  2013-05-11  7:06                         ` Eli Zaretskii
@ 2013-05-17 13:12                         ` Ted Zlatanov
  1 sibling, 0 replies; 28+ messages in thread
From: Ted Zlatanov @ 2013-05-17 13:12 UTC (permalink / raw)
  To: 14380

On Fri, 10 May 2013 21:44:12 +0100 João Távora <joaotavora@gmail.com> wrote: 

JT> Honestly I thought builtin tls was some kind of tls library in elisp.
JT> Needing external libraries doesn't make it very builtin :-), but
JT> that's just my opinion.

Implementing TLS in ELisp is possible, sure, but it would be extremely
slow and there's considerable risk in reimplementing that protocol.  We
chose to use GnuTLS instead, which is a fast reliable C implementation
and is available for W32 and all our other platforms.

JT> In any case, my description shows how it "almost worked" with whatever
JT> tls client it found (it was probably cygwin's, but I didn't check).
JT> You might want to avoid these bug reports by fixing it the way I
JT> suggested or some other way. Or not. All is OK from my part.

I've seen dozens of bugs related to "almost working" external TLS
binaries on all platforms.  It was one of the reasons I did the initial
GnuTLS integration with Emacs.  My vote is to require GnuTLS with Emacs
and to only support it, but there are some questions there, mainly for
W32 and Mac OS X: do we auto-update GnuTLS?  What happens when the
GnuTLS we install conflicts with another system install?  And so on...

Ted






^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: [gmane.emacs.bugs] bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
       [not found] <87k3mw79iv.fsf@lifelogs.com>
@ 2013-05-18 13:05 ` João Távora
  2013-05-19  3:17   ` Ted Zlatanov
       [not found]   ` <87zjvr64lt.fsf_-_@lifelogs.com>
  0 siblings, 2 replies; 28+ messages in thread
From: João Távora @ 2013-05-18 13:05 UTC (permalink / raw)
  To: Ted Zlatanov; +Cc: 14380

On Sat, May 18, 2013 at 1:33 PM, Ted Zlatanov <tzz@lifelogs.com> wrote:
> Sorry, I forgot to CC the two of you and the post just went to the bug
> list.  I didn't know if you'd see it so this is just a courtesy CC.
>
> Ted
>
>
>
> ---------- Forwarded message ----------
> From: Ted Zlatanov <tzz@lifelogs.com>
> To: 14380@debbugs.gnu.org
> Cc:
> Date: Fri, 17 May 2013 09:12:05 -0400
> Subject: bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
> On Fri, 10 May 2013 21:44:12 +0100 João Távora <joaotavora@gmail.com> wrote:
>
> JT> Honestly I thought builtin tls was some kind of tls library in elisp.
> JT> Needing external libraries doesn't make it very builtin :-), but
> JT> that's just my opinion.
>
> Implementing TLS in ELisp is possible, sure, but it would be extremely
> slow and there's considerable risk in reimplementing that protocol.  We
> chose to use GnuTLS instead, which is a fast reliable C implementation
> and is available for W32 and all our other platforms.

lol, I wasn't suggesting implementing it in elisp at all, I just read
built-in and assumed in was either statically linked in C or elisp. The
point it that needing external libraries which are not always bundled
doesn't make it very "builtin". But naming things, along with cache
invalidation,
is a a hard problem in programming :-)

> I've seen dozens of bugs related to "almost working" external TLS
> binaries on all platforms.

Yes, but have you looked closely at this particular one? The point is rather
to increase robustness. That is, `open-tls-stream` could/should promise
to cleanup the process buffer of its handshake garbage, so that future
functions that use that resource don't see it and don't get confused by it.

I'm assuming they don't need to see it, I might be wrong.

But if I'm right and that fix is performed then you've effectively extended
"imap just works" the set of W32 emacs users who type "M-x gnus" on a
vanilla emacs in a system with some cygwin installation in PATH. Maybe it's
a small set but I'm in it (when I'm at work).

> GnuTLS integration with Emacs.  My vote is to require GnuTLS with Emacs
> and to only support it, but there are some questions there, mainly for
> W32 and Mac OS X: do we auto-update GnuTLS?  What happens when the
> GnuTLS we install conflicts with another system install?  And so on...

That's all fine, I guess. I vote for that too :-)

Thanks,
João





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-05-18 13:05 ` bug#14380: [gmane.emacs.bugs] bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32 João Távora
@ 2013-05-19  3:17   ` Ted Zlatanov
       [not found]   ` <87zjvr64lt.fsf_-_@lifelogs.com>
  1 sibling, 0 replies; 28+ messages in thread
From: Ted Zlatanov @ 2013-05-19  3:17 UTC (permalink / raw)
  To: João Távora, Eli Zaretskii; +Cc: 14380, emacs-devel

(CC to emacs-devel as I think this discussion is relevant there)

On Sat, 18 May 2013 14:05:47 +0100 João Távora <joaotavora@gmail.com> wrote: 

JT> The point [is] that needing external libraries which are not always
JT> bundled doesn't make it very "builtin".

I'm not bringing GnuTLS into the Emacs source tree, which is the only
other way to make it built-in functionality.  I understand there are
issues with external dependencies and in fact I asked that we bundle the
GnuTLS W32 DLLs with the W32 Emacs builds.  That led to a long
discussions about how that makes security our responsibility and how we
then need to deal with GnuTLS updates, and I didn't have a strong desire
to become a W32 distribution expert since I barely know that platform.
No one else picked it up, and there we are with "install it yourself" as
the recommended way to get GnuTLS to work on W32.

>> I've seen dozens of bugs related to "almost working" external TLS
>> binaries on all platforms.

JT> Yes, but have you looked closely at this particular one? The point is rather
JT> to increase robustness. That is, `open-tls-stream` could/should promise
JT> to cleanup the process buffer of its handshake garbage, so that future
JT> functions that use that resource don't see it and don't get confused by it.

JT> I'm assuming they don't need to see it, I might be wrong.

I'm not able to fix this bug or work on bugs in the external SSL/TLS support.

JT> But if I'm right and that fix is performed then you've effectively extended
JT> "imap just works" the set of W32 emacs users who type "M-x gnus" on a
JT> vanilla emacs in a system with some cygwin installation in PATH. Maybe it's
JT> a small set but I'm in it (when I'm at work).

Wouldn't you rather get GnuTLS to work by default?  Otherwise we serve
the use case "I have no secure transport, so let me use a hack by
default."

>> GnuTLS integration with Emacs.  My vote is to require GnuTLS with Emacs
>> and to only support it, but there are some questions there, mainly for
>> W32 and Mac OS X: do we auto-update GnuTLS?  What happens when the
>> GnuTLS we install conflicts with another system install?  And so on...

JT> That's all fine, I guess. I vote for that too :-)

The big problem for me is that I don't have the time or platform
knowledge to write a GnuTLS auto-installer and updater for those two
problematic platforms.  The GnuTLS developers don't want to provide this
service either.  Who will be responsible to it?  What happens when a
security vulnerability hits the DLLs we distribute with Emacs?

My proposal would be to push out the next Emacs bundled with the latest
GnuTLS DLLs, only support GnuTLS, provide users with instructions on
updating them, and treat GnuTLS vulnerabilities as Emacs
vulnerabilities.  This is not ideal but IMO better than the current
situation.

Ted





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
       [not found]   ` <87zjvr64lt.fsf_-_@lifelogs.com>
@ 2013-05-19 11:45     ` João Távora
  2013-05-19 15:32     ` Eli Zaretskii
       [not found]     ` <CALDnm50KHS7wOKUpQJQHb4V_PLfQs6VkjEsRmPgY=R5x0eEuUg@mail.gmail.com>
  2 siblings, 0 replies; 28+ messages in thread
From: João Távora @ 2013-05-19 11:45 UTC (permalink / raw)
  To: Ted Zlatanov; +Cc: 14380, emacs-devel

On Sun, May 19, 2013 at 4:17 AM, Ted Zlatanov <tzz@lifelogs.com> wrote:
> GnuTLS W32 DLLs with the W32 Emacs builds.  That led to a long
> discussions about how that makes security our responsibility and how we

I see. Indeed, bundling security stuff with your app is increasing
its responsibility manifold.

> Wouldn't you rather get GnuTLS to work by default?  Otherwise we serve
> the use case "I have no secure transport, so let me use a hack by
> default."

I don't understand. What is the hack here? External binary for TLS?
But yes, GnuTLS by default is certainly better...

> service either.  Who will be responsible to it?  What happens when a
> security vulnerability hits the DLLs we distribute with Emacs?
>
> My proposal would be to push out the next Emacs bundled with the latest
> GnuTLS DLLs, only support GnuTLS, provide users with instructions on
> updating them, and treat GnuTLS vulnerabilities as Emacs
> vulnerabilities.  This is not ideal but IMO better than the current
> situation.

... but then you have all these headaches.

The fix I proposed aims for the status quo, that is: make external
TLS binary support slightly more robust. My test case is even smaller:

* W32
* cygwin carrying the responsibility burden
* vanilla emacs working with tls/imap/gnus.

Thanks for the time spent in analysing this,
-- 
João Távora





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
       [not found]   ` <87zjvr64lt.fsf_-_@lifelogs.com>
  2013-05-19 11:45     ` João Távora
@ 2013-05-19 15:32     ` Eli Zaretskii
  2013-05-20 13:56       ` Ted Zlatanov
       [not found]     ` <CALDnm50KHS7wOKUpQJQHb4V_PLfQs6VkjEsRmPgY=R5x0eEuUg@mail.gmail.com>
  2 siblings, 1 reply; 28+ messages in thread
From: Eli Zaretskii @ 2013-05-19 15:32 UTC (permalink / raw)
  To: Ted Zlatanov; +Cc: 14380, joaotavora

[I removed emacs-devel.]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Cc: 14380@debbugs.gnu.org, emacs-devel@gnu.org
> Date: Sat, 18 May 2013 23:17:02 -0400
> 
> (CC to emacs-devel as I think this discussion is relevant there)

There's no reason: all the people who you'd like to reach read the bug
list as well.

> The big problem for me is that I don't have the time or platform
> knowledge to write a GnuTLS auto-installer and updater for those two
> problematic platforms.  The GnuTLS developers don't want to provide this
> service either.  Who will be responsible to it?  What happens when a
> security vulnerability hits the DLLs we distribute with Emacs?
> 
> My proposal would be to push out the next Emacs bundled with the latest
> GnuTLS DLLs, only support GnuTLS, provide users with instructions on
> updating them, and treat GnuTLS vulnerabilities as Emacs
> vulnerabilities.  This is not ideal but IMO better than the current
> situation.

I see no problems with the current situation.  Installing precompiled
GnuTLS from a zip file is a snap.





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
       [not found]     ` <CALDnm50KHS7wOKUpQJQHb4V_PLfQs6VkjEsRmPgY=R5x0eEuUg@mail.gmail.com>
@ 2013-05-19 15:44       ` Eli Zaretskii
  2013-05-19 17:57         ` João Távora
  2013-05-19 23:05       ` Ted Zlatanov
       [not found]       ` <87txly4ll9.fsf@lifelogs.com>
  2 siblings, 1 reply; 28+ messages in thread
From: Eli Zaretskii @ 2013-05-19 15:44 UTC (permalink / raw)
  To: João Távora; +Cc: 14380, tzz

> From: João Távora <joaotavora@gmail.com>
> Date: Sun, 19 May 2013 12:45:12 +0100
> Cc: Eli Zaretskii <eliz@gnu.org>, 14380@debbugs.gnu.org, emacs-devel@gnu.org
> 
> The fix I proposed aims for the status quo, that is: make external
> TLS binary support slightly more robust.

I already said at lest twice in this thread: THIS WON'T WORK on
Windows (except in Cygwin Emacs).  The communications between the
external TLS client and Emacs are via signals, which aren't really
supported by Windows.  Solving this was one of the main reasons for
incorporating GnuTLS into Emacs.

I don't really understand what are we still discussing here.  Let me
describe how the current situation looks from my POV:

  . Emacs can be built with GnuTLS support if GnuTLS is installed on
    the end-user's machine, and that end user builds her own Emacs.
    This is the same as on Unix.  I hope no one will say this is "not
    built-in".

  . Windows users get special treatment in that precompiled binaries
    of Emacs are available for those who cannot or won't build their
    own.  These precompiled binaries are built with GnuTLS support to
    begin with.

  . As yet another bonus for Windows users, Emacs will happily start
    and run even if GnuTLS is not found on the end-user's machine;
    however, TLS will not be available in that case, of course (Emacs
    will announce that if required to use TLS).

So now you tell me how come these two bonuses are somehow regarded as
deficiencies?  Would it be better not to produce binaries at all, or
let them abort with a fatal error if GnuTLS is not installed?

Installing GnuTLS boils down to unzipping a single zip archive.  How
hard can that be for someone who uses Emacs??






^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-05-19 15:44       ` Eli Zaretskii
@ 2013-05-19 17:57         ` João Távora
  2013-05-19 19:01           ` Eli Zaretskii
  0 siblings, 1 reply; 28+ messages in thread
From: João Távora @ 2013-05-19 17:57 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 14380, tzz

On Sun, May 19, 2013 at 4:44 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: João Távora <joaotavora@gmail.com>
>> Date: Sun, 19 May 2013 12:45:12 +0100
>> Cc: Eli Zaretskii <eliz@gnu.org>, 14380@debbugs.gnu.org, emacs-devel@gnu.org
>>
>> The fix I proposed aims for the status quo, that is: make external
>> TLS binary support slightly more robust.
>
> I already said at lest twice in this thread: THIS WON'T WORK on
> Windows (except in Cygwin Emacs).  The communications between the

Look, there's no need to shout. I'm not using Cygwin emacs, I'm using
regular W32
binaries and am not even sure what tls binary emacs found or how. It
appears to be:

"openssl s_client -connect imaps.mycompany.com:993 -no_ssl2 -ign_eof"

My analysis of the code of `network-stream-open-tls' revealed (as do
the comments)
that it tries to cleanup the process buffer of previous garbage left there by
`open-tls-stream` (who nonetheless tries to place point correctly in
the process
buffer)

I'm **guessing** "openssl" is a cygwin binary, I didn't even check that.
I **reported** a bug since I considered unexpected behaviour occurred even with
the cleanest of "emacs -Q" run.
I **suggested** a fix because of two reasons: (1) I tried it and it
worked and has
been working since (2) in the context of the interaction between the
two functions
`network-stream-open-tls' and `open-tls-stream' it seemed reasonable
that the latter
cleans up after itself.

Maybe, in my reduced usage of gnus, I haven't gotten to a situation
where things would
break because of signal handling or whatever. Lucky me.

When things do break, I'll happily unzip dlls, I have nothing against that.

Thanks for all the info, feel free to close the bug if you haven't already
Over and out,
João





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-05-19 17:57         ` João Távora
@ 2013-05-19 19:01           ` Eli Zaretskii
  0 siblings, 0 replies; 28+ messages in thread
From: Eli Zaretskii @ 2013-05-19 19:01 UTC (permalink / raw)
  To: João Távora; +Cc: 14380, tzz

> From: João Távora <joaotavora@gmail.com>
> Date: Sun, 19 May 2013 18:57:43 +0100
> Cc: tzz@lifelogs.com, 14380@debbugs.gnu.org
> 
> On Sun, May 19, 2013 at 4:44 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> >> From: João Távora <joaotavora@gmail.com>
> >> Date: Sun, 19 May 2013 12:45:12 +0100
> >> Cc: Eli Zaretskii <eliz@gnu.org>, 14380@debbugs.gnu.org, emacs-devel@gnu.org
> >>
> >> The fix I proposed aims for the status quo, that is: make external
> >> TLS binary support slightly more robust.
> >
> > I already said at lest twice in this thread: THIS WON'T WORK on
> > Windows (except in Cygwin Emacs).  The communications between the
> 
> Look, there's no need to shout.

I'm sorry, but after saying that twice, what other methods do I have
to make sure I'm heard?

> I'm not using Cygwin emacs, I'm using regular W32 binaries

That's exactly what I was talking about: an external tls client will
not work with a native w32 Emacs.

> When things do break, I'll happily unzip dlls, I have nothing against that.

I suggest you do this without waiting for things to break.  The
built-in GnuTLS support does work on Windows, there are several happy
users of it here.  Why have a partial solution when you can have a
complete one?






^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
       [not found]     ` <CALDnm50KHS7wOKUpQJQHb4V_PLfQs6VkjEsRmPgY=R5x0eEuUg@mail.gmail.com>
  2013-05-19 15:44       ` Eli Zaretskii
@ 2013-05-19 23:05       ` Ted Zlatanov
       [not found]       ` <87txly4ll9.fsf@lifelogs.com>
  2 siblings, 0 replies; 28+ messages in thread
From: Ted Zlatanov @ 2013-05-19 23:05 UTC (permalink / raw)
  To: João Távora; +Cc: 14380, emacs-devel

On Sun, 19 May 2013 12:45:12 +0100 João Távora <joaotavora@gmail.com> wrote: 

JT> On Sun, May 19, 2013 at 4:17 AM, Ted Zlatanov <tzz@lifelogs.com> wrote:
>> Wouldn't you rather get GnuTLS to work by default?  Otherwise we serve
>> the use case "I have no secure transport, so let me use a hack by
>> default."

JT> I don't understand. What is the hack here? External binary for TLS?

Using an external binary to transport SSL or TLS is a hack IMO.

>> My proposal would be to push out the next Emacs bundled with the latest
>> GnuTLS DLLs, only support GnuTLS, provide users with instructions on
>> updating them, and treat GnuTLS vulnerabilities as Emacs
>> vulnerabilities.  This is not ideal but IMO better than the current
>> situation.

JT> ... but then you have all these headaches.

It's a headache I'm willing to endure for the sake of Emacs users.

The alternative, which João is enduring now, is to punt the problem.

This is a question for the Emacs maintainers: do you agree with me on
the above plan?  It would mean changing the way Mac OS X and W32 Emacs
builds are distributed, to include the GnuTLS libraries with the build,
and we'd have to implement a way (perhaps through the ELPA) to
distribute updates to these libraries.

JT> The fix I proposed aims for the status quo, that is: make external
JT> TLS binary support slightly more robust. My test case is even smaller:

JT> * W32
JT> * cygwin carrying the responsibility burden
JT> * vanilla emacs working with tls/imap/gnus.

Did you propose a patch?  I would commit a patch but can't write it
despite your great description of the problem.

Ted





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
       [not found]       ` <87txly4ll9.fsf@lifelogs.com>
@ 2013-05-20  2:08         ` Juanma Barranquero
  2013-05-20 22:07         ` João Távora
       [not found]         ` <CALDnm51-4aTee3NGoRp+=T6MuTV3xD9P4Ed1yZLU_E5k_r5hGw@mail.gmail.com>
  2 siblings, 0 replies; 28+ messages in thread
From: Juanma Barranquero @ 2013-05-20  2:08 UTC (permalink / raw)
  To: Ted Zlatanov; +Cc: 14380, João Távora, Emacs developers

On Mon, May 20, 2013 at 1:05 AM, Ted Zlatanov <tzz@lifelogs.com> wrote:

> It would mean changing the way Mac OS X and W32 Emacs
> builds are distributed, to include the GnuTLS libraries with the build,
> and we'd have to implement a way (perhaps through the ELPA) to
> distribute updates to these libraries.

Why would that be a good idea now if it wasn't seen as such before?

    J





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-05-19 15:32     ` Eli Zaretskii
@ 2013-05-20 13:56       ` Ted Zlatanov
  2013-05-20 16:28         ` Eli Zaretskii
  0 siblings, 1 reply; 28+ messages in thread
From: Ted Zlatanov @ 2013-05-20 13:56 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 14380, joaotavora

On Sun, 19 May 2013 18:32:37 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> My proposal would be to push out the next Emacs bundled with the latest
>> GnuTLS DLLs, only support GnuTLS, provide users with instructions on
>> updating them, and treat GnuTLS vulnerabilities as Emacs
>> vulnerabilities.  This is not ideal but IMO better than the current
>> situation.

EZ> I see no problems with the current situation.  Installing precompiled
EZ> GnuTLS from a zip file is a snap.

That's only a small part of the risk and responsibility we're shifting
onto the Emacs users.

Ted





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-05-20 13:56       ` Ted Zlatanov
@ 2013-05-20 16:28         ` Eli Zaretskii
  2013-05-24 19:48           ` Ted Zlatanov
  0 siblings, 1 reply; 28+ messages in thread
From: Eli Zaretskii @ 2013-05-20 16:28 UTC (permalink / raw)
  To: Ted Zlatanov; +Cc: 14380, joaotavora

> From: Ted Zlatanov <tzz@lifelogs.com>
> Cc: 14380@debbugs.gnu.org,  joaotavora@gmail.com
> Date: Mon, 20 May 2013 09:56:27 -0400
> 
> On Sun, 19 May 2013 18:32:37 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 
> 
> >> My proposal would be to push out the next Emacs bundled with the latest
> >> GnuTLS DLLs, only support GnuTLS, provide users with instructions on
> >> updating them, and treat GnuTLS vulnerabilities as Emacs
> >> vulnerabilities.  This is not ideal but IMO better than the current
> >> situation.
> 
> EZ> I see no problems with the current situation.  Installing precompiled
> EZ> GnuTLS from a zip file is a snap.
> 
> That's only a small part of the risk and responsibility we're shifting
> onto the Emacs users.

What risk? what responsibility?

A user who installs software on her computer is already trusted with
certain responsibilities, because a single mistyped command or a badly
built package can easily shut down a perfectly healthy system for
hours, if not days.  Users install dozens of packages needed to create
a workable environment for whatever they need to accomplish.  Why is
GnuTLS so special?

And mind you, in view of the latest sparring between GnuTLS developers
and the FSF (which I have no idea how ended, except that the license
was downgraded a bit and the official site moved), I'm not even sure
the FSF will agree to distribute GnuTLS with Emacs, on any platform.
Why should Emacs development enter this minefield?  And for what? for
solving a non-existing problem of installing a simple package?  Don't
we have better places to apply our time and energy?

Don't misunderstand me: if someone decides to provide regular builds
of GnuTLS ready to be downloaded and installed, I will applaud that
person.  Heck, it will be one less duty for me, for starters, as far
as the Windows binaries are concerned.  But please don't represent
this as a must for Emacs, because it isn't.





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
       [not found]       ` <87txly4ll9.fsf@lifelogs.com>
  2013-05-20  2:08         ` Juanma Barranquero
@ 2013-05-20 22:07         ` João Távora
       [not found]         ` <CALDnm51-4aTee3NGoRp+=T6MuTV3xD9P4Ed1yZLU_E5k_r5hGw@mail.gmail.com>
  2 siblings, 0 replies; 28+ messages in thread
From: João Távora @ 2013-05-20 22:07 UTC (permalink / raw)
  To: Ted Zlatanov; +Cc: 14380, emacs-devel

On Mon, May 20, 2013 at 12:05 AM, Ted Zlatanov <tzz@lifelogs.com> wrote:
>
> Using an external binary to transport SSL or TLS is a hack IMO.

I see, and reading a bit of tls.el it makes sense. It does all this
crazy regexp searching... This is a hack in my book too, albeit one
that's been working fine,

> Did you propose a patch?  I would commit a patch but can't write it
> despite your great description of the problem.
>

Here is a patch that should be equivalent to the defadvice I'm using.
As I said, it works for me. Also I didn't have a VC copy of emacs so I
used  `diff-buffer-with-file'

diff -u -L /usr/local/share/emacs/24.3/lisp/net/tls.el.gz -L
\#\<buffer\ tls.el.gz\> /tmp/jka-com1909LVh
/tmp/buffer-content-1909lpt
--- /usr/local/share/emacs/24.3/lisp/net/tls.el.gz
+++ #<buffer tls.el.gz>
@@ -286,7 +286,11 @@
 			     (format "Host name in certificate doesn't \
 match `%s'. Connect anyway? " host))))))
 	(setq done nil)
-	(delete-process process)))
+	(delete-process process))
+      ;; delete all the informational messages that could
+      ;; confuse futures users of `buffer'
+      ;;
+      (delete-region (point-min) (point)))
     (message "Opening TLS connection to `%s'...%s"
 	     host (if done "done" "failed"))
     (when use-temp-buffer


João
-- 
João Távora





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-05-20 16:28         ` Eli Zaretskii
@ 2013-05-24 19:48           ` Ted Zlatanov
  2013-05-24 20:27             ` Eli Zaretskii
  0 siblings, 1 reply; 28+ messages in thread
From: Ted Zlatanov @ 2013-05-24 19:48 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 14380, joaotavora

On Mon, 20 May 2013 19:28:40 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Cc: 14380@debbugs.gnu.org,  joaotavora@gmail.com
>> Date: Mon, 20 May 2013 09:56:27 -0400
>> 
>> On Sun, 19 May 2013 18:32:37 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 
>> 
>> >> My proposal would be to push out the next Emacs bundled with the latest
>> >> GnuTLS DLLs, only support GnuTLS, provide users with instructions on
>> >> updating them, and treat GnuTLS vulnerabilities as Emacs
>> >> vulnerabilities.  This is not ideal but IMO better than the current
>> >> situation.
>> 
EZ> I see no problems with the current situation.  Installing precompiled
EZ> GnuTLS from a zip file is a snap.
>> 
>> That's only a small part of the risk and responsibility we're shifting
>> onto the Emacs users.

EZ> What risk? what responsibility?

The risk is that their version of GnuTLS is out of date.  The
responsibility is to update it regularly.

EZ> A user who installs software on her computer is already trusted with
EZ> certain responsibilities, because a single mistyped command or a badly
EZ> built package can easily shut down a perfectly healthy system for
EZ> hours, if not days.  Users install dozens of packages needed to create
EZ> a workable environment for whatever they need to accomplish.  Why is
EZ> GnuTLS so special?

Installing and keeping GnuTLS up to date should not be the
responsibility of the user.

To put it another way, if you want that responsibility, you're in a very
small percentage of the Emacs user population.  Most users don't want it
and will neglect it badly.

EZ> And mind you, in view of the latest sparring between GnuTLS developers
EZ> and the FSF (which I have no idea how ended, except that the license
EZ> was downgraded a bit and the official site moved), I'm not even sure
EZ> the FSF will agree to distribute GnuTLS with Emacs, on any platform.
EZ> Why should Emacs development enter this minefield?

That's a reasonable question.  I think we have to face it regardless of
the outcome of this discussion because Emacs depends on GnuTLS for SSL
and TLS communications right now.

As far as I know GnuTLS status is back to "kosher."

EZ> And for what? for solving a non-existing problem of installing a
EZ> simple package?

Installing is easy.  Keeping it up to date isn't.  Security updates are
tedious and tedious things get overlooked.

EZ> Don't misunderstand me: if someone decides to provide regular builds
EZ> of GnuTLS ready to be downloaded and installed, I will applaud that
EZ> person.  Heck, it will be one less duty for me, for starters, as far
EZ> as the Windows binaries are concerned.  But please don't represent
EZ> this as a must for Emacs, because it isn't.

I see it as a responsibility we're avoiding.  But if we had these
regular builds, how would the user know about a critical update he
really must install?

See here http://bugs.python.org/issue17425 for an example of how the
Python community dealt with an security issue in the OpenSSL libraries
they ship for Windows.  I guess we have to answer the question of
whether that's a standard we as Emacs developers should aspire to, or
not.

Ted





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-05-24 19:48           ` Ted Zlatanov
@ 2013-05-24 20:27             ` Eli Zaretskii
  2013-05-10 12:49               ` João Távora
  0 siblings, 1 reply; 28+ messages in thread
From: Eli Zaretskii @ 2013-05-24 20:27 UTC (permalink / raw)
  To: Ted Zlatanov; +Cc: 14380, joaotavora

> From: Ted Zlatanov <tzz@lifelogs.com>
> Cc: 14380@debbugs.gnu.org,  joaotavora@gmail.com
> Date: Fri, 24 May 2013 15:48:20 -0400
> 
> EZ> What risk? what responsibility?
> 
> The risk is that their version of GnuTLS is out of date.

That happens with dozens of packages on each user's machine.  There's
nothing in GnuTLS that makes it unique in this regard.

Moreover, the latest and greatest GnuTLS sometimes simply won't build
on some systems.  Like with the latest release, for example.  How is
it a good idea to upgrade to a version that is broken?  And if the
latest version is not always the one to upgrade to, then who will make
the research required to tell users to which version to upgrade?  You?

I did that research for the single version whose Windows port I made
available.  I built it, fixed the build problems, tested it, fixed the
problems revealed by that, and after doing all that I could in good
faith tell people they can use that version without too much fear.
Why is it safer for users to upgrade to a newer version than to stay
with the one I tested?  Shouldn't whoever wants to tell them to
upgrade invest a similar effort in that newer version?  If she
doesn't, she is actually shifting the responsibility to the users
anyway!

> The responsibility is to update it regularly.

Or not.  Blindly upgrading could get users in trouble.

> EZ> A user who installs software on her computer is already trusted with
> EZ> certain responsibilities, because a single mistyped command or a badly
> EZ> built package can easily shut down a perfectly healthy system for
> EZ> hours, if not days.  Users install dozens of packages needed to create
> EZ> a workable environment for whatever they need to accomplish.  Why is
> EZ> GnuTLS so special?
> 
> Installing and keeping GnuTLS up to date should not be the
> responsibility of the user.

Says you.  But since there's no one else to pick up the gauntlet,
that's where this responsibility will need to rest.  If J.R. Hacker
needs GnuTLS today, he has no one else but himself to rely on.  All
we, the Emacs developers, do is just talk.

> To put it another way, if you want that responsibility, you're in a very
> small percentage of the Emacs user population.  Most users don't want it
> and will neglect it badly.

Again, nothing new or special here.

> As far as I know GnuTLS status is back to "kosher."

Not sure based on what you say this.

> I see it as a responsibility we're avoiding.  But if we had these
> regular builds, how would the user know about a critical update he
> really must install?
> 
> See here http://bugs.python.org/issue17425 for an example of how the
> Python community dealt with an security issue in the OpenSSL libraries
> they ship for Windows.  I guess we have to answer the question of
> whether that's a standard we as Emacs developers should aspire to, or
> not.

I'm sorry, but you are expecting from the Emacs development something
it can never provide in its present shape and form.  Tracking security
issues to this degree in even a single package is a very time
consuming job.  Unless we have several volunteers on board taking
responsibility for the various packages which Emacs supports, what you
seem to want is nothing more than a pipe dream.  I don't see any such
volunteers; in fact, I don't even see a single one.  If we had such an
individual, my year-old port would have been replaced by newer ones
already.  (Of course, the Windows build in GnuTLS is regularly broken,
so it's not really easy, either.)  Until that changes, all this talk
is just a huge waste of energy.  

If you think this kind of effort is possible, how about if you present
a complete realistic plan for having a secure Emacs, name individuals
who would test the releases of those packages for security issues, and
make sure any problems that are detected are promptly fixed on all
platforms we support, etc.?  Otherwise, let's just stop these endless
discussions and admit that we don't have the resources to live up to
it.





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-05-10 12:49               ` João Távora
  2013-05-10 14:00                 ` Eli Zaretskii
@ 2013-05-24 22:20                 ` Ted Zlatanov
  2013-05-25  6:49                   ` Eli Zaretskii
  1 sibling, 1 reply; 28+ messages in thread
From: Ted Zlatanov @ 2013-05-24 22:20 UTC (permalink / raw)
  To: 14380

On Fri, 24 May 2013 23:27:07 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Cc: 14380@debbugs.gnu.org,  joaotavora@gmail.com
>> Date: Fri, 24 May 2013 15:48:20 -0400
>> 
EZ> What risk? what responsibility?
>> 
>> The risk is that their version of GnuTLS is out of date.

EZ> That happens with dozens of packages on each user's machine.  There's
EZ> nothing in GnuTLS that makes it unique in this regard.

Yes, of course.  I don't know the other packages we require to enable
extra features on W32, sorry.  I think GnuTLS is somewhat unique in this
regard by being the only way to do secure communications with the
outside world, but it's worth considering putting the other packages
under the same mechanism as GnuTLS for installations and updates.

EZ> Moreover, the latest and greatest GnuTLS sometimes simply won't build
EZ> on some systems.  Like with the latest release, for example.  How is
EZ> it a good idea to upgrade to a version that is broken?  And if the
EZ> latest version is not always the one to upgrade to, then who will make
EZ> the research required to tell users to which version to upgrade?
EZ> You?

Yes, possibly.

EZ> I did that research for the single version whose Windows port I made
EZ> available.  I built it, fixed the build problems, tested it, fixed the
EZ> problems revealed by that, and after doing all that I could in good
EZ> faith tell people they can use that version without too much fear.
EZ> Why is it safer for users to upgrade to a newer version than to stay
EZ> with the one I tested?  Shouldn't whoever wants to tell them to
EZ> upgrade invest a similar effort in that newer version?  If she
EZ> doesn't, she is actually shifting the responsibility to the users
EZ> anyway!

You're right.  It's a lot of work.  I appreciate it very much.  I hope
to be able to find the resources to make these reviews happen.

>> Installing and keeping GnuTLS up to date should not be the
>> responsibility of the user.

EZ> Says you.  But since there's no one else to pick up the gauntlet,
EZ> that's where this responsibility will need to rest.  If J.R. Hacker
EZ> needs GnuTLS today, he has no one else but himself to rely on.  All
EZ> we, the Emacs developers, do is just talk.

I like to ask before I make changes, hence my request for votes in
emacs-devel.  Sorry if it seems like empty talk to you.

>> As far as I know GnuTLS status is back to "kosher."

EZ> Not sure based on what you say this.

Monitoring the GnuTLS mailing lists.  I don't mean there are no issues,
only that the FSF has not made a statement about changing its preference
for GnuTLS.

>> I see it as a responsibility we're avoiding.  But if we had these
>> regular builds, how would the user know about a critical update he
>> really must install?
>> 
>> See here http://bugs.python.org/issue17425 for an example of how the
>> Python community dealt with an security issue in the OpenSSL libraries
>> they ship for Windows.  I guess we have to answer the question of
>> whether that's a standard we as Emacs developers should aspire to, or
>> not.

EZ> I'm sorry, but you are expecting from the Emacs development something
EZ> it can never provide in its present shape and form.  Tracking security
EZ> issues to this degree in even a single package is a very time
EZ> consuming job.  Unless we have several volunteers on board taking
EZ> responsibility for the various packages which Emacs supports, what you
EZ> seem to want is nothing more than a pipe dream.  I don't see any such
EZ> volunteers; in fact, I don't even see a single one.  If we had such an
EZ> individual, my year-old port would have been replaced by newer ones
EZ> already.  (Of course, the Windows build in GnuTLS is regularly broken,
EZ> so it's not really easy, either.)  Until that changes, all this talk
EZ> is just a huge waste of energy.

EZ> If you think this kind of effort is possible, how about if you present
EZ> a complete realistic plan for having a secure Emacs, name individuals
EZ> who would test the releases of those packages for security issues, and
EZ> make sure any problems that are detected are promptly fixed on all
EZ> platforms we support, etc.?  Otherwise, let's just stop these endless
EZ> discussions and admit that we don't have the resources to live up to
EZ> it.

I'm trying to get the work started by first and foremost deciding if
Emacs as a project wants to do this at all.  This is a decision for the
maintainers and you've voted against it on emacs-devel, so let's see
what the vote count is and what the maintainers say.

If the maintainers are OK with this direction, I will start working on
automating the builds (which will need your help initially, if you're
willing, to replicate your build process), asking for volunteers, and
packaging the libraries.  I don't have a 5-year plan but hope to get far
enough to make it something sustainable.

Ted






^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-05-24 22:20                 ` Ted Zlatanov
@ 2013-05-25  6:49                   ` Eli Zaretskii
  0 siblings, 0 replies; 28+ messages in thread
From: Eli Zaretskii @ 2013-05-25  6:49 UTC (permalink / raw)
  To: 14380

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Fri, 24 May 2013 18:20:55 -0400
> 
> >> The risk is that their version of GnuTLS is out of date.
> 
> EZ> That happens with dozens of packages on each user's machine.  There's
> EZ> nothing in GnuTLS that makes it unique in this regard.
> 
> Yes, of course.  I don't know the other packages we require to enable
> extra features on W32, sorry.

Those are the image libraries and libxml2.  And, of course, any
prerequisite packages needed to build those: zlib, libintl, libiconv,
and the libraries, like libnettle, that GnuTLS depends on.

> I think GnuTLS is somewhat unique in this regard by being the only
> way to do secure communications with the outside world, but it's
> worth considering putting the other packages under the same
> mechanism as GnuTLS for installations and updates.

Security is not just about secure communications.  I'm sure you know
that.

> >> Installing and keeping GnuTLS up to date should not be the
> >> responsibility of the user.
> 
> EZ> Says you.  But since there's no one else to pick up the gauntlet,
> EZ> that's where this responsibility will need to rest.  If J.R. Hacker
> EZ> needs GnuTLS today, he has no one else but himself to rely on.  All
> EZ> we, the Emacs developers, do is just talk.
> 
> I like to ask before I make changes, hence my request for votes in
> emacs-devel.  Sorry if it seems like empty talk to you.

It is empty because it doesn't lead to any action.  We've been talking
about this for months already.

> >> As far as I know GnuTLS status is back to "kosher."
> 
> EZ> Not sure based on what you say this.
> 
> Monitoring the GnuTLS mailing lists.

The actual discussions were private, not on any public list.

> I'm trying to get the work started by first and foremost deciding if
> Emacs as a project wants to do this at all.  This is a decision for the
> maintainers and you've voted against it on emacs-devel, so let's see
> what the vote count is and what the maintainers say.

I think it's well beyond our capacities and resources, yes.





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
       [not found]         ` <CALDnm51-4aTee3NGoRp+=T6MuTV3xD9P4Ed1yZLU_E5k_r5hGw@mail.gmail.com>
@ 2013-06-05 15:06           ` Ted Zlatanov
       [not found]           ` <87obbk1trn.fsf@lifelogs.com>
  1 sibling, 0 replies; 28+ messages in thread
From: Ted Zlatanov @ 2013-06-05 15:06 UTC (permalink / raw)
  To: João Távora; +Cc: 14380, emacs-devel

On Mon, 20 May 2013 23:07:30 +0100 João Távora <joaotavora@gmail.com> wrote: 

JT> Here is a patch that should be equivalent to the defadvice I'm using.
JT> As I said, it works for me. Also I didn't have a VC copy of emacs so I
JT> used  `diff-buffer-with-file'

I applied something very similar.  It's a tiny change so it doesn't
require assignment papers.  If you'd like to contribute to GNU Emacs or
other GNU software in the future, I encourage you to file them.

revno: 112858
timestamp: Wed 2013-06-05 11:03:37 -0400
message:
  (open-tls-stream): Remove unneeded buffer contents when opening the connection.

This change didn't break anything in my testing.  Please test so I can
close the bug if it's fixed for you.

Ted

diff -u -L /usr/local/share/emacs/24.3/lisp/net/tls.el.gz -L
\#\<buffer\ tls.el.gz\> /tmp/jka-com1909LVh
/tmp/buffer-content-1909lpt
--- /usr/local/share/emacs/24.3/lisp/net/tls.el.gz
+++ #<buffer tls.el.gz>
@@ -286,7 +286,11 @@
			     (format "Host name in certificate doesn't \
 match `%s'. Connect anyway? " host))))))
	(setq done nil)
-	(delete-process process)))
+	(delete-process process))
+      ;; delete all the informational messages that could
+      ;; confuse futures users of `buffer'
+      ;;
+      (delete-region (point-min) (point)))
     (message "Opening TLS connection to `%s'...%s"
	     host (if done "done" "failed"))
     (when use-temp-buffer






^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
       [not found]           ` <87obbk1trn.fsf@lifelogs.com>
@ 2013-06-05 16:42             ` Glenn Morris
  2013-06-05 18:10               ` Ted Zlatanov
  2013-06-06  8:15             ` João Távora
  1 sibling, 1 reply; 28+ messages in thread
From: Glenn Morris @ 2013-06-05 16:42 UTC (permalink / raw)
  To: 14380; +Cc: Ted Zlatanov

Ted Zlatanov wrote:

> I applied something very similar.  It's a tiny change so it doesn't
> require assignment papers.

Thanks for applying it. If committing changes by others, please make the
ChangeLog entry in their name, and use commit --author.
And because this has a bug number, the ChangeLog entry can use that
rather than an emacs-devel link.

(Seems he already has an assignment BTW).





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-06-05 16:42             ` Glenn Morris
@ 2013-06-05 18:10               ` Ted Zlatanov
  2013-06-08  0:58                 ` Glenn Morris
  0 siblings, 1 reply; 28+ messages in thread
From: Ted Zlatanov @ 2013-06-05 18:10 UTC (permalink / raw)
  To: Glenn Morris; +Cc: 14380

On Wed, 05 Jun 2013 12:42:44 -0400 Glenn Morris <rgm@gnu.org> wrote: 

GM> Ted Zlatanov wrote:
>> I applied something very similar.  It's a tiny change so it doesn't
>> require assignment papers.

GM> Thanks for applying it. If committing changes by others, please make the
GM> ChangeLog entry in their name, and use commit --author.

This must be a recent policy, or at least wasn't requested before.  Sorry.

GM> And because this has a bug number, the ChangeLog entry can use that
GM> rather than an emacs-devel link.

Right.  Can you tell I haven't contributed in a while? :)

Should I edit the ChangeLog with the bug number or just leave it alone
now that it's wrong?

Ted





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
       [not found]           ` <87obbk1trn.fsf@lifelogs.com>
  2013-06-05 16:42             ` Glenn Morris
@ 2013-06-06  8:15             ` João Távora
  1 sibling, 0 replies; 28+ messages in thread
From: João Távora @ 2013-06-06  8:15 UTC (permalink / raw)
  To: Ted Zlatanov; +Cc: 14380, emacs-devel

Thanks!

To test, I will load just load the trunks "tls.el" in my existing W32
emacs. (I don't have time to get a windows emacs-building toolchain).

As Stefan mentioned, I have already signed papers.

João

On Wed, Jun 5, 2013 at 4:06 PM, Ted Zlatanov <tzz@lifelogs.com> wrote:
> On Mon, 20 May 2013 23:07:30 +0100 João Távora <joaotavora@gmail.com> wrote:
>
> JT> Here is a patch that should be equivalent to the defadvice I'm using.
> JT> As I said, it works for me. Also I didn't have a VC copy of emacs so I
> JT> used  `diff-buffer-with-file'
>
> I applied something very similar.  It's a tiny change so it doesn't
> require assignment papers.  If you'd like to contribute to GNU Emacs or
> other GNU software in the future, I encourage you to file them.
>
> revno: 112858
> timestamp: Wed 2013-06-05 11:03:37 -0400
> message:
>   (open-tls-stream): Remove unneeded buffer contents when opening the connection.
>
> This change didn't break anything in my testing.  Please test so I can
> close the bug if it's fixed for you.
>
> Ted
>
> diff -u -L /usr/local/share/emacs/24.3/lisp/net/tls.el.gz -L
> \#\<buffer\ tls.el.gz\> /tmp/jka-com1909LVh
> /tmp/buffer-content-1909lpt
> --- /usr/local/share/emacs/24.3/lisp/net/tls.el.gz
> +++ #<buffer tls.el.gz>
> @@ -286,7 +286,11 @@
>                              (format "Host name in certificate doesn't \
>  match `%s'. Connect anyway? " host))))))
>         (setq done nil)
> -       (delete-process process)))
> +       (delete-process process))
> +      ;; delete all the informational messages that could
> +      ;; confuse futures users of `buffer'
> +      ;;
> +      (delete-region (point-min) (point)))
>      (message "Opening TLS connection to `%s'...%s"
>              host (if done "done" "failed"))
>      (when use-temp-buffer
>



-- 
João Távora





^ permalink raw reply	[flat|nested] 28+ messages in thread

* bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32
  2013-06-05 18:10               ` Ted Zlatanov
@ 2013-06-08  0:58                 ` Glenn Morris
  0 siblings, 0 replies; 28+ messages in thread
From: Glenn Morris @ 2013-06-08  0:58 UTC (permalink / raw)
  To: 14380

Ted Zlatanov wrote:

> GM> Thanks for applying it. If committing changes by others, please make the
> GM> ChangeLog entry in their name, and use commit --author.
>
> This must be a recent policy, or at least wasn't requested before.  Sorry.

The ChangeLog thing is very old. The --commit thing is less well
advertised, but must be several years old by now I should think.

> Should I edit the ChangeLog with the bug number or just leave it alone
> now that it's wrong?

It's always fine to improve ChangeLog entries after the fact.
(I changed it.)





^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2013-06-08  0:58 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <87k3mw79iv.fsf@lifelogs.com>
2013-05-18 13:05 ` bug#14380: [gmane.emacs.bugs] bug#14380: 24.3; `network-stream-open-tls' fails in some imap servers on w32 João Távora
2013-05-19  3:17   ` Ted Zlatanov
     [not found]   ` <87zjvr64lt.fsf_-_@lifelogs.com>
2013-05-19 11:45     ` João Távora
2013-05-19 15:32     ` Eli Zaretskii
2013-05-20 13:56       ` Ted Zlatanov
2013-05-20 16:28         ` Eli Zaretskii
2013-05-24 19:48           ` Ted Zlatanov
2013-05-24 20:27             ` Eli Zaretskii
2013-05-10 12:49               ` João Távora
2013-05-10 14:00                 ` Eli Zaretskii
2013-05-10 16:00                   ` João Távora
2013-05-10 17:17                     ` Eli Zaretskii
2013-05-10 20:44                       ` João Távora
2013-05-11  7:06                         ` Eli Zaretskii
2013-05-17 13:12                         ` Ted Zlatanov
2013-05-24 22:20                 ` Ted Zlatanov
2013-05-25  6:49                   ` Eli Zaretskii
     [not found]     ` <CALDnm50KHS7wOKUpQJQHb4V_PLfQs6VkjEsRmPgY=R5x0eEuUg@mail.gmail.com>
2013-05-19 15:44       ` Eli Zaretskii
2013-05-19 17:57         ` João Távora
2013-05-19 19:01           ` Eli Zaretskii
2013-05-19 23:05       ` Ted Zlatanov
     [not found]       ` <87txly4ll9.fsf@lifelogs.com>
2013-05-20  2:08         ` Juanma Barranquero
2013-05-20 22:07         ` João Távora
     [not found]         ` <CALDnm51-4aTee3NGoRp+=T6MuTV3xD9P4Ed1yZLU_E5k_r5hGw@mail.gmail.com>
2013-06-05 15:06           ` Ted Zlatanov
     [not found]           ` <87obbk1trn.fsf@lifelogs.com>
2013-06-05 16:42             ` Glenn Morris
2013-06-05 18:10               ` Ted Zlatanov
2013-06-08  0:58                 ` Glenn Morris
2013-06-06  8:15             ` João Távora

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).