Re: gnutls for win32

Re: gnutls for win32

[Ted Zlatanov]

On Sat, 31 Dec 2011 00:55:26 +0200 Nikos Mavrogiannopoulos <n.mavrogiannopoulos@gmail.com> wrote: 

NM> Hello all and best wishes for new year,
NM>  I've put pre-built win32 dlls and the other gnutls applications for
NM> 3.0.9 in [0]. I've managed to automate the procedure, so it could be
NM> that next releases (at least the major ones) will have the
NM> corresponding windows dlls released as well.

NM> [0]. http://homes.esat.kuleuven.be/~nikos/gnutls-win32/

Great news!  That lets GNU Emacs use the latest GnuTLS on all the
supported platforms.

Currently we're in a feature freeze for the Emacs pretest, but after the
next Emacs release we'll make GnuTLS 3.x required.  That also lets us
ensure that we're not fighting old, already-fixed bugs.

Could the W32 releases go on the official GnuTLS site so we can point to
them in the GNU Emacs installation notes?  And could you, when able,
publish the build script so developers can make their own W32 builds?

Thanks
Ted

Re: gnutls for win32

[Nikos Mavrogiannopoulos]

2011/12/31 Ted Zlatanov <tzz@lifelogs.com>:

> NM> Hello all and best wishes for new year,
> NM>  I've put pre-built win32 dlls and the other gnutls applications for
> NM> 3.0.9 in [0]. I've managed to automate the procedure, so it could be
> NM> that next releases (at least the major ones) will have the
> NM> corresponding windows dlls released as well.
> NM> [0]. http://homes.esat.kuleuven.be/~nikos/gnutls-win32/
> Great news!  That lets GNU Emacs use the latest GnuTLS on all the
> supported platforms.
> Currently we're in a feature freeze for the Emacs pretest, but after the
> next Emacs release we'll make GnuTLS 3.x required.  That also lets us
> ensure that we're not fighting old, already-fixed bugs.
> Could the W32 releases go on the official GnuTLS site so we can point to
> them in the GNU Emacs installation notes?  And could you, when able,
> publish the build script so developers can make their own W32 builds?

Hi Ted,
 The build script is on the repository (cross.mk). I'll see what's the
optimal release method of windows binaries.

regards,
Nikos

_______________________________________________
Help-gnutls mailing list
Help-gnutls@gnu.org
https://lists.gnu.org/mailman/listinfo/help-gnutls

Re: gnutls for win32

[Eli Zaretskii]

> Date: Sun, 1 Jan 2012 13:10:22 +0200
> From: Nikos Mavrogiannopoulos <nmav@gnutls.org>
> Cc: gnutls-devel@gnu.org, emacs-devel@gnu.org
> 
> 2011/12/31 Ted Zlatanov <tzz@lifelogs.com>:
> 
> > NM> Hello all and best wishes for new year,
> > NM>  I've put pre-built win32 dlls and the other gnutls applications for
> > NM> 3.0.9 in [0]. I've managed to automate the procedure, so it could be
> > NM> that next releases (at least the major ones) will have the
> > NM> corresponding windows dlls released as well.
> > NM> [0]. http://homes.esat.kuleuven.be/~nikos/gnutls-win32/
> > Great news!  That lets GNU Emacs use the latest GnuTLS on all the
> > supported platforms.
> > Currently we're in a feature freeze for the Emacs pretest, but after the
> > next Emacs release we'll make GnuTLS 3.x required.  That also lets us
> > ensure that we're not fighting old, already-fixed bugs.
> > Could the W32 releases go on the official GnuTLS site so we can point to
> > them in the GNU Emacs installation notes?  And could you, when able,
> > publish the build script so developers can make their own W32 builds?
> 
> Hi Ted,
>  The build script is on the repository (cross.mk). I'll see what's the
> optimal release method of windows binaries.

FWIW, I've built GnuTLS 3.0.9 natively on Windows, using MinGW and
MSYS.  I will make the resulting binaries available shortly, as soon
as I'm done testing it with Emacs.

Note that currently Emacs cannot use GnuTLS 3.0.9 on Windows, because
of a couple of minor incompatibilities in Emacs itself.  I'm working
on a solution as we speak.

Re: gnutls for win32

[Ted Zlatanov]

On Sun, 01 Jan 2012 06:50:12 -0500 Eli Zaretskii <eliz@gnu.org> wrote: 

EZ> FWIW, I've built GnuTLS 3.0.9 natively on Windows, using MinGW and
EZ> MSYS.  I will make the resulting binaries available shortly, as soon
EZ> as I'm done testing it with Emacs.

Do you want to work with Christoph Scholtes to make a GnuTLS W32 build
part of the Emacs W32 build?  That would be terrific and we wouldn't
have to tell our users to download an extra DLL every time.

That allows us to make the choice of whether we should keep in sync with
the latest GnuTLS or refresh occasionally.  Testing will be harder if we
keep in sync, but has obvious benefits for both projects.

EZ> Note that currently Emacs cannot use GnuTLS 3.0.9 on Windows, because
EZ> of a couple of minor incompatibilities in Emacs itself.  I'm working
EZ> on a solution as we speak.

Thanks!

Ted

Re: gnutls for win32

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Sun, 01 Jan 2012 09:13:35 -0500
> Cc: help-gnutls@gnu.org, gnutls-devel@gnu.org
> 
> On Sun, 01 Jan 2012 06:50:12 -0500 Eli Zaretskii <eliz@gnu.org> wrote: 
> 
> EZ> FWIW, I've built GnuTLS 3.0.9 natively on Windows, using MinGW and
> EZ> MSYS.  I will make the resulting binaries available shortly, as soon
> EZ> as I'm done testing it with Emacs.
> 
> Do you want to work with Christoph Scholtes to make a GnuTLS W32 build
> part of the Emacs W32 build?

Sorry, I don't understand: what is the "GnuTLS W32 build part of the
Emacs W32 build"?  Emacs on Windows can already be built with GnuTLS.

> That would be terrific and we wouldn't have to tell our users to
> download an extra DLL every time.

GnuTLS is more than just one DLL.

> That allows us to make the choice of whether we should keep in sync with
> the latest GnuTLS or refresh occasionally.  Testing will be harder if we
> keep in sync, but has obvious benefits for both projects.

Unless there are known serious bugs, I see no particular reason to
upgrade to the next GnuTLS version, let alone keep in sync all the
time.  It adds unnecessary overhead to people involved in providing
the binary packages.  (E.g., it took me 2 days of my precious time to
build GnuTLS with MinGW, due to various complications and outright
bugs in the GnuTLS configury.  Why would I need to go through that
every month or so?)  Staying a whole major revision behind is indeed
undesirable, which is why I made the effort of building the latest
release.

Re: gnutls for win32

[Ted Zlatanov]

On Sun, 01 Jan 2012 18:10:14 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Sun, 01 Jan 2012 09:13:35 -0500
>> Cc: help-gnutls@gnu.org, gnutls-devel@gnu.org
>> 
>> On Sun, 01 Jan 2012 06:50:12 -0500 Eli Zaretskii <eliz@gnu.org> wrote: 
>> 
EZ> FWIW, I've built GnuTLS 3.0.9 natively on Windows, using MinGW and
EZ> MSYS.  I will make the resulting binaries available shortly, as soon
EZ> as I'm done testing it with Emacs.
>> 
>> Do you want to work with Christoph Scholtes to make a GnuTLS W32 build
>> part of the Emacs W32 build?

EZ> Sorry, I don't understand: what is the "GnuTLS W32 build part of the
EZ> Emacs W32 build"?  Emacs on Windows can already be built with GnuTLS.

That sentence could be parsed both ways, sorry.  I meant: do you want to
work with Christoph to incorporate building GnuTLS into building Emacs
itself?  But it sounds like that's extra work you don't need :)

>> That would be terrific and we wouldn't have to tell our users to
>> download an extra DLL every time.

EZ> GnuTLS is more than just one DLL.

Yes, I mean we want to make the installation easier, that's all.  Right
now they have to get the GnuTLS binaries separately.

>> That allows us to make the choice of whether we should keep in sync with
>> the latest GnuTLS or refresh occasionally.  Testing will be harder if we
>> keep in sync, but has obvious benefits for both projects.

EZ> Unless there are known serious bugs, I see no particular reason to
EZ> upgrade to the next GnuTLS version, let alone keep in sync all the
EZ> time.  It adds unnecessary overhead to people involved in providing
EZ> the binary packages.  (E.g., it took me 2 days of my precious time to
EZ> build GnuTLS with MinGW, due to various complications and outright
EZ> bugs in the GnuTLS configury.  Why would I need to go through that
EZ> every month or so?)  Staying a whole major revision behind is indeed
EZ> undesirable, which is why I made the effort of building the latest
EZ> release.

OK.  In that case, we should build some tests of the GnuTLS
functionality in Emacs so upgrading is easier and less stressful.  I can
keep track of the GnuTLS releases and bump the Emacs support when we
agree it's worthwhile.  (You or anyone else can volunteer to do
this if you want that role...)

Thanks
Ted

Re: gnutls for win32

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Sun, 01 Jan 2012 11:38:33 -0500
> 
> I meant: do you want to work with Christoph to incorporate building
> GnuTLS into building Emacs itself?

I don't think this is practical.  GnuTLS is a very large project (as
you know only too well), building it requires quite a few other
packages (that are its dependencies), and doing so on Windows requires
to have MSYS and many supporting tools installed, in addition and
alongside the MinGW setup.  Asking people who build Emacs to have all
that is way too much, IMO.

On top of that, the Windows build of GnuTLS is not a fire-and-forget
thing, due to various problems I won't go into here.  E.g., the
default static+dynamic build simply fails on Windows.

> Yes, I mean we want to make the installation easier, that's all.  Right
> now they have to get the GnuTLS binaries separately.

I think downloading a single zip archive and unzipping it is much
easier than adding GnuTLS to Emacs.

> OK.  In that case, we should build some tests of the GnuTLS
> functionality in Emacs so upgrading is easier and less stressful.

That would be good, yes.  In particular, it would allow people who
don't normally use GnuTLS related features in Emacs to reliably
produce tested binaries.

> I can keep track of the GnuTLS releases and bump the Emacs support
> when we agree it's worthwhile.  (You or anyone else can volunteer to
> do this if you want that role...)

It should be enough to post a message saying that you recommend
upgrading for this-and-that reason.

TIA

Re: gnutls for win32

[Ted Zlatanov]

On Sun, 01 Jan 2012 19:05:43 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Sun, 01 Jan 2012 11:38:33 -0500
>> 
>> I meant: do you want to work with Christoph to incorporate building
>> GnuTLS into building Emacs itself?

EZ> I don't think this is practical.  GnuTLS is a very large project (as
EZ> you know only too well), building it requires quite a few other
EZ> packages (that are its dependencies), and doing so on Windows requires
EZ> to have MSYS and many supporting tools installed, in addition and
EZ> alongside the MinGW setup.  Asking people who build Emacs to have all
EZ> that is way too much, IMO.

EZ> On top of that, the Windows build of GnuTLS is not a fire-and-forget
EZ> thing, due to various problems I won't go into here.  E.g., the
EZ> default static+dynamic build simply fails on Windows.

OK.

>> Yes, I mean we want to make the installation easier, that's all.  Right
>> now they have to get the GnuTLS binaries separately.

EZ> I think downloading a single zip archive and unzipping it is much
EZ> easier than adding GnuTLS to Emacs.

For every Emacs user?  That seems like a pain.  Can we include it in the
binaries Christoph is putting out, even if it's not built with Emacs
itself?

>> OK.  In that case, we should build some tests of the GnuTLS
>> functionality in Emacs so upgrading is easier and less stressful.

EZ> That would be good, yes.  In particular, it would allow people who
EZ> don't normally use GnuTLS related features in Emacs to reliably
EZ> produce tested binaries.

OK.  Since Emacs won't include the GnuTLS DLLs on Windows, and even if
it did it doesn't have the GnuTLS server functionality, this will
require some extra work.  It should probably hit Savannah.

>> I can keep track of the GnuTLS releases and bump the Emacs support
>> when we agree it's worthwhile.  (You or anyone else can volunteer to
>> do this if you want that role...)

EZ> It should be enough to post a message saying that you recommend
EZ> upgrading for this-and-that reason.

OK.

Ted

Re: gnutls for win32

[Juanma Barranquero]

2012/1/1 Ted Zlatanov <tzz@lifelogs.com>:

> For every Emacs user?  That seems like a pain.

Not every Emacs user, and certainly not every Windows Emacs user,
needs GnuTLS. We also don't include (most) image libraries.

    Juanma

Re: gnutls for win32

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Sun, 01 Jan 2012 16:17:54 -0500
> 
> EZ> I think downloading a single zip archive and unzipping it is much
> EZ> easier than adding GnuTLS to Emacs.
> 
> For every Emacs user?

Only those who need it.  I'm not sure what fraction would that be, but
if it's not very large, including GnuTLS in the binary package would
unnecessarily punish too many people, by letting them download a
significantly larger file.

> Can we include it in the binaries Christoph is putting out, even if
> it's not built with Emacs itself?

We could, but see above.  We currently don't even include libxpm,
although it's much, much smaller.

Re: gnutls for lose32

[Richard Stallman]

A reminder: please don't use the abbreviation "win" to refer to
Windows.  Please change that whenever you see it.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

GnuTLS for W32 (was: gnutls for win32)

[Ted Zlatanov]

On Sun, 01 Jan 2012 23:40:00 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Sun, 01 Jan 2012 16:17:54 -0500
>> 
EZ> I think downloading a single zip archive and unzipping it is much
EZ> easier than adding GnuTLS to Emacs.
>> 
>> For every Emacs user?

EZ> Only those who need it.  I'm not sure what fraction would that be, but
EZ> if it's not very large, including GnuTLS in the binary package would
EZ> unnecessarily punish too many people, by letting them download a
EZ> significantly larger file.

>> Can we include it in the binaries Christoph is putting out, even if
>> it's not built with Emacs itself?

EZ> We could, but see above.  We currently don't even include libxpm,
EZ> although it's much, much smaller.

On Sun, 1 Jan 2012 22:28:58 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> Not every Emacs user, and certainly not every Windows Emacs user,
JB> needs GnuTLS. We also don't include (most) image libraries.

I think nowadays it's a really good idea to include GnuTLS to encrypt
network connections within Emacs.  Unlike image libraries, opening
network connections unencrypted or through tunneling programs can
actually expose secret user data and introduces many unpleasant bugs and
workarounds, *especially* on W32.  I think the download size is tiny
either way by today's standards, but I understand the bloat concern.
Can we download and install the GnuTLS opportunistically, e.g. through
a GNU ELPA package?

I am obviously biased, so I'd like Lars, Stefan, Chong, and any other
concerned developers and users to give their opinion.

Thanks
Ted

Re: GnuTLS for W32 (was: gnutls for win32)

[Juanma Barranquero]

2012/1/2 Ted Zlatanov <tzz@lifelogs.com>:

> I think nowadays it's a really good idea to include GnuTLS to encrypt
> network connections within Emacs.

I've been using Emacs on Windows for fourteen years, and the only
network connections I start from Emacs are between server.el and
emacsclient (all of them local). I bet there are many other Windows
users that do not read e-mail, use ftp, visit web pages etc. from
Emacs.

    Juanma

Re: GnuTLS for W32

[Óscar Fuentes]

Juanma Barranquero <lekktu@gmail.com> writes:

> 2012/1/2 Ted Zlatanov <tzz@lifelogs.com>:
>
>> I think nowadays it's a really good idea to include GnuTLS to encrypt
>> network connections within Emacs.
>
> I've been using Emacs on Windows for fourteen years, and the only
> network connections I start from Emacs are between server.el and
> emacsclient (all of them local). I bet there are many other Windows
> users that do not read e-mail, use ftp, visit web pages etc. from
> Emacs.

Security default settings must be appropriate for those who are at risk.

Re: GnuTLS for W32

[Juanma Barranquero]

On Mon, Jan 2, 2012 at 02:33, Óscar Fuentes <ofv@wanadoo.es> wrote:

> Security default settings must be appropriate for those who are at risk.

That assumes that those at risk are many, or the cost of these
defaults are small. We can now discuss (without any real data, I
think) just how many Windows users of Emacs are at risk, and how
convenient or inconvenient is to assume distributing the GnuTLS binary
vs. pointing the users to an alternative download place.

And, just to put things in perspective, until now Emacs had no GnuTLS
support and we haven't seen a flood of network-related security
reports from Windows users.

    Juanma

Re: GnuTLS for W32

[Óscar Fuentes]

Juanma Barranquero <lekktu@gmail.com> writes:

> On Mon, Jan 2, 2012 at 02:33, Óscar Fuentes <ofv@wanadoo.es> wrote:
>
>> Security default settings must be appropriate for those who are at risk.
>
> That assumes that those at risk are many,

Do we implement security only when many users are at risk?

> or the cost of these defaults are small.  We can now discuss (without
> any real data, I think) just how many Windows users of Emacs are at
> risk, and how convenient or inconvenient is to assume distributing the
> GnuTLS binary vs. pointing the users to an alternative download place.

Including the GnuTLS binary with the official binary packages shouldn't
be too costly, if we consider how rare Emacs releases are. As for the
other option, GnuTLS support could be prominently advertised, not just
listed as another item on NEWS.

> And, just to put things in perspective, until now Emacs had no GnuTLS
> support and we haven't seen a flood of network-related security
> reports from Windows users.

Shrugh. Security-wise, this way of thinking is responsible for lots of
disasters.

I wouldn't detect if someone were eavesdropping my network
communications, nor would you.

Re: GnuTLS for W32

[Juanma Barranquero]

> Do we implement security only when many users are at risk?

Irrelevant. We've implemented security, we're talking about defaults.
And that's what cost-benefit analysis is for. The answer could well be
yes, if the alternative to "many" is "almost no-one".

> Including the GnuTLS binary with the official binary packages shouldn't
> be too costly, if we consider how rare Emacs releases are.

The moment a serious bug is detected in GnuTLS, you have to issue
updated packages and get the word out. It's not as easy as you put it.

> As for the
> other option, GnuTLS support could be prominently advertised, not just
> listed as another item on NEWS.

Agreed.

> Shrugh. Security-wise, this way of thinking is responsible for lots of
> disasters.

For some definition of "lots", sure.

> I wouldn't detect if someone were eavesdropping my network
> communications, nor would you.

Considering that I'm in a very small, non-WiFi network behind a rather
paranoid firewall, trust me: if someone is eavesdropping my network,
Emacs is the lesser of my troubles.

    Juanma

Re: GnuTLS for W32

[Óscar Fuentes]

Juanma Barranquero <lekktu@gmail.com> writes:

>> Do we implement security only when many users are at risk?
>
> Irrelevant. We've implemented security, we're talking about defaults.

What's the difference?

> And that's what cost-benefit analysis is for. The answer could well be
> yes, if the alternative to "many" is "almost no-one".

You can count me on. See below.

>> Including the GnuTLS binary with the official binary packages shouldn't
>> be too costly, if we consider how rare Emacs releases are.
>
> The moment a serious bug is detected in GnuTLS, you have to issue
> updated packages and get the word out. It's not as easy as you put it.

Granted, that's a considerable side-effect. I've looked at the release
history for GnuTLS and there are lots of them. I don't how many contain
fixes for serious bugs, though.

[snip]

>> Shrugh. Security-wise, this way of thinking is responsible for lots of
>> disasters.
>
> For some definition of "lots", sure.

Directly or indirectly, almost all of them, I would say.

>> I wouldn't detect if someone were eavesdropping my network
>> communications, nor would you.
>
> Considering that I'm in a very small, non-WiFi network behind a rather
> paranoid firewall, trust me: if someone is eavesdropping my network,
> Emacs is the lesser of my troubles.

AFAIK Emacs can use GnuTLS for talking to the outside world too. SMTP,
for instance. There are ISPs (like the one I use) that offer both
encrypted and plain login on their mail servers. That's pretty serious
stuff.

Re: GnuTLS for W32

[Juanma Barranquero]

On Mon, Jan 2, 2012 at 04:18, Óscar Fuentes <ofv@wanadoo.es> wrote:

> What's the difference?

We have offered the tools and we're leaving the issue in the hands of the users.

> Granted, that's a considerable side-effect. I've looked at the release
> history for GnuTLS and there are lots of them. I don't how many contain
> fixes for serious bugs, though.

If we start distributing GnuTLS binaries, it's suddenly our (=
"someone's") work to know when a release fixes serious bugs and when
it does not.

> Directly or indirectly, almost all of them, I would say.

Yes, it's already clear that's your opinion. Another way of seeing it
is, we haven't seen many security reports because the users have not
had many security incidents.

> AFAIK Emacs can use GnuTLS for talking to the outside world too. SMTP,
> for instance. There are ISPs (like the one I use) that offer both
> encrypted and plain login on their mail servers.

Of course. And then we're back to square one. Does anyone know how
many people *on Windows* uses Emacs to connect to the wide, wild
world? How many of them read e-mail from Emacs, for example? I don't
(I very much doubt there's any reliable statistics), and if I had to
bet, I'd bet on the side of few.

    Juanma

Re: gnutls for lose32

[Paul Eggert]

On 01/01/12 14:32, Richard Stallman wrote:
> A reminder: please don't use the abbreviation "win" to refer to
> Windows.  Please change that whenever you see it.

Thanks for the reminder.  I looked in the Emacs source code
for instances of this problem, and proposed a patch to fix them
in Bug#10421.

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10421

There are some other instances that come from upstream code
in Gnulib; I'll propose fixes for them upstream.

Re: GnuTLS for W32 (was: gnutls for win32)

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Sun, 01 Jan 2012 18:54:56 -0500
> 
> I think nowadays it's a really good idea to include GnuTLS to encrypt
> network connections within Emacs.  Unlike image libraries, opening
> network connections unencrypted or through tunneling programs can
> actually expose secret user data and introduces many unpleasant bugs and
> workarounds, *especially* on W32.  I think the download size is tiny
> either way by today's standards, but I understand the bloat concern.
> Can we download and install the GnuTLS opportunistically, e.g. through
> a GNU ELPA package?

Why are we talking only about Windows?  Do packages of Emacs 24
development snapshots on GNU/Linux come with GnuTLS in the same
package?  If they do, then I agree that the Windows binaries should
also include GnuTLS; but if not, I don't see why Windows should be
special in this regard.  We don't want to confuse users by providing
functionality only on some platforms.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Mon, 02 Jan 2012 03:35:02 +0100
> 
> I wouldn't detect if someone were eavesdropping my network
> communications, nor would you.

I hope your wireless connections are encrypted on the network level,
then (mine are).  GnuTLS will encrypt only some of the traffic, but
not all of it.

Re: GnuTLS for W32

[Jason Rumney]

Eli Zaretskii <eliz@gnu.org> writes:

> Why are we talking only about Windows?  Do packages of Emacs 24
> development snapshots on GNU/Linux come with GnuTLS in the same
> package?

The Debian and Ubuntu packages depend on the following:

libgnutls26 (>= 2.9.11-0)

Windows does not have a package system that supports dependency, so it
cannot be compared directly.

Re: GnuTLS for W32

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> I hope your wireless connections are encrypted on the network level,
> then (mine are).

Hopefully you have disabled WPS.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: gnutls for lose32

[Carsten Mattner]

On Mon, Jan 2, 2012 at 7:55 AM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 01/01/12 14:32, Richard Stallman wrote:
>> A reminder: please don't use the abbreviation "win" to refer to
>> Windows.  Please change that whenever you see it.
>
> Thanks for the reminder.  I looked in the Emacs source code
> for instances of this problem, and proposed a patch to fix them
> in Bug#10421.
>
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10421
>
> There are some other instances that come from upstream code
> in Gnulib; I'll propose fixes for them upstream.

I'm not sure this is a good idea.
"win32" is used in the same way as "posix" to refer to a set of
platform APIs. "win" alone should be expanded.
"w32" is something which few users will imagine to grep for
when searching for it.

"win32" and "windows" look reasonable to me, while I agree that
"win" is not descriptive enough.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: Óscar Fuentes <ofv@wanadoo.es>,  emacs-devel@gnu.org
> Date: Mon, 02 Jan 2012 11:42:36 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I hope your wireless connections are encrypted on the network level,
> > then (mine are).
> 
> Hopefully you have disabled WPS.

Never even considered it as a possibility.

Re: gnutls for lose32

[Juanma Barranquero]

On Mon, Jan 2, 2012 at 11:46, Carsten Mattner
<carstenmattner@googlemail.com> wrote:

> I'm not sure this is a good idea.
> "win32" is used in the same way as "posix" to refer to a set of
> platform APIs. "win" alone should be expanded.

And yet, imagine that Microsoft were to add some program, library or
product called "Windows Win", and that we had to refer to it in the
documentation. What would we call it? "Windows Lose"? Really? And
would that really help the users in any way, or just confuse them?

I won't deny the power of words. But sometimes, a name is just a name.
The crusade against "win32" is a bit silly IMHO; it's hard to believe
that anyone, outside of a few old timers, reads it as "a form of
praise".

    Juanma

Re: gnutls for lose32

[Paul Eggert]

On 01/02/12 02:46, Carsten Mattner wrote:

> "win32" is used in the same way as "posix" to refer to a set of
> platform APIs.

That's one name, but it's not the only one and it's not necessarily
even the most common one in practice.  As discussed in
<http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10421#23>,
Microsoft prefers the name "Windows API" to "Win32 API",
and "Windows API" is quite commonly used.  Using this common
name to talk about the API helps us to avoid the problem with "win".

For Emacs's own identifiers, common practice is to use "w32_"
as a prefix; perhaps that should be changed to something else
at some point (when 64-bit Windows takes over?), but this should
be an independent issue.  The proposed patch by and large
leaves "w32" alone, since "w32" doesn't run afoul of the "win"
issue.

Re: GnuTLS for W32

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> I hope your wireless connections are encrypted on the network level,
> then (mine are).  GnuTLS will encrypt only some of the traffic, but
> not all of it.

With all the attacks on wifi security being revealed weekly, I would
consider wifi-level encryption a moot point now.  If you don't do
end-to-end encryption now (i.e. TLS), and you use wifi, then it's likely
that someone who's really interested can probably listen in to whatever
you're doing on the network.

Sort of.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Date: Mon, 02 Jan 2012 13:26:15 +0100
> Mail-Copies-To: never
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I hope your wireless connections are encrypted on the network level,
> > then (mine are).  GnuTLS will encrypt only some of the traffic, but
> > not all of it.
> 
> With all the attacks on wifi security being revealed weekly, I would
> consider wifi-level encryption a moot point now.

You mean, attacks on WPS "encryption"?  Or on home networks that let
anyone in the neighborhood connect and surf at will?

Don't dismiss wifi encryption and security so quickly, just because a
bunch of people with unknown interests wrote something in some blog,
or because too many people who don't have a clue leave their networks
wide open to anyone.

> If you don't do end-to-end encryption now (i.e. TLS), and you use
> wifi, then it's likely that someone who's really interested can
> probably listen in to whatever you're doing on the network.

I never said that wifi encryption is a replacement for other
encryptions, just that someone who cares about eavesdropping should
_also_ have her wifi encrypted.

Re: gnutls for lose32

[Carsten Mattner]

On Mon, Jan 2, 2012 at 1:17 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 01/02/12 02:46, Carsten Mattner wrote:
>
>> "win32" is used in the same way as "posix" to refer to a set of
>> platform APIs.
>
> That's one name, but it's not the only one and it's not necessarily
> even the most common one in practice.  As discussed in
> <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10421#23>,
> Microsoft prefers the name "Windows API" to "Win32 API",

I have a hard time to decipher/interpret the following unambiguously:
"(Note that this was formerly called the Win32 API. The name
Windows API more accurately reflects its roots in 16-bit Windows
and its support on 64-bit Windows.)"

> and "Windows API" is quite commonly used.  Using this common
> name to talk about the API helps us to avoid the problem with "win".

It's the same imprecise definition as for posix.

> For Emacs's own identifiers, common practice is to use "w32_"
> as a prefix; perhaps that should be changed to something else
> at some point (when 64-bit Windows takes over?), but this should
> be an independent issue.  The proposed patch by and large
> leaves "w32" alone, since "w32" doesn't run afoul of the "win"
> issue.

I never understood what win64 is with the same code written
for "win32" compiled for x86_64 being a "win64" binary.
win32 was probably introduced during the Windows 95/Windows NT
move to differentiate 16-bit APIs. They shouldn't have introduced
a new name in the first place, but it's one of the companies which
strives to give stuff a new name and sell it as something brand new.
A preprocessor definition or compiler switch would have been
enough. This is probably what you get when marketing is involved
when naming internal technical stuff.

Re: gnutls for lose32

[Carsten Mattner]

On Mon, Jan 2, 2012 at 12:51 PM, Juanma Barranquero <lekktu@gmail.com> wrote:
> On Mon, Jan 2, 2012 at 11:46, Carsten Mattner
> <carstenmattner@googlemail.com> wrote:
>
>> I'm not sure this is a good idea.
>> "win32" is used in the same way as "posix" to refer to a set of
>> platform APIs. "win" alone should be expanded.
>
> And yet, imagine that Microsoft were to add some program, library or
> product called "Windows Win", and that we had to refer to it in the
> documentation. What would we call it? "Windows Lose"? Really? And
> would that really help the users in any way, or just confuse them?
>
> I won't deny the power of words. But sometimes, a name is just a name.
> The crusade against "win32" is a bit silly IMHO; it's hard to believe
> that anyone, outside of a few old timers, reads it as "a form of
> praise".

Is the intent to hide or deny the existence of Windows?

In any case, I believe this wastes resources better applied for real
Emacs work.

Re: gnutls for lose32

[Juanma Barranquero]

On Mon, Jan 2, 2012 at 14:09, Carsten Mattner
<carstenmattner@googlemail.com> wrote:

> Is the intent to hide or deny the existence of Windows?

Belittle, I suppose.

> In any case, I believe this wastes resources better applied for real
> Emacs work.

Agreed.

    Juanma

Re: gnutls for lose32

[Juanma Barranquero]

On Mon, Jan 2, 2012 at 14:09, Carsten Mattner
<carstenmattner@googlemail.com> wrote:

> Is the intent to hide or deny the existence of Windows?

And do not forget all the references to "MS-DOG", which I personally
find both infantile and a bit insulting. If denying any supposed
positive inference in "win32" merits such effort, fighting the
negative connotations of that particular idiomatic use of "dog" would
be even worthier, as dogs are quite wonderful creatures (recent
archaeological findings have pushed the estimated domestication of
wolves into dogs back from 12,000 to 30,000+ years, so we've been
friends time enough to start showing them some respect...).

    Juanma


(Sorry: I cooperate with a rescue org specializing in saving Spanish
Greyhounds, which quite likely are the most abused breed in the world,
and see dogs disrespected and brutalized almost daily. I'm touchy in
all things dog.)

Re: GnuTLS for W32

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> You mean, attacks on WPS "encryption"?

WPS is not an encryption, but a configuration system with very weak
authentication.

(Did you mean WEP?)

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: GnuTLS for W32

[Ted Zlatanov]

On Mon, 2 Jan 2012 05:02:45 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> If we start distributing GnuTLS binaries, it's suddenly our (=
JB> "someone's") work to know when a release fixes serious bugs and when
JB> it does not.

I stated I would do that monitoring.  In any case, it's better to have
the *ability* to issue updates than not to.  In that context, it seems
more sensible to package GnuTLS support as an ELPA package so it can be
upgraded without upgrading Emacs as a whole.  So perhaps we just need a
versioned "gnutls-w32" ELPA package to DTRT.  W32 users can choose to
install it or it can be installed by default, whichever we or the
distribution decides.  We can do similar solutions for other platforms
that need it, but I think W32 is the only one.

>> AFAIK Emacs can use GnuTLS for talking to the outside world too. SMTP,
>> for instance. There are ISPs (like the one I use) that offer both
>> encrypted and plain login on their mail servers.

JB> Of course. And then we're back to square one. Does anyone know how
JB> many people *on Windows* uses Emacs to connect to the wide, wild
JB> world? How many of them read e-mail from Emacs, for example? I don't
JB> (I very much doubt there's any reliable statistics), and if I had to
JB> bet, I'd bet on the side of few.

I'd guess we have at least 100 W32 users.  That number will certainly
grow if Emacs can offer better foolproof connectivity.  If you ever
tried to set up CLI helpers for W32 Emacs network connectivity before
GnuTLS came along, you know it was a big deterrent to adoption.

On Mon, 02 Jan 2012 03:47:03 -0500 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Sun, 01 Jan 2012 18:54:56 -0500
>> 
>> I think nowadays it's a really good idea to include GnuTLS to encrypt
>> network connections within Emacs.  Unlike image libraries, opening
>> network connections unencrypted or through tunneling programs can
>> actually expose secret user data and introduces many unpleasant bugs and
>> workarounds, *especially* on W32.  I think the download size is tiny
>> either way by today's standards, but I understand the bloat concern.
>> Can we download and install the GnuTLS opportunistically, e.g. through
>> a GNU ELPA package?

EZ> Why are we talking only about Windows?  Do packages of Emacs 24
EZ> development snapshots on GNU/Linux come with GnuTLS in the same
EZ> package?  If they do, then I agree that the Windows binaries should
EZ> also include GnuTLS; but if not, I don't see why Windows should be
EZ> special in this regard.  We don't want to confuse users by providing
EZ> functionality only on some platforms.

Absolutely; my post was about GnuTLS in general but so far, W32
packaging has been the issue (which was remedied by Nicos' work, which
started this thread).  Mac OS X and GNU/Linux packaging for GnuTLS is
not a problem.

Ted

Re: gnutls for lose32

[Ted Zlatanov]

On Mon, 2 Jan 2012 14:09:17 +0100 Carsten Mattner <carstenmattner@googlemail.com> wrote: 

CM> In any case, I believe this wastes resources better applied for real
CM> Emacs work.

I think we should stick to W32 because, so far, we have.

That way we can search docs and discussions for "W32" reliably.

Ted

Re: GnuTLS for W32

[Juanma Barranquero]

2012/1/2 Ted Zlatanov <tzz@lifelogs.com>:

> I stated I would do that monitoring.  In any case, it's better to have
> the *ability* to issue updates than not to.

We already have the ability to issue updates, because any update is
just getting the info to the user (whether they download a binary
tarball from us or somewhere else, the only real trouble is getting
the user to know that there's a security issue to be fixed in the
first place). In that regard, announcing a new release or announcing
that the user should upgrade their GnuTLS binary is largely
irrelevant.

But anyway, the issue is not just monitoring. Someone has to build
updated binary GnuTLS packages for Windows. That Eli just did it does
not mean we can burden him with the task forever.

> In that context, it seems
> more sensible to package GnuTLS support as an ELPA package so it can be
> upgraded without upgrading Emacs as a whole.  So perhaps we just need a
> versioned "gnutls-w32" ELPA package to DTRT.  W32 users can choose to
> install it or it can be installed by default, whichever we or the
> distribution decides.

I think my objections regarding the burden we heap upon ourselves are
equally valid (or not) for an ELPA package, with the only mitigating
factor that at least in that case a security upgrade does not force a
reissue of Emacs tarballs.

> I'd guess we have at least 100 W32 users.

Worldwide? ;-)

> That number will certainly
> grow if Emacs can offer better foolproof connectivity.  If you ever
> tried to set up CLI helpers for W32 Emacs network connectivity before
> GnuTLS came along, you know it was a big deterrent to adoption.

I suppose you're talking of corporate environments or the like. I
still find hard to believe that's a representative sample of Windows
users.

    Juanma

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: Lars Magne Ingebrigtsen <larsi@gnus.org>,  emacs-devel@gnu.org
> Date: Mon, 02 Jan 2012 15:03:15 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > You mean, attacks on WPS "encryption"?
> 
> WPS is not an encryption, but a configuration system with very weak
> authentication.

Right, thus the quotes.  I guess the joke wasn't a successful one...

> (Did you mean WEP?)

That, too.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Mon, 02 Jan 2012 11:16:34 -0500
> 
> On Mon, 2 Jan 2012 05:02:45 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 
> 
> JB> If we start distributing GnuTLS binaries, it's suddenly our (=
> JB> "someone's") work to know when a release fixes serious bugs and when
> JB> it does not.
> 
> I stated I would do that monitoring.

Yes, thanks.

> In any case, it's better to have the *ability* to issue updates than
> not to.  In that context, it seems more sensible to package GnuTLS
> support as an ELPA package so it can be upgraded without upgrading
> Emacs as a whole.

It makes sense, but how is ELPA easier than any other download
address?  All that's needed is to download a single zip archive and
unzip it.  Why does it matter if it comes from ELPA or from elsewhere?

>  So perhaps we just need a versioned "gnutls-w32" ELPA package to
> DTRT.  W32 users can choose to install it or it can be installed by
> default, whichever we or the distribution decides.

The w32 "distribution" is just a single zip file which you unzip
wherever you want it, and add the bin subdirectory to PATH.  A zip
file cannot "decide" anything.

The alternatives are:

 . have GnuTLS as part of the binary zip

 . have GnuTLS as a separate zip alongside the binary zip (we do this
   for libxpm)

 . have GnuTLS available for download from some other address

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Mon, 02 Jan 2012 11:16:34 -0500
> 
> so far, W32 packaging has been the issue (which was remedied by
> Nicos' work, which started this thread).

I've put the w32 binary package I compiled myself here:

  http://sourceforge.net/projects/ezwinports/files/gnutls-3.0.9-w32-bin.zip/download

Christoph helped to verify this package works with the current trunk,
including if the binary was built with headers from the older 2.10
version that is in widespread use on Windows.

Re: GnuTLS for W32

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> The w32 "distribution" is just a single zip file which you unzip
> wherever you want it, and add the bin subdirectory to PATH.  A zip
> file cannot "decide" anything.
>
> The alternatives are:
>
>  . have GnuTLS as part of the binary zip

Having the network connections from Emacs be encrypted has traditionally
been a tricky issue.  The user had to install additional binaries and do
additional manual configuration to make stuff work.

With Emacs 24, almost all of the "built-in" interfaces (web, mail, news,
etc) now do encryption by default, if possible.

On Linux, it sounds like the major distributions, at least, are going to
make Emacs depend on libgnutls, which means that most Linux users are
going to start seeing encryption working by default, with no
configuration.

So the question is: What do we want to do here about the binary
distributions that we do ourselves?  (And by "ourselves", I mean you,
obviously.  :-)

If the Windows version of Emacs 24 is the only one that doesn't have
network encryption enabled by default -- wouldn't that be less than
ideal?  If it's easy and possible to package up the gnutls libraries in
the distribution, wouldn't that be a win, overall?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

RE: gnutls for lose32

[Drew Adams]

> And do not forget all the references to "MS-DOG", which I personally
> find both infantile and a bit insulting. If denying any supposed
> positive inference in "win32" merits such effort, fighting the
> negative connotations of that particular idiomatic use of "dog" would
> be even worthier, as dogs are quite wonderful creatures (recent
> archaeological findings have pushed the estimated domestication of
> wolves into dogs back from 12,000 to 30,000+ years, so we've been
> friends time enough to start showing them some respect...).
> 
>     Juanma
> 
> (Sorry: I cooperate with a rescue org specializing in saving Spanish
> Greyhounds, which quite likely are the most abused breed in the world,
> and see dogs disrespected and brutalized almost daily. I'm touchy in
> all things dog.)

Love it.  I think it was the following documentary that pointed out the
possibility that humans might never have moved beyond hunter-gatherer to settled
agriculture and animal domestication were it not for dogs.  If true, that's
pretty important!

http://www.pbs.org/wgbh/nova/nature/dogs-decoded.html

 - Drew, whom no one on the Internet can tell is a dog

RE: gnutls for lose32

[Drew Adams]

> The crusade against "win32" is a bit silly IMHO; it's hard to believe
> that anyone, outside of a few old timers, reads it as "a form of
> praise".

+1, from one old-timer

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10421#26

Re: GnuTLS for W32

[Ted Zlatanov]

On Mon, 02 Jan 2012 19:39:12 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Mon, 02 Jan 2012 11:16:34 -0500
>> 
>> In any case, it's better to have the *ability* to issue updates than
>> not to.  In that context, it seems more sensible to package GnuTLS
>> support as an ELPA package so it can be upgraded without upgrading
>> Emacs as a whole.

EZ> It makes sense, but how is ELPA easier than any other download
EZ> address?  All that's needed is to download a single zip archive and
EZ> unzip it.  Why does it matter if it comes from ELPA or from
EZ> elsewhere?

ELPA updates can be done entirely from within Emacs.  Downloading and
unzipping a file is a nuisance.  Imagine if you asked GNU/Linux users to
do the same.  "But they have package managers..."  Exactly!  And so does
Emacs, now that package.el is here!  So let's use packages for
non-trivial work.

>> So perhaps we just need a versioned "gnutls-w32" ELPA package to
>> DTRT.  W32 users can choose to install it or it can be installed by
>> default, whichever we or the distribution decides.

EZ> The alternatives are:

EZ>  . have GnuTLS as part of the binary zip

EZ>  . have GnuTLS as a separate zip alongside the binary zip (we do this
EZ>    for libxpm)

EZ>  . have GnuTLS available for download from some other address

I think the right solution is to ask the user "you don't have the GnuTLS
package for W32, do you want to get it?" and then do it through ELPA,
when they try to use HTTPS for instance.  The same, actually, should be
done with libxpm and any other add-on DLLs.  So this work could become
much more useful than just for GnuTLS.

Can Emacs modify the DLL search path on W32 so it can install some DLL
from ELPA and then activate it dynamically?  Or does it require a
restart and modifying the global PATH?  Either way, can the process be
automated?

On Mon, 2 Jan 2012 18:31:24 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> 2012/1/2 Ted Zlatanov <tzz@lifelogs.com>:

>> I stated I would do that monitoring.  In any case, it's better to have
>> the *ability* to issue updates than not to.

JB> We already have the ability to issue updates, because any update is
JB> just getting the info to the user (whether they download a binary
JB> tarball from us or somewhere else, the only real trouble is getting
JB> the user to know that there's a security issue to be fixed in the
JB> first place). In that regard, announcing a new release or announcing
JB> that the user should upgrade their GnuTLS binary is largely
JB> irrelevant.

ELPA, however, can show the user all the packages that need to be
updated in one place, so they don't have to check for GnuTLS updates on
W32 specifically.  It should be set up to check on startup, IMHO.
That's a win (heh heh) for everyone, and fits with what users expect
nowadays, compared with other extensible software like Firefox and
Chrome.

JB> But anyway, the issue is not just monitoring. Someone has to build
JB> updated binary GnuTLS packages for Windows. That Eli just did it does
JB> not mean we can burden him with the task forever.

We can automate that.  Eli and Nicos have done the hard part, I think.
Ditto for an ELPA package, if we can do it once it can be automated.

Ted

Re: gnutls for lose32

[Richard Stallman]

    Is the intent to hide or deny the existence of Windows?

Our policy is not to use "win" as a reference to "windows".

Once you learn to write "w32" or "Windows32" or "Windows API",
it takes no extra effort.  So please do it.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: GnuTLS for W32

[Óscar Fuentes]

Ted Zlatanov <tzz@lifelogs.com> writes:

[snip]

> Can Emacs modify the DLL search path on W32 so it can install some DLL
> from ELPA and then activate it dynamically?  Or does it require a
> restart and modifying the global PATH?  Either way, can the process be
> automated?

The relevant MS Windows API function (LoadLibrary) accepts a full
pathname, so no need for restarts nor changing PATH.

[snip]

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Tue, 03 Jan 2012 01:48:20 +0100
> 
> > Can Emacs modify the DLL search path on W32 so it can install some DLL
> > from ELPA and then activate it dynamically?  Or does it require a
> > restart and modifying the global PATH?  Either way, can the process be
> > automated?
> 
> The relevant MS Windows API function (LoadLibrary) accepts a full
> pathname

That's a factual truth, but it would be a grave mistake on our part to
use absolute file names for loading dynamic libraries, because it will
mean a major inconvenience to users.  It is hard on Windows to pick up
a fixed directory where every user could easily put the library: the
only directories that are guaranteed to exist on every Windows system
are frequently locked up by security policies, the only disk drive
guaranteed to exist can be a remote drive or even a read-only drive,
etc.  It would be a step in the wrong direction.

Besides, even if we did follow this path, it wouldn't solve the
problem, because:

> so no need for restarts nor changing PATH.

There's much more to loading a new DLL in the middle of a session than
just the location of the new DLL.  I will address that in a separate
message.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Mon, 02 Jan 2012 17:35:21 -0500
> 
> EZ> It makes sense, but how is ELPA easier than any other download
> EZ> address?  All that's needed is to download a single zip archive and
> EZ> unzip it.  Why does it matter if it comes from ELPA or from
> EZ> elsewhere?
> 
> ELPA updates can be done entirely from within Emacs.

See below: currently, using this for anything not Lisp-based, like
installing a new dynamic library, is a pipe dream.  (Actually, even
downloading a new version of a Lisp library into a running session has
its problems, as discussed here recently; but I digress.)

> Downloading and unzipping a file is a nuisance.  Imagine if you
> asked GNU/Linux users to do the same.  "But they have package
> managers..."  Exactly!  And so does Emacs, now that package.el is
> here!

And I thought only Windows users were spoiled enough to demand
one-click installations.  "Downloading and unzipping is a
nuisance"... sheesh.

> Can Emacs modify the DLL search path on W32 so it can install some DLL
> from ELPA and then activate it dynamically?  Or does it require a
> restart and modifying the global PATH?  Either way, can the process be
> automated?

We currently lack infrastructure to replace a DLL that is already used
by a running Emacs process.  The code that loads a DLL and initializes
the function pointers used to access its APIs assumes this will be
done only once in a given session, so currently we do need a restart.

To be able to replace a DLL without restarting Emacs, we would need to
add code to free and unload a used DLL from the Emacs process, and
then load the new one.  This might be complicate if there are data
structures lying around that use the DLL facilities, like an existing
connection using GnuTLS or an image displayed using an image DLL.  We
will need to shut down the relevant Emacs features before replacing
the DLL.

There will also be complications with this if another Emacs process
runs the same binary and uses the same DLL, because I don't think you
can overwrite the DLL from under the feet of that other process.

If you are thinking about a limited goal of installing for the first
time a DLL that does not yet exist and is not loaded into Emacs, then
it can be done, although some code will need to be written for that as
well, and I'm not sure package.el is designed to handle such
"packages".  Moreover, I'm not sure such a limited goal would be in
the spirit of package managing, since upgrading an installed package
is an important part of that, perhaps more important than the initial
installation.

> JB> But anyway, the issue is not just monitoring. Someone has to build
> JB> updated binary GnuTLS packages for Windows. That Eli just did it does
> JB> not mean we can burden him with the task forever.
> 
> We can automate that.

Who exactly are "we" in this sentence?

> Eli and Nicos have done the hard part, I think.

I did nothing of the kind.  I just configured and built the stock
distribution, and fought whatever problems I found that prevented the
build and the test suite to run to completion.  The record of that
fight includes 18 "issues" which I will be reporting to GnuTLS
developers soon; until those are resolved, the build I did is anything
but an automatic thing.  And I don't see how anyone else's build can
be fully automated, unless the "issues" I bumped into are due to my
own misunderstandings.

In any case, my experience indicates that having a steady feed of
regular updates of Windows builds of any non-trivial package can never
be based on a single individual, no matter if his/her name is Eli,
Nicos, or something else.  People invest in this some time for as long
as they have it or are interested, and then go away to pursue other
interests or go on with their lives.  You cannot build a reliable
infrastructure on something that is not part of your project.

Bottom line, I feel that letting users download and unzip DLLs is by
far more practical for this purpose than what you suggest.  It also
has the advantage of already working.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Mon, 02 Jan 2012 17:35:21 -0500
> 
> We can automate that.  Eli and Nicos have done the hard part, I think.

Btw, for some reason Nicos disables creation of gnutls-openssl
library, and I don't understand why.  Doesn't Emacs need it?

(My primary motivation for building GnuTLS was to compile the latest
wget, btw.)

Re: GnuTLS for W32

[Ted Zlatanov]

On Tue, 03 Jan 2012 02:14:19 -0500 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Mon, 02 Jan 2012 17:35:21 -0500
>> 
EZ> It makes sense, but how is ELPA easier than any other download
EZ> address?  All that's needed is to download a single zip archive and
EZ> unzip it.  Why does it matter if it comes from ELPA or from
EZ> elsewhere?
>> 
>> ELPA updates can be done entirely from within Emacs.

EZ> See below: currently, using this for anything not Lisp-based, like
EZ> installing a new dynamic library, is a pipe dream.  (Actually, even
EZ> downloading a new version of a Lisp library into a running session has
EZ> its problems, as discussed here recently; but I digress.)

You're taking current technical limitations and applying them to a UI
comparison.  The package.el UI is far superior to a download+unzip
process, especially if that process needs to happen more than once and
for more than one package.

>> Downloading and unzipping a file is a nuisance.  Imagine if you
>> asked GNU/Linux users to do the same.  "But they have package
>> managers..."  Exactly!  And so does Emacs, now that package.el is
>> here!

EZ> And I thought only Windows users were spoiled enough to demand
EZ> one-click installations.  "Downloading and unzipping is a
EZ> nuisance"... sheesh.

It's 2012.  Emacs must be compared with Firefox and Chrome, not with
vi/vim.

If you don't see how passing the download+unzip burden to the user is a
nuisance to them and consider it "spoiling," I think that's unfortunate.

>> Can Emacs modify the DLL search path on W32 so it can install some DLL
>> from ELPA and then activate it dynamically?  Or does it require a
>> restart and modifying the global PATH?  Either way, can the process be
>> automated?

EZ> We currently lack infrastructure to replace a DLL that is already used
EZ> by a running Emacs process.  The code that loads a DLL and initializes
EZ> the function pointers used to access its APIs assumes this will be
EZ> done only once in a given session, so currently we do need a restart.

EZ> To be able to replace a DLL without restarting Emacs, we would need to
EZ> add code to free and unload a used DLL from the Emacs process, and
EZ> then load the new one.  This might be complicate if there are data
EZ> structures lying around that use the DLL facilities, like an existing
EZ> connection using GnuTLS or an image displayed using an image DLL.  We
EZ> will need to shut down the relevant Emacs features before replacing
EZ> the DLL.

EZ> There will also be complications with this if another Emacs process
EZ> runs the same binary and uses the same DLL, because I don't think you
EZ> can overwrite the DLL from under the feet of that other process.

EZ> If you are thinking about a limited goal of installing for the first
EZ> time a DLL that does not yet exist and is not loaded into Emacs, then
EZ> it can be done, although some code will need to be written for that as
EZ> well, and I'm not sure package.el is designed to handle such
EZ> "packages".  Moreover, I'm not sure such a limited goal would be in
EZ> the spirit of package managing, since upgrading an installed package
EZ> is an important part of that, perhaps more important than the initial
EZ> installation.

OK, so let's do a restart after a DLL package install or upgrade.  It's
better than nothing.  Can this process be reused for other DLLs like the
libxpm DLL?  Can we activate the new DLL after Emacs restarts, so the
old one will remain active as long as Emacs is using it and the user is
not forced to restart immediately?

I'm not sure if multiple Emacs processes are an important use case.  I
would guess it's an edge case on W32 and can probably be handled by
scanning the process table and insisting that all Emacs instances be
shut down.

JB> But anyway, the issue is not just monitoring. Someone has to build
JB> updated binary GnuTLS packages for Windows. That Eli just did it does
JB> not mean we can burden him with the task forever.
>> 
>> We can automate that.

EZ> Who exactly are "we" in this sentence?

emacs-devel; at least me specifically.

>> Eli and Nicos have done the hard part, I think.

EZ> I did nothing of the kind.  I just configured and built the stock
EZ> distribution, and fought whatever problems I found that prevented the
EZ> build and the test suite to run to completion.  The record of that
EZ> fight includes 18 "issues" which I will be reporting to GnuTLS
EZ> developers soon; until those are resolved, the build I did is anything
EZ> but an automatic thing.  And I don't see how anyone else's build can
EZ> be fully automated, unless the "issues" I bumped into are due to my
EZ> own misunderstandings.

EZ> In any case, my experience indicates that having a steady feed of
EZ> regular updates of Windows builds of any non-trivial package can never
EZ> be based on a single individual, no matter if his/her name is Eli,
EZ> Nicos, or something else.  People invest in this some time for as long
EZ> as they have it or are interested, and then go away to pursue other
EZ> interests or go on with their lives.  You cannot build a reliable
EZ> infrastructure on something that is not part of your project.

I'm not sure what you and Juanma want to say.  That because we don't
have salaried positions for tracking GnuTLS updates to Emacs, it's a
burden?  Or that we shouldn't even do it?  Or that we need more people
to run the Emacs infrastructure?  What's the problem and how can we
solve it?

EZ> Bottom line, I feel that letting users download and unzip DLLs is by
EZ> far more practical for this purpose than what you suggest.  It also
EZ> has the advantage of already working.

It certainly is easier for us, the developers.  But I think it's not the
best solution we can offer to the users.

Ted

Re: GnuTLS for W32

[Ted Zlatanov]

On Tue, 03 Jan 2012 02:48:19 -0500 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Mon, 02 Jan 2012 17:35:21 -0500
>> 
>> We can automate that.  Eli and Nicos have done the hard part, I think.

EZ> Btw, for some reason Nicos disables creation of gnutls-openssl
EZ> library, and I don't understand why.  Doesn't Emacs need it?

That's just the OpenSSL compatibility layer and Emacs doesn't currently
use it AFAIK.

Ted

Re: GnuTLS for W32

[Juanma Barranquero]

2012/1/3 Ted Zlatanov <tzz@lifelogs.com>:

> I'm not sure what you and Juanma want to say.  That because we don't
> have salaried positions for tracking GnuTLS updates to Emacs, it's a
> burden?  Or that we shouldn't even do it?  Or that we need more people
> to run the Emacs infrastructure?  What's the problem and how can we
> solve it?

What I want to say is that is a problem already fixed elsewhere, and I
see no point in diverting resources to fix it again poorly.

> It certainly is easier for us, the developers.  But I think it's not the
> best solution we can offer to the users.

The best thing we can offer to the users is invest our resources in
developing Emacs, not packaging what does not really need to be
packaged.

    Juanma

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Tue, 03 Jan 2012 08:06:51 -0500
> 
> On Tue, 03 Jan 2012 02:14:19 -0500 Eli Zaretskii <eliz@gnu.org> wrote: 
> 
> OK, so let's do a restart after a DLL package install or upgrade.  It's
> better than nothing.  Can this process be reused for other DLLs like the
> libxpm DLL?  Can we activate the new DLL after Emacs restarts, so the
> old one will remain active as long as Emacs is using it and the user is
> not forced to restart immediately?

We can, although it'd probably still need some code changes.  E.g.,
displaying the splash image at startup will automatically load an
image library, and we will have to defer that until after the DLL is
replaced.  (Assuming I understand you correctly that you propose to
run package.el in the new, restarted instance of Emacs and replace the
DLL before it is first used.)

> I'm not sure if multiple Emacs processes are an important use case.  I
> would guess it's an edge case on W32

Happens to me all the time when I'm working on some bug and have the
"regular" Emacs running alongside.  Also, I understand that some
people who use Gnus have it run in a separate session.

> I'm not sure what you and Juanma want to say.  That because we don't
> have salaried positions for tracking GnuTLS updates to Emacs, it's a
> burden?  Or that we shouldn't even do it?  Or that we need more people
> to run the Emacs infrastructure?  What's the problem and how can we
> solve it?

What I want to say is this:

> EZ> Bottom line, I feel that letting users download and unzip DLLs is by
> EZ> far more practical for this purpose than what you suggest.  It also
> EZ> has the advantage of already working.

What you propose is a laudable goal, but I think it has practical
complications that make its gain not worth the effort.  Please keep in
mind that Windows users of Emacs already know how to download and
unzip a distribution, because that's how they install Emacs in the
first place.

That said, if someone is motivated enough to invest the effort, I'll
applaud them.

Re: GnuTLS for W32

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> > Can Emacs modify the DLL search path on W32 so it can install some DLL
>> > from ELPA and then activate it dynamically?  Or does it require a
>> > restart and modifying the global PATH?  Either way, can the process be
>> > automated?
>> 
>> The relevant MS Windows API function (LoadLibrary) accepts a full
>> pathname
>
> That's a factual truth, but it would be a grave mistake on our part to
> use absolute file names for loading dynamic libraries, because it will
> mean a major inconvenience to users.  It is hard on Windows to pick up
> a fixed directory where every user could easily put the library: the
> only directories that are guaranteed to exist on every Windows system
> are frequently locked up by security policies, the only disk drive
> guaranteed to exist can be a remote drive or even a read-only drive,
> etc.  It would be a step in the wrong direction.

You are providing reasons for the package approach: if it is hard for
the user to put the dll in the correct directory, let Emacs do it.

> Besides, even if we did follow this path, it wouldn't solve the
> problem, because:
>
>> so no need for restarts nor changing PATH.
>
> There's much more to loading a new DLL in the middle of a session than
> just the location of the new DLL.  I will address that in a separate
> message.

I was not proposing to unload the old dll and load the new one on the
fly (BTW, I was not prososing anything at all, just providing technical
info). I know very well that that is unfeasible in general. What is
possible: if GnuTLS is absent and the user wants it, download the dll
and start using it right away. If GnuTLS needs to be upgraded, advertise
the fact to the user, ask for his permission, and proceed to
download(*), with an Emacs restart at the end (or some variation of this
sequence of actions).

* DLLs which are in use are not file-writable. Either the dll must be
  unloaded first (which may be unfeasible) or the new version must be
  downloaded to a temporary location and moved to the final place later.

Re: GnuTLS for W32

[Jason Rumney]

Eli Zaretskii <eliz@gnu.org> writes:

> The w32 "distribution" is just a single zip file which you unzip
> wherever you want it, and add the bin subdirectory to PATH.  A zip
> file cannot "decide" anything.
>
> The alternatives are:
>
>  . have GnuTLS as part of the binary zip
>
>  . have GnuTLS as a separate zip alongside the binary zip (we do this
>    for libxpm)

Not quite true. libxpm.dll is included in the binary zip for emacs. The
separate zip for libxpm is the sources, since we need to distribute the
sources for libraries we distribute according to the GPL (libxpm is
under a more permissive license I think, but since we distribute it as
part of Emacs, we have an obligation to follow GPL as well).

>  . have GnuTLS available for download from some other address

Re: GnuTLS for W32

[Ted Zlatanov]

On Tue, 03 Jan 2012 09:02:17 -0500 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Tue, 03 Jan 2012 08:06:51 -0500
>> 
>> On Tue, 03 Jan 2012 02:14:19 -0500 Eli Zaretskii <eliz@gnu.org> wrote: 
>> 
>> OK, so let's do a restart after a DLL package install or upgrade.  It's
>> better than nothing.  Can this process be reused for other DLLs like the
>> libxpm DLL?  Can we activate the new DLL after Emacs restarts, so the
>> old one will remain active as long as Emacs is using it and the user is
>> not forced to restart immediately?

EZ> We can, although it'd probably still need some code changes.  E.g.,
EZ> displaying the splash image at startup will automatically load an
EZ> image library, and we will have to defer that until after the DLL is
EZ> replaced.  (Assuming I understand you correctly that you propose to
EZ> run package.el in the new, restarted instance of Emacs and replace the
EZ> DLL before it is first used.)

The mechanism is not too important from a UI perspective, as long as it
DTRT, and I don't know the W32 platform well enough to specify how it
should operate.  If you think replacing the DLL is possible entirely
from within Emacs, I'm OK with that.

As a first cut, could we have a gnutls-w32 package, version 3.09
currently, which when activated will download and install the 3.09
GnuTLS DLL if it's missing?  Where should this DLL be written?

Does W32 have APIs or mechanisms to update DLLs safely?

As an alternate solution, could GnuTLS itself have an installer and an
updater?

>> I'm not sure if multiple Emacs processes are an important use case.  I
>> would guess it's an edge case on W32

EZ> Happens to me all the time when I'm working on some bug and have the
EZ> "regular" Emacs running alongside.  Also, I understand that some
EZ> people who use Gnus have it run in a separate session.

Emacs developers are always an edge case :)

>> I'm not sure what you and Juanma want to say.  That because we don't
>> have salaried positions for tracking GnuTLS updates to Emacs, it's a
>> burden?  Or that we shouldn't even do it?  Or that we need more people
>> to run the Emacs infrastructure?  What's the problem and how can we
>> solve it?

EZ> What I want to say is this:

EZ> Bottom line, I feel that letting users download and unzip DLLs is by
EZ> far more practical for this purpose than what you suggest.  It also
EZ> has the advantage of already working.

Unlike other libraries we support with Emacs, we have a responsibility
to the user to keep GnuTLS updated.  The worst case is that their
security is compromised, not that Emacs won't run.

On W32 this is a worse problem than anywhere else IIUC (Mac OS X has
MacPorts and GNU/Linux has a billion package managers).  So, we either
tell the user to download and unzip the GnuTLS DLLs every time there's a
critical fix, or we do it automatically for them from the package.el UI,
or we pass the responsibility to the GnuTLS developers to write an
updater.  I don't think we have the option, as responsible developers,
of just ignoring that responsibility.

On Tue, 3 Jan 2012 14:37:00 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> What I want to say is that is a problem already fixed elsewhere, and I
JB> see no point in diverting resources to fix it again poorly.

The problem of pushing out a critical GnuTLS fix to Emacs users on W32
is not fixed AFAIK.  We just have a DLL in a zip file.

Ted

Re: GnuTLS for W32

[Juanma Barranquero]

2012/1/3 Ted Zlatanov <tzz@lifelogs.com>:

> The problem of pushing out a critical GnuTLS fix to Emacs users on W32
> is not fixed AFAIK.  We just have a DLL in a zip file.

Again: the problem is letting them now that something needs to be
fixed. Downloading a zip is not the hard part.

    Juanma

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Tue, 03 Jan 2012 08:09:25 -0500
> 
> EZ> Btw, for some reason Nicos disables creation of gnutls-openssl
> EZ> library, and I don't understand why.  Doesn't Emacs need it?
> 
> That's just the OpenSSL compatibility layer and Emacs doesn't currently
> use it AFAIK.

OK, so Emacs will be happy, but some other program that does need
OpenSSL might not be.  IOW, that w32 build could be incomplete.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Tue, 03 Jan 2012 15:07:02 +0100
> 
> >> The relevant MS Windows API function (LoadLibrary) accepts a full
> >> pathname
> >
> > That's a factual truth, but it would be a grave mistake on our part to
> > use absolute file names for loading dynamic libraries, because it will
> > mean a major inconvenience to users.  It is hard on Windows to pick up
> > a fixed directory where every user could easily put the library: the
> > only directories that are guaranteed to exist on every Windows system
> > are frequently locked up by security policies, the only disk drive
> > guaranteed to exist can be a remote drive or even a read-only drive,
> > etc.  It would be a step in the wrong direction.
> 
> You are providing reasons for the package approach: if it is hard for
> the user to put the dll in the correct directory, let Emacs do it.

No, it is _not_ hard for the user to put the DLL in the correct
directory.  It is hard for _us_, the programmers of package.el, to
select a fixed directory that would work for all users, so that we
could hardcode its absolute file name in the Emacs sources.  An
entirely different issue.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Tue, 03 Jan 2012 10:00:15 -0500
> 
> The mechanism is not too important from a UI perspective, as long as it
> DTRT, and I don't know the W32 platform well enough to specify how it
> should operate.  If you think replacing the DLL is possible entirely
> from within Emacs, I'm OK with that.

Anything's possible: this is software, after all.  It is just that it
takes a non-trivial effort.

> As a first cut, could we have a gnutls-w32 package, version 3.09
> currently, which when activated will download and install the 3.09
> GnuTLS DLL if it's missing?

We can, if someone does the job.

> Where should this DLL be written?

To the directory where emacs.exe lives.

> Does W32 have APIs or mechanisms to update DLLs safely?

If it does, I'm not familiar with them (which doesn't mean they don't
exist; I don't consider myself an expert on the gazillion APIs
scattered across MS-Windows-land).

> As an alternate solution, could GnuTLS itself have an installer and an
> updater?

Alternative for what? for package.el or for replacing a DLL without
restarting Emacs?

> On W32 this is a worse problem than anywhere else IIUC (Mac OS X has
> MacPorts and GNU/Linux has a billion package managers).  So, we either
> tell the user to download and unzip the GnuTLS DLLs every time there's a
> critical fix, or we do it automatically for them from the package.el UI,
> or we pass the responsibility to the GnuTLS developers to write an
> updater.  I don't think we have the option, as responsible developers,
> of just ignoring that responsibility.

I don't ignore it, I built the W32 DLL in the first place, remember?
I just think this puts us 99% of the way to the goal, that's all.  But
if someone has motivation to go the extra 1%, my hat.

Re: GnuTLS for W32

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> You are providing reasons for the package approach: if it is hard for
>> the user to put the dll in the correct directory, let Emacs do it.
>
> No, it is _not_ hard for the user to put the DLL in the correct
> directory.

The user needs to know the correct directory (where "correct" implies
"with write access to it and Emacs can find the dll there"). That's
anything but trivial even for a computer-savvy user (BTW, since when
Emacs changed its policy and is targeted to geeks only again?)

> It is hard for _us_, the programmers of package.el, to
> select a fixed directory that would work for all users, so that we
> could hardcode its absolute file name in the Emacs sources.  An
> entirely different issue.

Elisp packages downloaded by package.el are already saved on a
well-known directory where Emacs has write access to. So the problem is
solved.

Re: GnuTLS for W32

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

[snip]

>> Where should this DLL be written?
>
> To the directory where emacs.exe lives.

Keep in mind that such operation may require Admin privileges. Writing
under the user's home directory doesn't require elevated
privileges. Ted, please see my other response to Eli. Writing on the
under the same directory where the Elisp packages retrieved by
package.el should no pose problems.

>> Does W32 have APIs or mechanisms to update DLLs safely?

The standard MS installers have a mechanism for detecting when a dll is
in use and, if the result is positive, arrange things for updating it on
the next restart or login. A poor's man method for detecting when a dll
is in use is just trying to overwrite it (after you know that you have
write access to the file).

[snip]

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Tue, 03 Jan 2012 18:48:17 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> You are providing reasons for the package approach: if it is hard for
> >> the user to put the dll in the correct directory, let Emacs do it.
> >
> > No, it is _not_ hard for the user to put the DLL in the correct
> > directory.
> 
> The user needs to know the correct directory (where "correct" implies
> "with write access to it and Emacs can find the dll there"). That's
> anything but trivial even for a computer-savvy user

The same user already unzipped the Emacs binary distro, so why exactly
would it be hard for her to unzip another zip file from the same
place?

> (BTW, since when Emacs changed its policy and is targeted to geeks
> only again?)

Since about forever?

> > It is hard for _us_, the programmers of package.el, to
> > select a fixed directory that would work for all users, so that we
> > could hardcode its absolute file name in the Emacs sources.  An
> > entirely different issue.
> 
> Elisp packages downloaded by package.el are already saved on a
> well-known directory where Emacs has write access to. So the problem is
> solved.

Solved my foot! we need to know that directory's absolute file name in
advance, to hardcode it into the C sources of Emacs.  How's that
going to work, if package.el doesn't know where that directory will be
until it is run by Emacs?

Re: GnuTLS for W32

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Tue, 03 Jan 2012 18:48:17 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> You are providing reasons for the package approach: if it is hard for
>> >> the user to put the dll in the correct directory, let Emacs do it.
>> >
>> > No, it is _not_ hard for the user to put the DLL in the correct
>> > directory.
>> 
>> The user needs to know the correct directory (where "correct" implies
>> "with write access to it and Emacs can find the dll there"). That's
>> anything but trivial even for a computer-savvy user
>
> The same user

Not necessarily the same user. But I admit that I'm nitpicking.

> already unzipped the Emacs binary distro, so why exactly
> would it be hard for her to unzip another zip file from the same
> place?

He must unzip the file on a very specific directory, not just on the
same directory where he installs every program.

BTW, zip files is a terrible way to install software on MS Windows. Only
geeks who already decided to try Emacs go through the process of
installing from a zip file. I find quite ironic that `make install'
creates the Emacs icon on the Start menu but with the standard binary
distribution the user need to figure out how to start Emacs.

>> (BTW, since when Emacs changed its policy and is targeted to geeks
>> only again?)
>
> Since about forever?

Then lots of time was wasted on writing documentation that is clearly
targeted to non-geeks.

>> > It is hard for _us_, the programmers of package.el, to
>> > select a fixed directory that would work for all users, so that we
>> > could hardcode its absolute file name in the Emacs sources.  An
>> > entirely different issue.
>> 
>> Elisp packages downloaded by package.el are already saved on a
>> well-known directory where Emacs has write access to. So the problem is
>> solved.
>
> Solved my foot! we need to know that directory's absolute file name in
> advance, to hardcode it into the C sources of Emacs.  How's that
> going to work, if package.el doesn't know where that directory will be
> until it is run by Emacs?

You don't need to know the directory at compile time. GnuTLS and
potentially other libraries (those that provide image support, for
instance) are perfectly fine if you load them on demand at run time. I
think that's what already happens with the GnuTLS on MS Windows.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Tue, 03 Jan 2012 19:34:55 +0100
> 
> >> Elisp packages downloaded by package.el are already saved on a
> >> well-known directory where Emacs has write access to. So the problem is
> >> solved.
> >
> > Solved my foot! we need to know that directory's absolute file name in
> > advance, to hardcode it into the C sources of Emacs.  How's that
> > going to work, if package.el doesn't know where that directory will be
> > until it is run by Emacs?
> 
> You don't need to know the directory at compile time. GnuTLS and
> potentially other libraries (those that provide image support, for
> instance) are perfectly fine if you load them on demand at run time.

Yes, but loaded from where?  They can either be in the same directory
where the Emacs executable lives, or on Path, or in the Windows's
system32 directory, or in some other place, but in the latter case we
must pass an absolute file name to LoadLibrary.

Re: GnuTLS for W32

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> >> Elisp packages downloaded by package.el are already saved on a
>> >> well-known directory where Emacs has write access to. So the problem is
>> >> solved.
>> >
>> > Solved my foot! we need to know that directory's absolute file name in
>> > advance, to hardcode it into the C sources of Emacs.  How's that
>> > going to work, if package.el doesn't know where that directory will be
>> > until it is run by Emacs?
>> 
>> You don't need to know the directory at compile time. GnuTLS and
>> potentially other libraries (those that provide image support, for
>> instance) are perfectly fine if you load them on demand at run time.
>
> Yes, but loaded from where?  They can either be in the same directory
> where the Emacs executable lives, or on Path, or in the Windows's
> system32 directory, or in some other place, but in the latter case we
> must pass an absolute file name to LoadLibrary.

If, at run time, you can find the elisp packages downloaded by
package.el, what's the problem with finding a dll on the same directory
(or a subdirectory of it, if you wish) ?

Re: GnuTLS for W32

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Why are we talking only about Windows?  Do packages of Emacs 24
> development snapshots on GNU/Linux come with GnuTLS in the same
> package?  If they do, then I agree that the Windows binaries should
> also include GnuTLS; but if not, I don't see why Windows should be
> special in this regard.  We don't want to confuse users by providing
> functionality only on some platforms.

And since the (major) binary GNU/Linux distributions will be providing
the gnutls libraries with Emacs, then we should, too?  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Tue, 03 Jan 2012 20:48:23 +0100
> 
> If, at run time, you can find the elisp packages downloaded by
> package.el, what's the problem with finding a dll on the same directory
> (or a subdirectory of it, if you wish) ?

Because load-path is used for loading Lisp, but not for loading DLLs.

Re: GnuTLS for W32

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> If, at run time, you can find the elisp packages downloaded by
>> package.el, what's the problem with finding a dll on the same directory
>> (or a subdirectory of it, if you wish) ?
>
> Because load-path is used for loading Lisp, but not for loading DLLs.

That has an obvious answer.

Re: GnuTLS for W32

[Stephen J. Turnbull]

Óscar Fuentes writes:

 > If, at run time, you can find the elisp packages downloaded by
 > package.el, what's the problem with finding a dll on the same directory
 > (or a subdirectory of it, if you wish) ?

None, AFAICS.  That's (actually, a sibling directory) is what XEmacs
does[1], and what Python does (same directory, if desired).


Footnotes: 
[1]  Except that we don't provide DLLs for download, for various
reasons, the most important of which is that binary distributions are
bug magnets that distract the maintainers disproportionately, and
non-maintainers are generally unwilling to touch.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Date: Wed, 04 Jan 2012 12:45:47 +0900
> Cc: emacs-devel@gnu.org
> 
> Óscar Fuentes writes:
> 
>  > If, at run time, you can find the elisp packages downloaded by
>  > package.el, what's the problem with finding a dll on the same directory
>  > (or a subdirectory of it, if you wish) ?
> 
> None, AFAICS.  That's (actually, a sibling directory) is what XEmacs
> does[1], and what Python does (same directory, if desired).

Please don't say that without telling the details.  There are people
reading this thread who don't know enough about Windows and are likely
to take this at face value.

Once again: the way the current C code is written, Emacs _cannot_ load
DLLs except from directories which Windows searches for dynamic
libraries.  Those directories do not include load-path, and therefore
no amount of coding in package.el alone will be able to make
package.el usable for downloading and installing DLLs (even if we
resolve the issue of doing so from within a running Emacs session).
Changes on the C level are needed to make that possible; but if we are
going to make C-level changes, it would be much better to have DLL
downloaded to the same directory where emacs.exe lives or (if the user
so wishes) to some directory on PATH, because that's where DLLs are
normally located.

An alternative is to modify PATH to include site-lisp or some other
directory on load-path, but that is a much worse idea, and it cannot
be done from a running Emacs session anyway.

> [1]  Except that we don't provide DLLs for download, for various
> reasons, the most important of which is that binary distributions are
> bug magnets that distract the maintainers disproportionately, and
> non-maintainers are generally unwilling to touch.

That point was made in this thread more than once, but Ted is still
pushing for it.  Which is fine by me, assuming that someone will step
forward and do the job.

Re: GnuTLS for W32

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Tue, 03 Jan 2012 20:48:23 +0100
>>
>> If, at run time, you can find the elisp packages downloaded by
>> package.el, what's the problem with finding a dll on the same directory
>> (or a subdirectory of it, if you wish) ?
>
> Because load-path is used for loading Lisp, but not for loading DLLs.

The code in a Lisp package can extract its package directory from the
variable `load-file-name'.  The Multi-file Packages node of the Lisp
manual gives an example of this.

I'm not saying we should distribute GnuTLS support via ELPA, only that
the Package system is no technical barrier.

Re: GnuTLS for W32

[Stephen J. Turnbull]

Eli Zaretskii writes:
 > > Óscar Fuentes writes:
 > > 
 > >  > If, at run time, you can find the elisp packages downloaded by
 > >  > package.el, what's the problem with finding a dll on the same directory
 > >  > (or a subdirectory of it, if you wish) ?
 > > 
 > > None, AFAICS.  That's (actually, a sibling directory) is what XEmacs
 > > does[1], and what Python does (same directory, if desired).
 > 
 > Please don't say that without telling the details.  There are people
 > reading this thread who don't know enough about Windows and are likely
 > to take this at face value.
 > 
 > Once again: the way the current C code is written, Emacs _cannot_ load
 > DLLs except from directories which Windows searches for dynamic
 > libraries.

OK, you may be right (I mean, for XEmacs too, I concede your expertise
on Emacs).  I don't understand the ins and outs of the Windows native
build, but it looks like support for a configurable modules directory
was removed from that port of XEmacs after 21.4 (which was when I was
reasonably familiar with the port), and what actually happens now is
that modules built at XEmacs build time are installed in the same
directory as the XEmacs binary.  It's quite possible the alleged
support was removed because it didn't actually work for the reasons
you give.

I don't know the details, but I'm pretty sure that Python does support
DLLs anywhere that a .py can be found, though, because where something
lives on PYTHONPATH is essential to figuring out what its name (as a
module in Python) is.

 > > [1]  Except that we don't provide DLLs for download, for various
 > > reasons, the most important of which is that binary distributions are
 > > bug magnets that distract the maintainers disproportionately, and
 > > non-maintainers are generally unwilling to touch.
 > 
 > That point was made in this thread more than once, but Ted is still
 > pushing for it.  Which is fine by me, assuming that someone will step
 > forward and do the job.

In my experience, such volunteers are only reliable in the long run if
they're paid to support Emacs by building installers etc.  But it's a
new year, so anything could happen!  Good luck finding the volunteer!

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Chong Yidong <cyd@gnu.org>
> Cc: Óscar Fuentes <ofv@wanadoo.es>,  emacs-devel@gnu.org
> Date: Wed, 04 Jan 2012 14:48:22 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Óscar Fuentes <ofv@wanadoo.es>
> >> Date: Tue, 03 Jan 2012 20:48:23 +0100
> >>
> >> If, at run time, you can find the elisp packages downloaded by
> >> package.el, what's the problem with finding a dll on the same directory
> >> (or a subdirectory of it, if you wish) ?
> >
> > Because load-path is used for loading Lisp, but not for loading DLLs.
> 
> The code in a Lisp package can extract its package directory from the
> variable `load-file-name'.  The Multi-file Packages node of the Lisp
> manual gives an example of this.

The issue is not figuring out the directory where the DLL lives.  The
issue is instructing the code that needs to load that DLL where to
look for it.

Currently, we load the DLLs by specifying their basename, without
leading directories, which causes the corresponding Windows API to
search for the DLL in known places, as documented in the MS
documentation.  To change that and give Lisp programs control of where
to look for DLLs, we need changes on the C level.

The other problem is how to upgrade a DLL from an Emacs session that
already uses the previous version of that DLL.  I think this is hard
without restarting Emacs, and not trivial even with restarting.

> I'm not saying we should distribute GnuTLS support via ELPA, only that
> the Package system is no technical barrier.

The barrier I was talking about has nothing to do with package.el.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: ofv@wanadoo.es,
>     emacs-devel@gnu.org
> Date: Wed, 04 Jan 2012 16:03:26 +0900
> 
> I don't know the details, but I'm pretty sure that Python does support
> DLLs anywhere that a .py can be found, though, because where something
> lives on PYTHONPATH is essential to figuring out what its name (as a
> module in Python) is.

That's also what I know (although my Python-related knowledge is
considerably smaller than yours).

But there is a difference between Python's .pyd DLLs and
general-purpose DLLs such as libgnutls.  The former are generally not
useful for anything but Python programs.  By contrast, the latter can
be used by programs that have nothing to do with Emacs.  For example,
GnuTLS is used by wget (in fact, building the latest wget was the
original reason why I built the latest GnuTLS on Windows).

For this reason, I think we should give Emacs users an option to put
the downloaded DLL in some directory that is not Emacs-specific, so
that other programs could use it.

>  > > [1]  Except that we don't provide DLLs for download, for various
>  > > reasons, the most important of which is that binary distributions are
>  > > bug magnets that distract the maintainers disproportionately, and
>  > > non-maintainers are generally unwilling to touch.
>  > 
>  > That point was made in this thread more than once, but Ted is still
>  > pushing for it.  Which is fine by me, assuming that someone will step
>  > forward and do the job.
> 
> In my experience, such volunteers are only reliable in the long run if
> they're paid to support Emacs by building installers etc.  But it's a
> new year, so anything could happen!  Good luck finding the volunteer!

Seconded.

Re: GnuTLS for W32

[Ted Zlatanov]

On Tue, 03 Jan 2012 19:21:14 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

EZ> No, it is _not_ hard for the user to put the DLL in the correct
EZ> directory.  It is hard for _us_, the programmers of package.el, to
EZ> select a fixed directory that would work for all users, so that we
EZ> could hardcode its absolute file name in the Emacs sources.  An
EZ> entirely different issue.

The addendum is "...and so GnuTLS is available to non-Emacs processes
too" which is something I did not consider carefully but your mention of
wget brought up.  We don't want multiple versions of the GnuTLS DLL all
over the place, that could be worse for security than a single outdated
version.  So the package.el install is not a good W32 solution to
deliver GnuTLS, after all.

On Tue, 03 Jan 2012 20:51:28 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> And since the (major) binary GNU/Linux distributions will be providing
LMI> the gnutls libraries with Emacs, then we should, too?  :-)

We should find a way, yes.  But we should be smart about it, and a
GnuTLS installer, separate from Emacs, would benefit far more users and
projects, and it would use the right APIs to deploy the DLL.

On Tue, 03 Jan 2012 19:10:14 +0100 Óscar Fuentes <ofv@wanadoo.es> wrote: 

ÓF> The standard MS installers have a mechanism for detecting when a dll is
ÓF> in use and, if the result is positive, arrange things for updating it on
ÓF> the next restart or login. A poor's man method for detecting when a dll
ÓF> is in use is just trying to overwrite it (after you know that you have
ÓF> write access to the file).

Because of this, I think we (emacs-devel and gnutls-devel) should just
put together a W32 installer for GnuTLS.  Obviously this would be
connected to the GnuTLS project but I think Nikos (whose name I've
misspelled as Nicos a few times) could use some help with such an
installer.  Is there another GNU project that provides a W32 DLL
*self-updating* installer, so I can get them started with that project?

I may end up owning that installer, we'll see what the GnuTLS developers
want.  But it would let GnuTLS issue updates, remove a release
dependency between their project and Emacs, and remove the tedious
requirement of downloading a zip file and installing it manually.

On the Emacs side on W32, if GnuTLS is not installed, we should tell the
user how to install it by pointing to the W32 installer (running it
automatically is probably too aggressive).  The more I think about it,
the more appropriate this solution seems for everyone concerned.

Ted

Re: GnuTLS for W32

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > For this reason, I think we should give Emacs users an option to put
 > the downloaded DLL in some directory that is not Emacs-specific, so
 > that other programs could use it.

Well, as you know I'm not a Windows person, but my understanding is
that one reason that DLL hell is called "DLL hell" is that programs
install their own versions of DLLs *in the system directories* that
for one reason or another are inappropriate for other programs.  So in
fact you're asking a lot of knowledge on the user's part to get this
right.

This is not unknown on other OSes.  For example, on most GNU/Linux
systems glibc will depend on a specific version of the kernel headers,
which may or may not correspond to the kernel actually running on the
system.  So glibc has its private version of the kernel headers, which
it then imposes on the whole system.  This actually seems to work
pretty well, but only because everybody knows that the most important
thing is for libc to work!  gnutls isn't going to have that priority.

I think that the thing to do is for Emacs to install private versions
of DLLs when it installs any at all, and if/when people complain about
"bloat", add some logic (maybe in the installer routine) to check to
see if there is an appropriate DLL already on the Windows DLL path.
(Yes, I understand that implementing this isn't going to be all that
simple either, but in principle I don't think Emacs should add to DLL
hell.)

Re: GnuTLS for W32

[Lars Magne Ingebrigtsen]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> I think that the thing to do is for Emacs to install private versions
> of DLLs when it installs any at all, and if/when people complain about
> "bloat", add some logic (maybe in the installer routine) to check to
> see if there is an appropriate DLL already on the Windows DLL path.

I agree.  Assuming that the size of these things is comparable on
Windows and Debian, the added size doesn't seem particularly daunting:

[larsi@stories ~]$ ls -lh /usr/lib/libgnutls.so.26.14.12 src/emacs/trunk/src/emacs /usr/lib/libxml2.so.2.7.8              
-rwxr-xr-x 3 larsi larsi  19M Jan  3 20:41 src/emacs/trunk/src/emacs
-rw-r--r-- 1 root  root  654K Mar 20  2010 /usr/lib/libgnutls.so.26.14.12
-rw-r--r-- 1 root  root  1.4M Jun  4  2011 /usr/lib/libxml2.so.2.7.8

And the .elc files seem to take a further 38MB, if I wc correctly.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: GnuTLS for W32

[Lennart Borgman]

Off topic: Lars Magne, why is it not possible to reach you at the mail
address you are using here. Gmail gives me this:

The recipient server did not accept our requests to connect. Learn
more at http://mail.google.com/support/bin/answer.py?answer=7720
[quimby.gnus.org. (10): No route to host]

Re: GnuTLS for W32

[Lars Magne Ingebrigtsen]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> Off topic: Lars Magne, why is it not possible to reach you at the mail
> address you are using here. Gmail gives me this:
>
> The recipient server did not accept our requests to connect. Learn
> more at http://mail.google.com/support/bin/answer.py?answer=7720
> [quimby.gnus.org. (10): No route to host]

I get plenty of email at my larsi@gnus.org address.  Perhaps it's a
Gmail bug?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: GnuTLS for W32

[joakim]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Tue, 03 Jan 2012 19:21:14 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 
>
> EZ> No, it is _not_ hard for the user to put the DLL in the correct
> EZ> directory.  It is hard for _us_, the programmers of package.el, to
> EZ> select a fixed directory that would work for all users, so that we
> EZ> could hardcode its absolute file name in the Emacs sources.  An
> EZ> entirely different issue.
>
> The addendum is "...and so GnuTLS is available to non-Emacs processes
> too" which is something I did not consider carefully but your mention of
> wget brought up.  We don't want multiple versions of the GnuTLS DLL all
> over the place, that could be worse for security than a single outdated
> version.  So the package.el install is not a good W32 solution to
> deliver GnuTLS, after all.
>
> On Tue, 03 Jan 2012 20:51:28 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 
>
> LMI> And since the (major) binary GNU/Linux distributions will be providing
> LMI> the gnutls libraries with Emacs, then we should, too?  :-)
>
> We should find a way, yes.  But we should be smart about it, and a
> GnuTLS installer, separate from Emacs, would benefit far more users and
> projects, and it would use the right APIs to deploy the DLL.
>
> On Tue, 03 Jan 2012 19:10:14 +0100 Óscar Fuentes <ofv@wanadoo.es> wrote: 
>
> ÓF> The standard MS installers have a mechanism for detecting when a dll is
> ÓF> in use and, if the result is positive, arrange things for updating it on
> ÓF> the next restart or login. A poor's man method for detecting when a dll
> ÓF> is in use is just trying to overwrite it (after you know that you have
> ÓF> write access to the file).
>
> Because of this, I think we (emacs-devel and gnutls-devel) should just
> put together a W32 installer for GnuTLS.  Obviously this would be
> connected to the GnuTLS project but I think Nikos (whose name I've
> misspelled as Nicos a few times) could use some help with such an
> installer.  Is there another GNU project that provides a W32 DLL
> *self-updating* installer, so I can get them started with that project?
>
> I may end up owning that installer, we'll see what the GnuTLS developers
> want.  But it would let GnuTLS issue updates, remove a release
> dependency between their project and Emacs, and remove the tedious
> requirement of downloading a zip file and installing it manually.
>
> On the Emacs side on W32, if GnuTLS is not installed, we should tell the
> user how to install it by pointing to the W32 installer (running it
> automatically is probably too aggressive).  The more I think about it,
> the more appropriate this solution seems for everyone concerned.

I haven't followed this thread too closely but I can offer advice on how
to use the NSIS installer (http://nsis.sourceforge.net/Main_Page) if
that is at all relevant (NSIS is free software and stuff)


>
> Ted
>

-- 
Joakim Verona

Re: GnuTLS for W32

[David Engster]

Lars Magne Ingebrigtsen writes:
> Lennart Borgman <lennart.borgman@gmail.com> writes:
>
>> Off topic: Lars Magne, why is it not possible to reach you at the mail
>> address you are using here. Gmail gives me this:
>>
>> The recipient server did not accept our requests to connect. Learn
>> more at http://mail.google.com/support/bin/answer.py?answer=7720
>> [quimby.gnus.org. (10): No route to host]
>
> I get plenty of email at my larsi@gnus.org address.  Perhaps it's a
> Gmail bug?

Lennart, when did you send the mail? quimby.gnus.org was down for
several days after christmas.

-David

Re: GnuTLS for W32

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: ofv@wanadoo.es,
>     emacs-devel@gnu.org
> Date: Wed, 04 Jan 2012 20:21:07 +0900
> 
> Eli Zaretskii writes:
> 
>  > For this reason, I think we should give Emacs users an option to put
>  > the downloaded DLL in some directory that is not Emacs-specific, so
>  > that other programs could use it.
> 
> Well, as you know I'm not a Windows person, but my understanding is
> that one reason that DLL hell is called "DLL hell"

An aside: the "DLL hell"s flames are much lower now than they used to
be, see

   http://en.wikipedia.org/wiki/Side-by-side_assembly

FWIW, I didn't have a single problem for years on my XP boxes.  YMMV,
of course.

> is that programs install their own versions of DLLs *in the system
> directories* that for one reason or another are inappropriate for
> other programs.  So in fact you're asking a lot of knowledge on the
> user's part to get this right.

Only as an option, which I expect to be used by someone who really
knows what they are doing.  By default, we should install into the
directory of emacs.exe.

(Btw, it doesn't have to be a system directory to be useful to other
programs.  A DLL that is not a standard component on Windows can be
put in some directory on PATH that was created and is maintained by
the user.  That's what I do, FWIW.)

> I think that the thing to do is for Emacs to install private versions
> of DLLs when it installs any at all

By default, certainly.

Re: GnuTLS for W32

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "Stephen J. Turnbull" <stephen@xemacs.org>
>> Cc: ofv@wanadoo.es,
>>     emacs-devel@gnu.org
>> Date: Wed, 04 Jan 2012 20:21:07 +0900
>> 
>> Eli Zaretskii writes:
>> 
>>  > For this reason, I think we should give Emacs users an option to put
>>  > the downloaded DLL in some directory that is not Emacs-specific, so
>>  > that other programs could use it.
>> 
>> Well, as you know I'm not a Windows person, but my understanding is
>> that one reason that DLL hell is called "DLL hell"
>
> An aside: the "DLL hell"s flames are much lower now than they used to
> be, see
>
>    http://en.wikipedia.org/wiki/Side-by-side_assembly

That requires an installer that follows MS guidelines. It's not as easy
as unzipping a file on a directory.

On the topic of how hard is to load a dll from an arbitrary location,
that is what I found on the Emacs sources:

/* The argument LIBRARIES is an alist that associates a symbol
   LIBRARY_ID, identifying an external DLL library known to Emacs, to
   a list of filenames under which the library is usually found.  In
   most cases, the argument passed as LIBRARIES is the variable
   `dynamic-library-alist', which is initialized to a list of common
   library names.  If the function loads the library successfully, it
   returns the handle of the DLL, and records the filename in the
   property :loaded-from of LIBRARY_ID; it returns NULL if the library
   could not be found, or when it was already loaded (because the
   handle is not recorded anywhere, and so is lost after use).  It
   would be trivial to save the handle too in :loaded-from, but
   currently there's no use case for it.  */
HMODULE
w32_delayed_load (Lisp_Object libraries, Lisp_Object library_id)
{
  HMODULE library_dll = NULL;

  CHECK_SYMBOL (library_id);

  if (CONSP (libraries) && NILP (Fassq (library_id, Vlibrary_cache)))
    {
      Lisp_Object found = Qnil;
      Lisp_Object dlls = Fassq (library_id, libraries);

      if (CONSP (dlls))
        for (dlls = XCDR (dlls); CONSP (dlls); dlls = XCDR (dlls))
          {
            CHECK_STRING_CAR (dlls);
            if ((library_dll = LoadLibrary (SDATA (XCAR (dlls)))))
              {
                found = XCAR (dlls);
                break;
              }
          }

      Fput (library_id, QCloaded_from, found);
    }

  return library_dll;
}


I think that Emacs is right now capable of loading a dll from an
arbitrary place. Just set dynamic-library-alist as it contains ('gnutls
. "/path/to/gnutls/gnutls.dll") (or whatever are the right names). In
any case, creating a new function that loads a .dll by name seems quite
straightforward, once we have seem how w32_delayed_load is implemented.

> FWIW, I didn't have a single problem for years on my XP boxes.  YMMV,
> of course.

On the last 5 years I the only occurrences of the dll hell I had were
related to Free software which uses naive installers. Cygwin, MSYS and
GnuWin32 does not get along very well, and even updating some of their
component may create havoc due to incompatibilities of the new dlls with
some of the old tools. The safe method is to update the whole lot.

Installing a dll intended to be used by Emacs on a shared place is
asking for trouble.

[snip]

Re: GnuTLS for W32

[Óscar Fuentes]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Tue, 03 Jan 2012 19:21:14 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 
>
> EZ> No, it is _not_ hard for the user to put the DLL in the correct
> EZ> directory.  It is hard for _us_, the programmers of package.el, to
> EZ> select a fixed directory that would work for all users, so that we
> EZ> could hardcode its absolute file name in the Emacs sources.  An
> EZ> entirely different issue.
>
> The addendum is "...and so GnuTLS is available to non-Emacs processes
> too" which is something I did not consider carefully but your mention of
> wget brought up.  We don't want multiple versions of the GnuTLS DLL all
> over the place, that could be worse for security than a single outdated
> version.  So the package.el install is not a good W32 solution to
> deliver GnuTLS, after all.

You are very wrong. For starters, MS Windows is not like most GNU/Linux
distribution that use a system for keeping all installed software in
sync. The wget example is a red herring. Sharing the gnutls dll is so
wrong at some many levels that I wont start discussing it.

[snip]

Re: GnuTLS for W32

[Juanma Barranquero]

On Wed, Jan 4, 2012 at 15:14, Óscar Fuentes <ofv@wanadoo.es> wrote:

> On the topic of how hard is to load a dll from an arbitrary location,
> that is what I found on the Emacs sources:

I think Eli's already aware of that function.

> I think that Emacs is right now capable of loading a dll from an
> arbitrary place. Just set dynamic-library-alist as it contains ('gnutls
> . "/path/to/gnutls/gnutls.dll") (or whatever are the right names).

We know that. Nobody discusses that you can use a full pathname to
load a DLL from an arbitrary place. Only the convenience of handling
the installation of DLLs in arbitrary places. (Well, not only; we're
mainly discussing the convenience of turning Emacs into a distribution
point for arbitrary Windows binaries from other projects; but I
digress.)

> In
> any case, creating a new function that loads a .dll by name seems quite
> straightforward, once we have seem how w32_delayed_load is implemented.

What do you mean?

    Juanma

Re: GnuTLS for W32

[Juanma Barranquero]

On Wed, Jan 4, 2012 at 15:14, Óscar Fuentes <ofv@wanadoo.es> wrote:

> On the topic of how hard is to load a dll from an arbitrary location,
> that is what I found on the Emacs sources:

And BTW, I prefer the patched version from bug#10424, which records
the full pathname too, and which I'll commit after 24.1 (at least this
change, the one in misc.el could be circumvented with better
customizability of tabulated-list-mode).

> /* The argument LIBRARIES is an alist that associates a symbol
>   LIBRARY_ID, identifying an external DLL library known to Emacs, to
>   a list of filenames under which the library is usually found.  In
>   most cases, the argument passed as LIBRARIES is the variable
>   `dynamic-library-alist', which is initialized to a list of common
>   library names.  If the function loads the library successfully, it
>   returns the handle of the DLL, and records the filename in the
>   property :loaded-from of LIBRARY_ID; it returns NULL if the library
>   could not be found, or when it was already loaded (because the
>   handle is not recorded anywhere, and so is lost after use).  It
>   would be trivial to save the handle too in :loaded-from, but
>   currently there's no use case for it.  */
> HMODULE
> w32_delayed_load (Lisp_Object libraries, Lisp_Object library_id)
> {
>  HMODULE library_dll = NULL;
>
>  CHECK_SYMBOL (library_id);
>
>  if (CONSP (libraries) && NILP (Fassq (library_id, Vlibrary_cache)))
>    {
>      Lisp_Object found = Qnil;
>      Lisp_Object dlls = Fassq (library_id, libraries);
>
>      if (CONSP (dlls))
>        for (dlls = XCDR (dlls); CONSP (dlls); dlls = XCDR (dlls))
>          {
>            CHECK_STRING_CAR (dlls);
>            if ((library_dll = LoadLibrary (SDATA (XCAR (dlls)))))
>              {
-                found = XCAR (dlls);
+                char name[MAX_PATH];
+                DWORD len;
+
+                len = GetModuleFileNameA (library_dll, name, sizeof (name));
+                found = Fcons (XCAR (dlls),
+                               (len > 0)
+                               /* Possibly truncated */
+                               ? make_specified_string (name, -1, len, 1)
+                               : Qnil);
>                break;
>              }
>          }
>
>      Fput (library_id, QCloaded_from, found);
>    }
>
>  return library_dll;
> }

    Juanma

Re: GnuTLS for W32

[Óscar Fuentes]

Juanma Barranquero <lekktu@gmail.com> writes:

> On Wed, Jan 4, 2012 at 15:14, Óscar Fuentes <ofv@wanadoo.es> wrote:
>
>> On the topic of how hard is to load a dll from an arbitrary location,
>> that is what I found on the Emacs sources:
>
> I think Eli's already aware of that function.

He's repeating that loading a dll with a pathname would require changes
at the C level.

>> I think that Emacs is right now capable of loading a dll from an
>> arbitrary place. Just set dynamic-library-alist as it contains ('gnutls
>> . "/path/to/gnutls/gnutls.dll") (or whatever are the right names).
>
> We know that. Nobody discusses that you can use a full pathname to
> load a DLL from an arbitrary place. Only the convenience of handling
> the installation of DLLs in arbitrary places. (Well, not only; we're
> mainly discussing the convenience of turning Emacs into a distribution
> point for arbitrary Windows binaries from other projects; but I
> digress.)

I don't know how a key component for an Emacs feature can be deemed as
"arbitrary".

As far as I'm concerned, there are two sane options here: suppose that
the user is a geek, advertise GnuTLS support citing the dll requirement
finishing with "now that's your problem". The other approach is to do
the job from Emacs, do it well, and automatically download and upgrade
the dll from Elpa.

Related to this, I'm convinced that Emacs packages too much on the
standard distribution. Things like org-mode, Gnus, Semantic... would be
better on Elpa. That would free resources from the Emacs developers
(less packaging, less maintenance, less administratrivia...), free
resources from the package developers (no merging back and forth) and,
finally, would provide the users updated packages. Oh, and would
diminish the pressure for releasing new Emacs versions.

>> In
>> any case, creating a new function that loads a .dll by name seems quite
>> straightforward, once we have seem how w32_delayed_load is implemented.
>
> What do you mean?

Just that if w32_delayed_load does not fit the bill, writing the
required function by tweaking w32_delayed_load is so easy that someone
like me who doesn't know the Emacs C dialect can do the job.

Re: GnuTLS for W32

[Ted Zlatanov]

On Wed, 04 Jan 2012 16:42:11 +0100 Óscar Fuentes <ofv@wanadoo.es> wrote: 

ÓF> As far as I'm concerned, there are two sane options here: suppose that
ÓF> the user is a geek, advertise GnuTLS support citing the dll requirement
ÓF> finishing with "now that's your problem". The other approach is to do
ÓF> the job from Emacs, do it well, and automatically download and upgrade
ÓF> the dll from Elpa.

If those are the only two options, obviously the second one is better.

On Wed, 04 Jan 2012 15:22:18 +0100 Óscar Fuentes <ofv@wanadoo.es> wrote: 

ÓF> Sharing the gnutls dll is so wrong at some many levels that I wont
ÓF> start discussing it.

I am puzzled by this.  Why is it wrong to share the GnuTLS DLL?

If you and Eli and other W32 experts say a standalone self-updating
installer that drops a shared GnuTLS DLL in a common area is a bad idea,
I won't argue.  But you have to at least explain your reasoning.

Ted

Re: GnuTLS for W32

[Juanma Barranquero]

2012/1/4 Ted Zlatanov <tzz@lifelogs.com>:

> I am puzzled by this.  Why is it wrong to share the GnuTLS DLL?
>
> If you and Eli and other W32 experts say a standalone self-updating
> installer that drops a shared GnuTLS DLL in a common area is a bad idea,

I think is a good idea. Fortunately, we don't have to discuss it,
because Óscar "won't start discussing it" ;-)

    Juanma

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Wed, 04 Jan 2012 15:22:18 +0100
> 
> > The addendum is "...and so GnuTLS is available to non-Emacs processes
> > too" which is something I did not consider carefully but your mention of
> > wget brought up.  We don't want multiple versions of the GnuTLS DLL all
> > over the place, that could be worse for security than a single outdated
> > version.  So the package.el install is not a good W32 solution to
> > deliver GnuTLS, after all.
> 
> You are very wrong. For starters, MS Windows is not like most GNU/Linux
> distribution that use a system for keeping all installed software in
> sync. The wget example is a red herring. Sharing the gnutls dll is so
> wrong at some many levels that I wont start discussing it.

FUD

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Wed, 04 Jan 2012 15:14:28 +0100
> 
> >    http://en.wikipedia.org/wiki/Side-by-side_assembly
> 
> That requires an installer that follows MS guidelines.

And granted, the DLLs that were causing the hell come with an
installer.

> It's not as easy as unzipping a file on a directory.

Nobody suggested that; it was an aside.

> I think that Emacs is right now capable of loading a dll from an
> arbitrary place.

With great care, yes.  dynamic-library-alist is carefully hand-crafted
to DTRT with several DLLs of different versions on the same machine.
Just pushing arbitrary file name there is not the best idea.

> Just set dynamic-library-alist as it contains ('gnutls
> . "/path/to/gnutls/gnutls.dll") (or whatever are the right names).

Putting DLLs in arbitrary places is not TRT.  We should put it in the
same directory where emacs.exe lives, and then there's no need to do
anything with dynamic-library-alist.

> Installing a dll intended to be used by Emacs on a shared place is
> asking for trouble.

Please give me the credit that when I do that I know what I'm doing.
I have been doing that for years with no problems at all.  I'm sure
others have succeeded in learning a few simple rules of how to do this
safely.  That's why an _option_ to do it available to knowledgeable
users would be a bonus.  The default place should be where emacs.exe
lives.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Wed, 04 Jan 2012 16:42:11 +0100
> 
> Juanma Barranquero <lekktu@gmail.com> writes:
> 
> > On Wed, Jan 4, 2012 at 15:14, Óscar Fuentes <ofv@wanadoo.es> wrote:
> >
> >> On the topic of how hard is to load a dll from an arbitrary location,
> >> that is what I found on the Emacs sources:
> >
> > I think Eli's already aware of that function.
> 
> He's repeating that loading a dll with a pathname would require changes
> at the C level.

And I still submit that is so.

> Just that if w32_delayed_load does not fit the bill, writing the
> required function by tweaking w32_delayed_load is so easy that someone
> like me who doesn't know the Emacs C dialect can do the job.

You are welcome to do that and present a complete solution for the
issue at hand.  Maybe then this thread will become anything but
hand-waving.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Date: Wed, 04 Jan 2012 12:33:11 +0100
> 
> "Stephen J. Turnbull" <stephen@xemacs.org> writes:
> 
> > I think that the thing to do is for Emacs to install private versions
> > of DLLs when it installs any at all, and if/when people complain about
> > "bloat", add some logic (maybe in the installer routine) to check to
> > see if there is an appropriate DLL already on the Windows DLL path.
> 
> I agree.  Assuming that the size of these things is comparable on
> Windows and Debian, the added size doesn't seem particularly daunting:
> 
> [larsi@stories ~]$ ls -lh /usr/lib/libgnutls.so.26.14.12 src/emacs/trunk/src/emacs /usr/lib/libxml2.so.2.7.8              
> -rwxr-xr-x 3 larsi larsi  19M Jan  3 20:41 src/emacs/trunk/src/emacs
> -rw-r--r-- 1 root  root  654K Mar 20  2010 /usr/lib/libgnutls.so.26.14.12
> -rw-r--r-- 1 root  root  1.4M Jun  4  2011 /usr/lib/libxml2.so.2.7.8

You missed few dependencies, but yes, the sizes are comparable:

  -rw-rw-rw- 1 Zaretzkii      None    73K 2012-01-02 19:44 libp11-kit-0.dll
  -rw-rw-rw- 1 Zaretzkii      None   148K 2012-01-02 19:44 libnettle-4-3.dll
  -rw-rw-rw- 1 Zaretzkii      None   306K 2012-01-02 19:44 libhogweed-2-1.dll
  -rw-rw-rw- 1 Zaretzkii      None   937K 2012-01-02 19:44 libgnutls-28.dll
  -rw-rw-rw- 1 Zaretzkii      None   417K 2012-01-02 19:44 libgmp-10.dll
  -rw-rw-rw- 1 Zaretzkii      None   966K 2011-06-19 21:52 libxml2.dll

Re: GnuTLS for W32

[Lennart Borgman]

On Wed, Jan 4, 2012 at 13:37, David Engster <deng@randomsample.de> wrote:
> Lars Magne Ingebrigtsen writes:
>> Lennart Borgman <lennart.borgman@gmail.com> writes:
>>
>>> Off topic: Lars Magne, why is it not possible to reach you at the mail
>>> address you are using here. Gmail gives me this:
>>>
>>> The recipient server did not accept our requests to connect. Learn
>>> more at http://mail.google.com/support/bin/answer.py?answer=7720
>>> [quimby.gnus.org. (10): No route to host]
>>
>> I get plenty of email at my larsi@gnus.org address.  Perhaps it's a
>> Gmail bug?
>
> Lennart, when did you send the mail? quimby.gnus.org was down for
> several days after christmas.

I sent it on 2011-12-28. So maybe that was the reason then. I will try
again and see.

Re: GnuTLS for W32

[Ted Zlatanov]

On Wed, 4 Jan 2012 18:00:49 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> 2012/1/4 Ted Zlatanov <tzz@lifelogs.com>:
>> I am puzzled by this.  Why is it wrong to share the GnuTLS DLL?
>> 
>> If you and Eli and other W32 experts say a standalone self-updating
>> installer that drops a shared GnuTLS DLL in a common area is a bad idea,

JB> I think is a good idea. Fortunately, we don't have to discuss it,
JB> because Óscar "won't start discussing it" ;-)

I'd like to discuss it with anyone willing.  Joakim offered to help with
the installer and I can probably follow a build recipe to make the DLLs,
so practically it seems like we could do this.  I just want to make sure
I don't find out in a few days or weeks that the installer approach is
flawed for some obscure W32 reason, which Óscar sort of implied.

On Wed, 04 Jan 2012 20:10:23 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

EZ> Maybe then this thread will become anything but hand-waving.

I think it makes sense to discuss approaches--I had not even thought of
the GnuTLS installer approach until we had this conversation.  Rushing
into development can be fun, but can also hide good ideas.

Ted

Re: GnuTLS for W32

[Óscar Fuentes]

Ted Zlatanov <tzz@lifelogs.com> writes:

> ÓF> Sharing the gnutls dll is so wrong at some many levels that I wont
> ÓF> start discussing it.
>
> I am puzzled by this.  Why is it wrong to share the GnuTLS DLL?

This is a common scenario on MS Windows: multiple providers of binary
packages, multiple installers with different install policies even for
the same installer, lots of directories on PATH (each application lives
on its own directory and often wants to be listed on PATH), varying
policies about where a non-privileged user is allowed to put binaries,
multiple incompatible binary macropackages that provides the same
executables and libraries with the same names (Cygwin, MSYS, GnuWin32
and what-not), a lack of culture of system administration, a growing
tendency to rely on self-updating packages... and the list goes on.

If a user is informed about the need to fix GnuTLS (through the local
newspaper, I guess) his first reaction would be "GnuWhat? Is it on my
machine?" Next, as every desktop computer user would do, he performs a
full HD file search for the library ("and BTW, how is it named,
exactly?") After locating the instance (or multiple instances) he needs
to figure out the correct procedure to update it ("Was this installed
along something else? Has this be put here by an installer of some sort?
Does that installer offer an update method? What depends on this dll?
What's the installed version, and what's the compatible update? Is it
available somewhere? If I use this newer version which I found with a
Google search, can something break apart?")

Sure, for us it all looks very easy, but I suffered DLL hell a few times
and it is very frustrating. Can't imagine how can it be for a novice or
a less computer-savvy user.

For a Windows binary package to be robust, it must be as self-contained
as possible. Quality-wise, one of the best decisions I ever made was to
distribute the C/C++ MS runtime dlls along with the rest of my binaries,
no matter they are already installed on virtually all MS Windows
machines. Certain long-standing, very nasty bugs simply went away.

[snip]

Re: GnuTLS for W32

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

[snip]

>> Just set dynamic-library-alist as it contains ('gnutls
>> . "/path/to/gnutls/gnutls.dll") (or whatever are the right names).
>
> Putting DLLs in arbitrary places is not TRT.

Nobody is suggesting an *arbitrary* place. Where ELPA puts its files is
anything but arbitrary.

> We should put it in the
> same directory where emacs.exe lives, and then there's no need to do
> anything with dynamic-library-alist.

Depending on how Emacs was installed, that would require elevated
privileges, something ELPA can not ask for, AFAIK. Please keep in mind
that we (or at least Ted and I) are discussing the feasibility of using
ELPA for providing and updating the required dll for Emacs STARTTLS
capability to work on Windows.

>> Installing a dll intended to be used by Emacs on a shared place is
>> asking for trouble.
>
> Please give me the credit that when I do that I know what I'm doing.
> I have been doing that for years with no problems at all.

Yes, putting the dll along the emacs executable on the same directory
works, that's what I do with my own software. However, my applications
require a privileged user to upgrade or install new components (unless
the user is Administrator on a pre MS Windows Vista machine, something
that is as common as malware targeting those same users).

> I'm sure
> others have succeeded in learning a few simple rules of how to do this
> safely.  That's why an _option_ to do it available to knowledgeable
> users would be a bonus.  The default place should be where emacs.exe
> lives.

Precisely, knowledgeable users would have no problem with putting GnuTLS
along with emacs.exe. A bit annoying, yes. Not very admin-friendly,
too. The non-knowledgeable users would have a problem, because once they
configure Emacs to send mail, they will receive an error
message. Hopefully the message will include detailed, up-to-date
instructions on how to obtain and install the dll, but it would be much
better if Emacs asked: "for this operation Emacs needs to download and
install a library from the Emacs code repository. Do you want to
proceed?" and then everything goes on smoothly. In the general case,
that dll can't be installed by ELPA along emacs.exe, for the reasons
already explained.

Re: GnuTLS for W32

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> He's repeating that loading a dll with a pathname would require changes
>> at the C level.
>
> And I still submit that is so.
>
>> Just that if w32_delayed_load does not fit the bill, writing the
>> required function by tweaking w32_delayed_load is so easy that someone
>> like me who doesn't know the Emacs C dialect can do the job.
>
> You are welcome to do that and present a complete solution for the
> issue at hand.  Maybe then this thread will become anything but
> hand-waving.

So w32_delayed_load plus dynamic-library-alist is not enough to load a
dll from a precise directory? What remains to be solved?

Re: GnuTLS for W32

[Juanma Barranquero]

On Wed, Jan 4, 2012 at 20:21, Óscar Fuentes <ofv@wanadoo.es> wrote:

> For a Windows binary package to be robust, it must be as self-contained
> as possible. Quality-wise, one of the best decisions I ever made was to
> distribute the C/C++ MS runtime dlls along with the rest of my binaries,
> no matter they are already installed on virtually all MS Windows
> machines. Certain long-standing, very nasty bugs simply went away.

Assuming you're right, that seems like a wonderful side project. Why
not start a project to build a prepackaged Emacs binary installer for
Windows? Aside from installing DLLs, it could set up other things
(after asking the user, of course), like CUA mode, and other binaries,
like a better ftp than the standard one.

I do really believe that a user-friendly installer for Windows is a
good idea. I strongly *don't* believe the Emacs project should bear
the load of it.

    Juanma

Re: GnuTLS for W32

[Ted Zlatanov]

On Wed, 04 Jan 2012 20:21:49 +0100 Óscar Fuentes <ofv@wanadoo.es> wrote: 

ÓF> Ted Zlatanov <tzz@lifelogs.com> writes:
ÓF> Sharing the gnutls dll is so wrong at some many levels that I wont
ÓF> start discussing it.
>> 
>> I am puzzled by this.  Why is it wrong to share the GnuTLS DLL?

ÓF> This is a common scenario on MS Windows: multiple providers of binary
ÓF> packages, multiple installers with different install policies even for
ÓF> the same installer, lots of directories on PATH (each application lives
ÓF> on its own directory and often wants to be listed on PATH), varying
ÓF> policies about where a non-privileged user is allowed to put binaries,
ÓF> multiple incompatible binary macropackages that provides the same
ÓF> executables and libraries with the same names (Cygwin, MSYS, GnuWin32
ÓF> and what-not), a lack of culture of system administration, a growing
ÓF> tendency to rely on self-updating packages... and the list goes on.

OK, thanks for explaining.  I'm not convinced it's *better for the user*
that we use ELPA to deliver the DLL, but it seems like a pain to make a
non-Emacs solution work reliably, so we're better off making our
solution only for Emacs.  How unfortunate.

ÓF> For a Windows binary package to be robust, it must be as self-contained
ÓF> as possible. Quality-wise, one of the best decisions I ever made was to
ÓF> distribute the C/C++ MS runtime dlls along with the rest of my binaries,
ÓF> no matter they are already installed on virtually all MS Windows
ÓF> machines. Certain long-standing, very nasty bugs simply went away.

I see.  So, assuming we agree on the ELPA package approach, maybe we
should ship W32 Emacs with a gnutls-w32 package and the DLL and all the
load paths set up already, so it can self-update and it works by
default.

Ted

Re: GnuTLS for W32

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I see.  So, assuming we agree on the ELPA package approach, maybe we
> should ship W32 Emacs with a gnutls-w32 package and the DLL and all the
> load paths set up already, so it can self-update and it works by
> default.

I don't really see that the ELPA solution really helps much here.
Wouldn't it be easier just to include the gnutls DLL in the Emacs zip
file?  Problem solved.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Wed, 04 Jan 2012 20:21:49 +0100
> 
> This is a common scenario on MS Windows: multiple providers of binary
> packages, multiple installers with different install policies even for
> the same installer, lots of directories on PATH (each application lives
> on its own directory and often wants to be listed on PATH), varying
> policies about where a non-privileged user is allowed to put binaries,
> multiple incompatible binary macropackages that provides the same
> executables and libraries with the same names (Cygwin, MSYS, GnuWin32
> and what-not), a lack of culture of system administration, a growing
> tendency to rely on self-updating packages... and the list goes on.
> 
> If a user is informed about the need to fix GnuTLS (through the local
> newspaper, I guess) his first reaction would be "GnuWhat? Is it on my
> machine?" Next, as every desktop computer user would do, he performs a
> full HD file search for the library ("and BTW, how is it named,
> exactly?") After locating the instance (or multiple instances) he needs
> to figure out the correct procedure to update it ("Was this installed
> along something else? Has this be put here by an installer of some sort?
> Does that installer offer an update method? What depends on this dll?
> What's the installed version, and what's the compatible update? Is it
> available somewhere? If I use this newer version which I found with a
> Google search, can something break apart?")
> 
> Sure, for us it all looks very easy, but I suffered DLL hell a few times
> and it is very frustrating. Can't imagine how can it be for a novice or
> a less computer-savvy user.

You are describing a situation that existed on Windows 9X, it no
longer exists on modern machines.  DLLs are either versioned in their
names or use the SxS mechanism.

Take the GnuTLS example: the previous DLL of version 2.x was named
libgnutls-26.dll, while the new 3.x one is libgnutls-28.dll.  Use your
friendly depends.exe program, and you will see that programs that
depend on one of them (were linked with its import library) will
refuse to load the other.  The same is true of libintl, libiconv, and
all the other libraries many Windows ports of GNU software need.
Poof! the problem doesn't exist.

Please stop spreading this FUD, you are tripping people like Ted who
don't know better into wrong conclusions based on what hurt you (and
me) several years ago.  THERE'S NO SUCH PROBLEM ANYMORE!

> For a Windows binary package to be robust, it must be as self-contained
> as possible.

True, but irrelevant.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Wed, 04 Jan 2012 20:39:27 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> [snip]
> 
> >> Just set dynamic-library-alist as it contains ('gnutls
> >> . "/path/to/gnutls/gnutls.dll") (or whatever are the right names).
> >
> > Putting DLLs in arbitrary places is not TRT.
> 
> Nobody is suggesting an *arbitrary* place. Where ELPA puts its files is
> anything but arbitrary.

Yes, it is, for DLLs and other binaries.  Those should go where
binaries live, not where Lisp packages live.

> > We should put it in the
> > same directory where emacs.exe lives, and then there's no need to do
> > anything with dynamic-library-alist.
> 
> Depending on how Emacs was installed, that would require elevated
> privileges, something ELPA can not ask for, AFAIK.

FUD.  Emacs is not installed by corporate policies on Windows, so this
won't happen except in very rare cases.

> Please keep in mind that we (or at least Ted and I) are discussing
> the feasibility of using ELPA for providing and updating the
> required dll for Emacs STARTTLS capability to work on Windows.

Excellent.  But please do it right, or your changes will be rejected
flat out.  There's a certain organization to the Emacs tree, including
on Windows; please follow it, so we could give users reasonably clear
explanations for them to be able to maintain the tree.

> Yes, putting the dll along the emacs executable on the same directory
> works, that's what I do with my own software. However, my applications
> require a privileged user to upgrade or install new components

Then it's _your_ problem, and I'm sure you already have a solution for
it.  Having a DLL near the .exe is the recommended practice and a safe
one at that.  Please follow it, if you want to do this job.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Wed, 04 Jan 2012 20:42:32 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> He's repeating that loading a dll with a pathname would require changes
> >> at the C level.
> >
> > And I still submit that is so.
> >
> >> Just that if w32_delayed_load does not fit the bill, writing the
> >> required function by tweaking w32_delayed_load is so easy that someone
> >> like me who doesn't know the Emacs C dialect can do the job.
> >
> > You are welcome to do that and present a complete solution for the
> > issue at hand.  Maybe then this thread will become anything but
> > hand-waving.
> 
> So w32_delayed_load plus dynamic-library-alist is not enough to load a
> dll from a precise directory? What remains to be solved?

I didn't say it was and I didn't say it wasn't.  I didn't research
the issue enough to know a complete recipe, just enough to realize
that changes _are_ needed.

Re: GnuTLS for W32

[Ted Zlatanov]

On Wed, 04 Jan 2012 21:41:20 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> I see.  So, assuming we agree on the ELPA package approach, maybe we
>> should ship W32 Emacs with a gnutls-w32 package and the DLL and all the
>> load paths set up already, so it can self-update and it works by
>> default.

LMI> I don't really see that the ELPA solution really helps much here.
LMI> Wouldn't it be easier just to include the gnutls DLL in the Emacs zip
LMI> file?  Problem solved.  :-)

I'm concerned about GnuTLS updates after the install.  An ELPA package
could do that, a simple DLL drop couldn't.

Ted

Re: GnuTLS for W32

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> You are describing a situation that existed on Windows 9X, it no
> longer exists on modern machines.  DLLs are either versioned in their
> names or use the SxS mechanism.

Before putting too much hope into SxS I encourage you to read the
wikipedia page about it that you linked on a previous post.

> Take the GnuTLS example: the previous DLL of version 2.x was named
> libgnutls-26.dll, while the new 3.x one is libgnutls-28.dll.

The 3.x version is named -28 ?

> Use your friendly depends.exe program, and you will see that programs
> that depend on one of them (were linked with its import library) will
> refuse to load the other.  The same is true of libintl, libiconv, and
> all the other libraries many Windows ports of GNU software need.

Appending a number to a name doesn't solve the problem.

> Poof! the problem doesn't exist.

A simple experiment: put a libgnutls-26.dll on the system32
directory. It is shared, right? Now install cygwin or msys, or any of
multiple standalone applications which are cygwin-based, and put its
binary directory before system32 on PATH. See it?

Another experiment: build an application such as Emacs with VC++ 6 or
MinGW with the default settings. Now make it to use a dll (GnuTLS, an
image library...)  compiled with a modern release of VS. Unless such
library follows a very strict policy about resource handling (and
possibly other aspects) you are asking for problems. I'm sure that I
don't need to explain the dangers of mixing different major versions of
the C runtime library.

Think on a user that discovers that sending mail with Emacs just works
because some other package installed GnuTLS on some directory listed on
PATH. Time later he decides to uninstall the application and afterwards
tries to send an e-mail, just to notice to his dismay that it doesn't
work anymore. Confusing, isn't it?

And then there is the issue with security fixes... If Emacs enters the
bussiness of secure protocols provide a mechanism for notifying the user
about updates, and make the upate process a no-brainer. Do you think it
is serious to let the user with a broken GnuTLS for years? Either take
full responsability or dismiss it completely: "Emacs supports StartTLS
through GnuTLS. The library usually can be downloaded <here>. Such
library is not part of Emacs and no effort is made related to checking
its suitability, correctness, etc."

> Please stop spreading this FUD, you are tripping people like Ted who
> don't know better into wrong conclusions based on what hurt you (and
> me) several years ago.  THERE'S NO SUCH PROBLEM ANYMORE!

Please stop using inflamatory language and offensive assertions. I could
say that your real-world experience distributing, installing and
supporting software across heterogenous environments looks quite
limited, but I'll rather suppose that you were very lucky so far.

[snip]

Re: GnuTLS for W32

[chad]

On Jan 4, 2012, at 2:12 PM, Ted Zlatanov wrote:

> LMI> I don't really see that the ELPA solution really helps much here.
> LMI> Wouldn't it be easier just to include the gnutls DLL in the Emacs zip
> LMI> file?  Problem solved.  :-)
> 
> I'm concerned about GnuTLS updates after the install.  An ELPA package
> could do that, a simple DLL drop couldn't.

Reading through the thread, it seems likely that there are two main groups of W32-GnuTLS-interested parties:  those that almost never update a `working' emacs (ala Drew Adams' report about of emacs 21.3 users), and those that update very frequently.  I'd guess that a `stable emacs' w32 installer would be very helpful for the former (and agree with Juanma that it would be good for emacs use in general), while I'd expect the other group to be more comfortable with piecemeal updates of emacs/packages/DLLs/etc. 

If this guess matches reality, then I have a suggestion:  by policy put the DLLs with the binary, make the installer DTRT, and then add some sort of `update alert' facility that checks ELPA to notify users about new versions.  This facility would not necessarily update the software itself, but would notify users that a new version of the software exists, along with notes about the changes (especially the severity of the changes) and instructions for updating.  The installer-based default would only (somehow) bug the user about severe problems (such as security breaches for GnuTLS), pointing them at a new installer.  Users who opted in could be notified of all changes (perhaps displaying ChangeLogs or vc status messages at the far end).

*Chad

Re: GnuTLS for W32

[Óscar Fuentes]

Juanma Barranquero <lekktu@gmail.com> writes:

> On Wed, Jan 4, 2012 at 20:21, Óscar Fuentes <ofv@wanadoo.es> wrote:
>
>> For a Windows binary package to be robust, it must be as self-contained
>> as possible. Quality-wise, one of the best decisions I ever made was to
>> distribute the C/C++ MS runtime dlls along with the rest of my binaries,
>> no matter they are already installed on virtually all MS Windows
>> machines. Certain long-standing, very nasty bugs simply went away.
>
> Assuming you're right,

You can bet on it. At the beginning there was several versions of
MSVCRT.DLL floating around, some of them notoriously buggy. Of course,
everybody installed the dll on system32. The problem was partially fixed
by SxS, which essentially ensured that applications that didn't embed
manifests (and hence didn't required a specific version of the dll) used
the default, "safe" one provided by MS with the OS. That started with
Windows XP, although it doesn't protect you from inadvertently picking a
dll inside a directory that comes first on PATH. With Windows 2000, I
had to face one of the more frustrating bugs on my career: a few users
reported crashes, freezes and data corruption (on a DB app!). It took me
months to discover the problem for one of the users: a mass storage
device driver and accompanying backup utility installed their own
custom-modified MSVCRT.DLL on system32, which somehow caused my app to
freeze when certain gui action was performed. They didn't bothered to
use a different version string or id on the resources of the library, so
it reported itself as one of the "good" dlls. Then I started to put my
runtime dlls on the same directory as the rest of my binaries, and the
problems of those users disappeared. Most of them haven't that storage
device. The issue costed me a several hundred work hours, mostly trying
to desperately find bugs inside my application.

> that seems like a wonderful side project. Why
> not start a project to build a prepackaged Emacs binary installer for
> Windows? Aside from installing DLLs, it could set up other things
> (after asking the user, of course), like CUA mode, and other binaries,
> like a better ftp than the standard one.
>
> I do really believe that a user-friendly installer for Windows is a
> good idea. I strongly *don't* believe the Emacs project should bear
> the load of it.

I think that's more or less what Lennart is already doing, isn't it?

OTOH an installer that could act as an update tool for the dlls could be
interesting.

Re: GnuTLS for W32

[Ted Zlatanov]

On Wed, 4 Jan 2012 14:47:02 -0800 chad <yandros@gmail.com> wrote: 

c> I have a suggestion: by policy put the DLLs with the binary, make the
c> installer DTRT, and then add some sort of `update alert' facility
c> that checks ELPA to notify users about new versions.  This facility
c> would not necessarily update the software itself, but would notify
c> users that a new version of the software exists, along with notes
c> about the changes (especially the severity of the changes) and
c> instructions for updating.  The installer-based default would only
c> (somehow) bug the user about severe problems (such as security
c> breaches for GnuTLS), pointing them at a new installer.  Users who
c> opted in could be notified of all changes (perhaps displaying
c> ChangeLogs or vc status messages at the far end).

I really like your suggestion.  It lets us write the DLL deployment code
later or never, depending on what users want, but at first we will only
do the acceptable minimum.  It can also work with a standalone GnuTLS
W32 installer, if we ever decide that's a better approach.

GnuTLS provides gnutls_check_version() which we can use to dynamically
find out the version of the currently loaded GnuTLS DLL, by calling it
with a NULL.  So the package's version check should be fairly easy to
write, and its version string will simply match the GnuTLS release it's
tracking.

I think, to get this working, we need a list of critical ELPA packages
that Emacs will check for updates on startup and alert the user to
upgrade.  By default that list should be empty on all platforms, except
on W32 it will contain the "gnutls-w32" package.  The actual package
will have to live on the GNU ELPA site, so that will require a network
connection to be opened... this will almost certainly displease some
Emacs users if we make it a default, but I do think it's the right one.

As a user I would like to have such a list, and will probably add a few
packages to it for my personal use.

Ted

Re: GnuTLS for W32

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

[snip]

>> Depending on how Emacs was installed, that would require elevated
>> privileges, something ELPA can not ask for, AFAIK.
>
> FUD.  Emacs is not installed by corporate policies on Windows, so this
> won't happen except in very rare cases.

Have you used MS Windows Vista or Seven lately? If you install Emacs on
"Program Files" Emacs itself does not have write rights to its own
directory (actually, sometimes the OS can fool the app into believing
that it is creating or changing files there, but in reality it is
diverting the operation to somewhere else. Look for "File and Registry
Virtualization" for further info). You must fidle with directory
permissions (that is, shutting down a security measure) for Emacs, or
the user in front of the computer, to alter its contents without
escalating privileges.

[snip]

Re: GnuTLS for W32

[Juanma Barranquero]

On Thu, Jan 5, 2012 at 00:00, Óscar Fuentes <ofv@wanadoo.es> wrote:

> It took me
> months to discover the problem for one of the users: a mass storage
> device driver and accompanying backup utility installed their own
> custom-modified MSVCRT.DLL on system32, which somehow caused my app to
> freeze when certain gui action was performed. They didn't bothered to
> use a different version string or id on the resources of the library, so
> it reported itself as one of the "good" dlls.

That just means that someone should be hit in the head with a printed
copy of the full MSDN site. Repeatedly.

> Then I started to put my
> runtime dlls on the same directory as the rest of my binaries, and the
> problems of those users disappeared.

Wonderful. But GnuTLS is not "our" runtime DLL, not more than msvcrt.dll is.

Please understand: I'm not really arguing against installing the
GnuTLS DLL along the emacs.exe binary (though, as a separate project,
I still think it's better to install it on its own). I'm arguing
against, and will continue to fight, *distributing* it in the first
place with Emacs. We *are* *not* a binary distribution project, and we
only do it for Windows because most Windows users do not have a
building environment. Going that route means less programming
resources and more administrivia.

> I think that's more or less what Lennart is already doing, isn't it?

Sort of. IMHO, Lennart's EmacsW32 is a fork, because he includes some
changes that aren't just customizations (emacsclient is a prime
example).

> OTOH an installer that could act as an update tool for the dlls could be
> interesting.

Yes, definitely. As long as developing it and maintaining it is
anybody else's (= "not the Emacs w32 people") responsibility :-)

    Juanma

Re: GnuTLS for W32

[Óscar Fuentes]

Juanma Barranquero <lekktu@gmail.com> writes:

> On Thu, Jan 5, 2012 at 00:00, Óscar Fuentes <ofv@wanadoo.es> wrote:
>
>> It took me
>> months to discover the problem for one of the users: a mass storage
>> device driver and accompanying backup utility installed their own
>> custom-modified MSVCRT.DLL on system32, which somehow caused my app to
>> freeze when certain gui action was performed. They didn't bothered to
>> use a different version string or id on the resources of the library, so
>> it reported itself as one of the "good" dlls.
>
> That just means that someone should be hit in the head with a printed
> copy of the full MSDN site. Repeatedly.

It *also* means that depending on unknown third parties is asking for
trouble.

>> Then I started to put my
>> runtime dlls on the same directory as the rest of my binaries, and the
>> problems of those users disappeared.
>
> Wonderful. But GnuTLS is not "our" runtime DLL, not more than msvcrt.dll is.
>
> Please understand: I'm not really arguing against installing the
> GnuTLS DLL along the emacs.exe binary (though, as a separate project,
> I still think it's better to install it on its own). I'm arguing
> against, and will continue to fight, *distributing* it in the first
> place with Emacs. We *are* *not* a binary distribution project, and we
> only do it for Windows because most Windows users do not have a
> building environment. Going that route means less programming
> resources and more administrivia.

AFAIK Windows binaries are distributed from the GNU servers just because
someone volunteers to do the job, not because it is a requisite for the
release. So it should be up to those volunteers to decide if they want
to include those libraries (GnuTLS, image support, etc) on the binary
package.

And if someone volunteers for maintaining a system that updates those
libraries, it is the job he picked.

A slightly different issue is to decide if changes to Emacs sources are
allowed to do that chore on certain way, but then it is up to the
volunteers again and solid reasons should be given to reject those
contributions.

>> I think that's more or less what Lennart is already doing, isn't it?
>
> Sort of. IMHO, Lennart's EmacsW32 is a fork, because he includes some
> changes that aren't just customizations (emacsclient is a prime
> example).

IIRC Lennart also distributes an unpatched Emacs.

>> OTOH an installer that could act as an update tool for the dlls could be
>> interesting.
>
> Yes, definitely. As long as developing it and maintaining it is
> anybody else's (= "not the Emacs w32 people") responsibility :-)

Ah, yes, the Emacs w32 people. Now I understand your stance better (and
maybe Eli's). I think it would be unfair and unreasonable to make you
responsible of doing the job related to those libraries, but I also
think that you are not obliged in any way to provide binaries of
anything. Keeping the sources on a ready state is enough, thank
you. (Not saying that you are *obliged* to do that either.)

Re: GnuTLS for W32

[Juanma Barranquero]

On Thu, Jan 5, 2012 at 03:00, Óscar Fuentes <ofv@wanadoo.es> wrote:

> It *also* means that depending on unknown third parties is asking for
> trouble.

Yes. Unknown.

> AFAIK Windows binaries are distributed from the GNU servers just because
> someone volunteers to do the job, not because it is a requisite for the
> release.

The binary tarballs for Windows are more or less "blessed". They are
not a requisite for the release, because strictly speaking, nothing
Windows-related is a requisite for the release (I suppose an exception
would be fixing bugs related to data loss or security issues).

> So it should be up to those volunteers to decide if they want
> to include those libraries (GnuTLS, image support, etc) on the binary
> package.

The moment the packages are accesible from the official site, there's
certain responsibilities. For example, to issue security upgrades as
fast as possible.

> A slightly different issue is to decide if changes to Emacs sources are
> allowed to do that chore on certain way, but then it is up to the
> volunteers again and solid reasons should be given to reject those
> contributions.

So far, none of the ways that had been proposed has been convincing,
and solid reasons have been given against them. It's just that we are
not agreeing on what "solid reasons" mean. As far as I'm concerned,
using ELPA to distribute Windows DLLs is gross beyond description, for
example. Any mechanism that makes Emacs try to auto-upgrade itself is
also a no-no (in my view, I don't know Stefan and Chong's opinion).

> IIRC Lennart also distributes an unpatched Emacs.

Yes, though I think he doesn't update it very often (I haven't checked
recently and I could be wrong).

> Ah, yes, the Emacs w32 people. Now I understand your stance better (and
> maybe Eli's). I think it would be unfair and unreasonable to make you
> responsible of doing the job related to those libraries, but I also
> think that you are not obliged in any way to provide binaries of
> anything.

Responsibility and obligation are disjoint concepts. I don't mind the
load, but I hate accepting (not personally, but as a project) the
responsibility to do things, like compiling GnuTLS binaries and
distributing them, that are utterly disconnected from Emacs
development per se. The moment we do that, people will expect we also
provide up-to-date binaries for image libs, libxml2, d-bus, you name
it.

    Juanma

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Wed, 04 Jan 2012 17:12:01 -0500
> 
> LMI> I don't really see that the ELPA solution really helps much here.
> LMI> Wouldn't it be easier just to include the gnutls DLL in the Emacs zip
> LMI> file?  Problem solved.  :-)
> 
> I'm concerned about GnuTLS updates after the install.  An ELPA package
> could do that, a simple DLL drop couldn't.

That's true, but if we assume that an urgent need to upgrade GnuTLS
will not be too frequent, we can update it with each release and in
binaries of development snapshots.  That would probably do 80% of the
job, if not more.

That said, I'm not against an ELPA based solution.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Wed, 04 Jan 2012 18:16:26 -0500
> 
> I think, to get this working, we need a list of critical ELPA packages
> that Emacs will check for updates on startup and alert the user to
> upgrade.  By default that list should be empty on all platforms, except
> on W32 it will contain the "gnutls-w32" package.

Again, why are we treating MS-Windows specially?  Why shouldn't Emacs
issue the same alert on GNU and Unix systems, if GnuTLS is found to be
unavailable?  The fact that several distributions have Emacs depend on
GnuTLS does not mean all of them do or will, and let's not forget that
even in the year 2012 users can build their own Emacs (without GnuTLS).

Re: GnuTLS for W32

[joakim]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Wed, 4 Jan 2012 18:00:49 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 
>
> JB> 2012/1/4 Ted Zlatanov <tzz@lifelogs.com>:
>>> I am puzzled by this.  Why is it wrong to share the GnuTLS DLL?
>>> 
>>> If you and Eli and other W32 experts say a standalone self-updating
>>> installer that drops a shared GnuTLS DLL in a common area is a bad idea,
>
> JB> I think is a good idea. Fortunately, we don't have to discuss it,
> JB> because Óscar "won't start discussing it" ;-)
>
> I'd like to discuss it with anyone willing.  Joakim offered to help with
> the installer and I can probably follow a build recipe to make the DLLs,
> so practically it seems like we could do this.  I just want to make sure
> I don't find out in a few days or weeks that the installer approach is
> flawed for some obscure W32 reason, which Óscar sort of implied.
>
> On Wed, 04 Jan 2012 20:10:23 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 
>
> EZ> Maybe then this thread will become anything but hand-waving.
>
> I think it makes sense to discuss approaches--I had not even thought of
> the GnuTLS installer approach until we had this conversation.  Rushing
> into development can be fun, but can also hide good ideas.

Here is my advice based on some commercial installers I worked on.

- use NSIS to install all binaries. Keep all binaries isolated from
everything else, so no fiddling in system directories unless it in
absolutely necessary in which case you should abandon all hope and carry
on to the bitter end.

- NSIS is really a tiny quirky Forth language so you can do a lot of
stuff, including downloading updates, patching, etc. but keep it simple.

- NSIS won't produce the standard MSI installers(not last time I looked
  anyway) but this is just as well since MSI installers are not what
  they are advertised to be

- You can install all of Emacs with a NSIS installer, and distribute
  further update NSIS installers for particular components through ELPA.

Don't underestimate the number of ways a windows installer can fuck up.

If I were to do this I would make a build bot that produced daily
binaries of the installer of a complete Emacs installation including the
dll files. I would not bother with partial updating of particular dll:s
at this time.

Later, I would implement libffi support in Emacs, by including Guile as
an Emacs dependency. Then we could have uniform soft dll support, and
get a step on the way on including Guile in the Emacs core. But I digress.

>
> Ted
>

-- 
Joakim Verona

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Wed, 04 Jan 2012 23:34:50 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > You are describing a situation that existed on Windows 9X, it no
> > longer exists on modern machines.  DLLs are either versioned in their
> > names or use the SxS mechanism.
> 
> Before putting too much hope into SxS I encourage you to read the
> wikipedia page about it that you linked on a previous post.

Believe me, I have.  With today's PC ("on the one hand .... but on the
other hand") stance, you can no longer have articles with definitive
opinions.  Articles need to be "balanced", so they will dig any minor
problem to show "objectivity".  Building evidence on that is silly.

All I can say that "it works for me" on no less than 4 different XP
systems used for 4 different purposes, for 7 years, with not a single
problem that I can remember.

> > Take the GnuTLS example: the previous DLL of version 2.x was named
> > libgnutls-26.dll, while the new 3.x one is libgnutls-28.dll.
> 
> The 3.x version is named -28 ?

Yes.  The number comes from the build.  GnuTLS has some weird scheme
for numbering the releases, take a look at their configure script.
But that's not really relevant here.

> > Use your friendly depends.exe program, and you will see that programs
> > that depend on one of them (were linked with its import library) will
> > refuse to load the other.  The same is true of libintl, libiconv, and
> > all the other libraries many Windows ports of GNU software need.
> 
> Appending a number to a name doesn't solve the problem.

It does solve most of it, because programs look only for DLLs they
were linked against.

> A simple experiment: put a libgnutls-26.dll on the system32
> directory. It is shared, right? Now install cygwin or msys, or any of
> multiple standalone applications which are cygwin-based, and put its
> binary directory before system32 on PATH. See it?

People who install both MinGW and MSYS/Cygwin on the same system have
a lot of rope to hang themselves, if they act stupidly.  Putting a
non-system DLL in system32 is one such stupid act; having MSYS on your
PATH is another.  MSYS installer has an option to stay away of PATH (I
think it's the default); if you read the installation instructions
carefully during installation, you won't get into this trouble.  I
know, because I have MSYS installed on one of my machines, and have no
trouble at all, although the amount of overlap in DLLs is
considerable.

In addition, MSYS names most (if not all) of its DLLs differently,
msys-FOO-NN.dll, which also alleviates this problem.

Finally, for the umpteenth time: the default should be to put the DLL
where emacs.exe lives.  Putting it elsewhere is a bonus option for
experts.  So I have no idea why you keep hitting on this subject; it's
a side track, as far as getting GnuTLS and its updates to Emacs is
concerned.

> Another experiment: build an application such as Emacs with VC++ 6 or
> MinGW with the default settings. Now make it to use a dll (GnuTLS, an
> image library...)  compiled with a modern release of VS. Unless such
> library follows a very strict policy about resource handling (and
> possibly other aspects) you are asking for problems.

These problems are theoretical, we never heard about them here.
dynamic-library-alist is carefully constructed to accept only names of
DLLs that we know are safe, which solves at least part of the
potential for trouble.

And again, if someone mixes MinGW with MSVC, they are in trouble
already and need a lot of discipline to avoid shooting themselves in
the foot.

We were talking about the majority of the users, but your examples are
all from the expert land.  I think it's not a coincidence: there
simply are no such problems in the vast majority of installations
nowadays, in practice.

> Think on a user that discovers that sending mail with Emacs just works
> because some other package installed GnuTLS on some directory listed on
> PATH. Time later he decides to uninstall the application and afterwards
> tries to send an e-mail, just to notice to his dismay that it doesn't
> work anymore. Confusing, isn't it?

It isn't confusing if Emacs displays a clear error message.

Again, this use case is for experts; by default the DLL should be with
emacs.exe.  Experts will know how to avoid that; most uninstallers ask
for an explicit permission to remove DLLs from public directories, and
experts know better than blindly clicking OK.

> > Please stop spreading this FUD, you are tripping people like Ted who
> > don't know better into wrong conclusions based on what hurt you (and
> > me) several years ago.  THERE'S NO SUCH PROBLEM ANYMORE!
> 
> Please stop using inflamatory language and offensive assertions.

Sorry, I cannot watch indifferently as people are talked into wrong
conclusions based on information that is several years obsolete.  It
is ridiculous to base design decision for a _future_ feature on
problems that last happened on Windows 2000, a system whose use today
is marginal at best.  Windows XP was released in 2001, and is already
obsolete, so any version prior to it is definitely so.

> I could say that your real-world experience distributing, installing
> and supporting software across heterogenous environments looks quite
> limited, but I'll rather suppose that you were very lucky so far.

You can call 7 years of safe use on 4 different machines luck if you
want.  I call it discipline and following safe practices.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Thu, 05 Jan 2012 00:00:57 +0100
> 
> You can bet on it. At the beginning there was several versions of
> MSVCRT.DLL floating around, some of them notoriously buggy. Of course,
> everybody installed the dll on system32. The problem was partially fixed
> by SxS, which essentially ensured that applications that didn't embed
> manifests (and hence didn't required a specific version of the dll) used
> the default, "safe" one provided by MS with the OS. That started with
> Windows XP, although it doesn't protect you from inadvertently picking a
> dll inside a directory that comes first on PATH. With Windows 2000, I
> had to face one of the more frustrating bugs on my career: a few users
> reported crashes, freezes and data corruption (on a DB app!). It took me
> months to discover the problem for one of the users: a mass storage
> device driver and accompanying backup utility installed their own
> custom-modified MSVCRT.DLL on system32, which somehow caused my app to
> freeze when certain gui action was performed. They didn't bothered to
> use a different version string or id on the resources of the library, so
> it reported itself as one of the "good" dlls. Then I started to put my
> runtime dlls on the same directory as the rest of my binaries, and the
> problems of those users disappeared. Most of them haven't that storage
> device. The issue costed me a several hundred work hours, mostly trying
> to desperately find bugs inside my application.

Conclusions based on experiences from Windows 2000 should be tossed as
irrelevant nowadays.  Citing this is a good "war story", but has no
bearing on design decisions for future features.

In addition, latest GnuTLS cannot be compiled with MinGW in a way that
will run on anything older than XP anyway.  (Maybe some non-trivial
tweaking could overcome that, but I didn't bother, and if Nikos built
the stock distribution, which is what I glean from his script, then
his binaries have the same limitation.)

So let's forget about Windows 2000; it's irrelevant for this thread,
if not for any other thread.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Thu, 05 Jan 2012 00:18:42 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> [snip]
> 
> >> Depending on how Emacs was installed, that would require elevated
> >> privileges, something ELPA can not ask for, AFAIK.
> >
> > FUD.  Emacs is not installed by corporate policies on Windows, so this
> > won't happen except in very rare cases.
> 
> Have you used MS Windows Vista or Seven lately? If you install Emacs on
> "Program Files" Emacs itself does not have write rights to its own
> directory (actually, sometimes the OS can fool the app into believing
> that it is creating or changing files there, but in reality it is
> diverting the operation to somewhere else. Look for "File and Registry
> Virtualization" for further info). You must fidle with directory
> permissions (that is, shutting down a security measure) for Emacs, or
> the user in front of the computer, to alter its contents without
> escalating privileges.

Irrelevant: we are talking about the same user who already installed
Emacs on that system, so that user will have already solved this
"problem" using one of the few available solutions, be it privilege
elevation, disabling UAC altogether, installing Emacs not in "Program
Files", or something else.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Thu, 05 Jan 2012 03:00:15 +0100
> 
> Juanma Barranquero <lekktu@gmail.com> writes:
> 
> > On Thu, Jan 5, 2012 at 00:00, Óscar Fuentes <ofv@wanadoo.es> wrote:
> >
> >> It took me
> >> months to discover the problem for one of the users: a mass storage
> >> device driver and accompanying backup utility installed their own
> >> custom-modified MSVCRT.DLL on system32, which somehow caused my app to
> >> freeze when certain gui action was performed. They didn't bothered to
> >> use a different version string or id on the resources of the library, so
> >> it reported itself as one of the "good" dlls.
> >
> > That just means that someone should be hit in the head with a printed
> > copy of the full MSDN site. Repeatedly.
> 
> It *also* means that depending on unknown third parties is asking for
> trouble.

If we put a DLL on ELPA or some other place and recommend it, it is no
longer "unknown".  The recommendation means someone has tested the DLL
and found it compatible with Emacs and free of nuisances.

Re: GnuTLS for W32

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1865 bytes --]

On 1/4/12 10:41 PM, Eli Zaretskii wrote:
>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Thu, 05 Jan 2012 00:00:57 +0100
>>
>> it reported itself as one of the "good" dlls. Then I started to put my
>> runtime dlls on the same directory as the rest of my binaries, and the
>> problems of those users disappeared. Most of them haven't that storage
>> device. The issue costed me a several hundred work hours, mostly trying
>> to desperately find bugs inside my application.
> 
> Conclusions based on experiences from Windows 2000 should be tossed as
> irrelevant nowadays.  Citing this is a good "war story", but has no
> bearing on design decisions for future features.

It underscores a general principle: ship applications as
self-contained units that don't try to muck with the rest of the
system. The only reason doing otherwise remotely feasible on Unixish
systems it the presence of package managers. On systems without
centralized package management, like Windows and OS X, shipping
self-contained packages is the only sane thing to do.

Microsoft even added COM features ("registration-free COM") to make
this approach easier.

With disks being as large as they are now, it makes no sense to try to
optimize for some resource sharing when you can just stick DLLs
alongside other downloaded files in perfect safety.

> In addition, latest GnuTLS cannot be compiled with MinGW in a way that
> will run on anything older than XP anyway.  (Maybe some non-trivial
> tweaking could overcome that, but I didn't bother, and if Nikos built
> the stock distribution, which is what I glean from his script, then
> his binaries have the same limitation.)
> 
> So let's forget about Windows 2000; it's irrelevant for this thread,
> if not for any other thread.

So we can, in fact, ditch ANSI support and use UNICODE everywhere?


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 235 bytes --]

Re: GnuTLS for W32

[Eli Zaretskii]

> Date: Wed, 04 Jan 2012 23:04:56 -0800
> From: Daniel Colascione <dancol@dancol.org>
> CC: Óscar Fuentes <ofv@wanadoo.es>, 
>  emacs-devel@gnu.org
> 
> > Conclusions based on experiences from Windows 2000 should be tossed as
> > irrelevant nowadays.  Citing this is a good "war story", but has no
> > bearing on design decisions for future features.
> 
> It underscores a general principle: ship applications as
> self-contained units that don't try to muck with the rest of the
> system.

I agree.

> > So let's forget about Windows 2000; it's irrelevant for this thread,
> > if not for any other thread.
> 
> So we can, in fact, ditch ANSI support and use UNICODE everywhere?

Unicode is not about Windows 2000; and I did say "for this thread".

Re: GnuTLS for W32

[Ted Zlatanov]

On Thu, 05 Jan 2012 00:36:57 -0500 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> 
>> I think, to get this working, we need a list of critical ELPA packages
>> that Emacs will check for updates on startup and alert the user to
>> upgrade.  By default that list should be empty on all platforms, except
>> on W32 it will contain the "gnutls-w32" package.

EZ> Again, why are we treating MS-Windows specially?  Why shouldn't Emacs
EZ> issue the same alert on GNU and Unix systems, if GnuTLS is found to be
EZ> unavailable?  The fact that several distributions have Emacs depend on
EZ> GnuTLS does not mean all of them do or will, and let's not forget that
EZ> even in the year 2012 users can build their own Emacs (without
EZ> GnuTLS).

You're right.  Do you agree with the general idea of checking for
critical updates on startup, though?

I would actually also like to bundle trusted certificates.  The
Diginotar compromise showed the need for managing the certs proactively,
and we can't rely on what's on W32 systems.  On other platforms we can
let the distribution choose the right cert bundle.  Right now, gnutls.el
will take a list of trustfiles or default to
/etc/ssl/certs/ca-certificates.crt, which is hardly ideal for all
platforms.

On Thu, 5 Jan 2012 03:36:38 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> The moment the packages are accesible from the official site, there's
JB> certain responsibilities. For example, to issue security upgrades as
JB> fast as possible.
...
JB> Responsibility and obligation are disjoint concepts. I don't mind the
JB> load, but I hate accepting (not personally, but as a project) the
JB> responsibility to do things, like compiling GnuTLS binaries and
JB> distributing them, that are utterly disconnected from Emacs
JB> development per se. The moment we do that, people will expect we also
JB> provide up-to-date binaries for image libs, libxml2, d-bus, you name
JB> it.

I think the risk of providing out-of-date libxml2 or libxpm is much
smaller than providing out-of-date GnuTLS.  So while I understand your
concern about this slippery slope, I think we can resist it, and regular
releases can address the general need for updates.

On Thu, 05 Jan 2012 00:24:56 -0500 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>

>> I'm concerned about GnuTLS updates after the install.  An ELPA package
>> could do that, a simple DLL drop couldn't.

EZ> That's true, but if we assume that an urgent need to upgrade GnuTLS
EZ> will not be too frequent, we can update it with each release and in
EZ> binaries of development snapshots.  That would probably do 80% of the
EZ> job, if not more.

Unfortunately security issues and their fixes are inherently urgent and
unpredictable (e.g. the Diginotar compromise).  Note above about my
desire to also provide cert bundles in this package, so it could do much
more than a DLL drop.  But if we go with Joakim's full installer idea,
that would do the job better than package.el could.

On Thu, 05 Jan 2012 06:40:27 +0100 joakim@verona.se wrote: 

j> If I were to do this I would make a build bot that produced daily
j> binaries of the installer of a complete Emacs installation including the
j> dll files. I would not bother with partial updating of particular dll:s
j> at this time.

If we tell the user to reinstall because GnuTLS is out of date, would
that be a big burden?  I guess that's a fourth option for distributing
and updating GnuTLS, which could be combined with the ELPA package for
notifications:

1) zipfile drop

2) ELPA package in the GNU ELPA archive with startup GnuTLS version check

3) GnuTLS standalone installer+updater

4) Emacs installer+updater

Combining (4) and (2) seems most convenient for the users: they will
have a single installer for all of Emacs (a convenience that goes beyond
this thread), and they'll get notified on all platforms when GnuTLS is
out of date.

Ted

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Thu, 05 Jan 2012 08:50:10 -0500
> 
> Do you agree with the general idea of checking for critical updates
> on startup, though?

I have nothing against it, provided that it's not a nuisance (e.g., I
don't always want to go on-line when I start Emacs).

> I would actually also like to bundle trusted certificates.

Where should they be gotten and how to integrate them with GnuTLS?

> Right now, gnutls.el will take a list of trustfiles or default to
> /etc/ssl/certs/ca-certificates.crt, which is hardly ideal for all
> platforms.

That directory most probably won't exist on Windows.

> If we tell the user to reinstall because GnuTLS is out of date, would
> that be a big burden?

It could be, since Emacs is a large distribution, and GnuTLS libraries
are much smaller in comparison.

Re: GnuTLS for W32

[Juanma Barranquero]

2012/1/5 Ted Zlatanov <tzz@lifelogs.com>:

> You're right.  Do you agree with the general idea of checking for
> critical updates on startup, though?

FWIW, I don't. That is a step (tiny, I know) in the "software as a
service" direction.

> I think the risk of providing out-of-date libxml2 or libxpm is much
> smaller than providing out-of-date GnuTLS.

If we don't provide GnuTLS libraries, the risk of providing
out-of-date ones is zero.

> So while I understand your
> concern about this slippery slope, I think we can resist it, and regular
> releases can address the general need for updates.

Why? We're not a binary shop. Why go that route?

> Combining (4) and (2) seems most convenient for the users: they will
> have a single installer for all of Emacs (a convenience that goes beyond
> this thread), and they'll get notified on all platforms when GnuTLS is
> out of date.

Does that mean that my Emacs is going to automatically try to
establish a network connection without asking me? Or that I'm gonna be
asked every time?

    Juanma

Re: GnuTLS for W32

[joakim]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Thu, 05 Jan 2012 00:36:57 -0500 Eli Zaretskii <eliz@gnu.org> wrote: 
>
>>> From: Ted Zlatanov <tzz@lifelogs.com>
>>> 
>>> I think, to get this working, we need a list of critical ELPA packages
>>> that Emacs will check for updates on startup and alert the user to
>>> upgrade.  By default that list should be empty on all platforms, except
>>> on W32 it will contain the "gnutls-w32" package.
>
> EZ> Again, why are we treating MS-Windows specially?  Why shouldn't Emacs
> EZ> issue the same alert on GNU and Unix systems, if GnuTLS is found to be
> EZ> unavailable?  The fact that several distributions have Emacs depend on
> EZ> GnuTLS does not mean all of them do or will, and let's not forget that
> EZ> even in the year 2012 users can build their own Emacs (without
> EZ> GnuTLS).
>
> You're right.  Do you agree with the general idea of checking for
> critical updates on startup, though?
>
> I would actually also like to bundle trusted certificates.  The
> Diginotar compromise showed the need for managing the certs proactively,
> and we can't rely on what's on W32 systems.  On other platforms we can
> let the distribution choose the right cert bundle.  Right now, gnutls.el
> will take a list of trustfiles or default to
> /etc/ssl/certs/ca-certificates.crt, which is hardly ideal for all
> platforms.
>
> On Thu, 5 Jan 2012 03:36:38 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 
>
> JB> The moment the packages are accesible from the official site, there's
> JB> certain responsibilities. For example, to issue security upgrades as
> JB> fast as possible.
> ...
> JB> Responsibility and obligation are disjoint concepts. I don't mind the
> JB> load, but I hate accepting (not personally, but as a project) the
> JB> responsibility to do things, like compiling GnuTLS binaries and
> JB> distributing them, that are utterly disconnected from Emacs
> JB> development per se. The moment we do that, people will expect we also
> JB> provide up-to-date binaries for image libs, libxml2, d-bus, you name
> JB> it.
>
> I think the risk of providing out-of-date libxml2 or libxpm is much
> smaller than providing out-of-date GnuTLS.  So while I understand your
> concern about this slippery slope, I think we can resist it, and regular
> releases can address the general need for updates.
>
> On Thu, 05 Jan 2012 00:24:56 -0500 Eli Zaretskii <eliz@gnu.org> wrote: 
>
>>> From: Ted Zlatanov <tzz@lifelogs.com>
>
>>> I'm concerned about GnuTLS updates after the install.  An ELPA package
>>> could do that, a simple DLL drop couldn't.
>
> EZ> That's true, but if we assume that an urgent need to upgrade GnuTLS
> EZ> will not be too frequent, we can update it with each release and in
> EZ> binaries of development snapshots.  That would probably do 80% of the
> EZ> job, if not more.
>
> Unfortunately security issues and their fixes are inherently urgent and
> unpredictable (e.g. the Diginotar compromise).  Note above about my
> desire to also provide cert bundles in this package, so it could do much
> more than a DLL drop.  But if we go with Joakim's full installer idea,
> that would do the job better than package.el could.
>
> On Thu, 05 Jan 2012 06:40:27 +0100 joakim@verona.se wrote: 
>
> j> If I were to do this I would make a build bot that produced daily
> j> binaries of the installer of a complete Emacs installation including the
> j> dll files. I would not bother with partial updating of particular dll:s
> j> at this time.
>
> If we tell the user to reinstall because GnuTLS is out of date, would
> that be a big burden?  I guess that's a fourth option for distributing
> and updating GnuTLS, which could be combined with the ELPA package for
> notifications:
>
> 1) zipfile drop
>
> 2) ELPA package in the GNU ELPA archive with startup GnuTLS version check
>
> 3) GnuTLS standalone installer+updater
>
> 4) Emacs installer+updater
>
> Combining (4) and (2) seems most convenient for the users: they will
> have a single installer for all of Emacs (a convenience that goes beyond
> this thread), and they'll get notified on all platforms when GnuTLS is
> out of date.

I think using a patcher should be sufficient:
http://wiz0u.free.fr/prog/WPatch/
That way we get a one stop solution for all changes.

Surely there must be an existing nsis installer for Emacs somewhere? If
there isn't I can provide a skeleton for someone else to tweak. Emacs
ought to be fairly easy to make an installer for since you basically
just copy the binaries somewhere and run out of tree. (I suppose, I'm
not familiar with how people run Emacs on Windows these days)

BTW the Nsis compiler can run on Gnu/Linux using Wine. There is a native
version as well but I never got it working properly. Also you will be
surprised at how arcane the Nsis language is(partly an artefact of the
domain), so one is often tempted to use something else. Last time I
checked the alternatives were worse, but that might have changed.



> Ted
>

-- 
Joakim Verona

Re: GnuTLS for W32

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

[snip]

>> I could say that your real-world experience distributing, installing
>> and supporting software across heterogenous environments looks quite
>> limited, but I'll rather suppose that you were very lucky so far.
>
> You can call 7 years of safe use on 4 different machines luck if you
> want.  I call it discipline and following safe practices.

Wait a minute. You are basing the points you so vehemently defend on
just personal experience with your own machines? Does that make sense at
all? Do you think that people is so disciplined and knowledgeable (and
lucky!) as you and that makes all the problems you don't have to become
irrelevant for Emacs?

That last paragraph of yours speaks tons about our different stances on
this issue. It is my direct responsability to keep my software up and
running for hundreds of users across dozens of sites and configurations
on machines I don't control, quite a few of them playing a critical
role, and deal with bug reports from tens of thousands of users
more. Acting as if the experience I gather from my desktop were enough
to set the bar for the rest of world would be so foolish that I would be
out of bussiness and sued after the three first months. Maybe Emacs
doesn't need so much careful thinking as mission-critical software does,
but basing design decissions onto very limited personal experiences
looks quite wrong too.

Re: GnuTLS for W32

[Lars Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> You're right.  Do you agree with the general idea of checking for
> critical updates on startup, though?

You didn't ask me, but I certainly do not.  

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: GnuTLS for W32

[Óscar Fuentes]

joakim@verona.se writes:

> Here is my advice based on some commercial installers I worked on.

[snipped a chunk of good advice]

> If I were to do this I would make a build bot that produced daily
> binaries of the installer of a complete Emacs installation including the
> dll files. I would not bother with partial updating of particular dll:s
> at this time.

I used NSIS lots of years ago. Finally abandoned it because its poor
support for partial updates. At the time some people were discussing the
possibility of adding a component for updating a package through the
Internet. Like a network installer but with the added capability of
re-running and updating the previously installed files. Guess that
nothing came out of it.

> Later, I would implement libffi support in Emacs, by including Guile as
> an Emacs dependency. Then we could have uniform soft dll support, and
> get a step on the way on including Guile in the Emacs core. But I digress.

I'm eager to see how well Guile competes with Emacs Lisp as a
convenience for extending Emacs :-)

Re: GnuTLS for W32

[chad]

On Jan 5, 2012, at 6:50 AM, Juanma Barranquero wrote:

> 2012/1/5 Ted Zlatanov <tzz@lifelogs.com>:
> 
>> You're right.  Do you agree with the general idea of checking for
>> critical updates on startup, though?
> 
> FWIW, I don't. That is a step (tiny, I know) in the "software as a
> service" direction.

I'm a little surprised to see the suggestion that `checking for critical updates' is objectionable because it's too much like `software as a service' in your mind.  Out of curiosity, do you assemble the pieces of your operating system by hand and manually check for updates yourself? Do you think that most GNU/Linux distributions are too much like `software as a service' for the same reasons?

> Does that mean that my Emacs is going to automatically try to
> establish a network connection without asking me? Or that I'm gonna be
> asked every time?

It would mean that the default Emacs would automatically try to establish a network connection without asking you, yes. If you believe that the default user is opposed to this, I'll suggest that you might not have noticed them all voting with their feet in favor of this at least a decade ago.  This isn't even the old `Windows Majority', even - I can't think of a computing system today that meets the criteria ``might run Emacs 24.2'' and ``does NOT somehow check the network for critical updates in the default installation''.

*Chad

Re: GnuTLS for W32

[Ted Zlatanov]

On Thu, 05 Jan 2012 16:37:04 +0100 Lars Ingebrigtsen <larsi@gnus.org> wrote: 

LI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> You're right.  Do you agree with the general idea of checking for
>> critical updates on startup, though?

LI> You didn't ask me, but I certainly do not.  

I certainly value your opinion.  Could you explain why you disagree with
checking critical packages (just GnuTLS currently)?  How would you
propose letting the user know they are out of date, instead of this?

On Thu, 5 Jan 2012 15:50:40 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> 2012/1/5 Ted Zlatanov <tzz@lifelogs.com>:

>> You're right.  Do you agree with the general idea of checking for
>> critical updates on startup, though?

JB> FWIW, I don't. That is a step (tiny, I know) in the "software as a
JB> service" direction.

Not at all.  It's just a convenience based on our desire to take
responsibility for the security of the software we provide.

>> Combining (4) and (2) seems most convenient for the users: they will
>> have a single installer for all of Emacs (a convenience that goes beyond
>> this thread), and they'll get notified on all platforms when GnuTLS is
>> out of date.

JB> Does that mean that my Emacs is going to automatically try to
JB> establish a network connection without asking me? Or that I'm gonna be
JB> asked every time?

It will be configurable and transparent when possible, but yes, at some
point it may ask you once.

If we have a W32 installer I'd make it a checkbox during the install.

On Thu, 05 Jan 2012 09:14:11 -0500 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> I would actually also like to bundle trusted certificates.

EZ> Where should they be gotten and how to integrate them with GnuTLS?

(Note this is speculative, I don't know for sure we should do this, but
certainly on W32 the cert bundle has to come from somewhere.)

I think it's safest to use Mozilla's cert bundle but I may sync with
Debian's bundle instead.

They don't integrate with GnuTLS as a library, but rather they are given
to it by gnutls.el.  So it would be maintenance and special cases in
gnutls.el, not in C code.

Our list of certs may diverge from what's built into the OS (e.g. RHEL
vs. Debian vs. Mac OS X).  There's no way to fix that, we have to let
the user choose, and by default use the OS cert bundle when it's
feasible.

>> If we tell the user to reinstall because GnuTLS is out of date, would
>> that be a big burden?

EZ> It could be, since Emacs is a large distribution, and GnuTLS libraries
EZ> are much smaller in comparison.

Maybe the wpatch Joakim mentioned would help here.  But yeah, I see the
problem, and yet everyone (IIUC) is saying a bundled install is the
safest way instead of trying to update DLLs directly.

Ted

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Thu, 05 Jan 2012 16:17:12 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> [snip]
> 
> >> I could say that your real-world experience distributing, installing
> >> and supporting software across heterogenous environments looks quite
> >> limited, but I'll rather suppose that you were very lucky so far.
> >
> > You can call 7 years of safe use on 4 different machines luck if you
> > want.  I call it discipline and following safe practices.
> 
> Wait a minute. You are basing the points you so vehemently defend on
> just personal experience with your own machines? Does that make sense at
> all? Do you think that people is so disciplined and knowledgeable (and
> lucky!) as you and that makes all the problems you don't have to become
> irrelevant for Emacs?

Most "people" do not install multiple packages, such as MinGW and
MSYS, that could potentially conflict on the same machine.

And yes, I think my experience is fairly common to those who do.

> That last paragraph of yours speaks tons about our different stances on
> this issue. It is my direct responsability to keep my software up and
> running for hundreds of users across dozens of sites and configurations
> on machines I don't control, quite a few of them playing a critical
> role, and deal with bug reports from tens of thousands of users
> more.

Does that involve installing MinGW and MSYS/Cygwin on the same
machine?  Or mixing MSVC 6 with MSVC 13?

> Acting as if the experience I gather from my desktop were enough
> to set the bar for the rest of world would be so foolish

I never set that this bar is for everyone.  For the umpteenth time: BY
DEFAULT THE DLLS SHOULD GO TO THE SAME DIRECTORY AS EMACS.EXE.  Let go!

Re: GnuTLS for W32

[Lars Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I certainly value your opinion.  Could you explain why you disagree with
> checking critical packages (just GnuTLS currently)?

If every program did stuff like that, using a computer would be
untenable.

$ gcc file.c
Checking for new versions of gcc...
New version of gcc found.  Do you want to upgrade? (y/n)

> How would you propose letting the user know they are out of date,
> instead of this?

I wouldn't.

Emacs under Linux doesn't check for stuff like that.  If the user
believes that there's something critical going on that needs updating,
the user says "apt-get update; apt-get upgrade".  If the user believes
that Emacs under Windows needs updating, the user will download the zip
file as usual.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: GnuTLS for W32

[Ted Zlatanov]

On Thu, 05 Jan 2012 19:29:27 +0100 Lars Ingebrigtsen <larsi@gnus.org> wrote: 

LI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> I certainly value your opinion.  Could you explain why you disagree with
>> checking critical packages (just GnuTLS currently)?

LI> If every program did stuff like that, using a computer would be
LI> untenable.

LI> $ gcc file.c
LI> Checking for new versions of gcc...
LI> New version of gcc found.  Do you want to upgrade? (y/n)

GCC, unlike Emacs, is not a self-sufficient environment.  Compare Emacs
to Chrome and Firefox... which, not surprisingly, do tell you about
updates, and they are both layout engines coupled with an interpreter,
like Emacs.

Also, I agree it's silly to interrupt batch mode with questions, as your
example shows.  This update check (which, again, I'm only proposing for
GnuTLS, and can be easily disabled) would be turned off in
batch/noninteractive modes.

>> How would you propose letting the user know they are out of date,
>> instead of this?

LI> I wouldn't.

LI> Emacs under Linux doesn't check for stuff like that.

Emacs' package.el can check and update packages.

LI> If the user believes that there's something critical going on that
LI> needs updating, the user says "apt-get update; apt-get upgrade".  If
LI> the user believes that Emacs under Windows needs updating, the user
LI> will download the zip file as usual.

The user doesn't know, usually, that there's been a critical GnuTLS
release that affects them.  Unlike normal updates, ignoring this can
actually compromise their security, not just corrupt or expose their
data.  This is a crucial distinction.  So I want Emacs to notify the
user their GnuTLS is out of date, or else something else should
(e.g. the self-contained GnuTLS updater for W32 I proposed).

Ted

Re: GnuTLS for W32

[Juanma Barranquero]

On Thu, Jan 5, 2012 at 17:19, chad <yandros@gmail.com> wrote:

> Out of curiosity, do you assemble the pieces of your operating system by hand

No, because I'm not offered that alternative in the system I use.

> and manually check for updates yourself?

I disable automatic checking and do manual checks every now and then, yes.

> Do you think that most GNU/Linux distributions are too much like `software as a service' for the same reasons?

Certainly I don't like much the way GNU/Linux distributions are going.

> It would mean that the default Emacs would automatically try to establish a network connection without asking you, yes.

If my vote counts, I'll vote against that being the default.

> If you believe that the default user is opposed to this, I'll suggest that you might not have noticed them all voting with their feet in favor of this at least a decade ago.  This isn't even the old `Windows Majority', even - I can't think of a computing system today that meets the criteria ``might run Emacs 24.2'' and ``does NOT somehow check the network for critical updates in the default installation''.

Last I checked, Emacs wasn't yet an operating system. There are plenty
of applications, many of them quite new, with no automatic checking
and/or upgrading.

    Juanma

Re: GnuTLS for W32

[Juanma Barranquero]

2012/1/5 Ted Zlatanov <tzz@lifelogs.com>:

> Not at all.  It's just a convenience based on our desire to take
> responsibility for the security of the software we provide.

Starting with the fact that we shouldn't be providing that software in
the first place...

> It will be configurable and transparent when possible, but yes, at some
> point it may ask you once.

Even if I don't use, and don't want to use, and don't plan to use, and
don't even want to know that the binary somehow included, the GnuTLS
DLL?

> If we have a W32 installer I'd make it a checkbox during the install.

If someone creates an installer for Emacs, I don't care what the defaults are.

    Juanma

Re: GnuTLS for W32

[Juanma Barranquero]

On Thu, Jan 5, 2012 at 19:29, Lars Ingebrigtsen <larsi@gnus.org> wrote:

> If every program did stuff like that, using a computer would be
> untenable.
>
> $ gcc file.c
> Checking for new versions of gcc...
> New version of gcc found.  Do you want to upgrade? (y/n)

Thanks! I was starting to feel alone in this.

>> How would you propose letting the user know they are out of date,
>> instead of this?
>
> I wouldn't.

+1,000

    Juanma

Re: GnuTLS for W32

[Richard Riley]

Juanma Barranquero <lekktu@gmail.com> writes:

> 2012/1/5 Ted Zlatanov <tzz@lifelogs.com>:
>
>> Not at all.  It's just a convenience based on our desire to take
>> responsibility for the security of the software we provide.
>
> Starting with the fact that we shouldn't be providing that software in
> the first place...
>
>> It will be configurable and transparent when possible, but yes, at some
>> point it may ask you once.
>
> Even if I don't use, and don't want to use, and don't plan to use, and
> don't even want to know that the binary somehow included, the GnuTLS
> DLL?
>
>> If we have a W32 installer I'd make it a checkbox during the install.
>
> If someone creates an installer for Emacs, I don't care what the
> defaults are.

Maybe not, but just about everyone else will. Its why they are
"defaults" : sensible choices.

>
>     Juanma
>

Re: GnuTLS for W32

[Ted Zlatanov]

On Thu, 5 Jan 2012 21:36:55 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> 2012/1/5 Ted Zlatanov <tzz@lifelogs.com>:
>> Not at all.  It's just a convenience based on our desire to take
>> responsibility for the security of the software we provide.

JB> Starting with the fact that we shouldn't be providing that software in
JB> the first place...

I mean Emacs.  Maybe you do too, I don't know anymore.

>> It will be configurable and transparent when possible, but yes, at some
>> point it may ask you once.

JB> Even if I don't use, and don't want to use, and don't plan to use, and
JB> don't even want to know that the binary somehow included, the GnuTLS
JB> DLL?

Yes, even you personally.

Ted

Re: GnuTLS for W32

[Juanma Barranquero]

2012/1/5 Ted Zlatanov <tzz@lifelogs.com>:

> I mean Emacs.  Maybe you do too, I don't know anymore.

I meant the GnuTLS DLL, but the Emacs binary too, if possible.
Unfortunately, on Windows is almost a requirement.

> Yes, even you personally.

Well, duh.

    Juanma

Re: GnuTLS for W32

[Juanma Barranquero]

On Thu, Jan 5, 2012 at 21:39, Richard Riley <rileyrg@gmail.com> wrote:

> Maybe not, but just about everyone else will. Its why they are
> "defaults" : sensible choices.

What I meant is that I won't use it** so I don't care which defaults
are defined. All of them are equally good to me.

**(Assuming we're talking about an installer for Emacs. I would use an
installer for GnuTLS as a separate package.)

    Juanma

Re: GnuTLS for W32

[chad]

On Jan 5, 2012, at 12:30 PM, Juanma Barranquero wrote:

> On Thu, Jan 5, 2012 at 17:19, chad <yandros@gmail.com> wrote:
> 
>> Out of curiosity, do you assemble the pieces of your operating system by hand
> 
> No, because I'm not offered that alternative in the system I use.
> 
>> and manually check for updates yourself?
> 
> I disable automatic checking and do manual checks every now and then, yes.

When you do manual checks, do you run a program that checks for updates, downloads them, and then installs them; or do you load up a web browser, visit some project pages from memory/bookmarks/etc, and start downloading and unpacking zip files?

I certainly don't think that everyone should be *required* to run an automatic critical-update-checker, but we're not talking about that - we're talking about the default setting.  That might involve you being asked a question once ever (something that's been built into emacs at least since I started using in in the 18.43 days), or adding a tiny bit of elisp to your set-up before being asked.

>> Do you think that most GNU/Linux distributions are too much like `software as a service' for the same reasons?
> 
> Certainly I don't like much the way GNU/Linux distributions are going.

Ok, I sympathize (I tend to disable auto-updaters on windows systems myself), but wasn't the question. I assume that you mention SoaS because you think that such systems are opposed to the FSF's and/or GNU project's goals, not just because you don't like them.

You asked, ``if your vote counts'', and - to me, at least - your vote definitely counts.  I'm trying to understand your reasoning for objecting to a default setting that would notify the user about critical issues.  Either I'm not understanding what you're saying, or you're saying that the default users shouldn't have a feature that many (I'd say `vast majority', but `many' is enough) because it might cause you to have to type `n' a few times, and that doesn't match what I expect from seeing your efforts on emacs-devel.

>> If you believe that the default user is opposed to this, I'll suggest that you might not have noticed them all voting with their feet in favor of this at least a decade ago.  This isn't even the old `Windows Majority', even - I can't think of a computing system today that meets the criteria ``might run Emacs 24.2'' and ``does NOT somehow check the network for critical updates in the default installation''.
> 
> There are plenty of applications, many of them quite new, with no automatic checking and/or upgrading.

I don't want to start a flame-war, but I really don't think this statement is true of user software.  Basically everything not hand-hacked on modern GNU/Linux, Mac OS X, or Windows system has an automatic checking (or checking and upgrading) system in place, built in to the application (web browsers, office suites, document/imaging systems, and games, for example) or the operating system.  To my knowledge, emacs is the *only* software I use under windows that doesn't do this, but I don't use windows very often, and mostly just for playing certain computer games. Can you suggest a few `user' applications that don't?

Perhaps this is a matter of nomenclature, but in my opinion, if the operating system's default-run package manager performs such a function, I believe that it counts.  Is that's the distinction you're drawing?

*Chad

Re: GnuTLS for W32

[Ted Zlatanov]

On Thu, 5 Jan 2012 23:43:29 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> 2012/1/5 Ted Zlatanov <tzz@lifelogs.com>:
>>> Not at all.  It's just a convenience based on our desire to take
>>> responsibility for the security of the software we provide.

JB> Starting with the fact that we shouldn't be providing that software in
JB> the first place...

JB> 2012/1/5 Ted Zlatanov <tzz@lifelogs.com>:
>> I mean Emacs.  Maybe you do too, I don't know anymore.

JB> I meant the GnuTLS DLL, but the Emacs binary too, if possible.
JB> Unfortunately, on Windows is almost a requirement.

I meant Emacs, the software, not just its binary form.  Forget the
binaries; you and Lars are protesting a startup check that critical
packages like GnuTLS are not out of date.  I'm saying that's a
convenience I think we should impose on our users at the cost of a
single y/n/never_bug_me_again prompt.  I can't think of a better way to
notify them that an Emacs component is out of date and possibly
compromising their security.  I believe it's our responsibility to do
this.

Ted

Re: GnuTLS for W32

[Juanma Barranquero]

On Fri, Jan 6, 2012 at 00:14, chad <yandros@gmail.com> wrote:

> When you do manual checks, do you run a program that checks for updates,
> downloads them, and then installs them; or do you load up a web browser,
> visit some project pages from memory/bookmarks/etc, and start downloading
> and unpacking zip files?

Well, for the operating system I do the first, because it is what the
system allows. For other software, I do visit the page, unload zip
files or whatever and unpack them.

> I'm trying to understand your reasoning for objecting to a default setting that
> would notify the user about critical issues.  Either I'm not understanding what
> you're saying, or you're saying that the default users shouldn't have a feature
> that many (I'd say `vast majority', but `many' is enough) because it might
> cause you to have to type `n' a few times, and that doesn't match what I
> expect from seeing your efforts on emacs-devel.

My objection is at a more fundamental level: we should not be
distributing binaries. For Windows, we are forced (more or less),
because most Windows users do not have a build environment, so we
should distribute the minimal binary that can possibly work and leave
options to the user.

The objection is twofold: on one hand, the more we do, the less
"customizable" the system is. Of course a dedicated user can change
anything, but defaults tend to be widely used and rarely questioned,
at least on systems with (relatively) unexperienced users, like
Windows. On the other hand, and as I've already said three or four
times, this is a software development project, not a packaging one. We
don't build an "Emacs distribution", we distribute Emacs source
tarballs. That's what I think we should continue doing. I see a lot of
people arguing how secure and convenient will be to have automatic
upgrades, and I wonder why nobody but me finds weird that we are
dedicating so much energy to discuss *binaries* in the first place. At
which moment did we switch goals? Once we have this wonderful system
for the Windows binaries, are we going to start distributing binary
tarballs for RedHat, Ubuntu or gNewSense? Is that what we want to do?
Certainly is not what I want to do, and it pains me seeing resources
diverted to that.

> To my knowledge, emacs is the *only* software I use under windows that doesn't do this, but I don't use windows very often, and mostly just for playing certain computer games. Can you suggest a few `user' applications that don't?

FreeOffice. Battle for Wesnoth. TrueCrypt. Just out of my head, I
haven't checked the software installed in my computer.

    Juanma

Re: GnuTLS for W32

[Juanma Barranquero]

2012/1/6 Ted Zlatanov <tzz@lifelogs.com>:

> I meant Emacs, the software, not just its binary form.  Forget the
> binaries; you and Lars are protesting a startup check that critical
> packages like GnuTLS are not out of date.

When you say that, you are not talking about gnutls.el, you are
talking about the GnuTLS binary, so no, I cannot forget the binaries.
That's the whole point of the discussion (at least, of the part of the
discussion I'm involved in).

> I can't think of a better way to
> notify them that an Emacs component is out of date and possibly
> compromising their security.

The GnuTLS binary is *not* an "Emacs component".

    Juanma

Re: GnuTLS for W32

[Richard Riley]

Juanma Barranquero <lekktu@gmail.com> writes:

> 2012/1/6 Ted Zlatanov <tzz@lifelogs.com>:
>
>> I meant Emacs, the software, not just its binary form.  Forget the
>> binaries; you and Lars are protesting a startup check that critical
>> packages like GnuTLS are not out of date.
>
> When you say that, you are not talking about gnutls.el, you are
> talking about the GnuTLS binary, so no, I cannot forget the binaries.
> That's the whole point of the discussion (at least, of the part of the
> discussion I'm involved in).
>
>> I can't think of a better way to
>> notify them that an Emacs component is out of date and possibly
>> compromising their security.
>
> The GnuTLS binary is *not* an "Emacs component".

Silly word play. It's a required component for Emacs in the context of
this discussion.

>
>     Juanma
>

Re: GnuTLS for W32

[Richard Riley]

Juanma Barranquero <lekktu@gmail.com> writes:

> On Fri, Jan 6, 2012 at 00:14, chad <yandros@gmail.com> wrote:
>
>> When you do manual checks, do you run a program that checks for updates,
>> downloads them, and then installs them; or do you load up a web browser,
>> visit some project pages from memory/bookmarks/etc, and start downloading
>> and unpacking zip files?
>
> Well, for the operating system I do the first, because it is what the
> system allows. For other software, I do visit the page, unload zip
> files or whatever and unpack them.
>
>> I'm trying to understand your reasoning for objecting to a default setting that
>> would notify the user about critical issues.  Either I'm not understanding what
>> you're saying, or you're saying that the default users shouldn't have a feature
>> that many (I'd say `vast majority', but `many' is enough) because it might
>> cause you to have to type `n' a few times, and that doesn't match what I
>> expect from seeing your efforts on emacs-devel.
>
> My objection is at a more fundamental level: we should not be
> distributing binaries. For Windows, we are forced (more or less),
> because most Windows users do not have a build environment, so we
> should distribute the minimal binary that can possibly work and leave
> options to the user.

The distribution should have everything the user might *optionally*
want. This kind of minimalist thinking doesnt exactly help new users
adopt emacs when they have no clue about building or fetching/installing
such components themsleves. Let the advanced users opt out, but dont cut
off the beginners and new users. Its silly. If distributing binaries
local to the emacs install helps people get going then why not? You can
disable this. You're an advanced user.

Re: GnuTLS for W32

[Juanma Barranquero]

On Fri, Jan 6, 2012 at 00:55, Richard Riley <rileyrg@gmail.com> wrote:

> Silly word play. It's a required component for Emacs in the context of
> this discussion.

No, it's not. Some people are arguing that it should be, and some are
treating it as if it already were, but it is not. There has not ever
been a single official release of Emacs including GnuTLS binaries.

The fact that Emacs can use it does not make it a component. libpng or
giflib are not Emacs components either.

    Juanma

Re: GnuTLS for W32

[Juanma Barranquero]

On Fri, Jan 6, 2012 at 00:58, Richard Riley <rileyrg@gmail.com> wrote:

> The distribution should have everything the user might *optionally* want.

That's just an opinion, and one I happen not to agree with.

> This kind of minimalist thinking doesnt exactly help new users
> adopt emacs when they have no clue about building or fetching/installing
> such components themsleves.

GnuTLS is not required to "adopt Emacs". I would say that, for a
Windows user, adding the image libraries would be more useful that
GnuTLS, because I bet most of them are not going to start using Emacs
to read e-mail or surf the web.

> If distributing binaries
> local to the emacs install helps people get going then why not?

For Windows users, enabling CUA mode by default will "help" them more.
But we don't do it.

But, as for "why not"... Why? Why us? Why cannot the people who is so
interested in doing it just set a side project to build an Emacs
installer, and be done with it?

> You can disable this. You're an advanced user.

Irrelevant.

    Juanma

Re: GnuTLS for W32

[Juanma Barranquero]

On Fri, Jan 6, 2012 at 00:32, Juanma Barranquero <lekktu@gmail.com> wrote:

> FreeOffice.

LibreOffice, I mean.

Steel Bank Common Lisp. InkScape. Wireshark. ActivePerl. Bazaar.
Dropbox. Apache. MySQL. GIMP...

    Juanma

Re: GnuTLS for W32

[Ted Zlatanov]

On Fri, 6 Jan 2012 00:38:41 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> 2012/1/6 Ted Zlatanov <tzz@lifelogs.com>:
>> I meant Emacs, the software, not just its binary form.  Forget the
>> binaries; you and Lars are protesting a startup check that critical
>> packages like GnuTLS are not out of date.

JB> When you say that, you are not talking about gnutls.el, you are
JB> talking about the GnuTLS binary, so no, I cannot forget the binaries.
JB> That's the whole point of the discussion (at least, of the part of the
JB> discussion I'm involved in).

No, what I was proposing was a startup check that the "gnutls-critical"
package is up to date, meaning what the user has installed is the
latest on the GNU ELPA.  This does not mean the latest GnuTLS is
installed.

The "gnutls-critical" package may do more afterwards, depending on the
OS.  On W32 it may trigger a patch eventually.  At first it will just
display a warning, as Chad suggested.  On GNU/Linux I think it should
leave the package management alone but still display a warning.

>> I can't think of a better way to notify them that an Emacs component
>> is out of date and possibly compromising their security.

JB> The GnuTLS binary is *not* an "Emacs component".

I think the C glue to GnuTLS is an Emacs component, deeply embedded.
The point of an exploit is that it can cross the barrier between "not a
component/not our problem" and "oh crap."

On Fri, 6 Jan 2012 01:05:36 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> GnuTLS is not required to "adopt Emacs". I would say that, for a
JB> Windows user, adding the image libraries would be more useful that
JB> GnuTLS, because I bet most of them are not going to start using Emacs
JB> to read e-mail or surf the web.

I believe `open-network-stream' can use GnuTLS for HTTPS connections,
which matters for a lot of cases, e.g. package.el.  I agree about the
image libraries, though, they should also be included in an installer.

JB> But, as for "why not"... Why? Why us? Why cannot the people who is so
JB> interested in doing it just set a side project to build an Emacs
JB> installer, and be done with it?

I need the "gnutls-critical" startup check or some other way to tell the
user their GnuTLS version is at risk *by default*.  This will be useful
on Mac OS X as well in some cases, as I mentioned.  That's all I need
from emacs-devel (so Stefan or Chong's approval, I guess); the rest of
the work will be on the GNU ELPA "gnutls-critical" package and a W32
installer, and does not need to involve anyone uninterested.

Ted

Re: GnuTLS for W32

[Juanma Barranquero]

2012/1/6 Ted Zlatanov <tzz@lifelogs.com>:

> No, what I was proposing was a startup check that the "gnutls-critical"
> package is up to date, meaning what the user has installed is the
> latest on the GNU ELPA.

At the end of the "gnutls-critical" chain, the intention is, either to
update non-binaries (gnutls.c, gnutls.el), or binaries (the DLL). In
the first case, I don't know why do we need such a special mechanism
(security releases have been handled before, just by issuing a new
release or an updated tarball); in the second case, you already know
my objections, so I won't repeat them again.

> The "gnutls-critical" package may do more afterwards, depending on the
> OS.  On W32 it may trigger a patch eventually.  At first it will just
> display a warning, as Chad suggested.

And then, we're going to implement something similar for image
libraries, because they can also have security-related bugs. Aren't
we?

We could also make our own MinGW/MSYS distribution, for people that
builds their own Windows Emacs. We would automatically upgrade it in
case there's a security issue. And let's not forget binutils, and
texinfo. Yes, I'm being facetious. Or not, I'm not sure anymore.

> I think the C glue to GnuTLS is an Emacs component, deeply embedded.
> The point of an exploit is that it can cross the barrier between "not a
> component/not our problem" and "oh crap."

Lots of code in Emacs calls external tools (from grep to nslookup to
make). Anyone of them could turn into an "oh crap" moment. But we
don't feel the impulse to distribute grep and make sure it is up to
date.

> I believe `open-network-stream' can use GnuTLS for HTTPS connections,
> which matters for a lot of cases, e.g. package.el.

I disagree with "a lot of cases". There are a few Emacs components
that connect to the network, but it is perfectly possible (and, I
think, even common) not to need them on Windows.

> I agree about the image libraries, though, they should also be included in an installer.

As long as you say "an installer" and do not say "automatically
check", I'm fine.

> I need the "gnutls-critical" startup check or some other way to tell the
> user their GnuTLS version is at risk *by default*.

s/need/want/.

    Juanma

Re: GnuTLS for W32

[chad]

On Jan 5, 2012, at 4:09 PM, Juanma Barranquero wrote:

> On Fri, Jan 6, 2012 at 00:32, Juanma Barranquero <lekktu@gmail.com> wrote:
> 
> LibreOffice, I mean.
> 
> Steel Bank Common Lisp. InkScape. Wireshark. ActivePerl. Bazaar.
> Dropbox. Apache. MySQL. GIMP...

DropBox updates itself without telling you:

	How do I upgrade to the latest version of the Dropbox application?

	[...] Dropbox will silently update itself in the background.

LibreOffice has automatic updates:

	http://help.libreoffice.org/Common/Online_Update

Bazaar tells you to use your package manager (although not on Windows):

	Use your package manager to upgrade to the latest version.

I stopped looking there.

Most of those are typically (that is, for the typical user/administrator) checked and managed by the operating system's package manager, as far as I can tell. Sure, people can manage them by hand (I have and still do myself), but that's certainly not *typical*.  When I still lived inside of GNU/Linux, I used kernel distributions only long enough to configure and build my own, but I certainly don't expect that kind of behavior from the typical Windows, Mac OS X, or Ubuntu user.  I also don't think it should be the default.  As I understand it, you seem to be saying that it should.

To be clear: I'm asking you why you object to an easily-disabled system that checks for the existence of updates deemed critical by a human being, and warns the user if it finds them. I'm not talking about distributing binaries (although there certainly are people who are saying it would be convenient if someone did that, too).

*Chad

Re: GnuTLS for W32

[Juanma Barranquero]

On Fri, Jan 6, 2012 at 02:05, chad <yandros@gmail.com> wrote:

> DropBox updates itself without telling you:
>
>        How do I upgrade to the latest version of the Dropbox application?
>
>        [...] Dropbox will silently update itself in the background.

No. DropBox tells that, but it is false. This morning I had to
manually upgrade it. Apparently it auto upgrades just for very minor
changes, but not to pass from 1.1.X to 1.2.X. The same happens, for
example, with ESET Smart Security: it updates the engine for minor
changes, but when they switched from release 4 to release 5, it didn't
even warn me.

> LibreOffice has automatic updates:

Not enabled by default: "Online Update is a module that can be
selected or deselected to be installed. Choose the customized
installation in the Setup of LibreOffice."

> Bazaar tells you to use your package manager (although not on Windows):

I'm glad we agree.

> I stopped looking there.

Please, continue.

> To be clear: I'm asking you why you object to an easily-disabled system that checks for the existence of updates deemed critical by a human being

I don't object to that. I object to that being part of Emacs. By all
means, build an installer for GnuTLS and put all automatic checking
you want on it. Make it part of the Windows startup if you want. But
don't make Emacs connect to Internet every time it starts just to
check that yes, that GnuTLS that you are not really using for anything
at all is still safe to use (or ignore).

    Juanma

Re: GnuTLS for W32

[Óscar Fuentes]

Juanma Barranquero <lekktu@gmail.com> writes:

[snip]

Please note that notifying the user about security vulnerabilities and
critical fixes has nothing to do with the source/binary discussion. Even
on a source-only world, it is a nice (maybe the correct word nowadays
would be "required") service to users to proactively inform them.

Re: GnuTLS for W32

[Juanma Barranquero]

On Fri, Jan 6, 2012 at 02:24, Óscar Fuentes <ofv@wanadoo.es> wrote:

> Please note that notifying the user about security vulnerabilities and
> critical fixes has nothing to do with the source/binary discussion.

No, but the source/binary discussion is being swept under the carpet
as if there were nothing to discuss and it was *evident* that we
should distribute the binaries. Perhaps you have noticed that I
disagree with that idea.

> Even
> on a source-only world, it is a nice (maybe the correct word nowadays
> would be "required") service to users to proactively inform them.

Yes. And I bet that in most source-only projects, like Emacs, the
usual way is to put a notice in the project web page, publish an
announcement in the relevant mailing lists and newsgroups, or send
people an e-mail if they are subscribed to e-mail notifications.

Which are perfectly fine ways to do it and won't get the tiniest
objection from me.

    Juanma

Re: GnuTLS for W32

[Óscar Fuentes]

Juanma Barranquero <lekktu@gmail.com> writes:

> On Fri, Jan 6, 2012 at 02:24, Óscar Fuentes <ofv@wanadoo.es> wrote:
>
>> Please note that notifying the user about security vulnerabilities and
>> critical fixes has nothing to do with the source/binary discussion.
>
> No, but the source/binary discussion is being swept under the carpet
> as if there were nothing to discuss and it was *evident* that we
> should distribute the binaries. Perhaps you have noticed that I
> disagree with that idea.

I have no objections to your disagreement towards this and your POV wrt
the Emacs project distributing binaries. Furthermore, if that's the
sentiment on the rest of the w32 emacs people and the project leaders,
I'll urge you to stop distributing MS Windows binaries. I'm pretty sure
that there will be no shortage of Emacs binaries for MS Windows.

>> Even
>> on a source-only world, it is a nice (maybe the correct word nowadays
>> would be "required") service to users to proactively inform them.
>
> Yes. And I bet that in most source-only projects, like Emacs, the
> usual way is to put a notice in the project web page, publish an
> announcement in the relevant mailing lists and newsgroups, or send
> people an e-mail if they are subscribed to e-mail notifications.

That's not proactive. I don't periodically visit the project the web
page of all sensitive software I use searching for critical updates, nor
subscribe to mailing lists, e-mail notifications (is there one for
Emacs?) nor read newsgroups. That's totally unrealistic as it would take
lots of time. Really, I can't see how you object to automatic checks for
critical updates. Even less can I understand how you object to that
feature in principle, not just as personal preference.

[snip]

Re: GnuTLS for W32

[Juanma Barranquero]

On Fri, Jan 6, 2012 at 03:37, Óscar Fuentes <ofv@wanadoo.es> wrote:

> Furthermore, if that's the
> sentiment on the rest of the w32 emacs people and the project leaders,
> I'll urge you to stop distributing MS Windows binaries. I'm pretty sure
> that there will be no shortage of Emacs binaries for MS Windows.

Oh, I'm not against distributing our binaries. Emphasis in "our".

> That's not proactive.

What's the proactive way to do it in a source-only project? Are you
suggesting that all projects do include some kind of run-time check? I
already gave a short list of some software in my computer that does
not take proactive action. I could add lots more, like git, mercurial,
MinGW, Take Command, Python, Evernote, etc. Some of them have a menu
option to check for updates, or installer programs with an update
option, but they don't do it automatically.

> Really, I can't see how you object to automatic checks for
> critical updates.

Because we don't have (or very rarely have) critical updates. Let
GnuTLS announce their critical updates any way they see fit.

> Even less can I understand how you object to that
> feature in principle, not just as personal preference.

It's a change from one model of development and distribution to
another one. I like the way Emacs is right now. Source. You compile
it, and have a binary that does not depend on some mythical,
externally maintained resource.

I don't like ELPA much, either, though I understand how can be useful.
But I'm distressed by the impulse to move things from Emacs to ELPA.

    Juanma

Re: GnuTLS for W32

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> The user doesn't know, usually, that there's been a critical GnuTLS
> release that affects them.  Unlike normal updates, ignoring this can
> actually compromise their security, not just corrupt or expose their
> data.

$ ssh gnu.org
Checking for updates to ssh...  please wait
Apparently somebody has made a brute-force attack feasible against
the encryption algorithm ssh was going to use against the remote server.
Download and install a new version of ssh?

> This is a crucial distinction.  So I want Emacs to notify the
> user their GnuTLS is out of date, or else something else should
> (e.g. the self-contained GnuTLS updater for W32 I proposed).

I don't really see that there's much of a difference between bugs in
libgnutls and in the Emacs binary proper.  If a major security hole was
discovered in Emacs, then presumably a new Emacs release would be made.
If a major libgnutls hole was discovered, then presumably someone would
zip up a new Windows release.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: GnuTLS for W32

[chad]

On Jan 5, 2012, at 7:15 PM, Lars Magne Ingebrigtsen wrote:
> If a major libgnutls hole was discovered, then presumably someone would
> zip up a new Windows release.

How would the users know to download it?

*Chad

Re: GnuTLS for W32

[Óscar Fuentes]

Juanma Barranquero <lekktu@gmail.com> writes:

> On Fri, Jan 6, 2012 at 03:37, Óscar Fuentes <ofv@wanadoo.es> wrote:
>
>> Furthermore, if that's the
>> sentiment on the rest of the w32 emacs people and the project leaders,
>> I'll urge you to stop distributing MS Windows binaries. I'm pretty sure
>> that there will be no shortage of Emacs binaries for MS Windows.
>
> Oh, I'm not against distributing our binaries. Emphasis in "our".

Then I'm misunderstanding. IIRC you said, more or less: "we are a source
code shop and MS Windows is an exception because those users would have
a hard time getting an Emacs running on their machines". I fully
sympathize with the "we are a source code shop", but at the same time
I'll like to remark that the rest is no longer true.

>> That's not proactive.
>
> What's the proactive way to do it in a source-only project? Are you
> suggesting that all projects do include some kind of run-time check?

We are on emacs-devel, not on all-projects-devel.

But now that you ask, yes, I'll appreciate that all projects would
include a system for notifying me that its software is putting my
machine at risk.

> I already gave a short list of some software in my computer that does
> not take proactive action. I could add lots more, like git, mercurial,
> MinGW, Take Command, Python, Evernote, etc. Some of them have a menu
> option to check for updates, or installer programs with an update
> option, but they don't do it automatically.

The key here is to determine what the Right Thing is. Have you
considered the possibility that some or most of those projects doesn't
have the automatic notification not because they think it is a bad idea,
but because some other reason?

>> Really, I can't see how you object to automatic checks for
>> critical updates.
>
> Because we don't have (or very rarely have) critical updates.

That's like saying that smoke detectors are unneeded because fires
rarely occur, if at all, on most housings.

> Let GnuTLS announce their critical updates any way they see fit.
>
>> Even less can I understand how you object to that
>> feature in principle, not just as personal preference.
>
> It's a change from one model of development and distribution to
> another one. I like the way Emacs is right now. Source. You compile
> it, and have a binary that does not depend on some mythical,
> externally maintained resource.

You are sidetracking from my question by going back to the GnuTLS
dll. I'm genuinely interested in your reasoning for rejecting an
automatic notification system built into Emacs. Something you can use to
warn users that a problem was found that would pose a risk to their data
(a security breach, data corruption, whatever). That's independent from
how the user obtained its binary package.

[snip]

Re: GnuTLS for W32

[Juanma Barranquero]

On Fri, Jan 6, 2012 at 04:56, Óscar Fuentes <ofv@wanadoo.es> wrote:

> Then I'm misunderstanding. IIRC you said, more or less: "we are a source
> code shop and MS Windows is an exception because those users would have
> a hard time getting an Emacs running on their machines". I fully
> sympathize with the "we are a source code shop", but at the same time
> I'll like to remark that the rest is no longer true.

I don't know what causes your misunderstanding. I would prefer not to
distribute binaries, can accept distributing our own because they are
useful, dislike the idea of distributing other project's binaries
unless they are *strictly* required to run our Emacs binary.

> We are on emacs-devel, not on all-projects-devel.

You were the one talking about "source-only worlds".

> But now that you ask, yes, I'll appreciate that all projects would
> include a system for notifying me that its software is putting my
> machine at risk.

If GnuTLS has a security issue, I wouldn't say that Emacs puts my
machine at risk. GnuTLS does.

> The key here is to determine what the Right Thing is. Have you
> considered the possibility that some or most of those projects doesn't
> have the automatic notification not because they think it is a bad idea,
> but because some other reason?

Why the second guessing? I was told that almost all software packages
today did automatic upgrading, and I mentioned some that do not. I
don't know why they don't offer it, and neither do you.

> That's like saying that smoke detectors are unneeded because fires
> rarely occur, if at all, on most housings.

Nonsense. It's like saying that a smoke detector is not needed in this
particular house because it is built with fireproof materials and the
likelihood of a fire is almost zero.

Do you have an smoke detector in your home? I don't. I don't have a
fire extinguisher, either.

> You are sidetracking from my question by going back to the GnuTLS dll.

No, I'm not.

> I'm genuinely interested in your reasoning for rejecting an
> automatic notification system built into Emacs.

That's what I've answered.

> Something you can use to
> warn users that a problem was found that would pose a risk to their data
> (a security breach, data corruption, whatever). That's independent from
> how the user obtained its binary package.

There are zillions of ways their data could be lost. Are you going to
add a program to Emacs to test the hard drive for bad spots? That kind
of checks (updates, I mean, not the disk test tool ;-) instill false
security. It's like the people who has an AV installed and thinks that
it is protected because the AV software has not detected anything.

    Juanma

Re: GnuTLS for W32

[chad]

[-- Attachment #1: Type: text/plain, Size: 1299 bytes --]

On Jan 5, 2012, at 8:11 PM, Juanma Barranquero wrote:
> 
> Why the second guessing? I was told that almost all software packages
> today did automatic upgrading, and I mentioned some that do not. I
> don't know why they don't offer it, and neither do you.

I suspect the answer is ``most of them do, either built in to the app, via a default package system, and/or via the operating system''.  You

> Do you have an smoke detector in your home? I don't. I don't have a
> fire extinguisher, either.

I do, as does basically everyone I know.  They're required in various ways here in the US. Perhaps we're ruining it for the rest of you. :)

Regardless, you have made it clear that you object to oranges because they might someday turn into apples.  To be explicit, here is the `sweeping under the carpet' to which you have repeatedly voiced objection:

On Jan 4, 2012, at 3:16 PM, Ted Zlatanov wrote:
> I really like your suggestion.  It lets us write the DLL deployment code
> later or never, depending on what users want, but at first we will only
> do the acceptable minimum.  It can also work with a standalone GnuTLS
> W32 installer, if we ever decide that's a better approach.


Apparently `later or never' is too much like `evident that we should' for you.

*Chad



[-- Attachment #2: Type: text/html, Size: 1832 bytes --]

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Richard Riley <rileyrg@gmail.com>
> Date: Fri, 06 Jan 2012 00:55:46 +0100
> 
> >> I can't think of a better way to
> >> notify them that an Emacs component is out of date and possibly
> >> compromising their security.
> >
> > The GnuTLS binary is *not* an "Emacs component".
> 
> Silly word play. It's a required component for Emacs in the context of
> this discussion.

As are grep, find, ls, ispell/aspell, and many others.

That way lies madness.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: Richard Riley <rileyrg@gmail.com>
> Date: Fri, 06 Jan 2012 00:58:23 +0100
> 
> The distribution should have everything the user might *optionally*
> want.

You are either not serious or didn't think about the consequences of
this approach.  On any system that is not GNU/Linux, this will require
us to package half of the GNU project into the binary distro.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: chad <yandros@gmail.com>
> Date: Thu, 5 Jan 2012 21:49:38 -0800
> 
> > Do you have an smoke detector in your home? I don't. I don't have a
> > fire extinguisher, either.
> 
> I do, as does basically everyone I know.  They're required in various ways here in the US. Perhaps we're ruining it for the rest of you. :)

Then I suggest that the check for updates be turned on by default for
US locales.

Re: GnuTLS for W32

[Juanma Barranquero]

On Fri, Jan 6, 2012 at 08:12, Eli Zaretskii <eliz@gnu.org> wrote:

> Then I suggest that the check for updates be turned on by default for
> US locales.

+1 :-)

    Juanma

Re: GnuTLS for W32

[Juanma Barranquero]

On Fri, Jan 6, 2012 at 06:49, chad <yandros@gmail.com> wrote:

> I suspect the answer is ``most of them do, either built in to the app, via a
> default package system, and/or via the operating system''.

Please tell me of a software project whose security issues would be
more far reaching that Apache. And still, not a single Apache I've
ever installed has tried to automatically check for updates.

> Regardless, you have made it clear that you object to oranges because they
> might someday turn into apples.

No. I object to oranges because, last time I checked, we were in the
business of selling seafood, not vegetables and fruit.

    Juanma

Re: GnuTLS for W32

[Ted Zlatanov]

On Fri, 6 Jan 2012 01:59:32 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> 2012/1/6 Ted Zlatanov <tzz@lifelogs.com>:
>> No, what I was proposing was a startup check that the "gnutls-critical"
>> package is up to date, meaning what the user has installed is the
>> latest on the GNU ELPA.

JB> At the end of the "gnutls-critical" chain, the intention is, either to
JB> update non-binaries (gnutls.c, gnutls.el), or binaries (the DLL).

The intention is to do whatever is appropriate on the platform to let
the user know they need to update and make the update easy.

>> The "gnutls-critical" package may do more afterwards, depending on the
>> OS.  On W32 it may trigger a patch eventually.  At first it will just
>> display a warning, as Chad suggested.

JB> And then, we're going to implement something similar for image
JB> libraries, because they can also have security-related bugs. Aren't
JB> we?

I'm not.  The risk is not worth the effort with image libraries.  The
risk outweighs the effort with GnuTLS, in my opinion.

>> I think the C glue to GnuTLS is an Emacs component, deeply embedded.
>> The point of an exploit is that it can cross the barrier between "not a
>> component/not our problem" and "oh crap."

JB> Lots of code in Emacs calls external tools (from grep to nslookup to
JB> make). Anyone of them could turn into an "oh crap" moment. But we
JB> don't feel the impulse to distribute grep and make sure it is up to
JB> date.

You're ignoring the "deeply embedded" part.  Obviously external
utilities are not able to compromise Emacs like internal C glue.  Can
you stick to comparable components like the libxml2 glue?

>> I believe `open-network-stream' can use GnuTLS for HTTPS connections,
>> which matters for a lot of cases, e.g. package.el.

JB> I disagree with "a lot of cases". There are a few Emacs components
JB> that connect to the network, but it is perfectly possible (and, I
JB> think, even common) not to need them on Windows.

If you don't think the package manager is important to our users, you've
got your head stuck in the sand.

>> I need the "gnutls-critical" startup check or some other way to tell the
>> user their GnuTLS version is at risk *by default*.

JB> s/need/want/.

I appreciate your attention to detail, but "need" is the verb I meant to
write there.

On Fri, 06 Jan 2012 04:15:28 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:

>> The user doesn't know, usually, that there's been a critical GnuTLS
>> release that affects them.  Unlike normal updates, ignoring this can
>> actually compromise their security, not just corrupt or expose their
>> data.

LMI> $ ssh gnu.org
LMI> Checking for updates to ssh...  please wait
LMI> Apparently somebody has made a brute-force attack feasible against
LMI> the encryption algorithm ssh was going to use against the remote server.
LMI> Download and install a new version of ssh?

Are we talking GNU/Linux, where the package manager will handle this
update?  Or, say, Putty on W32, where such an auto-update makes more
sense (I don't know if Putty updates itself but that's not the point)?

SSH clients are not extensible layout engines with embedded interpreters
and flexible package managers.  As I keep saying, compare Emacs to
Firefox and Chrome, not to `vim' or `ssh' and `grep'.  It hasn't been
just an editor in a long while.  Eclipse is another good comparison
point.

But you raise an interesting point: even without client updates, the
server admin may disable the algorithm (manually or through a
sshd_config update), and the SSH protocol will try another algorithm as
configured on both sides.  And what if there's no algorithm they can
agree upon?  The connection fails mysteriously.  So yes, it matters to
the user sometimes that an algorithm has been compromised.

>> This is a crucial distinction.  So I want Emacs to notify the
>> user their GnuTLS is out of date, or else something else should
>> (e.g. the self-contained GnuTLS updater for W32 I proposed).

LMI> I don't really see that there's much of a difference between bugs in
LMI> libgnutls and in the Emacs binary proper.  If a major security hole was
LMI> discovered in Emacs, then presumably a new Emacs release would be made.
LMI> If a major libgnutls hole was discovered, then presumably someone would
LMI> zip up a new Windows release.

I just want a way to tell the users about it.  I don't care how we
deliver the update, if at all.  That should depend on the OS and as I
said should not be done by emacs-devel.

On Fri, 6 Jan 2012 05:11:47 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> If GnuTLS has a security issue, I wouldn't say that Emacs puts my
JB> machine at risk. GnuTLS does.

That's oversimplifying the problem, but yes, this is the fundamental
question.  I think, considering how Emacs is used and positioned as a
software package, we should take responsibility to notify the user on
the W32 platform and maybe on Mac OS X.  Probably not on GNU/Linux,
since we can assume on that platform the package manager's policies are
what the user wants, even if those policies put the user at risk.

This is my personal opinion.  You and Lars are clearly against that
approach.  I won't make any changes to Emacs in this direction until
either you're convinced otherwise, or the maintainers make a decision.

JB> Are you going to add a program to Emacs to test the hard drive for
JB> bad spots? That kind of checks (updates, I mean, not the disk test
JB> tool ;-) instill false security.

I was planning on that next.  How did you know?

JB> It's like the people who has an AV installed and thinks that it is
JB> protected because the AV software has not detected anything.

No, it's not like that at all.  Intrusion detection and security
advisories are completely different things.

Ted

Re: GnuTLS for W32

[Juanma Barranquero]

2012/1/6 Ted Zlatanov <tzz@lifelogs.com>:

> The intention is to do whatever is appropriate on the platform to let
> the user know they need to update and make the update easy.

There's no single, general definition of "appropriate".

> I'm not.  The risk is not worth the effort with image libraries.

I don't understand why. Buffer overruns exploited through
carefully-crafted images have been used before. I would fear that (as
a vector for malware) much more than someone eavesdropping my
communications.

> You're ignoring the "deeply embedded" part.  Obviously external
> utilities are not able to compromise Emacs like internal C glue.  Can
> you stick to comparable components like the libxml2 glue?

See the image libraries comment above.

> If you don't think the package manager is important to our users, you've
> got your head stuck in the sand.

I don't know about "our" users, but certainly is unimportant to many
Emacs users (starting with myself).

And, please, let's not turn this discussion into a description of the
relative positions of our respective heads or other body parts.

> I appreciate your attention to detail, but "need" is the verb I meant to
> write there.

I don't doubt it. My correction turned what you said into what I
believe is real.

> SSH clients are not extensible layout engines with embedded interpreters
> and flexible package managers.  As I keep saying, compare Emacs to
> Firefox and Chrome, not to `vim' or `ssh' and `grep'.  It hasn't been
> just an editor in a long while.  Eclipse is another good comparison
> point.

Compare it to Apache, which can be infinitely extended via external
modules and it's mission-critical for so many business.

> That's oversimplifying the problem, but yes, this is the fundamental
> question.

You think it's an oversimplification, I think it's approaching it in a
realistic way.

> I was planning on that next.  How did you know?

With the head under the sand I had plenty of time to think, and I
started having premonitions.

> No, it's not like that at all.  Intrusion detection and security
> advisories are completely different things.

I thought it was evident I was not comparing situations, but
inadequate feelings of security.

    Juanma

Re: GnuTLS for W32

[Ted Zlatanov]

On Fri, 6 Jan 2012 15:35:44 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> 2012/1/6 Ted Zlatanov <tzz@lifelogs.com>:

>> I'm not.  The risk is not worth the effort with image libraries.

JB> I don't understand why. Buffer overruns exploited through
JB> carefully-crafted images have been used before. I would fear that (as
JB> a vector for malware) much more than someone eavesdropping my
JB> communications.

I don't think those are as risky, but even if I did it doesn't really
support your point.

>> SSH clients are not extensible layout engines with embedded interpreters
>> and flexible package managers.  As I keep saying, compare Emacs to
>> Firefox and Chrome, not to `vim' or `ssh' and `grep'.  It hasn't been
>> just an editor in a long while.  Eclipse is another good comparison
>> point.

JB> Compare it to Apache, which can be infinitely extended via external
JB> modules and it's mission-critical for so many business.

That's a very tough comparison.  I'm listing "extensible layout engines
with embedded interpreters and flexible package managers" which Apache
has never been, never mind that it's not interactive with the user since
it's a daemon.

>> No, it's not like that at all.  Intrusion detection and security
>> advisories are completely different things.

JB> I thought it was evident I was not comparing situations, but
JB> inadequate feelings of security.

Yes, I understood your point.  While you're right that AV software can
provide a false feeling of security, security advisories do not provide
a feeling of security at all.  They let you know when your security is
at risk.  None of my proposal aims to make users feel secure, but rather
to tell them when they are not.

Ted

Re: GnuTLS for W32

[Juanma Barranquero]

2012/1/6 Ted Zlatanov <tzz@lifelogs.com>:

> I don't think those are as risky, but even if I did it doesn't really
> support your point.

It does support my point that if we do it for GnuTLS, we should do it
for other libraries. Of course, I believe we shouldn't.

> That's a very tough comparison.  I'm listing "extensible layout engines
> with embedded interpreters and flexible package managers" which Apache
> has never been, never mind that it's not interactive with the user since
> it's a daemon.

It could still check and write a log entry for the administrator. Or
it could have a module, accessible only to some authenticated user,
that listed recommended upgrades. It doesn't.

And it's not necessarily a daemon. You can start it in batch mode to
check things. It could have an option to check upgrades in that case.
It doesn't, either.

> None of my proposal aims to make users feel secure, but rather
> to tell them when they are not.

And, as soon as they have upgraded, the users feel secure again.
Particularly on Windows, where most people is not very
security-conscious.

Anyway, I think the dead equine has been beaten to a pulp and turned
into fertilizer. We don't really advance anything rehashing the same
arguments again and again, IMHO. YMMV.

    Juanma

Re: GnuTLS for W32

[Ted Zlatanov]

On Fri, 6 Jan 2012 16:47:56 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> Anyway, I think the dead equine has been beaten to a pulp and turned
JB> into fertilizer. We don't really advance anything rehashing the same
JB> arguments again and again, IMHO. YMMV.

I appreciate your opinions and hope we can find some middle ground that
will satisfy everyone's expectations.

Ted

Re: GnuTLS for W32

[Stephen J. Turnbull]

Juanma Barranquero writes:
 > 2012/1/5 Ted Zlatanov <tzz@lifelogs.com>:
 > 
 > > You're right.  Do you agree with the general idea of checking for
 > > critical updates on startup, though?
 > 
 > FWIW, I don't. That is a step (tiny, I know) in the "software as a
 > service" direction.

No, it's not.  Providing software *as* a service is a different issue
from providing software *with* a service.

There is an issue of "ET phoning home" here, which is not an easy one
for free software (at least as I understand Richard's positions on
these things).  But please don't call it "software as a service,"
which is a business model based on decreasing the range of choices
and information available to the user; Ted is suggesting increasing
both, while decreasing certain costs to some users at relatively low
(but not insignificant!) cost to the Emacs project.

Re: GnuTLS for W32

[Stephen J. Turnbull]

Juanma Barranquero writes:

 > Last I checked, Emacs wasn't yet an operating system.

Comparing kinds of software here is bogus.  The question is "what do
users want?"  Obviously, there is a vocal faction (and I believe it's
large) of Emacs users who don't like processes running on their hosts
that they didn't explicitly start, and especially not networking
processes.  But there are also an awful lot of people who would
appreciate this service, I'm sure.

I think is it quite reasonable to provide the service, default the
automatic check OFF in the sources that the Emacs project publishes,
and let redistributors make their own choices.

However, there are alternative ways to provide notification services,
such as providing an RSS feed for the update page, and making that
easy to access manually (eg, from the Help or File menus).

Re: GnuTLS for W32

[Juanma Barranquero]

On Sat, Jan 7, 2012 at 02:36, Stephen J. Turnbull <stephen@xemacs.org> wrote:

> Comparing kinds of software here is bogus.

It wasn't me who proposed comparing software.

> The question is "what do users want?"

I think that's a strange metric. Users might want all kind of things
that we wouldn't want to offer, because of lack of resources, project
policies, etc. I think the question is "What can we offer (and want to
offer) while maximizing usefulness to the users?"

> I think is it quite reasonable to provide the service, default the
> automatic check OFF in the sources that the Emacs project publishes,
> and let redistributors make their own choices.

In this scenario, if the redistributor enables checking for GnuTLS
updates, who will distribute the DLL? The GNU project, or the
redistributor? If the latter, I'm OK with this proposal (the OFF part
is important, of course).

> However, there are alternative ways to provide notification services,
> such as providing an RSS feed for the update page, and making that
> easy to access manually (eg, from the Help or File menus).

Right.

    Juanma

Re: GnuTLS for W32

[Stephen J. Turnbull]

Richard Riley writes:
 > Juanma Barranquero <lekktu@gmail.com> writes:

 > > The GnuTLS binary is *not* an "Emacs component".
 > 
 > Silly word play.

Not at all.  It's an admirably precise statement of Juanma's opinion,
and it corresponds to what I believe to be the project's policy on
such things.

 > It's a required component for Emacs in the context of this
 > discussion.

Of course it's not.  Nothing related to Windows can be required of
Emacs, only permitted given the needed volunteers.

Even if a similar practice were a good idea for GNU/Linux systems, it
would still be the project's choice whether to consider such a DLL a
"component of Emacs" or not.  The former means the project accepts
responsibility for it (in some "moral" sense even though the license
disclaims all legal responsibility), the latter means it's delivered
"as is" and defects are the user's problem entirely.

Re: GnuTLS for W32

[Stephen J. Turnbull]

Óscar Fuentes writes:

 > But now that you ask, yes, I'll appreciate that all projects would
 > include a system for notifying me that its software is putting my
 > machine at risk.

Óscar, I'll tell you right now: all of the software on all of your
machines is putting your systems at risk.  If those systems are
connected to the Internet (including by "sneakernet"), that risk is
nonnegligible.  You know that.

What one[1] really wants is something like "I'll appreciate that all
projects will inform me that features of their software that I use has
a known and relatively high security risk."  But identifying features
that you use is impossible; at best the software can determine what
features you have used in the past.  The software also cannot
determine what you mean by "relatively high"; it can only use some
"objective" criterion of exploitability, which might or might not
matter to you.  The bar has to be higher than zero (or you'd just add
my first paragraph to the startup message, no need to check), so some
users (and I gather you are a member of that group) will not get as
many warnings as they like.  But others will get too many, and shut
off a system that they would find beneficial if the bar were set higher.

 > You are sidetracking from my question by going back to the GnuTLS
 > dll. I'm genuinely interested in your reasoning for rejecting an
 > automatic notification system built into Emacs.

Did he reject such a system, or simply insist that it not be turned on
by default?  I don't see how he can reject the system itself, if
somebody else volunteers to create and maintain it.  Rejection is
different from what he actually said, which is that he thinks those
volunteers would be doing a better service for Emacs by developing
Emacs instead of trying to keep up with the security details (which
are normally not public, as you know) of an independent project.

 > Something you can use to warn users that a problem was found that
 > would pose a risk to their data (a security breach, data
 > corruption, whatever).

Something includes "email", "website", "RSS feed", etc; you just want
to feed that information to all users, including many who don't want
it, and some who believe in turning off all services that they don't
need, and won't approve of having Emacs turn it on for them by default.


Footnotes: 
[1]  Cf. larsi's infinitely extensible example of why he doesn't like
checks at startup.  Maybe you would be happy to see that, but I doubt
very many people would.

Re: GnuTLS for W32

[Stephen J. Turnbull]

Juanma Barranquero writes:
 > On Fri, Jan 6, 2012 at 08:12, Eli Zaretskii <eliz@gnu.org> wrote:
 > 
 > > Then I suggest that the check for updates be turned on by default for
 > > US locales.
 > 
 > +1 :-)

Please, no.  That will make the Tea Partyers attack Emacs.

Re: GnuTLS for W32

[Óscar Fuentes]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Óscar, I'll tell you right now: all of the software on all of your
> machines is putting your systems at risk.

Yes, and we are all gonna die, but somehow find more interesting a
diagnostic of brain cancer than the statistics about life expectancy.

Please excuse me for not commenting on the rest of your post, but I've
already devoted too much energy to this discussion and it ceased to be
productive since almost the beginning.

[snip]

Re: GnuTLS for W32

[Stephen J. Turnbull]

Juanma Barranquero writes:

 > In this scenario, if the redistributor enables checking for GnuTLS
 > updates, who will distribute the DLL? The GNU project, or the
 > redistributor? If the latter, I'm OK with this proposal (the OFF part
 > is important, of course).

The latter, of course.

Re: GnuTLS for W32

[Richard Riley]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Richard Riley writes:
>  > Juanma Barranquero <lekktu@gmail.com> writes:
>
>  > > The GnuTLS binary is *not* an "Emacs component".
>  > 
>  > Silly word play.
>
> Not at all.  It's an admirably precise statement of Juanma's opinion,
> and it corresponds to what I believe to be the project's policy on
> such things.

It is word play when everyone knows its not an emacs developed
component. The whole point is whether to ship this external
components. Had you not snipped, my point would clearer.

Ted said:

,----
| > I can't think of a better way to
| > notify them that an Emacs component is out of date and possibly
| > compromising their security.
`----

and the reply was

,----
| The GnuTLS binary is *not* an "Emacs component".
`----

It was quite clear in the context what was meant by "emacs component" :
a component which contributes to a working and complete emacs
installation.


>
>  > It's a required component for Emacs in the context of this
>  > discussion.
>
> Of course it's not.  Nothing related to Windows can be required of
> Emacs, only permitted given the needed volunteers.
>

More word play. Sorry. Clearly no one thinks its an emacs component per
se : but IS a component required by emacs on windows for certain subsets
of emacs users. Its why this thread is taking place. Its not unique to
emacs to have to or want to ship a support 3rd party element in order to
facilitate installation. The location etc has been covered elsewhere.

I dont want to bicker and be petty but felt Ted was getting a bit of a
hard time for merely wanting to make things better for the end user and,
more importantly, the new adopter.

Re: GnuTLS for W32

[Juanma Barranquero]

On Sat, Jan 7, 2012 at 03:31, Stephen J. Turnbull <stephen@xemacs.org> wrote:

> Did he reject such a system, or simply insist that it not be turned on
> by default?  I don't see how he can reject the system itself, if
> somebody else volunteers to create and maintain it.  Rejection is
> different from what he actually said, which is that he thinks those
> volunteers would be doing a better service for Emacs by developing
> Emacs instead of trying to keep up with the security details (which
> are normally not public, as you know) of an independent project.

Right. I don't reject adding such a system, with a note in NEWS
saying: "if you want automatic warning of updates, customize the
variable `emacs-check-for-updates'"- However, I do fear that, once the
system is in place, for 24.2 or 25.1 we will heard people say that the
default should be t.

(I'm leaving aside the issue of distributing binaries.)

    Juanma

Re: GnuTLS for W32

[Chong Yidong]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Fri, 6 Jan 2012 16:47:56 +0100 Juanma Barranquero <lekktu@gmail.com> wrote:
>
> JB> Anyway, I think the dead equine has been beaten to a pulp and turned
> JB> into fertilizer. We don't really advance anything rehashing the same
> JB> arguments again and again, IMHO. YMMV.
>
> I appreciate your opinions and hope we can find some middle ground that
> will satisfy everyone's expectations.

Here are my thoughts:

- First of all, any change involving distributing GnuTLS with Emacs
  should be post-24.1.

- Phoning home on startup by default is out of the question.  There are
  lots of users with the "open Emacs many times" usage pattern, even
  though that usage pattern is discouraged.  Accessing the network for
  each startup would be unreasonable, quite apart from the privacy
  concerns (GNU knows each time you launch Emacs!)

- I am open to improvements to package.el to implement _periodic_ update
  checking, and improvements to check for updates in M-x list-packages.
  It is probably not too difficult to add some infrastructure to
  highlight "strongly recommended updates" in the Package Menu.

- I agree with Lars' point that

> I don't really see that there's much of a difference between bugs in
> libgnutls and in the Emacs binary proper.  If a major security hole was
> discovered in Emacs, then presumably a new Emacs release would be made.
> If a major libgnutls hole was discovered, then presumably someone would
> zip up a new Windows release.

  If a really serious security flaw is found in GnuPG, and we are
  distributing GnuPG with Emacs, we should make an Emacs security
  release, exactly as though it was a security flaw in Emacs itself.

Re: GnuTLS for W32

[Juanma Barranquero]

On Sat, Jan 7, 2012 at 11:24, Chong Yidong <cyd@gnu.org> wrote:

> - First of all, any change involving distributing GnuTLS with Emacs
>  should be post-24.1.
>
> - Phoning home on startup by default is out of the question.  [...]
>
> - I am open to improvements to package.el to implement _periodic_ update
>  checking, [...]

I 100% agree with all the points above.

>  If a really serious security flaw is found in GnuPG, and we are
>  distributing GnuPG with Emacs, we should make an Emacs security
>  release, exactly as though it was a security flaw in Emacs itself.

I think that's clear. But IMO, in the case we are discussing, we
should not distribute the GnuTLS DLL, just as we don't distribute
libpng or libxml2. If we make it available through ELPA (which I don't
like, but like much more than the alternative), then of course
security releases of the relevant package would be available through
ELPA too.

    Juanma

Re: GnuTLS for W32

[Ted Zlatanov]

On Sat, 07 Jan 2012 18:24:39 +0800 Chong Yidong <cyd@gnu.org> wrote: 

CY> - First of all, any change involving distributing GnuTLS with Emacs
CY>   should be post-24.1.

OK; see below.

CY> - Phoning home on startup by default is out of the question.  There are
CY>   lots of users with the "open Emacs many times" usage pattern, even
CY>   though that usage pattern is discouraged.  Accessing the network for
CY>   each startup would be unreasonable, quite apart from the privacy
CY>   concerns (GNU knows each time you launch Emacs!)

CY> - I am open to improvements to package.el to implement _periodic_ update
CY>   checking, and improvements to check for updates in M-x list-packages.
CY>   It is probably not too difficult to add some infrastructure to
CY>   highlight "strongly recommended updates" in the Package Menu.

OK.  How about a new variable `package-critical-packages' which is empty
by default?  When it has elements, Emacs will check on startup if those
packages have been updated, and after the 24.1 release we can add
highlighting to the package list, plus some UI to add/remove packages to
the critical list.  I would really like to get the basic functionality,
off by default, into 24.1.  I think the risk is minimal and the benefit
to users is significant.  The UI will also be simpler, just y/n to the
update (no need for the "never bother me about this again" choice),
since we know that any packages in the critical list were added by the
user.

I think periodic checks won't work well in the Emacs world, but perhaps
I am misunderstanding what you mean.

CY> - I agree with Lars' point that

>> I don't really see that there's much of a difference between bugs in
>> libgnutls and in the Emacs binary proper.  If a major security hole was
>> discovered in Emacs, then presumably a new Emacs release would be made.
>> If a major libgnutls hole was discovered, then presumably someone would
>> zip up a new Windows release.

CY>   If a really serious security flaw is found in GnuPG, and we are
CY>   distributing GnuPG with Emacs, we should make an Emacs security
CY>   release, exactly as though it was a security flaw in Emacs itself.

OK.  Since the consensus seems to be that the platform-specific
installer's maintainers, not emacs-devel, should deal with installing
GnuTLS and other third-party libraries, the responsibility for such
security releases should be with the installer's maintainers, and each
platform will have to figure out its own way to notify the user that
there's a critical security update.

If you agree, this work doesn't have to wait for the 24.1 release since
it won't require changes to Emacs.  For a W32 installer I can work with
Joakim.  For Mac OS X I don't know if the NS port, when bundled as an
app, can include its own GnuTLS and other libraries, or if we'll require
a real installer.  On both those platforms self-updating should be
possible.  Does all of that make sense?

Ted

Re: GnuTLS for W32

[Ted Zlatanov]

On Sat, 07 Jan 2012 06:40:38 +0100 Richard Riley <rileyrg@gmail.com> wrote: 

RR> I dont want to bicker and be petty but felt Ted was getting a bit of
RR> a hard time for merely wanting to make things better for the end
RR> user and, more importantly, the new adopter.

Not at all, everyone in the discussion made good points and helped me
(and others, I hope) see things more clearly.  It's not always obvious
to any single developer, especially me with my limited W32 experience,
what's the right approach on a particular platform and how to balance
that with the general Emacs needs.

Ted

Re: GnuTLS for W32

[Ted Zlatanov]

On Sat, 7 Jan 2012 10:30:27 +0100 Juanma Barranquero <lekktu@gmail.com> wrote: 

JB> Right. I don't reject adding such a system, with a note in NEWS
JB> saying: "if you want automatic warning of updates, customize the
JB> variable `emacs-check-for-updates'"- However, I do fear that, once the
JB> system is in place, for 24.2 or 25.1 we will heard people say that the
JB> default should be t.

I think the final decision should be up to the Emacs maintainers.  So
far Stefan and Chong have said there is no chance of enabling automatic
update checks by default, so I think you can stop worrying.

Ted

Re: GnuTLS for W32

[Richard Riley]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Sat, 07 Jan 2012 06:40:38 +0100 Richard Riley <rileyrg@gmail.com> wrote: 
>
> RR> I dont want to bicker and be petty but felt Ted was getting a bit of
> RR> a hard time for merely wanting to make things better for the end
> RR> user and, more importantly, the new adopter.
>
> Not at all, everyone in the discussion made good points and helped me

Possibly you are more battle hardened in the discussions here ;)

There is a clear group specific technique to fall back on : and much of
it is word games. Well, thats how I read it .. and invariably its down
to the point I noted about fixating on phrases like "emacs component":
anyways - nothing more from me on the subject other than to say whatever
can be done ot support the inexperienced or new user the better - the
more eperienced users can toggle defaults to how they want them. 

Re: GnuTLS for W32

[Juanma Barranquero]

2012/1/7 Ted Zlatanov <tzz@lifelogs.com>:

> I think the final decision should be up to the Emacs maintainers.

Of course.

> So
> far Stefan and Chong have said there is no chance of enabling automatic
> update checks by default, so I think you can stop worrying.

Consider me officially un-worried.

    Juanma

Re: GnuTLS for W32

[Juanma Barranquero]

On Sat, Jan 7, 2012 at 15:51, Richard Riley <rileyrg@gmail.com> wrote:

> There is a clear group specific technique to fall back on : and much of
> it is word games. Well, thats how I read it .. and invariably its down
> to the point I noted about fixating on phrases like "emacs component":

FWIW, I wasn't involving in word games, and I was not fixated in such
a phrase. I rejected the notion, clear and simple. It had nothing to
do with the wording.

    Juanma

Re: GnuTLS for W32

[Reiner Steib]

On Fri, Jan 06 2012, Juanma Barranquero wrote:

> 2012/1/6 Ted Zlatanov <tzz@lifelogs.com>:
>
>> The intention is to do whatever is appropriate on the platform to let
>> the user know they need to update and make the update easy.
>
> There's no single, general definition of "appropriate".
>
>> I'm not.  The risk is not worth the effort with image libraries.
>
> I don't understand why. Buffer overruns exploited through
> carefully-crafted images have been used before. 

yes, see e.g. CVE-2011-0408, http://www.kb.cert.org/vuls/id/388984,
http://www.google.com/search?q=libpng+arbitrary+code+execution

> I would fear that (as a vector for malware) much more than someone
> eavesdropping my communications.

I agree.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: GnuTLS for W32

[Stephen J. Turnbull]

Richard Riley writes:

 > It was quite clear in the context what was meant by "emacs component" :
 > a component which contributes to a working and complete emacs
 > installation.

In context, I don't disagree with that definition.  I merely deny that
GnuTLS is needed for a complete Emacs distribution in that sense from
the point of view of the GNU Emacs project.  I believe that Juanma and
Eli would agree with me that those two points are the main ones,
though we respectfully disagree with each other on many other grounds.

 > I dont want to bicker and be petty but felt Ted was getting a bit
 > of a hard time for merely wanting to make things better for the end
 > user and, more importantly, the new adopter.

But this is not petty, it's fundamental!  Proponents who argue that
distribution of GnuTLS binaries is needed for Windows users are
ignoring a very important distinction: users (new adopters) of the GNU
System (and at lower priority, free OSes in general) vs. users of
non-free OSes and OSes whose attractiveness is based on non-free
software.  Supporting the former is the policy of GNU Emacs;
supporting the latter is not (though of course individual developers
are free to contribute to such support if they like).

I'm not sure that the above even applies to Ted, though.  AIUI, you
misread Ted's main argument, which is that an automatic notification
service for security updates (not necessarily involving distribution
of GnuTLS binaries for Windows users, though it *could* eventually do
so) would be beneficial for (almost) all Emacs users and therefore the
project.

Re: GnuTLS for W32

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Date: Sun, 08 Jan 2012 16:40:34 +0900
> 
> Richard Riley writes:
> 
>  > It was quite clear in the context what was meant by "emacs component" :
>  > a component which contributes to a working and complete emacs
>  > installation.
> 
> In context, I don't disagree with that definition.  I merely deny that
> GnuTLS is needed for a complete Emacs distribution in that sense from
> the point of view of the GNU Emacs project.  I believe that Juanma and
> Eli would agree with me that those two points are the main ones,
> though we respectfully disagree with each other on many other grounds.

I do agree, mainly because the concept of a "working and complete
emacs installation" defined above will lead us to absurdity, whereby
we will need to provide many programs and packages Emacs uses in some
of its features, which on Windows are not available out of the box.
GnuTLS is not different from Grep or Find or Ispell/Aspell or Awk, to
name just a few (and that's even before you consider the tools needed
to build programs from Emacs).

Building these add-ons and providing installers for them must be a job
of a separate group of volunteers, not of the Emacs project.  The job
of the Emacs project is to provide infrastructure for integration with
those add-ons, such as dynamic-library-alist populated with the names
of the supported DLLs and the machinery to load the DLLs on demand.

>  > I dont want to bicker and be petty but felt Ted was getting a bit
>  > of a hard time for merely wanting to make things better for the end
>  > user and, more importantly, the new adopter.
> 
> But this is not petty, it's fundamental!  Proponents who argue that
> distribution of GnuTLS binaries is needed for Windows users are
> ignoring a very important distinction: users (new adopters) of the GNU
> System (and at lower priority, free OSes in general) vs. users of
> non-free OSes and OSes whose attractiveness is based on non-free
> software.  Supporting the former is the policy of GNU Emacs;
> supporting the latter is not (though of course individual developers
> are free to contribute to such support if they like).

That is true, but we don't even distribute GnuTLS for GNU systems, so
talking about doing that for Windows is _really_ far-fetched.

Re: GnuTLS for W32

[Ted Zlatanov]

On Sun, 08 Jan 2012 16:40:34 +0900 "Stephen J. Turnbull" <stephen@xemacs.org> wrote: 

SJT> I'm not sure that the above even applies to Ted, though.  AIUI, you
SJT> misread Ted's main argument, which is that an automatic notification
SJT> service for security updates (not necessarily involving distribution
SJT> of GnuTLS binaries for Windows users, though it *could* eventually do
SJT> so) would be beneficial for (almost) all Emacs users and therefore the
SJT> project.

What I've now propsed, after Chong clarified the maintainers' position,
is the same service, enabled by default but with no packages (so an
interested user only has to customize one variable and it does nothing
otherwise).  I think that's the middle ground we've been looking for.

I only need to know, from Stefan or Chong, if that proposal is OK and if
it should go into the current trunk or if it should wait until 24.1 is out.

On Sun, 08 Jan 2012 03:34:46 -0500 Eli Zaretskii <eliz@gnu.org> wrote: 

EZ> Building these add-ons and providing installers for them must be a job
EZ> of a separate group of volunteers, not of the Emacs project.  The job
EZ> of the Emacs project is to provide infrastructure for integration with
EZ> those add-ons, such as dynamic-library-alist populated with the names
EZ> of the supported DLLs and the machinery to load the DLLs on demand.

>> But this is not petty, it's fundamental!  Proponents who argue that
>> distribution of GnuTLS binaries is needed for Windows users are
>> ignoring a very important distinction: users (new adopters) of the GNU
>> System (and at lower priority, free OSes in general) vs. users of
>> non-free OSes and OSes whose attractiveness is based on non-free
>> software.  Supporting the former is the policy of GNU Emacs;
>> supporting the latter is not (though of course individual developers
>> are free to contribute to such support if they like).

EZ> That is true, but we don't even distribute GnuTLS for GNU systems, so
EZ> talking about doing that for Windows is _really_ far-fetched.

Let's agree the GNU Emacs project will not be distributing GnuTLS (or
any other third-party library) binaries for any platform.  I think I was
clear on that from the beginning but maybe I did not state it well.

My proposals for distributing binaries were either a GNU ELPA package; a
standalone GnuTLS installer; or an Emacs installer (anything but
downloading a zip file!).  Through our discussion I've seen that an
Emacs installer/patcher as Joakim suggested is the best choice, with a
GNU ELPA package set up as an opt-in update notifier if my notification
proposal above is approved.

Ted

Re: GnuTLS for W32

[Stefan Monnier]

> What I've now propsed, after Chong clarified the maintainers' position,
> is the same service, enabled by default but with no packages (so an
> interested user only has to customize one variable and it does nothing
> otherwise).  I think that's the middle ground we've been looking for.

> I only need to know, from Stefan or Chong, if that proposal is OK and if
> it should go into the current trunk or if it should wait until 24.1 is out.

I don't see a problem with it, but it's too late for 24.1.


        Stefan

NaCl support for Emacs (was: GnuTLS for W32)

[Ted Zlatanov]

Speaking of security libraries...

I'm interested in bringing in support for the NaCl cryptographic library
for Emacs, after 24.1 is out.  There is info on NaCl here:

http://nacl.cr.yp.to/index.html

The library is in the public domain, with no usage or distribution
restrictions.  I'm not sure if that means we can or can't include the
support.

My rationale for supporting this library is that it's fast, very simple
on the client side, and provides good security for arbitrary binary
payloads.  There are many places within Emacs where that's appropriate,
whereas heavyweight network-oriented security like GnuTLS is either not
appropriate or not usable.  An example is EPA/EPG, which currently
relies on the external GPG utility.  Emacs could provide similar
functionality (perhaps integrated with EPA/EPG, perhaps standalone)
without relying on external utilities if it has NaCl support.

NaCl can't replace GnuTLS for many reasons, but I think by itself it
would be a good addition to Emacs.  The C API is really simple compared
to GnuTLS, so the actual C support will not take long to write.

Thanks
Ted

Re: GnuTLS for W32

[Ted Zlatanov]

On Sun, 08 Jan 2012 20:04:53 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>> What I've now propsed, after Chong clarified the maintainers' position,
>> is the same service, enabled by default but with no packages (so an
>> interested user only has to customize one variable and it does nothing
>> otherwise).  I think that's the middle ground we've been looking for.

>> I only need to know, from Stefan or Chong, if that proposal is OK and if
>> it should go into the current trunk or if it should wait until 24.1 is out.

SM> I don't see a problem with it, but it's too late for 24.1.

OK, thanks.  I'll add it after the release.

Ted

Re: NaCl support for Emacs

[Stefan Monnier]

> I'm interested in bringing in support for the NaCl cryptographic library
> for Emacs, after 24.1 is out.  There is info on NaCl here:

While it might be an interesting feature to provide for future Elisp
packages, its immediate usefulness is much less obvious, so the kind of
compile-time linking model we use for things like libgnutls would not be
appropriate (e.g. Debian wouldn't want to add nacl as a dependency if
it's not actually used).

OTOH that might be a good motivation to add support for dynamic loading
of extension libraries.


        Stefan

Re: NaCl support for Emacs

[Carsten Mattner]

On Mon, Jan 9, 2012 at 4:30 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> I'm interested in bringing in support for the NaCl cryptographic library
>> for Emacs, after 24.1 is out.  There is info on NaCl here:
>
> While it might be an interesting feature to provide for future Elisp
> packages, its immediate usefulness is much less obvious, so the kind of
> compile-time linking model we use for things like libgnutls would not be
> appropriate (e.g. Debian wouldn't want to add nacl as a dependency if
> it's not actually used).
>
> OTOH that might be a good motivation to add support for dynamic loading
> of extension libraries.

Only if NaCl's "Automatic CPU-specific tuning" can be done at run-time
and not only at compile-time. Ted, what's the status with that?

Re: NaCl support for Emacs

[Ted Zlatanov]

On Mon, 09 Jan 2012 10:30:23 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>> I'm interested in bringing in support for the NaCl cryptographic library
>> for Emacs, after 24.1 is out.  There is info on NaCl here:

SM> While it might be an interesting feature to provide for future Elisp
SM> packages, its immediate usefulness is much less obvious

Many places in the Emacs core (C and ELisp) could use a fast easy
encryption library for arbitrary data that supports public and
secret-key encryption, in addition to EPA/EPG that I already mentioned.
In-memory storage of secrets (auth-source.el and many places in Gnus)
and emacsclient-style RPC control of Emacs come to mind immediately.

The biggest advantage over GnuTLS in my opinion is the much simpler
interface.  There are less than 30 functions, as opposed to hundreds in
GnuTLS, and they are modeless which makes the API easy to learn and use.

SM> so the kind of compile-time linking model we use for things like
SM> libgnutls would not be appropriate (e.g. Debian wouldn't want to add
SM> nacl as a dependency if it's not actually used).

SM> OTOH that might be a good motivation to add support for dynamic loading
SM> of extension libraries.

It would be cool if I could put out a GNU ELPA package that provided the
NaCl library hookup, but NaCl is not written to be dynamically loaded.
NaCl is all C/C++, just 160K in a compressed tarball and 300K as a
static libnacl.a file.  The code is available through a GitHub clone,
see https://github.com/jeremywohl/nacl.git or you can get the tarball
from http://hyperelliptic.org/nacl/nacl-20110221.tar.bz2

I don't know how much work it would be to adapt NaCl through a custom
dynamic library or if it's better to make it part of the Emacs C tree.
Let me know what you think.

Ted

Re: NaCl support for Emacs

[Ted Zlatanov]

On Mon, 9 Jan 2012 17:43:58 +0100 Carsten Mattner <carstenmattner@googlemail.com> wrote: 

CM> On Mon, Jan 9, 2012 at 4:30 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>>> I'm interested in bringing in support for the NaCl cryptographic library
>>> for Emacs, after 24.1 is out.  There is info on NaCl here:
>> 
>> While it might be an interesting feature to provide for future Elisp
>> packages, its immediate usefulness is much less obvious, so the kind of
>> compile-time linking model we use for things like libgnutls would not be
>> appropriate (e.g. Debian wouldn't want to add nacl as a dependency if
>> it's not actually used).
>> 
>> OTOH that might be a good motivation to add support for dynamic loading
>> of extension libraries.

CM> Only if NaCl's "Automatic CPU-specific tuning" can be done at run-time
CM> and not only at compile-time. Ted, what's the status with that?

If we manage it as a GNU ELPA package with an included tarball, so it's
downloaded and compiled locally, sure.  But otherwise yeah, it's not so
nice.  NaCl is a nice library with no community, AFAICT, and that's
really my biggest concern about integrating with it.  There's no place
to propose changes or get updates.

Ted

Re: NaCl support for Emacs (was: GnuTLS for W32)

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Mon, 09 Jan 2012 09:26:21 -0500
> 
> I'm interested in bringing in support for the NaCl cryptographic library
> for Emacs, after 24.1 is out.  There is info on NaCl here:
> 
> http://nacl.cr.yp.to/index.html

Why not libnettle?  We already use it, albeit indirectly, because
latest versions of GnuTLS depend on it.

There's also libgcrypt, which is a dependency of libxml2.

If the functionalities are comparable, bringing in yet another, third,
dependency of the same kind doesn't make sense, IMO.

> My rationale for supporting this library is that it's fast, very simple
> on the client side, and provides good security for arbitrary binary
> payloads.  There are many places within Emacs where that's appropriate,
> whereas heavyweight network-oriented security like GnuTLS is either not
> appropriate or not usable.  An example is EPA/EPG, which currently
> relies on the external GPG utility.  Emacs could provide similar
> functionality (perhaps integrated with EPA/EPG, perhaps standalone)
> without relying on external utilities if it has NaCl support.

Isn't GPG built on top of a library that itself sits on top of
libgcrypt?  If so, it would make sense to use these libraries instead
of yet another one.

With each new external dependency, we (a) increase the number of
external know-how needed to maintain Emacs; (b) increase the
complexity of building a feature-rich Emacs on anything but the few
most popular GNU/Linux systems; and (c) increase the amount of energy
Emacs maintainers/contributors need to spend on external projects --
to build them regularly, participate in discussions, contribute
patches, etc.

I say, let's bring these dependencies and energy spent on other
projects to the absolute minimum, and if we already depend on some
functionality, even if it isn't the latest and the greatest, let's use
it for as long as it satisfies our needs.

Re: NaCl support for Emacs

[Ted Zlatanov]

On Mon, 09 Jan 2012 19:09:34 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Mon, 09 Jan 2012 09:26:21 -0500
>> 
>> I'm interested in bringing in support for the NaCl cryptographic library
>> for Emacs, after 24.1 is out.  There is info on NaCl here:
>> 
>> http://nacl.cr.yp.to/index.html

EZ> Why not libnettle?  We already use it, albeit indirectly, because
EZ> latest versions of GnuTLS depend on it.

EZ> There's also libgcrypt, which is a dependency of libxml2.

EZ> If the functionalities are comparable, bringing in yet another, third,
EZ> dependency of the same kind doesn't make sense, IMO.

The NaCl API is simplest and it seems to be really fast, that's what
attracted me to it.

I agree, though, that libnettle or libgcrypt may be a better choice.
NaCl is not required at all.  I simply didn't think of libnettle and
libgcrypt.

EZ> Isn't GPG built on top of a library that itself sits on top of
EZ> libgcrypt?  If so, it would make sense to use these libraries instead
EZ> of yet another one.

Unfortunately there's no way to use GPG's functionality without invoking
GPG itself, so libgcrypt is no better than libnettle in this context.

EZ> With each new external dependency, we (a) increase the number of
EZ> external know-how needed to maintain Emacs; (b) increase the
EZ> complexity of building a feature-rich Emacs on anything but the few
EZ> most popular GNU/Linux systems; and (c) increase the amount of energy
EZ> Emacs maintainers/contributors need to spend on external projects --
EZ> to build them regularly, participate in discussions, contribute
EZ> patches, etc.

EZ> I say, let's bring these dependencies and energy spent on other
EZ> projects to the absolute minimum, and if we already depend on some
EZ> functionality, even if it isn't the latest and the greatest, let's use
EZ> it for as long as it satisfies our needs.

OK, I agree.  I'll look at libgcrypt and libnettle and see if they are
easily exposed.

Ted

Re: NaCl support for Emacs (was: GnuTLS for W32)

[Eli Zaretskii]

> Date: Mon, 09 Jan 2012 19:09:34 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
> There's also libgcrypt, which is a dependency of libxml2.

Sorry, this part is wrong, I confused libxml2 with libxslt.

All the rest still stands, however, as libnettle is brought in by
GnuTLS.

Re: NaCl support for Emacs

[Carsten Mattner]

2012/1/9 Ted Zlatanov <tzz@lifelogs.com>:
> On Mon, 9 Jan 2012 17:43:58 +0100 Carsten Mattner <carstenmattner@googlemail.com> wrote:
>
> CM> On Mon, Jan 9, 2012 at 4:30 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>>>> I'm interested in bringing in support for the NaCl cryptographic library
>>>> for Emacs, after 24.1 is out.  There is info on NaCl here:
>>>
>>> While it might be an interesting feature to provide for future Elisp
>>> packages, its immediate usefulness is much less obvious, so the kind of
>>> compile-time linking model we use for things like libgnutls would not be
>>> appropriate (e.g. Debian wouldn't want to add nacl as a dependency if
>>> it's not actually used).
>>>
>>> OTOH that might be a good motivation to add support for dynamic loading
>>> of extension libraries.
>
> CM> Only if NaCl's "Automatic CPU-specific tuning" can be done at run-time
> CM> and not only at compile-time. Ted, what's the status with that?
>
> If we manage it as a GNU ELPA package with an included tarball, so it's
> downloaded and compiled locally, sure.  But otherwise yeah, it's not so
> nice.  NaCl is a nice library with no community, AFAICT, and that's

That sounds like a good plan :).

> really my biggest concern about integrating with it.  There's no place
> to propose changes or get updates.

NaCl's design goals, implementation and patent cleanness make it attractive
to anyone who's had to make use of any kind of cipher functionality.
If there's no forum, I suggest addressing the authors listed in
http://cr.yp.to/highspeed/coolnacl-20111201.pdf
If you're bound by FIPS rules, your choices are limited and different.
I wouldn't put much weight on that.

djb has time and time again proven that his work is solid and provides less
attack surface.

Part of the reason that there's much asm code involved may be that
NaCl avoids timing attacks by design.

I'd definitely favor NaCl, just because they provide a simple API with known
safe defaults. Way safer than using OpenSSL without a the required
crypto background.
Most bugs surface in combination of the different tools from a crypto lib,
because too much code is written without being aware of all the semantics of
the used ciphers and modes.

Re: NaCl support for Emacs

[Ted Zlatanov]

On Mon, 9 Jan 2012 18:48:58 +0100 Carsten Mattner <carstenmattner@googlemail.com> wrote: 

CM> NaCl's design goals, implementation and patent cleanness make it attractive
CM> to anyone who's had to make use of any kind of cipher functionality.

I think it makes sense to provide a GNU ELPA package with NaCl,
absolutely.  But it's provided as a static library and I don't know how
to convert it to dynamically loaded, and Emacs doesn't have facilities
to load arbitrary dynamic libraries at runtime.  The rest (packaging,
on-activation compile) I can handle in the package itself.  If you want
to help out... or anyone else... let me know off-list.

CM> I'd definitely favor NaCl, just because they provide a simple API with known
CM> safe defaults. Way safer than using OpenSSL without a the required
CM> crypto background.
CM> Most bugs surface in combination of the different tools from a crypto lib,
CM> because too much code is written without being aware of all the semantics of
CM> the used ciphers and modes.

Look at libnettle as Eli suggested, I'm leaning towards it since we
already have it through GnuTLS, for the Emacs core.  I still would like
to provide the NaCl support as a GNU ELPA package.

Thanks
Ted

Re: NaCl support for Emacs

[Carsten Mattner]

2012/1/9 Ted Zlatanov <tzz@lifelogs.com>:
> On Mon, 9 Jan 2012 18:48:58 +0100 Carsten Mattner <carstenmattner@googlemail.com> wrote:
>
> CM> NaCl's design goals, implementation and patent cleanness make it attractive
> CM> to anyone who's had to make use of any kind of cipher functionality.
>
> I think it makes sense to provide a GNU ELPA package with NaCl,
> absolutely.  But it's provided as a static library and I don't know how
> to convert it to dynamically loaded, and Emacs doesn't have facilities
> to load arbitrary dynamic libraries at runtime.  The rest (packaging,
> on-activation compile) I can handle in the package itself.  If you want
> to help out... or anyone else... let me know off-list.

Does wrapping the .a file in a dynamic lib make sense?

Re: NaCl support for Emacs

[joakim]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> I'm interested in bringing in support for the NaCl cryptographic library
>> for Emacs, after 24.1 is out.  There is info on NaCl here:
>
> While it might be an interesting feature to provide for future Elisp
> packages, its immediate usefulness is much less obvious, so the kind of
> compile-time linking model we use for things like libgnutls would not be
> appropriate (e.g. Debian wouldn't want to add nacl as a dependency if
> it's not actually used).
>
> OTOH that might be a good motivation to add support for dynamic loading
> of extension libraries.

I've been toying with the idea of using linking Guile to Emacs so as to
access Guiles libffi integration in Emacs. Granted, if all you wanted
was Emacs + libffi than Emacs + Guile + libffi would seem stupid. OTOH I
think it would be a good project to nudge the two platforms closer.

>
>
>         Stefan

-- 
Joakim Verona

Re: NaCl support for Emacs

[Stefan Monnier]

> Many places in the Emacs core (C and ELisp) could use a fast easy
> encryption library for arbitrary data that supports public and
> secret-key encryption, in addition to EPA/EPG that
> I already mentioned.

Could be.  There's no hard evidence for it yet.

> I don't know how much work it would be to adapt NaCl through a custom
> dynamic library or if it's better to make it part of the Emacs C tree.

I don't want such a thing in the Emacs C tree.


        Stefan

Re: NaCl support for Emacs

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Why not libnettle?  We already use it, albeit indirectly, because
> latest versions of GnuTLS depend on it.

The libnettle interface functions look sensible (from a very quick
perusal of the examples):

http://www.lysator.liu.se/~nisse/nettle/nettle.html#Example

So adding built-in support for encryption via libnettle in Emacs looks
pretty sensible, I think.

libnettle also does digests, which we already have via the secure_hash
function/gnulib.  Would it make long-term sense to make libnettle a
prerequisite and drop all the gnulib digest code from the tree?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: NaCl support for Emacs

[Ted Zlatanov]

On Mon, 09 Jan 2012 17:23:22 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>> Many places in the Emacs core (C and ELisp) could use a fast easy
>> encryption library for arbitrary data that supports public and
>> secret-key encryption, in addition to EPA/EPG that
>> I already mentioned.

SM> Could be.  There's no hard evidence for it yet.

I listed three places I think could use it, how much harder does the
evidence have to be?

- auth-source's cache of file contents

- EPA/EPG or something like it that does not rely on the external GPG
  utility

- general ELisp storage of secret data

You may disagree with these, but we've had discussions over the years of
each of them and at least some people agreed with me that those would be
useful.

>> I don't know how much work it would be to adapt NaCl through a custom
>> dynamic library or if it's better to make it part of the Emacs C tree.

SM> I don't want such a thing in the Emacs C tree.

Understood, and as I said libnettle is a better fit as it's already
brought in by GnuTLS.

Ted

Re: NaCl support for Emacs

[Stefan Monnier]

>>> Many places in the Emacs core (C and ELisp) could use a fast easy
>>> encryption library for arbitrary data that supports public and
>>> secret-key encryption, in addition to EPA/EPG that
>>> I already mentioned.
SM> Could be.  There's no hard evidence for it yet.
> I listed three places I think could use it, how much harder does the
> evidence have to be?
> - auth-source's cache of file contents
> - EPA/EPG or something like it that does not rely on the external GPG
>   utility
> - general ELisp storage of secret data

I don't think Emacs should reinvent every wheel.  GPG does this job well
and using it means that those files can be decrypted without Emacs.

I'm sure there are cases where doing the encryption in Emacs can make
sense, but I'd first like to see actual code (aka "hard evidence").


        Stefan

Re: NaCl support for Emacs

[Ted Zlatanov]

On Mon, 09 Jan 2012 20:30:48 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>>>> Many places in the Emacs core (C and ELisp) could use a fast easy
>>>> encryption library for arbitrary data that supports public and
>>>> secret-key encryption, in addition to EPA/EPG that
>>>> I already mentioned.
SM> Could be.  There's no hard evidence for it yet.
>> I listed three places I think could use it, how much harder does the
>> evidence have to be?
>> - auth-source's cache of file contents
>> - EPA/EPG or something like it that does not rely on the external GPG utility
>> - general ELisp storage of secret data

SM> I don't think Emacs should reinvent every wheel.  GPG does this job well
SM> and using it means that those files can be decrypted without Emacs.

Calling out to an external process is less secure than using built-in
encryption primitives.  So while in general you're right, in this case
I'll respectfully disagree.  It may be convenient but it's not secure.

SM> I'm sure there are cases where doing the encryption in Emacs can make
SM> sense, but I'd first like to see actual code (aka "hard evidence").

Argh.  The auth-source cache is already implemented as a hack, is that
hard enough evidence?  Quoting the relevant bit from
`auth-source-netrc-parse':

#+begin_src lisp
;; cache all netrc files (used to be just .gpg files)
;; Store the contents of the file heavily encrypted in memory.
;; (note for the irony-impaired: they are just obfuscated)
(aput 'auth-source-netrc-cache file
      (list :mtime (nth 5 (file-attributes file))
            :secret (lexical-let ((v (mapcar '1+ (buffer-string))))
                      (lambda () (apply 'string (mapcar '1- v))))))
#+end_src

Similarly auth-source.el returns passwords in a `lexical-let' wrapper so
they can't be leaked into the logs, they have to be evaluated, but it
doesn't attempt to obfuscate them like the code above.

Ted

Re: NaCl support for Emacs

[Ted Zlatanov]

On Mon, 9 Jan 2012 19:21:36 +0100 Carsten Mattner <carstenmattner@googlemail.com> wrote: 

CM> 2012/1/9 Ted Zlatanov <tzz@lifelogs.com>:
>> On Mon, 9 Jan 2012 18:48:58 +0100 Carsten Mattner <carstenmattner@googlemail.com> wrote:
>> 
CM> NaCl's design goals, implementation and patent cleanness make it attractive
CM> to anyone who's had to make use of any kind of cipher functionality.
>> 
>> I think it makes sense to provide a GNU ELPA package with NaCl,
>> absolutely.  But it's provided as a static library and I don't know how
>> to convert it to dynamically loaded, and Emacs doesn't have facilities
>> to load arbitrary dynamic libraries at runtime.  The rest (packaging,
>> on-activation compile) I can handle in the package itself.  If you want
>> to help out... or anyone else... let me know off-list.

CM> Does wrapping the .a file in a dynamic lib make sense?

If you could point me to a tutorial on this, I'd appreciate it, but I
think we're too far off-topic for emacs-devel.  If anyone else is
interested in packaging NaCl on the GNU ELPA, let me know directly.

Joakim's suggestion of bringing in libffi is good, but that's a lot of
work!  I hope someone is interested in doing it.

Thanks
Ted

Re: NaCl support for Emacs

[Richard Riley]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Mon, 09 Jan 2012 20:30:48 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 
>
>>>>> Many places in the Emacs core (C and ELisp) could use a fast easy
>>>>> encryption library for arbitrary data that supports public and
>>>>> secret-key encryption, in addition to EPA/EPG that
>>>>> I already mentioned.
> SM> Could be.  There's no hard evidence for it yet.
>>> I listed three places I think could use it, how much harder does the
>>> evidence have to be?
>>> - auth-source's cache of file contents
>>> - EPA/EPG or something like it that does not rely on the external GPG utility
>>> - general ELisp storage of secret data
>
> SM> I don't think Emacs should reinvent every wheel.  GPG does this job well
> SM> and using it means that those files can be decrypted without Emacs.
>
> Calling out to an external process is less secure than using built-in
> encryption primitives.  So while in general you're right, in this case
> I'll respectfully disagree.  It may be convenient but it's not secure.

probably naive Q but : this library encrypts/decrypts identically to the
external gpg? e.g a file I encrypt with gpg can be read using this new
proposal and vice versa?

libnettle for Emacs (was: NaCl support for Emacs)

[Ted Zlatanov]

On Tue, 10 Jan 2012 02:54:48 +0100 Richard Riley <rileyrg@gmail.com> wrote: 

RR> probably naive Q but : this library [libnettle] encrypts/decrypts
RR> identically to the external gpg? e.g a file I encrypt with gpg can
RR> be read using this new proposal and vice versa?

Nope, libnettle is not compatible with GPG.  The intention is not to
cooperate with GPG (which EPA/EPG can already do) but to have an
internal solution to Emacs' encryption needs when the producer and the
consumer are both Emacs (not necessarily the same process).

Ted

Re: libnettle for Emacs

[Richard Riley]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Tue, 10 Jan 2012 02:54:48 +0100 Richard Riley <rileyrg@gmail.com> wrote: 
>
> RR> probably naive Q but : this library [libnettle] encrypts/decrypts
> RR> identically to the external gpg? e.g a file I encrypt with gpg can
> RR> be read using this new proposal and vice versa?
>
> Nope, libnettle is not compatible with GPG.  The intention is not to
> cooperate with GPG (which EPA/EPG can already do) but to have an
> internal solution to Emacs' encryption needs when the producer and the
> consumer are both Emacs (not necessarily the same process).

Ah, so its an addition. Fair enough. Well so long as gpg stays supported
then. gpg stuff is used all over the desktop for mail encryption, doc
encryption and handled in the keyring managers for things like gnome and
cached using stuff like gpg-agent as you'll know better than me ;)
epa/epg is used in stuff like org-crypt so all my :crypt: entries are
encrypted using my gpg keys. pani over ;) regards,r .

Re: NaCl support for Emacs

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 617 bytes --]

On 1/9/12 5:43 PM, Ted Zlatanov wrote:
> SM> I don't think Emacs should reinvent every wheel.  GPG does this job well
> SM> and using it means that those files can be decrypted without Emacs.
> 
> Calling out to an external process is less secure than using built-in
> encryption primitives.  So while in general you're right, in this case
> I'll respectfully disagree.  It may be convenient but it's not secure.

If an attacker can read the bytes sent over a pipe between your Emacs
and its GPG subprocess, you've already lost. I'm not sure what
reasonable definition of "secure" you meant to use here.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 235 bytes --]

Re: NaCl support for Emacs

[Stefan Monnier]

> Argh.  The auth-source cache is already implemented as a hack, is that
> hard enough evidence?  Quoting the relevant bit from
> `auth-source-netrc-parse':

> #+begin_src lisp
> ;; cache all netrc files (used to be just .gpg files)
> ;; Store the contents of the file heavily encrypted in memory.
> ;; (note for the irony-impaired: they are just obfuscated)
> (aput 'auth-source-netrc-cache file
>       (list :mtime (nth 5 (file-attributes file))
>             :secret (lexical-let ((v (mapcar '1+ (buffer-string))))
>                       (lambda () (apply 'string (mapcar '1- v))))))
> #+end_src

Not only I'm not worried about that, but I'm not sure libnettle (or any
other encryption library) would help you fix the underlying problem:
Emacs needs to be able to recover the password for later use anyway, so
anything we do can only ever be obfuscation, AFAIK.  Maybe there's some
clever way to do better, but again, for lack of hard evidence
I'm unconvinced.


        Stefan

Re: NaCl support for Emacs (was: GnuTLS for W32)

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Mon, 09 Jan 2012 09:26:21 -0500
> 
> I'm interested in bringing in support for the NaCl cryptographic library
> for Emacs, after 24.1 is out.  There is info on NaCl here:
> 
> http://nacl.cr.yp.to/index.html

Call me weird or old-fashioned, but that library's build procedure
looks odd, to say the least.

It's as if Make and Autoconf never existed: the entire build is done
by a series of shell scripts, all of them called `do', which have
specific knowledge about several platforms hardcoded into them (do the
developers really intend to maintain all this mess?).

On top of that, the top-level `do' modifies PATH and LD_LIBRAY_PATH
(what?  why??) as it sees fit, and redirects all its output to a log
file, so you basically run blind, like in bad old DOS days (can you
say "tail -f"?).

Also, the scripts run all kinds of non-standard tools and compiler
switches, and although they seem to ignore the resulting errors, how
would J.R. Hacker become confident that the build succeeded and she
can use the results?  E.g., what do you get out of this snippet of the
build log:

  === Tue Jan 10 03:28:17 EST 2012 === checking gcc -m32 -fomit-frame-pointer
  /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.4.3/libgcc.a when searching for -lgcc
  /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.4.3/libgcc.a when searching for -lgcc
  /usr/bin/ld: cannot find -lgcc
  collect2: ld returned 1 exit status

Is it good? is it bad? can I trust the results?

Finally, there's no Makefile anywhere in sight, and no installation
script that I could find.  Bye-bye, "make install-strip", hello manual
installation.

Sorry, but I wouldn't touch such "software" with a 3-mile stick.  Not
for Emacs, anyway.

Re: NaCl support for Emacs (was: GnuTLS for W32)

[Carsten Mattner]

On Tue, Jan 10, 2012 at 11:01 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Mon, 09 Jan 2012 09:26:21 -0500
>>
>> I'm interested in bringing in support for the NaCl cryptographic library
>> for Emacs, after 24.1 is out.  There is info on NaCl here:
>>
>> http://nacl.cr.yp.to/index.html
>
> Call me weird or old-fashioned, but that library's build procedure
> looks odd, to say the least.
>
> It's as if Make and Autoconf never existed: the entire build is done
> by a series of shell scripts, all of them called `do', which have
> specific knowledge about several platforms hardcoded into them (do the
> developers really intend to maintain all this mess?).
>
> On top of that, the top-level `do' modifies PATH and LD_LIBRAY_PATH
> (what?  why??) as it sees fit, and redirects all its output to a log
> file, so you basically run blind, like in bad old DOS days (can you
> say "tail -f"?).
>
> Also, the scripts run all kinds of non-standard tools and compiler
> switches, and although they seem to ignore the resulting errors, how
> would J.R. Hacker become confident that the build succeeded and she
> can use the results?  E.g., what do you get out of this snippet of the
> build log:
>
>  === Tue Jan 10 03:28:17 EST 2012 === checking gcc -m32 -fomit-frame-pointer
>  /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.4.3/libgcc.a when searching for -lgcc
>  /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.4.3/libgcc.a when searching for -lgcc
>  /usr/bin/ld: cannot find -lgcc
>  collect2: ld returned 1 exit status
>
> Is it good? is it bad? can I trust the results?
>
> Finally, there's no Makefile anywhere in sight, and no installation
> script that I could find.  Bye-bye, "make install-strip", hello manual
> installation.
>
> Sorry, but I wouldn't touch such "software" with a 3-mile stick.  Not
> for Emacs, anyway.

For better or worse Dan's software is packaged and licensed as he
sees fit. It is somtimes unconventional, but is something which can be
handled to get at the good stuff within :). People tolerate JavaScript to
get stuff done. We tolerate C as a (limited overhead) porting layer
between us and the machine.

Some reading material:
http://cr.yp.to/redo.html
https://github.com/apenwarr/redo
http://thedjbway.b0llix.net/future.html

Re: NaCl support for Emacs

[Ted Zlatanov]

On Mon, 09 Jan 2012 19:01:48 -0800 Daniel Colascione <dancol@dancol.org> wrote: 

DC> On 1/9/12 5:43 PM, Ted Zlatanov wrote:
SM> I don't think Emacs should reinvent every wheel.  GPG does this job well
SM> and using it means that those files can be decrypted without Emacs.
>> 
>> Calling out to an external process is less secure than using built-in
>> encryption primitives.  So while in general you're right, in this case
>> I'll respectfully disagree.  It may be convenient but it's not secure.

DC> If an attacker can read the bytes sent over a pipe between your Emacs
DC> and its GPG subprocess, you've already lost. I'm not sure what
DC> reasonable definition of "secure" you meant to use here.

I'm being polite.

Ted

Re: NaCl support for Emacs

[Ted Zlatanov]

On Mon, 09 Jan 2012 22:21:19 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> Argh.  The auth-source cache is already implemented as a hack, is that
>> hard enough evidence?  Quoting the relevant bit from
>> `auth-source-netrc-parse':

>> #+begin_src lisp
>> ;; cache all netrc files (used to be just .gpg files)
>> ;; Store the contents of the file heavily encrypted in memory.
>> ;; (note for the irony-impaired: they are just obfuscated)
>> (aput 'auth-source-netrc-cache file
>> (list :mtime (nth 5 (file-attributes file))
>> :secret (lexical-let ((v (mapcar '1+ (buffer-string))))
>> (lambda () (apply 'string (mapcar '1- v))))))
>> #+end_src

SM> Not only I'm not worried about that, but I'm not sure libnettle (or any
SM> other encryption library) would help you fix the underlying problem:
SM> Emacs needs to be able to recover the password for later use anyway, so
SM> anything we do can only ever be obfuscation, AFAIK.  Maybe there's some
SM> clever way to do better, but again, for lack of hard evidence
SM> I'm unconvinced.

With true encryption with libnettle, we can encrypt the secret in
memory, on the wire, and on disk so a casual attacker doesn't have the
chance to grab it.  This should hook into the Lisp object printer, for
instance, so it's effortless to print and read encrypted objects.

I'm worried about treating obfuscation as "good enough" security.  That
has a history of backfiring.  Would it convince you to show an attack
that succeeds with obfuscation but fails with true encryption?

I know Emacs is not designed with security in mind.  We have to start
somewhere; this will at least harden the outer shell.  You may not be
worried about it, but I am.

Ted

Re: NaCl support for Emacs

[Ted Zlatanov]

On Tue, 10 Jan 2012 06:45:49 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> On Mon, 09 Jan 2012 19:01:48 -0800 Daniel Colascione <dancol@dancol.org> wrote: 
DC> On 1/9/12 5:43 PM, Ted Zlatanov wrote:

>>> Calling out to an external process is less secure than using built-in
>>> encryption primitives.  So while in general you're right, in this case
>>> I'll respectfully disagree.  It may be convenient but it's not secure.

DC> If an attacker can read the bytes sent over a pipe between your Emacs
DC> and its GPG subprocess, you've already lost. I'm not sure what
DC> reasonable definition of "secure" you meant to use here.

TZ> I'm being polite.

I sent this off too quickly accidentally.  I was writing that I don't
want to say Emacs is insecure currently, only that it can be made more
so.

To answer your question, the risk of calling an external process is not
limited to just the IPC (although that can be compromised too, depending
on the platform and its security model).  On Unix an attacker can
replace /usr/bin/gpg, for instance--that's much easier than compromising
the kernel.  The risk is in the external dependency, not GPG in
particular.  My point is, if we can gain some security by using
libnettle, which is already part of Emacs when it's compiled with
GnuTLS, then it makes sense to do it.  The cost is minimal.

Ted

Re: NaCl support for Emacs

[Carsten Mattner]

2012/1/10 Ted Zlatanov <tzz@lifelogs.com>:
> On Mon, 09 Jan 2012 22:21:19 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
>>> Argh.  The auth-source cache is already implemented as a hack, is that
>>> hard enough evidence?  Quoting the relevant bit from
>>> `auth-source-netrc-parse':
>
>>> #+begin_src lisp
>>> ;; cache all netrc files (used to be just .gpg files)
>>> ;; Store the contents of the file heavily encrypted in memory.
>>> ;; (note for the irony-impaired: they are just obfuscated)
>>> (aput 'auth-source-netrc-cache file
>>> (list :mtime (nth 5 (file-attributes file))
>>> :secret (lexical-let ((v (mapcar '1+ (buffer-string))))
>>> (lambda () (apply 'string (mapcar '1- v))))))
>>> #+end_src
>
> SM> Not only I'm not worried about that, but I'm not sure libnettle (or any
> SM> other encryption library) would help you fix the underlying problem:
> SM> Emacs needs to be able to recover the password for later use anyway, so
> SM> anything we do can only ever be obfuscation, AFAIK.  Maybe there's some
> SM> clever way to do better, but again, for lack of hard evidence
> SM> I'm unconvinced.
>
> With true encryption with libnettle, we can encrypt the secret in
> memory, on the wire, and on disk so a casual attacker doesn't have the

isn't the secret to decrypt it available in emacs process space
for ready retrieval?

usually you also overwrite that memory to prevent leakage as
soon as possible.

unlocking a keychain and keeping that "safe" open for the time
a user is using the environment is common practice.

aren't you going to implement something like gnome's or kde's
locked keychain?

there will be at least a couple users demanding integration
with existing keychain systems (kde, osx, gnome, ...).
git has recently implemented support for various systems
with an abstraction layer and a caching "daemon".

> chance to grab it.  This should hook into the Lisp object printer, for
> instance, so it's effortless to print and read encrypted objects.
>
> I'm worried about treating obfuscation as "good enough" security.  That
> has a history of backfiring.  Would it convince you to show an attack
> that succeeds with obfuscation but fails with true encryption?
>
> I know Emacs is not designed with security in mind.  We have to start
> somewhere; this will at least harden the outer shell.  You may not be
> worried about it, but I am.

when Emacs was written legend tells that passwords were not in
common use in the timeshare environment Richard worked with.

Re: NaCl support for Emacs

[Ted Zlatanov]

On Tue, 10 Jan 2012 13:51:13 +0100 Carsten Mattner <carstenmattner@googlemail.com> wrote: 

CM> isn't the secret to decrypt it available in emacs process space for
CM> ready retrieval?

Not necessarily.  But even if it is, the attacker has to know where to
find the private key and run non-trivial code to use it.  The risk is
smaller than plopping the secret data in plain view.

CM> usually you also overwrite that memory to prevent leakage as
CM> soon as possible.

Yes, and we've discussed how ELisp makes that hard, so this will require
work at the C level.  It's not trivial, absolutely.

CM> unlocking a keychain and keeping that "safe" open for the time
CM> a user is using the environment is common practice.

CM> aren't you going to implement something like gnome's or kde's
CM> locked keychain?

That's a possibility but not my target currently.

CM> there will be at least a couple users demanding integration with
CM> existing keychain systems (kde, osx, gnome, ...).

We have KDE+GNOME in auth-source already, through the Secrets API.  We
also had an attempt to provide an interface to the Mac OS X keychain
last year, but I don't think it was fruitful.

CM> git has recently implemented support for various systems with an
CM> abstraction layer and a caching "daemon".

Yes, I've followed Jeff King's patches with great interest, although I
was absolutely swamped last year and could not test them as I wanted to.
I intend to work on integrating VC and Magit with Git's credentials,
probably with auth-source support.

Ted

Re: NaCl support for Emacs

[Carsten Mattner]

2012/1/10 Ted Zlatanov <tzz@lifelogs.com>:
> On Tue, 10 Jan 2012 13:51:13 +0100 Carsten Mattner <carstenmattner@googlemail.com> wrote:
>
> CM> isn't the secret to decrypt it available in emacs process space for
> CM> ready retrieval?
>
> Not necessarily.  But even if it is, the attacker has to know where to
> find the private key and run non-trivial code to use it.  The risk is
> smaller than plopping the secret data in plain view.

When it comes to security, you have to clearly document things like this,
so that it's at least clear what's going on, and there is no false sense
of safety.
I tend to view security from a "the worst will happen" angle.
That way you can try to minimize surprises by being aware of
all the attack vectors.

> CM> usually you also overwrite that memory to prevent leakage as
> CM> soon as possible.
>
> Yes, and we've discussed how ELisp makes that hard, so this will require
> work at the C level.  It's not trivial, absolutely.
>
> CM> unlocking a keychain and keeping that "safe" open for the time
> CM> a user is using the environment is common practice.
>
> CM> aren't you going to implement something like gnome's or kde's
> CM> locked keychain?
>
> That's a possibility but not my target currently.
>
> CM> there will be at least a couple users demanding integration with
> CM> existing keychain systems (kde, osx, gnome, ...).
>
> We have KDE+GNOME in auth-source already, through the Secrets API.  We
> also had an attempt to provide an interface to the Mac OS X keychain
> last year, but I don't think it was fruitful.
>
> CM> git has recently implemented support for various systems with an
> CM> abstraction layer and a caching "daemon".
>
> Yes, I've followed Jeff King's patches with great interest, although I
> was absolutely swamped last year and could not test them as I wanted to.
> I intend to work on integrating VC and Magit with Git's credentials,
> probably with auth-source support.

Also on my TODO list.

Re: NaCl support for Emacs

[Stefan Monnier]

> Not necessarily.  But even if it is, the attacker has to know where to
> find the private key and run non-trivial code to use it.  The risk is
> smaller than plopping the secret data in plain view.

I call that "obfuscation".  The non-trivial code is not a deterrent either
because it comes with Emacs ;-)


        Stefan

Re: NaCl support for Emacs (was: GnuTLS for W32)

[Stephen J. Turnbull]

Carsten Mattner writes:

 > For better or worse Dan's software is packaged and licensed as he
 > sees fit.

If it's anything like Dan's usual "license", you're going to need a
real lawyer to decide if it's free enough for Emacs.  IIRC, Dan often
does not permit distribution of modified versions, which is right out.

Re: NaCl support for Emacs (was: GnuTLS for W32)

[Carsten Mattner]

On Wed, Jan 11, 2012 at 6:09 AM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
> Carsten Mattner writes:
>
>  > For better or worse Dan's software is packaged and licensed as he
>  > sees fit.
>
> If it's anything like Dan's usual "license", you're going to need a
> real lawyer to decide if it's free enough for Emacs.  IIRC, Dan often
> does not permit distribution of modified versions, which is right out.

It's public domain. Is that a problem?

Re: NaCl support for Emacs (was: GnuTLS for W32)

[Stephen J. Turnbull]

Carsten Mattner writes:

 > It's public domain. Is that a problem?

Not if it really is.  I'm just saying that Bernstein's licensing and
distribution practices for much of his software, if applied here,
would be problematic, and it deserves checking before doing too much
work on adding support for it to Emacs.

Re: NaCl support for Emacs

[Harald Hanche-Olsen]

["Stephen J. Turnbull" <stephen@xemacs.org> (2012-01-11 12:26:22 UTC)]

> Not if it really is.  I'm just saying that Bernstein's licensing and
> distribution practices for much of his software, if applied here,
> would be problematic, and it deserves checking before doing too much
> work on adding support for it to Emacs.

That used to be true, several years ago. But in late 2007, he
announced that he was changing that, and all his software – past and
future – would be in the public domain [1]. (It may be that some old
distributions of his still have the old license files, though.)

[1] http://video.google.com/videoplay?docid=-3147768955127254412&hl=en

As far as I know the term "public domain", there is really only one
kind, so you cannot attach license conditions to something in the
public domain. That would be a contradiction in terms, like hair on a
bald head. (In some jurisdictions, such as Norway, there are moral
rights attached to a work that cannot be given away, such as the right
to be identified as the author of the work. But that is another
matter.)

- Harald

Re: NaCl support for Emacs

[Carsten Mattner]

On Wed, Jan 11, 2012 at 1:49 PM, Harald Hanche-Olsen
<hanche@math.ntnu.no> wrote:
> ["Stephen J. Turnbull" <stephen@xemacs.org> (2012-01-11 12:26:22 UTC)]
>
>> Not if it really is.  I'm just saying that Bernstein's licensing and
>> distribution practices for much of his software, if applied here,
>> would be problematic, and it deserves checking before doing too much
>> work on adding support for it to Emacs.
>
> That used to be true, several years ago. But in late 2007, he
> announced that he was changing that, and all his software – past and
> future – would be in the public domain [1]. (It may be that some old
> distributions of his still have the old license files, though.)
>
> [1] http://video.google.com/videoplay?docid=-3147768955127254412&hl=en
>
> As far as I know the term "public domain", there is really only one
> kind, so you cannot attach license conditions to something in the
> public domain. That would be a contradiction in terms, like hair on a
> bald head. (In some jurisdictions, such as Norway, there are moral
> rights attached to a work that cannot be given away, such as the right
> to be identified as the author of the work. But that is another
> matter.)

Even if it had a non public-domain license, that license may not be
enforceable.

We already have a hard time as developers of free code when it comes
to deciding whether a combination of two pieces of code is legal.
I wish there was a matrix listing all combinations of "free" licenses.

Re: NaCl support for Emacs

[Stefan Monnier]

> It's public domain.  Is that a problem?

IIUC that depends on how he made it "public domain".  More specifically,
IIRC, it is impossible to really place in the public domain something
you created recently (i.e. you can't relinquish your copyright rights so
easily).  OTOH, you can get a morally equivalent result, by using an
appropriate license, see http://creativecommons.org/about/cc0.


        Stefan

Re: NaCl support for Emacs

[Carsten Mattner]

On Wed, Jan 11, 2012 at 3:07 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> It's public domain.  Is that a problem?
>
> IIUC that depends on how he made it "public domain".  More specifically,
> IIRC, it is impossible to really place in the public domain something
> you created recently (i.e. you can't relinquish your copyright rights so
> easily).  OTOH, you can get a morally equivalent result, by using an
> appropriate license, see http://creativecommons.org/about/cc0.

So selecting MIT or X11 is a better choice then, isn't it?
IANAL and believe we're speculating here somewhat.

Re: NaCl support for Emacs

[Stephen J. Turnbull]

Harald Hanche-Olsen writes:

 > That used to be true, several years ago. But in late 2007, he
 > announced that he was changing that, and all his software – past and
 > future – would be in the public domain [1].

Oh, that's good!

 > (It may be that some old distributions of his still have the old
 > license files, though.)

Well, that was one of the problems with some of those distributions
... there were no license files!

 > As far as I know the term "public domain", there is really only one
 > kind, so you cannot attach license conditions to something in the
 > public domain.

Sure, but that's assuming that Carsten knows what the public domain
is.  No offense intended, I just don't know him.  OTOH, Bernstein's
reputation as a copyright crank is well-deserved, you know, even if
it's been several years since he changed his mind.  Unfortunately,
that attitude has meant that his software, which is universally
respected for its security and performance, has been relegated to the
backwaters of FLOSS (because it usually wasn't).

Re: NaCl support for Emacs

[Carsten Mattner]

On Wed, Jan 11, 2012 at 4:47 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
> Harald Hanche-Olsen writes:
>  > As far as I know the term "public domain", there is really only one
>  > kind, so you cannot attach license conditions to something in the
>  > public domain.
>
> Sure, but that's assuming that Carsten knows what the public domain
> is.  No offense intended, I just don't know him.  OTOH, Bernstein's

I'm not offended, but find it funny that you'd have to know me
to know whether I know what public domain is.
I get what you intend to express with that.
Mildly amused and yes you're right, I'm no licensing expert.

> reputation as a copyright crank is well-deserved, you know, even if
> it's been several years since he changed his mind.  Unfortunately,
> that attitude has meant that his software, which is universally
> respected for its security and performance, has been relegated to the
> backwaters of FLOSS (because it usually wasn't).

All I know about public domain is that it's not a license per se
and something similar to how old recordings of Bach are free for
most useses after n years. Which is something a couple politicians
intend to extend so that n is more than 70 or 100 (don't remember the
values say for the US).

Re: NaCl support for Emacs

[Stephen J. Turnbull]

Stefan Monnier writes:
 > > It's public domain.  Is that a problem?
 > 
 > IIUC that depends on how he made it "public domain".  More specifically,
 > IIRC, it is impossible to really place in the public domain something
 > you created recently (i.e. you can't relinquish your copyright rights so
 > easily).

AFAIK Richard has stated explicitly that public domain software may be
incorporated in Emacs, but the legal staff recommends (and he agrees)
that the author needs to file a statement of dedication or something
like that with the FSF.  I don't recall him saying you need to be
careful about how old the software is.

Larry Rosen told me that a public statement of dedication (including
as a permission notice in the source) should be sufficient, up to the
question of proving in court if the author removed the evidence (such
as the tarballs on his site).

So, yes, some lawyers do say that in the U.S. there is no explicit
basis in legislation or case law for public domain dedications, and
therefore the only reliable way for a work to enter the public domain
is via lapse of copyright (which thanks to Disney and Sonny Bono's
widow won't happen again in our lifetimes, but it's the principle of
the thing, you know!)  But most lawyers (of the dozen or so I've
talked to on a "not legal advice" basis, usually with wine in hand :-)
seem to think that an appropriately written dedication should be
sufficient.

Re: NaCl support for Emacs

[Stephen J. Turnbull]

Carsten Mattner writes:

 > All I know about public domain is that it's not a license per se

Right.  But more precise is that it is the absence of copyright, so no
license is needed (up to the considerations of moral rights that
Harald mentioned).

 > and something similar to how old recordings of Bach are free for
 > most useses after n years.

Not just similar; those old recordings *are* in the public domain.

 > Which is something a couple politicians intend to extend so that n
 > is more than 70 or 100 (don't remember the values say for the US).

Something like 95 years, and renewable.  So Disney will be free to sue
kindergartens at least into the *next* century. :-(

Re: NaCl support for Emacs (was: GnuTLS for W32)

[Richard Stallman]

If there is a license that needs evaluation, and it isn't in
gnu.org/licenses/license-list.html, please mail it to me.
I and my advisors will study it together.


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/