100% CPU on TCP servers

100% CPU on TCP servers

[Juanma Barranquero]

Lennart just discovered that

 (make-network-process :name "test" :server t :service t)

consumes 100% CPU, at least on Windows. Any idea what can be happening?

-- 
                    /L/e/k/t/u

Re: 100% CPU on TCP servers

[Juanma Barranquero]

(A question for process-savvy people)

It seems like 

>  (make-network-process :name "test" :server t :service t)

on Windows makes the server process to call server_accept_connection()
continuously (in a 2.8 GHz Pentium IV I've measured around 10,200
calls in 3,5 s, almost 2,900 calls per second). It's no wonder Emacs
is munching 50% CPU.

Any idea why that can be happening?

-- 
                    /L/e/k/t/u

Re: 100% CPU on TCP servers

[Kim F. Storm]

Juanma Barranquero <lekktu@gmail.com> writes:

> (A question for process-savvy people)
>
> It seems like 
>
>>  (make-network-process :name "test" :server t :service t)
>
> on Windows makes the server process to call server_accept_connection()
> continuously (in a 2.8 GHz Pentium IV I've measured around 10,200
> calls in 3,5 s, almost 2,900 calls per second). It's no wonder Emacs
> is munching 50% CPU.
>
> Any idea why that can be happening?

Perhaps some unhandled error...  What does the following say
when executed in *scratch* buffer:

(progn
  (defun my-log (server client message)
    (setq mm (cons message mm)))
  (setq mm
    (make-network-process :name "test" :server t :service t :log 'my-log))
  (sleep-for 5)
  mm)

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: 100% CPU on TCP servers

[Juanma Barranquero]

On 9/9/05, Kim F. Storm <storm@cua.dk> wrote:
> What does the following say
> when executed in *scratch* buffer:

Just

  #<process test>

as expected (and CPU time is at 50%; that's on a Windows XP Home).

-- 
                    /L/e/k/t/u

Re: 100% CPU on TCP servers

[Jason Rumney]

Juanma Barranquero <lekktu@gmail.com> writes:

> (A question for process-savvy people)
>
> It seems like 
>
>>  (make-network-process :name "test" :server t :service t)
>
> on Windows makes the server process to call server_accept_connection()
> continuously (in a 2.8 GHz Pentium IV I've measured around 10,200
> calls in 3,5 s, almost 2,900 calls per second). It's no wonder Emacs
> is munching 50% CPU.
>
> Any idea why that can be happening?

Probably a bug in sys_select() in w32proc.c

Re: 100% CPU on TCP servers

[Kim F. Storm]

Jason Rumney <jasonr@gnu.org> writes:

> Juanma Barranquero <lekktu@gmail.com> writes:
>
>> (A question for process-savvy people)
>>
>> It seems like 
>>
>>>  (make-network-process :name "test" :server t :service t)
>>
>> on Windows makes the server process to call server_accept_connection()
>> continuously (in a 2.8 GHz Pentium IV I've measured around 10,200
>> calls in 3,5 s, almost 2,900 calls per second). It's no wonder Emacs
>> is munching 50% CPU.
>>
>> Any idea why that can be happening?
>
> Probably a bug in sys_select() in w32proc.c

Sounds like it, indeed!

Juanma or Jason, could you try to single step through it to see why
it fails to wait on the server socket...

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: 100% CPU on TCP servers

[Kim F. Storm]

Jason Rumney <jasonr@gnu.org> writes:

> Juanma Barranquero <lekktu@gmail.com> writes:
>
>> (A question for process-savvy people)
>>
>> It seems like 
>>
>>>  (make-network-process :name "test" :server t :service t)
>>
>> on Windows makes the server process to call server_accept_connection()
>> continuously (in a 2.8 GHz Pentium IV I've measured around 10,200
>> calls in 3,5 s, almost 2,900 calls per second). It's no wonder Emacs
>> is munching 50% CPU.
>>
>> Any idea why that can be happening?
>
> Probably a bug in sys_select() in w32proc.c

I investigate it a little bit and found that it seems that w32 server
sockets must use WSAAsyncSelect + FD_ACCEPT to request notifications
of incoming connections -- but exactly how that is done is not for me
to look at...

In its current form, it sys_select says the socket is ready, so we
call server_accept_connection which again calls accept() which returns
-1 with errno == WSAEWOULDBLOCK (10035) indicating that there is no
pending connection to accept.

Can somebody pls. look into doing this the right way?


-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: 100% CPU on TCP servers

[Juanma Barranquero]

On 10/12/05, Kim F. Storm <storm@cua.dk> wrote:

> Can somebody pls. look into doing this the right way?

I won't be able for the next several weeks. I'm moving in three days
and it'll take a while to settle.

--
                    /L/e/k/t/u

Re: 100% CPU on TCP servers (on Windoze).

[Kim F. Storm]

Juanma Barranquero <lekktu@gmail.com> writes:

> (A question for process-savvy people)
>
> It seems like 
>
>>  (make-network-process :name "test" :server t :service t)
>
> on Windows makes the server process to call server_accept_connection()
> continuously (in a 2.8 GHz Pentium IV I've measured around 10,200
> calls in 3,5 s, almost 2,900 calls per second). It's no wonder Emacs
> is munching 50% CPU.
>
> Any idea why that can be happening?

I took a look at this issue, and it seems quite trivial to fix.

Would someone try to apply the following patch and tell me whether
it makes a difference  (pls. try the above example before and after
applying the patch).  If it doesn't compile, pls try to fix it!!

I don't quite understand the pfn_ stuff in w32.c -- maybe the
WSAEventSelect function need to be loaded in the same way to be
available.  Someone who knows this stuff, please DTRT.

Of course, it would be great if you could actually try to connect
to the server socket to see if it can really accept the connection
and do something useful with it.

*** w32.h	06 Feb 2006 18:21:50 +0100	1.19
--- w32.h	14 Jul 2006 00:09:55 +0200	
***************
*** 93,98 ****
--- 93,99 ----
  /* fd_info flag definitions */
  #define FILE_READ               0x0001
  #define FILE_WRITE              0x0002
+ #define FILE_LISTEN		0x0004
  #define FILE_BINARY             0x0010
  #define FILE_LAST_CR            0x0020
  #define FILE_AT_EOF             0x0040


*** w32.c	20 May 2006 22:52:11 +0200	1.102
--- w32.c	14 Jul 2006 00:25:21 +0200	
***************
*** 3295,3300 ****
--- 3295,3305 ----
        int rc = pfn_listen (SOCK_HANDLE (s), backlog);
        if (rc == SOCKET_ERROR)
  	set_errno ();
+       else
+ 	{
+ 	  fd_info[s].flags |= FILE_LISTEN;
+ 	  WSAEventSelect (SOCK_HANDLE (s), fd_info[s].cp->char_avail, FD_ACCEPT);
+ 	}
        return rc;
      }
    h_errno = ENOTSOCK;
***************
*** 3332,3342 ****
      }
  
    check_errno ();
!   if (fd_info[s].flags & FILE_SOCKET)
      {
        SOCKET t = pfn_accept (SOCK_HANDLE (s), addr, addrlen);
        if (t != INVALID_SOCKET)
! 	return socket_to_fd (t);
  
        set_errno ();
        return -1;
--- 3337,3352 ----
      }
  
    check_errno ();
!   if (fd_info[s].flags & FILE_LISTEN)
      {
        SOCKET t = pfn_accept (SOCK_HANDLE (s), addr, addrlen);
        if (t != INVALID_SOCKET)
! 	{
! 	  int fd = socket_to_fd (t);
! 	  if (fd >= 0)
! 	    WSAEventSelect (SOCK_HANDLE (fd), fd_info[fd].cp->char_avail, FD_READ | FD_CLOSE);
! 	  return fd;
! 	}
  
        set_errno ();
        return -1;

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: 100% CPU on TCP servers (on Windoze).

[Jason Rumney]

Kim F. Storm wrote:
> I don't quite understand the pfn_ stuff in w32.c -- maybe the
> WSAEventSelect function need to be loaded in the same way to be
> available.  Someone who knows this stuff, please DTRT.
>   
The dynamic loading of network functions is obsolete. The original idea 
was to support versions of Windows that do not contain the Winsock 
library by default. Since NT 3.51, it has been included by default 
(prior to that it was an optional component), and since at least Emacs 
20 we gave up trying to support such old versions of Windows NT, since 
Windows NT was not widely used until 4.0, and any NT 3.1 or 3.5 
installations have probably been replaced by now.

So this can probably be cleaned up by using those functions normally.

Re: 100% CPU on TCP servers (on Windoze).

[Kim F. Storm]

Jason Rumney <jasonr@gnu.org> writes:

> Kim F. Storm wrote:
>> I don't quite understand the pfn_ stuff in w32.c -- maybe the
>> WSAEventSelect function need to be loaded in the same way to be
>> available.  Someone who knows this stuff, please DTRT.
>>   
> The dynamic loading of network functions is obsolete. The original
> idea was to support versions of Windows that do not contain the
> Winsock library by default. Since NT 3.51, it has been included by
> default (prior to that it was an optional component), and since at
> least Emacs 20 we gave up trying to support such old versions of
> Windows NT, since Windows NT was not widely used until 4.0, and any NT
> 3.1 or 3.5 installations have probably been replaced by now.
>
> So this can probably be cleaned up by using those functions normally.

I found a Windoze system where I could build emacs, and it didn't
like properly becuase WSAEventSelect could not be found by the linker.

Then I tried to load it dynamically like the other pfn_ stuff,
and with that change it linked and seemed to run perfectly
as I thought it would.

So I have installed the changes, and now server sockets seem
to behave just fine.

Now, maybe someone would like to make emacsclient and server.el
work on Windoze.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: 100% CPU on TCP servers (on Windoze).

[Juanma Barranquero]

On 7/14/06, Kim F. Storm <storm@cua.dk> wrote:

> Now, maybe someone would like to make emacsclient and server.el
> work on Windoze.

I did some work on this, which I left aside because of the 100% CPU
bug. I'd be willing to continue with it (assuming the bug is fixed),
but I cannot promise I won't miss the "start of the pretesting"
deadline, if it is that imminent as it does appear.

-- 
                    /L/e/k/t/u

Re: 100% CPU on TCP servers (on Windoze).

[Jason Rumney]

Kim F. Storm wrote:
>>>  (make-network-process :name "test" :server t :service t)
>>>       
> I took a look at this issue, and it seems quite trivial to fix.
>
> Would someone try to apply the following patch and tell me whether
> it makes a difference  (pls. try the above example before and after
> applying the patch).  If it doesn't compile, pls try to fix it!!
>   
Before your patch, the example works, but uses 50% CPU. I confirmed it 
worked by connecting to the socket and sending data to Emacs.

After your patch, and some additional patching to dynamically load 
WSAEventSelect (the build process needs changing to eliminate this 
requirement), the example fails with the following stack trace:

Debugger entered--Lisp error: (file-error "make server process failed" 
"invalid argument" :name "test" :server t :service t)
  make-network-process(:name "test" :server t :service t)
  eval((make-network-process :name "test" :server t :service t))
  eval-last-sexp-1(t)
  eval-last-sexp(t)
  eval-print-last-sexp()
  call-interactively(eval-print-last-sexp)

Re: 100% CPU on TCP servers (on Windoze).

[Kim F. Storm]

Jason Rumney <jasonr@gnu.org> writes:

> Kim F. Storm wrote:
>>>>  (make-network-process :name "test" :server t :service t)
>>>>       
>> I took a look at this issue, and it seems quite trivial to fix.
>>
>> Would someone try to apply the following patch and tell me whether
>> it makes a difference  (pls. try the above example before and after
>> applying the patch).  If it doesn't compile, pls try to fix it!!
>>   
> Before your patch, the example works, but uses 50% CPU. I confirmed it
> worked by connecting to the socket and sending data to Emacs.
>
> After your patch, and some additional patching to dynamically load
> WSAEventSelect (the build process needs changing to eliminate this
> requirement), the example fails with the following stack trace:
>
> Debugger entered--Lisp error: (file-error "make server process failed"
> "invalid argument" :name "test" :server t :service t)
>  make-network-process(:name "test" :server t :service t)
>  eval((make-network-process :name "test" :server t :service t))
>  eval-last-sexp-1(t)
>  eval-last-sexp(t)
>  eval-print-last-sexp()
>  call-interactively(eval-print-last-sexp)

Strange -- it works fine when I try it (with the version I 
checked into CVS).

Can you pls. try with that version.


-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: 100% CPU on TCP servers (on Windoze).

[Jason Rumney]

Kim F. Storm wrote:
>> After your patch, and some additional patching to dynamically load
>> WSAEventSelect (the build process needs changing to eliminate this
>> requirement), the example fails with the following stack trace:
>>
>> Debugger entered--Lisp error: (file-error "make server process failed"
>> "invalid argument" :name "test" :server t :service t)
>>  make-network-process(:name "test" :server t :service t)
>>  eval((make-network-process :name "test" :server t :service t))
>>  eval-last-sexp-1(t)
>>  eval-last-sexp(t)
>>  eval-print-last-sexp()
>>  call-interactively(eval-print-last-sexp)
>>     
>
> Strange -- it works fine when I try it (with the version I 
> checked into CVS).
>
> Can you pls. try with that version.
>   
I get the same results with your version (which differs from mine only 
in the position of two lines defining and loading pfn_WSAEventService).

Re: 100% CPU on TCP servers (on Windoze).

[Eli Zaretskii]

> Date: Fri, 14 Jul 2006 11:43:00 +0100
> From: Jason Rumney <jasonr@gnu.org>
> Cc: Juanma Barranquero <lekktu@gmail.com>, Emacs Devel <emacs-devel@gnu.org>
> 
> Kim F. Storm wrote:
> >> After your patch, and some additional patching to dynamically load
> >> WSAEventSelect (the build process needs changing to eliminate this
> >> requirement), the example fails with the following stack trace:
> >>
> >> Debugger entered--Lisp error: (file-error "make server process failed"
> >> "invalid argument" :name "test" :server t :service t)
> >>  make-network-process(:name "test" :server t :service t)
> >>  eval((make-network-process :name "test" :server t :service t))
> >>  eval-last-sexp-1(t)
> >>  eval-last-sexp(t)
> >>  eval-print-last-sexp()
> >>  call-interactively(eval-print-last-sexp)
> >>     
> >
> > Strange -- it works fine when I try it (with the version I 
> > checked into CVS).
> >
> > Can you pls. try with that version.
> >   
> I get the same results with your version

Same here.

Kim, could you please confirm that you used the precise recipe posted
by Juanma, viz.:

  (make-network-process :name "test" :server t :service t)

I fired up "emacs -q", typed this line into *scratch", then typed C-j
after the right paren.  Is that what you did?

If you did exactly that, please tell the details about the system
where you tested (the version of the OS, at the very least).

Re: 100% CPU on TCP servers (on Windoze).

[Juanma Barranquero]

On 7/14/06, Kim F. Storm <storm@cua.dk> wrote:

> Can you pls. try with that version.

ELISP> (make-network-process :name "test" :server t :service t)
*** Eval error ***  make server process failed: invalid argument,
:name, test, :server, t, :service, t
ELISP>

-- 
                    /L/e/k/t/u

Re: 100% CPU on TCP servers (on Windoze).

[Kim F. Storm]

"Juanma Barranquero" <lekktu@gmail.com> writes:

> On 7/14/06, Kim F. Storm <storm@cua.dk> wrote:
>
>> Can you pls. try with that version.
>
> ELISP> (make-network-process :name "test" :server t :service t)
> *** Eval error ***  make server process failed: invalid argument,
> :name, test, :server, t, :service, t
> ELISP>

I don't know what happened (except that maybe I was starting the
wrong binary) -- but I see the error too now.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: 100% CPU on TCP servers (on Windoze).

[Kim F. Storm]

storm@cua.dk (Kim F. Storm) writes:

> "Juanma Barranquero" <lekktu@gmail.com> writes:
>
>> On 7/14/06, Kim F. Storm <storm@cua.dk> wrote:
>>
>>> Can you pls. try with that version.
>>
>> ELISP> (make-network-process :name "test" :server t :service t)
>> *** Eval error ***  make server process failed: invalid argument,
>> :name, test, :server, t, :service, t
>> ELISP>
>
> I don't know what happened (except that maybe I was starting the
> wrong binary) -- but I see the error too now.

Well, it is LOAD_PROC (WSAEventSelect) which fails -- and then
disables winsock support, so any socket() call fails.

Any idea how to fix that?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: 100% CPU on TCP servers (on Windoze).

[Juanma Barranquero]

On 7/14/06, Kim F. Storm <storm@cua.dk> wrote:

> Any idea how to fix that?

You can make it work with the attached patch. Apparently,
WSAEventSelect is exported from ws2_32.dll but not from wsock32.dll. I
don't know whether all targets (I mean, all supported Windows
environments) have or use ws2_32.dll, though.

Unfortunately, with your patch and the attached one, I still see a 50%
CPU use on the reported example.

-- 
                    /L/e/k/t/u


Index: src/w32.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32.c,v
retrieving revision 1.103
diff -u -2 -r1.103 w32.c
--- src/w32.c   14 Jul 2006 09:29:32 -0000      1.103
+++ src/w32.c   14 Jul 2006 12:22:29 -0000
@@ -2771,5 +2771,5 @@
                               "SetHandleInformation");

-  winsock_lib = LoadLibrary ("wsock32.dll");
+  winsock_lib = LoadLibrary ("ws2_32.dll");

   if (winsock_lib != NULL)

Re: 100% CPU on TCP servers (on Windoze).

[Eli Zaretskii]

> Date: Fri, 14 Jul 2006 14:27:27 +0200
> From: "Juanma Barranquero" <lekktu@gmail.com>
> Cc: Emacs Devel <emacs-devel@gnu.org>
> 
> I don't know whether all targets (I mean, all supported Windows
> environments) have or use ws2_32.dll, though.

Jason, can you comment on using wsock32.dll vs ws2_32.dll?  Thanks.

Re: 100% CPU on TCP servers (on Windoze).

[Jason Rumney]

Eli Zaretskii wrote:
> Jason, can you comment on using wsock32.dll vs ws2_32.dll?  Thanks.

It looks like all versions of Windows since NT 3.51 and Windows 95 come 
with ws2_32.dll, so there is no longer a need for us to use wsock32.dll 
since we haven't supported NT versions prior to 3.51 for some time.

Re: 100% CPU on TCP servers (on Windoze).

[Kim F. Storm]

Jason Rumney <jasonr@gnu.org> writes:

> Eli Zaretskii wrote:
>> Jason, can you comment on using wsock32.dll vs ws2_32.dll?  Thanks.
>
> It looks like all versions of Windows since NT 3.51 and Windows 95
> come with ws2_32.dll, so there is no longer a need for us to use
> wsock32.dll since we haven't supported NT versions prior to 3.51 for
> some time.

Sounds good!

I have installed the new patch.


Maybe we should a notice somewhere:

** Windows 95 and networking.

To use Emacs' networking features on Windows 95, you must install the
"Windows Socket 2" update available from MicroSoft's support Web.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: 100% CPU on TCP servers (on Windoze).

[Jason Rumney]

Kim F. Storm wrote:
> Maybe we should a notice somewhere:
>
> ** Windows 95 and networking.
>
> To use Emacs' networking features on Windows 95, you must install the
> "Windows Socket 2" update available from MicroSoft's support Web.
>
>   
Yes, that seems a good idea. PROBLEMS and/or NEWS are probably 
appropriate values for "somewhere". My guess is that if Windows 95 users 
still exist then they probably already have that (I am sure it will have 
been included with some version of Internet Explorer for example, 
probably 4.0), but a note in PROBLEMS should cover it just in case.

Re: 100% CPU on TCP servers (on Windoze).

[Eli Zaretskii]

> Cc: Eli Zaretskii <eliz@gnu.org>,  Juanma Barranquero <lekktu@gmail.com>,
> 	  emacs-devel@gnu.org
> From: storm@cua.dk (Kim F. Storm)
> Date: Fri, 14 Jul 2006 16:22:21 +0200
> 
> I have installed the new patch.

Thanks.

> Maybe we should a notice somewhere:
> 
> ** Windows 95 and networking.
> 
> To use Emacs' networking features on Windows 95, you must install the
> "Windows Socket 2" update available from MicroSoft's support Web.

A good idea, IMO.

Re: 100% CPU on TCP servers (on Windoze).

[Stefan Monnier]

>> Cc: Eli Zaretskii <eliz@gnu.org>,  Juanma Barranquero <lekktu@gmail.com>,
>> emacs-devel@gnu.org
>> From: storm@cua.dk (Kim F. Storm)
>> Date: Fri, 14 Jul 2006 16:22:21 +0200
>> 
>> I have installed the new patch.

> Thanks.

>> Maybe we should a notice somewhere:
>> 
>> ** Windows 95 and networking.
>> 
>> To use Emacs' networking features on Windows 95, you must install the
>> "Windows Socket 2" update available from MicroSoft's support Web.

> A good idea, IMO.

If we can signal an error at the point of use, then we can include this info
in the error message.


        Stefan

Re: 100% CPU on TCP servers (on Windoze).

[Stuart D. Herring]

> It looks like all versions of Windows since NT 3.51 and Windows 95 come
> with ws2_32.dll, so there is no longer a need for us to use wsock32.dll
> since we haven't supported NT versions prior to 3.51 for some time.

Original versions of Windows 95 didn't have ws2_32.dll, but it was
compatible: you could get it (from Microsoft, at no cost) and it worked
(and many other networking applications need it).  Since Windows 98 (and
whatever version of NT) it's been standard.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: 100% CPU on TCP servers (on Windoze).

[Kim F. Storm]

"Juanma Barranquero" <lekktu@gmail.com> writes:

> On 7/14/06, Kim F. Storm <storm@cua.dk> wrote:
>
>> Any idea how to fix that?
>
> You can make it work with the attached patch. Apparently,
> WSAEventSelect is exported from ws2_32.dll but not from wsock32.dll. I
> don't know whether all targets (I mean, all supported Windows
> environments) have or use ws2_32.dll, though.
>
> Unfortunately, with your patch and the attached one, I still see a 50%
> CPU use on the reported example.

Thanks.  Here is something which actually seems to work -- provided
we can use ws2_32.dll:

Index: w32.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32.c,v
retrieving revision 1.103
diff -c -r1.103 w32.c
*** w32.c	14 Jul 2006 09:29:32 -0000	1.103
--- w32.c	14 Jul 2006 13:25:07 -0000
***************
*** 2701,2706 ****
--- 2701,2708 ----
  void (PASCAL *pfn_WSASetLastError) (int iError);
  int (PASCAL *pfn_WSAGetLastError) (void);
  int (PASCAL *pfn_WSAEventSelect) (SOCKET s, HANDLE hEventObject, long lNetworkEvents);
+ HANDLE (PASCAL *pfn_WSACreateEvent) (void);
+ int (PASCAL *pfn_WSACloseEvent) (HANDLE hEvent);
  int (PASCAL *pfn_socket) (int af, int type, int protocol);
  int (PASCAL *pfn_bind) (SOCKET s, const struct sockaddr *addr, int namelen);
  int (PASCAL *pfn_connect) (SOCKET s, const struct sockaddr *addr, int namelen);
***************
*** 2770,2776 ****
      = (void *) GetProcAddress (GetModuleHandle ("kernel32.dll"),
  			       "SetHandleInformation");
  
!   winsock_lib = LoadLibrary ("wsock32.dll");
  
    if (winsock_lib != NULL)
      {
--- 2772,2778 ----
      = (void *) GetProcAddress (GetModuleHandle ("kernel32.dll"),
  			       "SetHandleInformation");
  
!   winsock_lib = LoadLibrary ("Ws2_32.dll");
  
    if (winsock_lib != NULL)
      {
***************
*** 2784,2789 ****
--- 2786,2793 ----
        LOAD_PROC( WSASetLastError );
        LOAD_PROC( WSAGetLastError );
        LOAD_PROC( WSAEventSelect );
+       LOAD_PROC( WSACreateEvent );
+       LOAD_PROC( WSACloseEvent );
        LOAD_PROC( socket );
        LOAD_PROC( bind );
        LOAD_PROC( connect );
***************
*** 3298,3307 ****
        if (rc == SOCKET_ERROR)
  	set_errno ();
        else
! 	{
! 	  fd_info[s].flags |= FILE_LISTEN;
! 	  pfn_WSAEventSelect (SOCK_HANDLE (s), fd_info[s].cp->char_avail, FD_ACCEPT);
! 	}
        return rc;
      }
    h_errno = ENOTSOCK;
--- 3302,3308 ----
        if (rc == SOCKET_ERROR)
  	set_errno ();
        else
! 	fd_info[s].flags |= FILE_LISTEN;
        return rc;
      }
    h_errno = ENOTSOCK;
***************
*** 3342,3357 ****
    if (fd_info[s].flags & FILE_LISTEN)
      {
        SOCKET t = pfn_accept (SOCK_HANDLE (s), addr, addrlen);
!       if (t != INVALID_SOCKET)
! 	{
! 	  int fd = socket_to_fd (t);
! 	  if (fd >= 0)
! 	    pfn_WSAEventSelect (SOCK_HANDLE (fd), fd_info[fd].cp->char_avail, FD_READ | FD_CLOSE);
! 	  return fd;
! 	}
  
!       set_errno ();
!       return -1;
      }
    h_errno = ENOTSOCK;
    return -1;
--- 3343,3357 ----
    if (fd_info[s].flags & FILE_LISTEN)
      {
        SOCKET t = pfn_accept (SOCK_HANDLE (s), addr, addrlen);
!       int fd = -1;
!       if (t == INVALID_SOCKET)
! 	set_errno ();
!       else
! 	fd = socket_to_fd (t);
  
!       fd_info[s].cp->status = STATUS_READ_ACKNOWLEDGED;
!       ResetEvent (fd_info[s].cp->char_avail);
!       return fd;
      }
    h_errno = ENOTSOCK;
    return -1;
***************
*** 3653,3658 ****
--- 3653,3688 ----
    return cp->status;
  }
  
+ int _sys_wait_accept (int fd)
+ {
+   HANDLE hEv;
+   child_process * cp;
+   int rc;
+ 
+   if (fd < 0 || fd >= MAXDESC)
+     return STATUS_READ_ERROR;
+ 
+   cp = fd_info[fd].cp;
+ 
+   if (cp == NULL || cp->fd != fd || cp->status != STATUS_READ_READY)
+     return STATUS_READ_ERROR;
+ 
+   cp->status = STATUS_READ_FAILED;
+ 
+   hEv = pfn_WSACreateEvent ();
+   rc = pfn_WSAEventSelect (SOCK_HANDLE (fd), hEv, FD_ACCEPT);
+   if (rc != SOCKET_ERROR)
+     {
+       rc = WaitForSingleObject (hEv, INFINITE);
+       pfn_WSAEventSelect (SOCK_HANDLE (fd), NULL, 0);
+       pfn_WSACloseEvent (hEv);
+       if (rc == WAIT_OBJECT_0)
+ 	cp->status = STATUS_READ_SUCCEEDED;
+     }
+ 
+   return cp->status;
+ }
+ 
  int
  sys_read (int fd, char * buffer, unsigned int count)
  {
Index: w32.h
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32.h,v
retrieving revision 1.20
diff -c -r1.20 w32.h
*** w32.h	14 Jul 2006 09:29:22 -0000	1.20
--- w32.h	14 Jul 2006 13:24:48 -0000
***************
*** 137,142 ****
--- 137,145 ----
  extern void globals_of_w32menu (void);
  extern void syms_of_fontset (void);
  
+ extern int _sys_read_ahead (int fd);
+ extern int _sys_wait_accept (int fd);
+ 
  #endif /* EMACS_W32_H */
  
  /* arch-tag: 02c36b00-312b-4c4d-a1d9-f905c5e968f0
Index: w32proc.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/w32proc.c,v
retrieving revision 1.66
diff -c -r1.66 w32proc.c
*** w32proc.c	6 Feb 2006 15:23:22 -0000	1.66
--- w32proc.c	14 Jul 2006 13:23:28 -0000
***************
*** 1,6 ****
  /* Process support for GNU Emacs on the Microsoft W32 API.
     Copyright (C) 1992, 1995, 1999, 2000, 2001, 2002, 2003, 2004,
!                  2005, 2006 Free Software Foundation, Inc.
  
  This file is part of GNU Emacs.
  
--- 1,6 ----
  /* Process support for GNU Emacs on the Microsoft W32 API.
     Copyright (C) 1992, 1995, 1999, 2000, 2001, 2002, 2003, 2004,
! 		 2005, 2006 Free Software Foundation, Inc.
  
  This file is part of GNU Emacs.
  
***************
*** 280,286 ****
      {
        int rc;
  
!       rc = _sys_read_ahead (cp->fd);
  
        /* The name char_avail is a misnomer - it really just means the
  	 read-ahead has completed, whether successfully or not. */
--- 280,289 ----
      {
        int rc;
  
!       if (fd_info[cp->fd].flags & FILE_LISTEN)
! 	rc = _sys_wait_accept (cp->fd);
!       else
! 	rc = _sys_read_ahead (cp->fd);
  
        /* The name char_avail is a misnomer - it really just means the
  	 read-ahead has completed, whether successfully or not. */

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: 100% CPU on TCP servers (on Windoze).

[Eli Zaretskii]

> From: storm@cua.dk (Kim F. Storm)
> Date: Fri, 14 Jul 2006 15:24:23 +0200
> Cc: Emacs Devel <emacs-devel@gnu.org>
> 
> Here is something which actually seems to work -- provided
> we can use ws2_32.dll:

We could test that dynamically: if LoadLibrary fails for ws2_32.dll,
fall back on wsock32.dll, and use the old code.  The rest of the code
can test if pfn_WSAEventSelect and pfn_WSACreateEvent are non-NULL to
DTRT.

I asked Jason to comment on this, let's wait for his response.

Re: 100% CPU on TCP servers (on Windoze).

[Kim F. Storm]

Eli Zaretskii <eliz@gnu.org> writes:

> I asked Jason to comment on this, let's wait for his response.

MSDN says that WSAEventSelect is in ws2_32.dll -- and that it is
supported since Windows 95 (but only if you install the Windows Socket
2 update, still available a M$ support web).

Do we need to care about Windows 95 which are not "up to date"?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: 100% CPU on TCP servers (on Windoze).

[Juanma Barranquero]

On 7/14/06, Kim F. Storm <storm@cua.dk> wrote:

> Thanks.  Here is something which actually seems to work -- provided
> we can use ws2_32.dll:

Yeah, it (apparently) works! Great!

I'll dig the old server.el/emacsclient.c code and the thread to see
what's left to do. IIRC, I had it mostly working, but I had gotten rid
of the Unix socket stuff, and someone (Stefan, perhaps?) asked for it
to be kept.

-- 
                    /L/e/k/t/u

Re: 100% CPU on TCP servers (on Windoze).

[Kim F. Storm]

"Juanma Barranquero" <lekktu@gmail.com> writes:

> On 7/14/06, Kim F. Storm <storm@cua.dk> wrote:
>
>> Thanks.  Here is something which actually seems to work -- provided
>> we can use ws2_32.dll:
>
> Yeah, it (apparently) works! Great!
>
> I'll dig the old server.el/emacsclient.c code and the thread to see
> what's left to do. IIRC, I had it mostly working, but I had gotten rid
> of the Unix socket stuff, and someone (Stefan, perhaps?) asked for it
> to be kept.

You can use (featurep 'make-network-process '(:family local)) to see
if unix sockets are available -- use them if available, otherwise
switch to TCP or UDP, and save the server's port number in a file.

Likewise in emacsclient, first try to see if the unix socket file
is present -- if not, look for a file containing the local port number
for the server.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: 100% CPU on TCP servers (on Windoze).

[Juanma Barranquero]

On 7/15/06, Kim F. Storm <storm@cua.dk> wrote:

> You can use (featurep 'make-network-process '(:family local)) to see
> if unix sockets are available -- use them if available,

Wouldn't that preclude using emacsclient from a Windows machine to
connect to an Emacs running on GNU/Linux?

Many issues were already discussed in this (admittedly long) thread
from a year ago:

http://lists.gnu.org/archive/html/emacs-devel/2005-07/msg00913.html
http://lists.gnu.org/archive/html/emacs-devel/2005-08/msg00140.html

-- 
                    /L/e/k/t/u

Re: 100% CPU on TCP servers (on Windoze).

[Kim F. Storm]

"Juanma Barranquero" <lekktu@gmail.com> writes:

> On 7/15/06, Kim F. Storm <storm@cua.dk> wrote:
>
>> You can use (featurep 'make-network-process '(:family local)) to see
>> if unix sockets are available -- use them if available,
>
> Wouldn't that preclude using emacsclient from a Windows machine to
> connect to an Emacs running on GNU/Linux?

Yes, but for now I would be more than satisfied if we can just make it
work on the local system only -- that is what we have on GNU/Linux, so
Windows doesn't have to be more than that.  And doing more is opening
a can of worms,

>
> Many issues were already discussed in this (admittedly long) thread
> from a year ago:
>
> http://lists.gnu.org/archive/html/emacs-devel/2005-07/msg00913.html
> http://lists.gnu.org/archive/html/emacs-devel/2005-08/msg00140.html

I looked through those messages, and I don't see anything which
suggests that cross-system connections are to be considered at this
time.

Give it another shot (keeping AF_UNIX as primary choice, and using a
standard file name for the _local_ server file) and see what you can do.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: 100% CPU on TCP servers (on Windoze).

[Juanma Barranquero]

On 7/15/06, Kim F. Storm <storm@cua.dk> wrote:

> I looked through those messages, and I don't see anything which
> suggests that cross-system connections are to be considered at this
> time.

The sub-thread starting from

http://lists.gnu.org/archive/html/emacs-devel/2005-08/msg00356.html

refers to this specific issue, among others.

-- 
                    /L/e/k/t/u

Re: 100% CPU on TCP servers (on Windoze).

[Eli Zaretskii]

> From: storm@cua.dk (Kim F. Storm)
> Date: Sat, 15 Jul 2006 02:50:42 +0200
> Cc: Emacs Devel <emacs-devel@gnu.org>
> 
> "Juanma Barranquero" <lekktu@gmail.com> writes:
> 
> > On 7/15/06, Kim F. Storm <storm@cua.dk> wrote:
> >
> >> You can use (featurep 'make-network-process '(:family local)) to see
> >> if unix sockets are available -- use them if available,
> >
> > Wouldn't that preclude using emacsclient from a Windows machine to
> > connect to an Emacs running on GNU/Linux?
> 
> Yes, but for now I would be more than satisfied if we can just make it
> work on the local system only

Indeed.  And what's more, I'd be very uneasy about letting the server
accept connections from outside my machine, so if that is ever made a
possibility, I agree with Stefan that it should be an option.

Re: 100% CPU on TCP servers (on Windoze).

[Juanma Barranquero]

On 7/15/06, Eli Zaretskii <eliz@gnu.org> wrote:

> Indeed.  And what's more, I'd be very uneasy about letting the server
> accept connections from outside my machine, so if that is ever made a
> possibility, I agree with Stefan that it should be an option.

The scheme I was thinking of implementing, proposed by Stefan the last
time around, would force the outside machine to have access to a
configuration file written by the server Emacs in order to know the
authentication key. So it's a matter of having the right permissions.
We're not talking about openly available TCP connections from some
arbitrary machine.

-- 
                    /L/e/k/t/u

Re: 100% CPU on TCP servers (on Windoze).

[Stefan Monnier]

>> Indeed.  And what's more, I'd be very uneasy about letting the server
>> accept connections from outside my machine, so if that is ever made a
>> possibility, I agree with Stefan that it should be an option.

> The scheme I was thinking of implementing, proposed by Stefan the last
> time around, would force the outside machine to have access to a
> configuration file written by the server Emacs in order to know the
> authentication key. So it's a matter of having the right permissions.
> We're not talking about openly available TCP connections from some
> arbitrary machine.

Modulo security holes, of course, which is the crux of the matter.


        Stefan

Re: 100% CPU on TCP servers (on Windoze).

[Juanma Barranquero]

On 7/15/06, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> Modulo security holes, of course, which is the crux of the matter.

In the mechanism discussed earlier there wasn't any specific way of
forcing connections to be local, other than using AF_UNIX sockets (on
Unix, GNU/Linux, etc.) or making the .emacs.server file (with the
authentication code) private via file permissions (which would also
work on Windows).

How do you propose exactly the non-remote policy to be implemented,
and more important, with which interface? An elisp variable to force
only connections from 127.0.0.1, a list of valid IPs, or what?

-- 
                    /L/e/k/t/u

Re: 100% CPU on TCP servers (on Windoze).

[Eli Zaretskii]

> Date: Sat, 15 Jul 2006 17:30:01 +0200
> From: "Juanma Barranquero" <lekktu@gmail.com>
> Cc: emacs-devel@gnu.org
> 
> On 7/15/06, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
> > Modulo security holes, of course, which is the crux of the matter.
> 
> In the mechanism discussed earlier there wasn't any specific way of
> forcing connections to be local, other than using AF_UNIX sockets (on
> Unix, GNU/Linux, etc.) or making the .emacs.server file (with the
> authentication code) private via file permissions (which would also
> work on Windows).

I've read that part of the past discussions, but I still would feel
uneasy about allowing outside connections.  My firewall blocks all
outside connections for very good reasons.

So I think this possibility should be optional and off by default.

Re: 100% CPU on TCP servers (on Windoze).

[Stefan Monnier]

>> > Modulo security holes, of course, which is the crux of the matter.
>> In the mechanism discussed earlier there wasn't any specific way of
>> forcing connections to be local, other than using AF_UNIX sockets (on
>> Unix, GNU/Linux, etc.) or making the .emacs.server file (with the
>> authentication code) private via file permissions (which would also
>> work on Windows).

When Unix sockets are not available, I'd bind to 127.0.0.1 only
(additionally to the use of the .emacs.d/server file so as to protect
oneself from other users on the same machine, which is possible even under
w32).

> I've read that part of the past discussions, but I still would feel
> uneasy about allowing outside connections.  My firewall blocks all
> outside connections for very good reasons.

I think I agree that the default should be to only allow local connections.
And I even think that remote connections can be kept for after Emacs-22 if
they introduce any non-trivial difficulty.


        Stefan

Re: 100% CPU on TCP servers (on Windoze).

[Richard Stallman]

    Indeed.  And what's more, I'd be very uneasy about letting the server
    accept connections from outside my machine, so if that is ever made a
    possibility, I agree with Stefan that it should be an option.

And it should be disabled by default!

Re: 100% CPU on TCP servers (on Windoze).

[Stefan Monnier]

>> You can use (featurep 'make-network-process '(:family local)) to see
>> if unix sockets are available -- use them if available,

> Wouldn't that preclude using emacsclient from a Windows machine to
> connect to an Emacs running on GNU/Linux?

Whether or not remote control is allowed (via a TCP port) should be
controlled by a variable.


        Stefan

Re: 100% CPU on TCP servers (on Windoze).

[Juanma Barranquero]

On 7/15/06, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> Whether or not remote control is allowed (via a TCP port) should be
> controlled by a variable.

That's what I'd like to do. And, in this case, using file sockets if
present (as Kim suggests) wouldn't work.

-- 
                    /L/e/k/t/u

Re: 100% CPU on TCP servers (on Windoze).

[Stefan Monnier]

>> Whether or not remote control is allowed (via a TCP port) should be
>> controlled by a variable.

> That's what I'd like to do. And, in this case, using file sockets if
> present (as Kim suggests) wouldn't work.

Huh?  I understand you can write the code in such a way that it wouldn't
work, but why would you since you can also write it in such a way that
it'll work.


        Stefan

Re: 100% CPU on TCP servers (on Windoze).

[Juanma Barranquero]

On 7/15/06, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> Huh?  I understand you can write the code in such a way that it wouldn't
> work, but why would you since you can also write it in such a way that
> it'll work.

AFAICS, Kim proposed that:

 - If the local system supports AF_UNIX sockets, use that;
 - If not, use AF_INET sockets

which wouldn't allow an Emacs running on GNU/Linux to accept
connections from a Windows machine (where AF_UNIX is not supported).

OTOH, I think it should be configurable, via an elisp or environment variable.

-- 
                    /L/e/k/t/u

Re: 100% CPU on TCP servers (on Windoze).

[Stefan Monnier]

>> Huh?  I understand you can write the code in such a way that it wouldn't
>> work, but why would you since you can also write it in such a way that
>> it'll work.

> AFAICS, Kim proposed that:

> - If the local system supports AF_UNIX sockets, use that;
> - If not, use AF_INET sockets

That's for local connections.  If you want to allow remote connections, of
course you'll use a TCP socket either way (maybe additionally to a unix
socket).
I see no problem here.


        Stefan

Re: 100% CPU on TCP servers (on Windoze).

[Juanma Barranquero]

On 7/15/06, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> That's for local connections.  If you want to allow remote connections, of
> course you'll use a TCP socket either way (maybe additionally to a unix
> socket).
> I see no problem here.

Uh? Are you proposing that the server can use simultaneously a Unix
socket for local connections and a TCP socket for remote ones? Or
(what I thought) that the server be configurable to use either Unix
sockets (which would naturally limit the connections to local ones) or
TCP sockets?

-- 
                    /L/e/k/t/u

Re: 100% CPU on TCP servers (on Windoze).

[Stefan Monnier]

>> That's for local connections.  If you want to allow remote connections, of
>> course you'll use a TCP socket either way (maybe additionally to a unix
>> socket).
>> I see no problem here.

> Uh? Are you proposing that the server can use simultaneously a Unix
> socket for local connections and a TCP socket for remote ones?

Why not, if that's necessary (don't know that it is, and don't know that it
isn't either).

> Or (what I thought) that the server be configurable to use either Unix
> sockets (which would naturally limit the connections to local ones) or
> TCP sockets?

The config var should enable/disable remote connections.
The use of Unix sockets vs TCP sockets is an implementation detail.


        Stefan