What does the error "Process <URL> not running" mean?

all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed

* What does the error "Process <URL> not running" mean?
@ 2022-02-02 19:52 emacsq
  2022-02-03  0:14 ` Emanuel Berg via Users list for the GNU Emacs text editor
                   ` (2 more replies)
  0 siblings, 3 replies; 48+ messages in thread
From: emacsq @ 2022-02-02 19:52 UTC (permalink / raw)
  To: help-gnu-emacs@gnu.org

When fetching several urls with url-retrieve, some of them returns with this error:

    Process <the url> not running

What does this error mean? I asked emacs to fetch an url, not start a process, so why does it tell me this and how can I know from this what
the actual error is?

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-02 19:52 What does the error "Process <URL> not running" mean? emacsq
@ 2022-02-03  0:14 ` Emanuel Berg via Users list for the GNU Emacs text editor
  2022-02-03  5:02 ` emacsq
  2022-02-03  6:57 ` Eli Zaretskii
  2 siblings, 0 replies; 48+ messages in thread
From: Emanuel Berg via Users list for the GNU Emacs text editor @ 2022-02-03  0:14 UTC (permalink / raw)
  To: help-gnu-emacs

emacsq wrote:

> When fetching several urls with url-retrieve, some of them
> returns with this error:
>
>     Process <the url> not running
>
> What does this error mean? I asked emacs to fetch an url,
> not start a process

Yeah but I think that's what it does ...

> so why does it tell me this and how can I know from this
> what the actual error is?

What did you do exactly?

-- 
underground experts united
https://dataswamp.org/~incal

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-02 19:52 What does the error "Process <URL> not running" mean? emacsq
  2022-02-03  0:14 ` Emanuel Berg via Users list for the GNU Emacs text editor
@ 2022-02-03  5:02 ` emacsq
  2022-02-03  5:59   ` Tassilo Horn
                     ` (2 more replies)
  2022-02-03  6:57 ` Eli Zaretskii
  2 siblings, 3 replies; 48+ messages in thread
From: emacsq @ 2022-02-03  5:02 UTC (permalink / raw)
  To: help-gnu-emacs@gnu.org

> > What does this error mean? I asked emacs to fetch an url,
> > not start a process
>
> Yeah but I think that's what it does ...

It may do that in the background, I meant from the user
point of view I told emacs to fetch an url, so it telling
me some process is not running does not help me as a user
to know what the problem is.

> > so why does it tell me this and how can I know from this
> > what the actual error is?
>
> What did you do exactly?

I have a document with hundreds of urls in it which needs
to be checked if they are still  alive and well, so I
fetched them one by one with a code like this:

(condition-case errmsg
    (url-retrieve
     "http://www.example.com"

     (lambda (status)
       (let ((error (plist-get status :error)))
         (if error
             (print error))

         ...do something, then check next url...))

     )
  (error (print errmsg)))

It works for most of the urls, it finds 404, non-existent
domains and other errors, but for some urls it returns the
error "Process http://www.example.com not running" which
I don't know what it means.

I tried finding this "process ... not running" text in the
lisp code, but I couldn't find it.

It's on windows, BTW.




^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-03  5:02 ` emacsq
@ 2022-02-03  5:59   ` Tassilo Horn
       [not found]     ` <5Opci81-9PcO6g1ljSQMCeDxh03cTnw4IEQWRzc2QInT4V52Ph2nB2WyyrBkSR11O8A9W4Coh1Wz=5FJ9v-JeqnFC3melWsBOK2-hU9f2tyWQ=3D@protonmail.com>
  2022-02-03  6:18     ` emacsq
  2022-02-03  7:34   ` Emanuel Berg via Users list for the GNU Emacs text editor
  2022-02-03 15:27   ` [External] : " Drew Adams
  2 siblings, 2 replies; 48+ messages in thread
From: Tassilo Horn @ 2022-02-03  5:59 UTC (permalink / raw)
  To: emacsq; +Cc: help-gnu-emacs

emacsq <laszlomail@protonmail.com> writes:

>> What did you do exactly?
>
> I have a document with hundreds of urls in it which needs to be
> checked if they are still alive and well, so I fetched them one by one
> with a code like this:
>
> (condition-case errmsg
>     (url-retrieve
>      "http://www.example.com"
>
>      (lambda (status)
>        (let ((error (plist-get status :error)))
>          (if error
>              (print error))
>
>          ...do something, then check next url...))
>
>      )
>   (error (print errmsg)))

I'd start by printing the complete status plist and see if that sheds
some light.  I assume that you might be hitting some kind of "max number
of child processes" limit since you are talking about hundreds of URLs,
and `url-retrieve' is asynchronous.

You might want to check if `url-retrieve-synchronously' works.  If it
does (but takes much longer, obviously), then the above is most probably
the cause.  In that case, you could try to structure your code so that
it doesn't retrieve all URLs in one go but in chunks of, say, 20 and
starts the next chunk not before the previous is finished.

Bye,
Tassilo

^ permalink raw reply	[flat|nested] 48+ messages in thread

[parent not found: <5Opci81-9PcO6g1ljSQMCeDxh03cTnw4IEQWRzc2QInT4V52Ph2nB2WyyrBkSR11O8A9W4Coh1Wz=5FJ9v-JeqnFC3melWsBOK2-hU9f2tyWQ=3D@protonmail.com>]

[parent not found: <Ckg2a8v34NSsrKnpfXhAXAOdIOvwsFqlv12BoMp9tr1AiSOzlmoJFCSu7I3hi5c2c-Kv3A3L175kXYQZHG8PVQfp7EpUWK8Vj21kFYx21Mo=3D@protonmail.com>]

[parent not found: <TalBxoV5qsEqukLpxb9v9C3cjNPPN86=5FH3ruOHNa1PAofx8PLEGFBsC=5FVE8Jye18zcNRLar9mbeAoNguQshYRZ-kjdPWLK=5FTE0uggtyb2Lg=3D@protonmail.com>]

* Re: What does the error "Process <URL> not running" mean?
  2022-02-03  5:59   ` Tassilo Horn
       [not found]     ` <5Opci81-9PcO6g1ljSQMCeDxh03cTnw4IEQWRzc2QInT4V52Ph2nB2WyyrBkSR11O8A9W4Coh1Wz=5FJ9v-JeqnFC3melWsBOK2-hU9f2tyWQ=3D@protonmail.com>
@ 2022-02-03  6:18     ` emacsq
  2022-02-03  8:08       ` emacsq
  1 sibling, 1 reply; 48+ messages in thread
From: emacsq @ 2022-02-03  6:18 UTC (permalink / raw)
  To: Tassilo Horn; +Cc: help-gnu-emacs




>
> In that case, you could try to structure your code so that
> it doesn't retrieve all URLs in one go but in chunks of, say, 20 and
> starts the next chunk not before the previous is finished.

It checks them one by one, that's what the

>          ...do something, then check next url...))
>

line means in my code.

So when the async result returns for the current url then
I start checking the next url from the callback.




^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-03  6:18     ` emacsq
@ 2022-02-03  8:08       ` emacsq
  2022-02-03  8:49         ` Emanuel Berg via Users list for the GNU Emacs text editor
                           ` (2 more replies)
  0 siblings, 3 replies; 48+ messages in thread
From: emacsq @ 2022-02-03  8:08 UTC (permalink / raw)
  To: Tassilo Horn; +Cc: help-gnu-emacs

> src/process.c

Yes, I see it, thanks.

What I'd like to know is what this error tells me. Is this
something the user can do something about? Or is it some
internal error in the emacs network stack? If it is then
shouldn't it say something like "It should not happen,
please notify the emacs developers"?

url-retrieve is a high level function, so I'd expect it to
return normal errors like "can't connect to url", so the
user knows it's a network error, and if it's something internal
emacs bug then it should say "internal error: ...", so the
user knows he can't do anything about it,  because it's a
bug in emacs.

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-03  8:08       ` emacsq
@ 2022-02-03  8:49         ` Emanuel Berg via Users list for the GNU Emacs text editor
  2022-02-03  9:27         ` Eli Zaretskii
  2022-02-03 10:10         ` emacsq
  2 siblings, 0 replies; 48+ messages in thread
From: Emanuel Berg via Users list for the GNU Emacs text editor @ 2022-02-03  8:49 UTC (permalink / raw)
  To: help-gnu-emacs

emacsq wrote:

>> src/process.c
>
> Yes, I see it, thanks.
>
> What I'd like to know is what this error tells me. Is this
> something the user can do something about? Or is it some
> internal error in the emacs network stack?

Do you know if it originates from line 6427

  error ("Process %s not running", SDATA (p->name));

or line 7131

  error ("Process %s not running", SDATA (XPROCESS (proc)->name));

?

-- 
underground experts united
https://dataswamp.org/~incal




^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-03  8:08       ` emacsq
  2022-02-03  8:49         ` Emanuel Berg via Users list for the GNU Emacs text editor
@ 2022-02-03  9:27         ` Eli Zaretskii
  2022-02-03 10:10         ` emacsq
  2 siblings, 0 replies; 48+ messages in thread
From: Eli Zaretskii @ 2022-02-03  9:27 UTC (permalink / raw)
  To: help-gnu-emacs

> Date: Thu, 03 Feb 2022 08:08:38 +0000
> From: emacsq <laszlomail@protonmail.com>
> Cc: help-gnu-emacs@gnu.org
> 
> > src/process.c
> 
> Yes, I see it, thanks.
> 
> What I'd like to know is what this error tells me. Is this
> something the user can do something about? Or is it some
> internal error in the emacs network stack? If it is then
> shouldn't it say something like "It should not happen,
> please notify the emacs developers"?

The error means that the connection either failed to be established,
or was unexpectedly closed by the other end.



^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-03  8:08       ` emacsq
  2022-02-03  8:49         ` Emanuel Berg via Users list for the GNU Emacs text editor
  2022-02-03  9:27         ` Eli Zaretskii
@ 2022-02-03 10:10         ` emacsq
  2022-02-03 10:35           ` Eli Zaretskii
  2022-02-03 15:05           ` emacsq
  2 siblings, 2 replies; 48+ messages in thread
From: emacsq @ 2022-02-03 10:10 UTC (permalink / raw)
  To: Tassilo Horn; +Cc: help-gnu-emacs

>
> The error means that the connection either failed to be established,
> or was unexpectedly closed by the other end.

Thanks, but then shouldn't the error message say 'connection error' or
something like that if it's a network process, instead of saying
'process x is not running'?

Connection error is more informative as an error message for url-retrieve
which deals with connecting other servers.




^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-03 10:10         ` emacsq
@ 2022-02-03 10:35           ` Eli Zaretskii
  2022-02-03 13:32             ` Robert Pluim
  2022-02-03 15:05           ` emacsq
  1 sibling, 1 reply; 48+ messages in thread
From: Eli Zaretskii @ 2022-02-03 10:35 UTC (permalink / raw)
  To: help-gnu-emacs

> Date: Thu, 03 Feb 2022 10:10:34 +0000
> From: emacsq <laszlomail@protonmail.com>
> Cc: help-gnu-emacs@gnu.org
> 
> > The error means that the connection either failed to be established,
> > or was unexpectedly closed by the other end.
> 
> Thanks, but then shouldn't the error message say 'connection error' or
> something like that if it's a network process, instead of saying
> 'process x is not running'?

Feel free to file a bug report about this, maybe it will be possible
to mention "connection" in the error message.



^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-03 10:35           ` Eli Zaretskii
@ 2022-02-03 13:32             ` Robert Pluim
  0 siblings, 0 replies; 48+ messages in thread
From: Robert Pluim @ 2022-02-03 13:32 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: help-gnu-emacs

>>>>> On Thu, 03 Feb 2022 12:35:57 +0200, Eli Zaretskii <eliz@gnu.org> said:

    >> Date: Thu, 03 Feb 2022 10:10:34 +0000
    >> From: emacsq <laszlomail@protonmail.com>
    >> Cc: help-gnu-emacs@gnu.org
    >> 
    >> > The error means that the connection either failed to be established,
    >> > or was unexpectedly closed by the other end.
    >> 
    >> Thanks, but then shouldn't the error message say 'connection error' or
    >> something like that if it's a network process, instead of saying
    >> 'process x is not running'?

    Eli> Feel free to file a bug report about this, maybe it will be possible
    Eli> to mention "connection" in the error message.

That shouldn't be too hard. Does anyone have a sample url showing the problem?

Robert
-- 



^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-03 10:10         ` emacsq
  2022-02-03 10:35           ` Eli Zaretskii
@ 2022-02-03 15:05           ` emacsq
  2022-02-03 20:07             ` Tassilo Horn
  1 sibling, 1 reply; 48+ messages in thread
From: emacsq @ 2022-02-03 15:05 UTC (permalink / raw)
  To: Tassilo Horn; +Cc: help-gnu-emacs

> That shouldn't be too hard. Does anyone have a sample url showing the problem?

In my experience, it's not a specific error which happens all
the time, it just occurs sometimes. Like connection closed
unexpectedly.

I fetched hundreds of urls with url-retrieve sequentially, so
the next url is fetched when the previous returned, and it works
most of the time, but sometimes you get this "process x not
running" error and then you get this same error for the next
10-20 urls for some reason, and then it works again well for
a while.

So it's an intermittent error. I don't know if it's a problem
in emacs' networking layer or it's something else.

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-03 15:05           ` emacsq
@ 2022-02-03 20:07             ` Tassilo Horn
       [not found]               ` <3qySp5xSA2V0n9C8vwql9UbGKia8POa7OZcDnXg6e8jvW59uKuICMg8MMi5o-drq2sIcWWOejQJhal9aBXZaZM09a6oyenNylYnn5Qjp-H8=3D@protonmail.com>
                                 ` (2 more replies)
  0 siblings, 3 replies; 48+ messages in thread
From: Tassilo Horn @ 2022-02-03 20:07 UTC (permalink / raw)
  To: emacsq; +Cc: help-gnu-emacs

emacsq <laszlomail@protonmail.com> writes:

>> That shouldn't be too hard. Does anyone have a sample url showing the problem?
>
> In my experience, it's not a specific error which happens all the
> time, it just occurs sometimes. Like connection closed unexpectedly.
>
> I fetched hundreds of urls with url-retrieve sequentially, so the next
> url is fetched when the previous returned, and it works most of the
> time, but sometimes you get this "process x not running" error and
> then you get this same error for the next 10-20 urls for some reason,
> and then it works again well for a while.

I've just tried to reproduce with this code

--8<---------------cut here---------------start------------->8---
;; -*- lexical-binding: t; -*-

(defun th/get-urls ()
  (with-temp-buffer
    (insert-file-contents "~/tmp/craft-popular-urls")
    (goto-char (point-min))
    (let (urls)
      (while (re-search-forward "^\\(.+\\)$" nil t)
        (push (match-string 1) urls))
      urls)))

(defun th/url-test (urls)
  (let ((url (pop urls)))
    (url-retrieve url
                  (lambda (status)
                    (when-let ((err (plist-get status :error)))
                      (message "Error for %S: %S" url err))
                    (kill-buffer)
                    (th/url-test urls)))))

(th/url-test (th/get-urls))
--8<---------------cut here---------------end--------------->8---

where ~/tmp/craft-popular-urls is
https://gist.githubusercontent.com/demersdesigns/4442cd84c1cc6c5ccda9b19eac1ba52b/raw/cf06109a805b661dd12133f9aa4473435e478569/craft-popular-urls.
However, I don't get such an error, only some 404s or connection-failed.
I also tried running the invocation 20 times in a loop so that there are
in fact about 20 processes running in parallel with no "luck".  This is
on GNU/Linux, though.

Does the above code with your url list also result in that error?

Bye,
Tassilo



^ permalink raw reply	[flat|nested] 48+ messages in thread

[parent not found: <3qySp5xSA2V0n9C8vwql9UbGKia8POa7OZcDnXg6e8jvW59uKuICMg8MMi5o-drq2sIcWWOejQJhal9aBXZaZM09a6oyenNylYnn5Qjp-H8=3D@protonmail.com>]

[parent not found: <3qySp5xSA2V0n9C8vwql9UbGKia8POa7OZcDnXg6e8jvW59uKuICMg8MMi5o-drq2sIcWWOejQJhal9aBXZaZM09a6oyenNylYn n5Qjp-H8=3D@protonmail.com>]

[parent not found: <6Ox0QxOSiVddeNsCaACeldkV9F-Nh9dM-rRERWveYqhG8t126cIm2MGmadX7Uy8YL-IQX9-Y=5F=5FZjAwEKArVB5v81UoZWZ7U4=5F1R70ywhZZY=3D@protonmail.com>]

[parent not found: <kQ5WkoUp-ir-MeV9eFbiOXYaopRMN0pcwlurXw1SD72U?= =?us-ascii?Q?PJ-pcb1MUYYK3T5ucsurROJMoHHQJ0wR9=5FVRECOWyuXe95B3PJgZ-phCdgdUpHk=3D@protonmail.com>]

[parent not found: <8jy--UTf4wNbxysxHArjzE3ADfF5mB=5FZsfnFd7sgKpf=5FGM=5F9O5YqVK1PH1QbnoizHbb6HonK-BeQEQx0OpCmSRnMSpJzNTcHHXASGOoiD9I=3D@protonmail.com>]

[parent not found: <waNRXNl-vCnaC-u0F0hzugc4OjmLVw35PmjluKisu6G9euFh2IXTKrhR=5FVY1CJ0II2LP49ybLHsvIiu-d6TuwIa0vdN rLJ8ULCaBk=5FR38O8=3D@protonmail.com>]

[parent not found: <waNRXNl-vCnaC-u0F0hzugc4OjmLVw35PmjluKisu6G9euFh2IXTKrhR=5FVY1CJ0II2LP49ybLHsvIiu-d6TuwIa0vdNrLJ8ULCaBk=5FR38O8=3D@protonmail.com>]

* Re: What does the error "Process <URL> not running" mean?
  2022-02-03 20:07             ` Tassilo Horn
       [not found]               ` <3qySp5xSA2V0n9C8vwql9UbGKia8POa7OZcDnXg6e8jvW59uKuICMg8MMi5o-drq2sIcWWOejQJhal9aBXZaZM09a6oyenNylYnn5Qjp-H8=3D@protonmail.com>
       [not found]               ` <3qySp5xSA2V0n9C8vwql9UbGKia8POa7OZcDnXg6e8jvW59uKuICMg8MMi5o-drq2sIcWWOejQJhal9aBXZaZM09a6oyenNylYn n5Qjp-H8=3D@protonmail.com>
@ 2022-02-03 20:34               ` emacsq
  2022-02-03 20:47                 ` emacsq
  2 siblings, 1 reply; 48+ messages in thread
From: emacsq @ 2022-02-03 20:34 UTC (permalink / raw)
  To: Tassilo Horn; +Cc: help-gnu-emacs

>
> Does the above code with your url list also result in that error?
>

First, I tried your list with my code. It's worse than with my list. :)
With my list it happens only occassionally, but with your list it
comes out pretty quickly.

If you don't experince this then I guess it's some local networking thing
on my machine, probably not emacs' fault.

I should run some network trace to see why and where the connections
break down exactly. I may do that when I have the time.

My output:

http://www.youtube.com			YouTube
http://www.facebook.com			no title
http://www.baidu.com			百度一下，你就知道
http://www.yahoo.com			Yahoo | Mail, Weather, Search, Politics, News, Finance, Sports & Videos
http://www.amazon.com			Amazon.com. Spend less. Smile more.
http://www.wikipedia.org			Wikipedia
http://www.qq.com			  Ѷ  ҳ
http://www.google.co.in			Google
http://www.twitter.com			no title
http://www.live.com			Outlook – free personal email and calendar from Microsoft
http://www.taobao.com			天貓淘寶海外，花更少，買到寶！
http://www.bing.com			Bing
http://www.instagram.com			Instagram
http://www.weibo.com			微博国际
http://www.sina.com.cn			新浪首页
http://www.linkedin.com			LinkedIn: Log In or Sign Up
http://www.yahoo.co.jp			Yahoo! JAPAN
http://www.vk.com			(error http 429)
http://www.google.de			Process www.google.de not running
http://www.yandex.ru			Process www.yandex.ru not running
http://www.hao123.com			Process www.hao123.com not running
http://www.google.co.uk			Process www.google.co.uk not running
http://www.reddit.com			Process www.reddit.com not running
http://www.ebay.com			Process www.ebay.com not running
http://www.google.fr			Process www.google.fr not running
http://www.t.co			Process www.t.co not running
http://www.tmall.com			Process www.tmall.com not running
http://www.google.com.br			Process www.google.com.br not running
http://www.360.cn			Process www.360.cn not running
http://www.sohu.com			Process www.sohu.com not running
http://www.amazon.co.jp			Process www.amazon.co.jp not running
http://www.pinterest.com			Process www.pinterest.com not running
http://www.netflix.com			Process www.netflix.com not running
http://www.google.it			Process www.google.it not running
http://www.google.ru			Process www.google.ru not running
http://www.microsoft.com			Process www.microsoft.com not running
http://www.google.es			Process www.google.es not running
http://www.wordpress.com			Process www.wordpress.com not running
http://www.gmw.cn			Process www.gmw.cn not running
http://www.tumblr.com			Process www.tumblr.com not running
http://www.paypal.com			Process www.paypal.com not running
http://www.blogspot.com			Process www.blogspot.com not running
http://www.imgur.com			Process www.imgur.com not running
http://www.stackoverflow.com			Process www.stackoverflow.com not running
http://www.aliexpress.com			Process www.aliexpress.com not running
http://www.naver.com			Process www.naver.com not running
http://www.ok.ru			Process www.ok.ru not running
http://www.apple.com			Process www.apple.com not running
http://www.github.com			Process www.github.com not running
http://www.chinadaily.com.cn			Process www.chinadaily.com.cn not running
http://www.imdb.com			Process www.imdb.com not running
http://www.google.co.kr			Process www.google.co.kr not running
http://www.fc2.com			Process www.fc2.com not running
http://www.jd.com			Process www.jd.com not running
http://www.blogger.com			Process www.blogger.com not running
http://www.163.com			Process www.163.com not running
http://www.google.ca			Process www.google.ca not running
http://www.whatsapp.com			Process www.whatsapp.com not running
http://www.amazon.in			Process www.amazon.in not running
http://www.office.com			Process www.office.com not running
http://www.tianya.cn			Process www.tianya.cn not running
http://www.google.co.id			Process www.google.co.id not running
http://www.youku.com			Process www.youku.com not running
http://www.rakuten.co.jp			Process www.rakuten.co.jp not running
http://www.craigslist.org			Process www.craigslist.org not running
http://www.amazon.de			Process www.amazon.de not running
http://www.nicovideo.jp			Process www.nicovideo.jp not running
http://www.google.pl			Process www.google.pl not running
http://www.soso.com			Process www.soso.com not running
http://www.bilibili.com			Process www.bilibili.com not running
http://www.dropbox.com			Process www.dropbox.com not running
http://www.xinhuanet.com			Process www.xinhuanet.com not running
http://www.outbrain.com			Process www.outbrain.com not running
http://www.pixnet.net			Process www.pixnet.net not running
http://www.alibaba.com			Process www.alibaba.com not running
http://www.alipay.com			Process www.alipay.com not running
http://www.microsoftonline.com			www.microsoftonline.com/80 getaddrinfo error 11001
http://www.booking.com			Process www.booking.com not running
http://www.googleusercontent.com			Process www.googleusercontent.com not running
http://www.google.com.au			Process www.google.com.au not running
http://www.popads.net			Process www.popads.net not running
http://www.cntv.cn			Process www.cntv.cn not running
http://www.zhihu.com			Process www.zhihu.com not running
http://www.amazon.co.uk			Process www.amazon.co.uk not running
http://www.diply.com			Process www.diply.com not running
http://www.coccoc.com			Process www.coccoc.com not running
http://www.cnn.com			Process www.cnn.com not running
http://www.bbc.co.uk			Process www.bbc.co.uk not running
http://www.twitch.tv			Process www.twitch.tv not running
http://www.wikia.com			Process www.wikia.com not running
http://www.google.co.th			Process www.google.co.th not running
http://www.go.com			Process www.go.com not running
http://www.google.com.ph			Process www.google.com.ph not running
http://www.doubleclick.net			www.doubleclick.net/80 getaddrinfo error 11004
http://www.onet.pl			Process www.onet.pl not running
http://www.googleadservices.com			Process www.googleadservices.com not running
http://www.accuweather.com			Process www.accuweather.com not running
http://www.googleweblight.com			Process www.googleweblight.com not running
http://www.answers.yahoo.com			Process www.answers.yahoo.com not running

finished

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-03 20:34               ` emacsq
@ 2022-02-03 20:47                 ` emacsq
  2022-02-06 16:24                   ` emacsq
  0 siblings, 1 reply; 48+ messages in thread
From: emacsq @ 2022-02-03 20:47 UTC (permalink / raw)
  To: Tassilo Horn; +Cc: help-gnu-emacs


> If you don't experince this then I guess it's some local networking
thing
> on my machine, probably not emacs' fault.

Just a wild guess: maybe the windows firewall blocks them if the
same application tries to access many different domains in a short
time, because it thinks it's suspicious?

Just an idea. I'll check it when I have the time.





^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-03 20:47                 ` emacsq
@ 2022-02-06 16:24                   ` emacsq
  2022-02-06 16:54                     ` Eli Zaretskii
  2022-02-06 18:25                     ` emacsq
  0 siblings, 2 replies; 48+ messages in thread
From: emacsq @ 2022-02-06 16:24 UTC (permalink / raw)
  To: Tassilo Horn; +Cc: help-gnu-emacs

> Just an idea. I'll check it when I have the time.

Apparently, it's too many open files:

("make client process failed" "Too many open files" :name "example.com" ...)

Which is curious, because it's the same with url-queue-retrieve
which allows 6 parallel processes by default.

I checked and on windows (where I tried) the default file
handle limit is 512 for a process. (which can be increased with
_setmaxstdio)

512 is not much, but if only 6 network calls are going on at the
same time then it should be enough. Unless, the emacs windows'
networking code does not close file handles in a timely manner.

Emacs letting the open handles lingering for a while or not
closing them for some reason (handle leaking) could explain
why fetching many urls can run into the handle limit even if
there are not many parallel fetches at the same time.

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-06 16:24                   ` emacsq
@ 2022-02-06 16:54                     ` Eli Zaretskii
  2022-02-06 18:25                     ` emacsq
  1 sibling, 0 replies; 48+ messages in thread
From: Eli Zaretskii @ 2022-02-06 16:54 UTC (permalink / raw)
  To: help-gnu-emacs

> Date: Sun, 06 Feb 2022 16:24:15 +0000
> From: emacsq <laszlomail@protonmail.com>
> Cc: help-gnu-emacs@gnu.org
> 
> 
> > Just an idea. I'll check it when I have the time.
> 
> Apparently, it's too many open files:
> 
> ("make client process failed" "Too many open files" :name "example.com" ...)
> 
> Which is curious, because it's the same with url-queue-retrieve
> which allows 6 parallel processes by default.
> 
> I checked and on windows (where I tried) the default file
> handle limit is 512 for a process. (which can be increased with
> _setmaxstdio)

I don't think _setmaxstdio is relevant here: it only affects the
number of FILE objects a process can have at any given time (hence the
"stdio" part: it alludes to the stdio.h header).  IOW, it only affects
stream I/O functions: fscanf, fprintf, fread, fwrite, etc.  And we
don't use those in networking implementation in Emacs, we use
low-level functions that go through file descriptors and even
lower-level handles.

It is much more probable that you are hitting here the 32 subprocesses
limit we impose on MS-Windows (for boring technical reasons related to
how we emulate 'pselect' and SIGCHLD there).  If a Lisp program
attempts to create more than 32 sub-processes/network connections at
the same time, it will indeed get "Too many open files" (EMFILE).
That limit cannot be lifted unless we reimplement core parts of
subprocess support on MS-Windows.  (If you are interested, read the
large comment around line 850 in w32proc.c.)

> 512 is not much, but if only 6 network calls are going on at the
> same time then it should be enough. Unless, the emacs windows'
> networking code does not close file handles in a timely manner.
> 
> Emacs letting the open handles lingering for a while or not
> closing them for some reason (handle leaking) could explain
> why fetching many urls can run into the handle limit even if
> there are not many parallel fetches at the same time.

I don't think that's the case: we close the handles as soon as the
network connection is closed.

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-06 16:24                   ` emacsq
  2022-02-06 16:54                     ` Eli Zaretskii
@ 2022-02-06 18:25                     ` emacsq
  2022-02-06 19:20                       ` emacsq
  1 sibling, 1 reply; 48+ messages in thread
From: emacsq @ 2022-02-06 18:25 UTC (permalink / raw)
  To: Tassilo Horn; +Cc: help-gnu-emacs

> It is much more probable that you are hitting here the 32 subprocesses
> limit we impose on MS-Windows (for boring technical reasons related to
> how we emulate 'pselect' and SIGCHLD there).  If a Lisp program
> attempts to create more than 32 sub-processes/network connections at
> the same time, it will indeed get "Too many open files" (EMFILE).

Now I'm using url-queue-retrieve where the default number of
parallel processes is 6, so if the queue works well then there
shouldn't be too many subprocesses.

> I don't think that's the case: we close the handles as soon as the
> network connection is closed.

I tried url-queue-retrieve with url-queue-parallel-processes set
to 1 and there was no 'not running' error then.

But during fetching the urls there were lots of open processes,
when I checked list-processes, so either the connections linger,
or the 'open' state of network processes in list-processes
means something else than an open network connection.

This is with url-queue-parallel-processes set to 1 and when
I started the queue list-processes was empty:


<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open
<various site urls> --      open






^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-06 18:25                     ` emacsq
@ 2022-02-06 19:20                       ` emacsq
  2022-02-07 17:35                         ` emacsq
  0 siblings, 1 reply; 48+ messages in thread
From: emacsq @ 2022-02-06 19:20 UTC (permalink / raw)
  To: Tassilo Horn; +Cc: help-gnu-emacs

> But during fetching the urls there were lots of open processes,
> when I checked list-processes, so either the connections linger,
> or the 'open' state of network processes in list-processes
> means something else than an open network connection.

If somebody wants to test this, here's a little test code.
Parallel is set to 1, so one url is fetched at a time

If you eval this then it starts fetching. Check list-processes
multiple times while it's running. There are many open
connections, despite fetching urls one by one:

(progn
  (setq url-queue-parallel-processes 1)
  (dolist (url '("http://www.wikipedia.org"
                 "http://www.qq.com"
                 "http://www.google.co.in"
                 "http://www.twitter.com"
                 "http://www.live.com"
                 "http://www.taobao.com"
                 "http://www.bing.com"
                 "http://www.instagram.com"
                 "http://www.weibo.com"
                 "http://www.sina.com.cn"
                 "http://www.linkedin.com"
                 "http://www.yahoo.co.jp"
                 "http://www.msn.com"
                 "http://www.vk.com"
                 "http://www.google.de"
                 "http://www.yandex.ru"
                 "http://www.hao123.com"
                 "http://www.google.co.uk"
                 "http://www.reddit.com"
                 "http://www.ebay.com"
                 "http://www.google.fr"
                 "http://www.t.co"
                 "http://www.tmall.com"
                 "http://www.google.com.br"
                 "http://www.360.cn"
                 "http://www.sohu.com"
                 "http://www.amazon.co.jp"
                 "http://www.pinterest.com"
                 "http://www.netflix.com"
                 "http://www.google.it"
                 "http://www.google.ru"
                 "http://www.microsoft.com"
                 "http://www.google.es"
                 "http://www.wordpress.com"
                 "http://www.gmw.cn"
                 "http://www.tumblr.com"
                 "http://www.paypal.com"
                 "http://www.blogspot.com"
                 "http://www.imgur.com"
                 "http://www.stackoverflow.com"
                 "http://www.aliexpress.com"
                 "http://www.naver.com"
                 "http://www.ok.ru"
                 "http://www.apple.com"
                 "http://www.github.com"
                 "http://www.chinadaily.com.cn"
                 "http://www.imdb.com"
                 "http://www.google.co.kr"
                 "http://www.fc2.com"
                 "http://www.jd.com"
                 "http://www.blogger.com"
                 "http://www.163.com"
                 "http://www.google.ca"
                 "http://www.whatsapp.com"
                 "http://www.amazon.in"
                 "http://www.office.com"
                 "http://www.tianya.cn"
                 "http://www.google.co.id"
                 "http://www.youku.com"
                 "http://www.rakuten.co.jp"
                 "http://www.craigslist.org"
                 "http://www.amazon.de"
                 "http://www.nicovideo.jp"
                 "http://www.google.pl"
                 "http://www.soso.com"
                 "http://www.bilibili.com"
                 "http://www.dropbox.com"
                 "http://www.xinhuanet.com"
                 "http://www.outbrain.com"
                 "http://www.pixnet.net"
                 "http://www.alibaba.com"
                 "http://www.alipay.com"
                 "http://www.microsoftonline.com"
                 "http://www.booking.com"
                 "http://www.googleusercontent.com"
                 "http://www.google.com.au"
                 "http://www.popads.net"
                 "http://www.cntv.cn"
                 "http://www.zhihu.com"
                 "http://www.amazon.co.uk"
                 "http://www.diply.com"
                 "http://www.coccoc.com"
                 "http://www.cnn.com"
                 "http://www.bbc.co.uk"
                 "http://www.twitch.tv"
                 "http://www.wikia.com"
                 "http://www.google.co.th"
                 "http://www.go.com"
                 "http://www.google.com.ph"
                 "http://www.doubleclick.net"
                 "http://www.onet.pl"
                 "http://www.googleadservices.com"
                 "http://www.accuweather.com"
                 "http://www.googleweblight.com"
                 "http://www.answers.yahoo.com"
                 ))
    (url-queue-retrieve
     url
     (lambda (status url)
       (message "%s fetched" url))
     (list url))))




^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-06 19:20                       ` emacsq
@ 2022-02-07 17:35                         ` emacsq
  2022-02-07 19:48                           ` Tassilo Horn
  0 siblings, 1 reply; 48+ messages in thread
From: emacsq @ 2022-02-07 17:35 UTC (permalink / raw)
  To: Tassilo Horn; +Cc: help-gnu-emacs


If someone here is on linux, could you run the test below?

I wonder if it's a windows-only thing and linux
will show only one open connection at a time, or
it's the same on linux and previous network connections
linger on for a while regardless of OS.

------- Original Message -------

If somebody wants to test this, here's a little test code.
Parallel is set to 1, so one url is fetched at a time

If you eval this then it starts fetching. Check list-processes
multiple times while it's running. There are many open
connections, despite fetching urls one by one:

(progn
  (setq url-queue-parallel-processes 1)
  (dolist (url '("http://www.wikipedia.org"
                 "http://www.qq.com"
                 "http://www.google.co.in"
                 "http://www.twitter.com"
                 "http://www.live.com"
                 "http://www.taobao.com"
                 "http://www.bing.com"
                 "http://www.instagram.com"
                 "http://www.weibo.com"
                 "http://www.sina.com.cn"
                 "http://www.linkedin.com"
                 "http://www.yahoo.co.jp"
                 "http://www.msn.com"
                 "http://www.vk.com"
                 "http://www.google.de"
                 "http://www.yandex.ru"
                 "http://www.hao123.com"
                 "http://www.google.co.uk"
                 "http://www.reddit.com"
                 "http://www.ebay.com"
                 "http://www.google.fr"
                 "http://www.t.co"
                 "http://www.tmall.com"
                 "http://www.google.com.br"
                 "http://www.360.cn"
                 "http://www.sohu.com"
                 "http://www.amazon.co.jp"
                 "http://www.pinterest.com"
                 "http://www.netflix.com"
                 "http://www.google.it"
                 "http://www.google.ru"
                 "http://www.microsoft.com"
                 "http://www.google.es"
                 "http://www.wordpress.com"
                 "http://www.gmw.cn"
                 "http://www.tumblr.com"
                 "http://www.paypal.com"
                 "http://www.blogspot.com"
                 "http://www.imgur.com"
                 "http://www.stackoverflow.com"
                 "http://www.aliexpress.com"
                 "http://www.naver.com"
                 "http://www.ok.ru"
                 "http://www.apple.com"
                 "http://www.github.com"
                 "http://www.chinadaily.com.cn"
                 "http://www.imdb.com"
                 "http://www.google.co.kr"
                 "http://www.fc2.com"
                 "http://www.jd.com"
                 "http://www.blogger.com"
                 "http://www.163.com"
                 "http://www.google.ca"
                 "http://www.whatsapp.com"
                 "http://www.amazon.in"
                 "http://www.office.com"
                 "http://www.tianya.cn"
                 "http://www.google.co.id"
                 "http://www.youku.com"
                 "http://www.rakuten.co.jp"
                 "http://www.craigslist.org"
                 "http://www.amazon.de"
                 "http://www.nicovideo.jp"
                 "http://www.google.pl"
                 "http://www.soso.com"
                 "http://www.bilibili.com"
                 "http://www.dropbox.com"
                 "http://www.xinhuanet.com"
                 "http://www.outbrain.com"
                 "http://www.pixnet.net"
                 "http://www.alibaba.com"
                 "http://www.alipay.com"
                 "http://www.microsoftonline.com"
                 "http://www.booking.com"
                 "http://www.googleusercontent.com"
                 "http://www.google.com.au"
                 "http://www.popads.net"
                 "http://www.cntv.cn"
                 "http://www.zhihu.com"
                 "http://www.amazon.co.uk"
                 "http://www.diply.com"
                 "http://www.coccoc.com"
                 "http://www.cnn.com"
                 "http://www.bbc.co.uk"
                 "http://www.twitch.tv"
                 "http://www.wikia.com"
                 "http://www.google.co.th"
                 "http://www.go.com"
                 "http://www.google.com.ph"
                 "http://www.doubleclick.net"
                 "http://www.onet.pl"
                 "http://www.googleadservices.com"
                 "http://www.accuweather.com"
                 "http://www.googleweblight.com"
                 "http://www.answers.yahoo.com"
                 ))
    (url-queue-retrieve
     url
     (lambda (status url)
       (message "%s fetched" url))
     (list url))))



^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-07 17:35                         ` emacsq
@ 2022-02-07 19:48                           ` Tassilo Horn
       [not found]                             ` <=5FcUGEWwwd0FeA1-yUQ5OJfKvG6Y2m4lFHC0m42EOzULv02BYrQMv6BYH-YMsVim1q3G8b3ZNnmPMGB0ZgQP7=5F2Ib5vLqAHln3bVDUOqT3eU=3D@protonmail.com>
                                               ` (5 more replies)
  0 siblings, 6 replies; 48+ messages in thread
From: Tassilo Horn @ 2022-02-07 19:48 UTC (permalink / raw)
  To: emacsq; +Cc: help-gnu-emacs

emacsq <laszlomail@protonmail.com> writes:

> If someone here is on linux, could you run the test below?
>
> I wonder if it's a windows-only thing and linux
> will show only one open connection at a time, or
> it's the same on linux and previous network connections
> linger on for a while regardless of OS.

I've tried it (with just 5 of your urls) on GNU/Linux and see the same.
The network processes seem to stay in open status for quite some time
even though I also killed the buffer in the callback function.  It's now
about 5 minutes since my test and still 4 of initially 8 or 9 network
processes are listed as open in `list-processes'.

I have no idea if that's intended or not.

Bye,
Tassilo



^ permalink raw reply	[flat|nested] 48+ messages in thread

[parent not found: <=5FcUGEWwwd0FeA1-yUQ5OJfKvG6Y2m4lFHC0m42EOzULv02BYrQMv6BYH-YMsVim1q3G8b3ZNnmPMGB0ZgQP7=5F2Ib5vLqAHln3bVDUOqT3eU=3D@protonmail.com>]

[parent not found: <=3D5FcUGEWwwd0FeA1-yUQ5OJfKvG6Y2m4lFHC0m42EOzULv02BYrQMv6BY H-YMsVim1q3G8b3ZNnmPMGB0ZgQP7=3D5F2Ib5vLqAHln3bVDUOqT3eU=3D3D@protonmail.com>]

[parent not found: <=3D5FcUGEWwwd0FeA1-yUQ5OJfKvG6Y2m4lFHC0m42EOzULv02BYrQMv6BYH-YMsVim1q3G8b3ZNnmPMGB0ZgQP7=3D5F2Ib5vLqAHln3bVDUOqT3eU=3D3D@protonmail.com>]

* Re: What does the error "Process <URL> not running" mean?
  2022-02-07 19:48                           ` Tassilo Horn
                                               ` (2 preceding siblings ...)
       [not found]                             ` <=3D5FcUGEWwwd0FeA1-yUQ5OJfKvG6Y2m4lFHC0m42EOzULv02BYrQMv6BYH-YMsVim1q3G8b3ZNnmPMGB0ZgQP7=3D5F2Ib5vLqAHln3bVDUOqT3eU=3D3D@protonmail.com>
@ 2022-02-07 20:10                             ` Eli Zaretskii
  2022-02-08  5:55                               ` Tassilo Horn
  2022-02-08 13:24                               ` Stefan Monnier via Users list for the GNU Emacs text editor
  2022-02-07 20:33                             ` Stefan Monnier via Users list for the GNU Emacs text editor
  2022-02-07 20:36                             ` emacsq
  5 siblings, 2 replies; 48+ messages in thread
From: Eli Zaretskii @ 2022-02-07 20:10 UTC (permalink / raw)
  To: help-gnu-emacs

> From: Tassilo Horn <tsdh@gnu.org>
> Date: Mon, 07 Feb 2022 20:48:08 +0100
> Cc: help-gnu-emacs@gnu.org
> 
> emacsq <laszlomail@protonmail.com> writes:
> 
> > If someone here is on linux, could you run the test below?
> >
> > I wonder if it's a windows-only thing and linux
> > will show only one open connection at a time, or
> > it's the same on linux and previous network connections
> > linger on for a while regardless of OS.
> 
> I've tried it (with just 5 of your urls) on GNU/Linux and see the same.
> The network processes seem to stay in open status for quite some time
> even though I also killed the buffer in the callback function.  It's now
> about 5 minutes since my test and still 4 of initially 8 or 9 network
> processes are listed as open in `list-processes'.
> 
> I have no idea if that's intended or not.

Isn't that the timeout till the other side closes the connection?



^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-07 20:10                             ` Eli Zaretskii
@ 2022-02-08  5:55                               ` Tassilo Horn
  2022-02-08 12:33                                 ` Eli Zaretskii
  2022-02-08 13:24                               ` Stefan Monnier via Users list for the GNU Emacs text editor
  1 sibling, 1 reply; 48+ messages in thread
From: Tassilo Horn @ 2022-02-08  5:55 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: help-gnu-emacs

Eli Zaretskii <eliz@gnu.org> writes:

>> I've tried it (with just 5 of your urls) on GNU/Linux and see the
>> same.  The network processes seem to stay in open status for quite
>> some time even though I also killed the buffer in the callback
>> function.  It's now about 5 minutes since my test and still 4 of
>> initially 8 or 9 network processes are listed as open in
>> `list-processes'.
>> 
>> I have no idea if that's intended or not.
>
> Isn't that the timeout till the other side closes the connection?

That's certainly possible.  At least I can say the following:

1. I've retrieved 5 urls
2. but got about 9 network processes (so multiple for some hosts)
3. and the processes stayed open for different times where the
   difference was more that the period between retrievals.

2 seems to be caused due to redirection, i.e., I fetched the http site
which redirected to the https site.

3 could be that the other sides have different timeouts.  Some
connections disappear after maybe 30 seconds, others survive 5 minutes
or even longer.  Eventually, they'll all disappear.

Bye,
Tassilo

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-08  5:55                               ` Tassilo Horn
@ 2022-02-08 12:33                                 ` Eli Zaretskii
  0 siblings, 0 replies; 48+ messages in thread
From: Eli Zaretskii @ 2022-02-08 12:33 UTC (permalink / raw)
  To: help-gnu-emacs

> From: Tassilo Horn <tsdh@gnu.org>
> Cc: help-gnu-emacs@gnu.org
> Date: Tue, 08 Feb 2022 06:55:11 +0100
> 
> > Isn't that the timeout till the other side closes the connection?
> 
> That's certainly possible.  At least I can say the following:
> 
> 1. I've retrieved 5 urls
> 2. but got about 9 network processes (so multiple for some hosts)
> 3. and the processes stayed open for different times where the
>    difference was more that the period between retrievals.
> 
> 2 seems to be caused due to redirection, i.e., I fetched the http site
> which redirected to the https site.
> 
> 3 could be that the other sides have different timeouts.  Some
> connections disappear after maybe 30 seconds, others survive 5 minutes
> or even longer.  Eventually, they'll all disappear.

I would try using 'netstat' to display the state of each of these
connections during the time they stay open while you expect them to
die.  If you see them in shutdown handshake, then you'll know.



^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-07 20:10                             ` Eli Zaretskii
  2022-02-08  5:55                               ` Tassilo Horn
@ 2022-02-08 13:24                               ` Stefan Monnier via Users list for the GNU Emacs text editor
  2022-02-08 16:20                                 ` Robert Pluim
  1 sibling, 1 reply; 48+ messages in thread
From: Stefan Monnier via Users list for the GNU Emacs text editor @ 2022-02-08 13:24 UTC (permalink / raw)
  To: help-gnu-emacs

Eli Zaretskii [2022-02-07 22:10:09] wrote:
>> I've tried it (with just 5 of your urls) on GNU/Linux and see the same.
>> The network processes seem to stay in open status for quite some time
>> even though I also killed the buffer in the callback function.  It's now
>> about 5 minutes since my test and still 4 of initially 8 or 9 network
>> processes are listed as open in `list-processes'.
>> 
>> I have no idea if that's intended or not.
>
> Isn't that the timeout till the other side closes the connection?

So the URL library (under w32) should try and eagerly cull its set of
open connections to bound the number of open connections it keeps well
below the 32 limit.

        Stefan

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-08 13:24                               ` Stefan Monnier via Users list for the GNU Emacs text editor
@ 2022-02-08 16:20                                 ` Robert Pluim
  2022-02-08 17:49                                   ` Stefan Monnier via Users list for the GNU Emacs text editor
  2022-02-08 18:35                                   ` Eli Zaretskii
  0 siblings, 2 replies; 48+ messages in thread
From: Robert Pluim @ 2022-02-08 16:20 UTC (permalink / raw)
  To: help-gnu-emacs; +Cc: Stefan Monnier

>>>>> On Tue, 08 Feb 2022 08:24:40 -0500, Stefan Monnier via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> said:

    Stefan> Eli Zaretskii [2022-02-07 22:10:09] wrote:
    >>> I've tried it (with just 5 of your urls) on GNU/Linux and see the same.
    >>> The network processes seem to stay in open status for quite some time
    >>> even though I also killed the buffer in the callback function.  It's now
    >>> about 5 minutes since my test and still 4 of initially 8 or 9 network
    >>> processes are listed as open in `list-processes'.
    >>> 
    >>> I have no idea if that's intended or not.
    >> 
    >> Isn't that the timeout till the other side closes the connection?

    Stefan> So the URL library (under w32) should try and eagerly cull its set of
    Stefan> open connections to bound the number of open connections it keeps well
    Stefan> below the 32 limit.

Iʼd rather someone with the requisite skills and tolerance for Windows
changed the Emacs select emulation to remove the limit (perhaps by
using the Gnulib 'select' module).

Robert
-- 



^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-08 16:20                                 ` Robert Pluim
@ 2022-02-08 17:49                                   ` Stefan Monnier via Users list for the GNU Emacs text editor
  2022-02-08 18:35                                   ` Eli Zaretskii
  1 sibling, 0 replies; 48+ messages in thread
From: Stefan Monnier via Users list for the GNU Emacs text editor @ 2022-02-08 17:49 UTC (permalink / raw)
  To: help-gnu-emacs

Robert Pluim [2022-02-08 17:20:41] wrote:
>>>>>> On Tue, 08 Feb 2022 08:24:40 -0500, Stefan Monnier via Users list for
>     Stefan> So the URL library (under w32) should try and eagerly cull its set of
>     Stefan> open connections to bound the number of open connections it keeps well
>     Stefan> below the 32 limit.
> Iʼd rather someone with the requisite skills and tolerance for Windows
> changed the Emacs select emulation to remove the limit (perhaps by
> using the Gnulib 'select' module).

I think both are desirable: I think even under POSIX systems it makes
sense to bound the number of connections we keep open (that limit
should be significantly higher than 32, OTOH, but once that code is
written, it's trivial to (if (eq system-type 'windows-nt) 8 256).

        Stefan

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-08 16:20                                 ` Robert Pluim
  2022-02-08 17:49                                   ` Stefan Monnier via Users list for the GNU Emacs text editor
@ 2022-02-08 18:35                                   ` Eli Zaretskii
  2022-02-09  8:43                                     ` Robert Pluim
  1 sibling, 1 reply; 48+ messages in thread
From: Eli Zaretskii @ 2022-02-08 18:35 UTC (permalink / raw)
  To: help-gnu-emacs

> From: Robert Pluim <rpluim@gmail.com>
> Date: Tue, 08 Feb 2022 17:20:41 +0100
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>
> 
> Iʼd rather someone with the requisite skills and tolerance for Windows
> changed the Emacs select emulation to remove the limit (perhaps by
> using the Gnulib 'select' module).

Why do you think the Gnulib module is free from the same (or similar)
limitation?  Isn't FD_SETSIZE = 64 in winsock2.h?

Also, the pselect emulation in Emacs also emulates SIGCHLD from
sub-processes, something the Gnulib module does not AFAIR.

And finally, Gnulib dropped Windows 9X support long ago, so its module
will not run on those old versions (and actually is unlikely to run
even on Windows XP, as they've abandoned that as well).

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-08 18:35                                   ` Eli Zaretskii
@ 2022-02-09  8:43                                     ` Robert Pluim
  2022-02-09  9:34                                       ` Po Lu
  2022-02-09 14:17                                       ` Eli Zaretskii
  0 siblings, 2 replies; 48+ messages in thread
From: Robert Pluim @ 2022-02-09  8:43 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: help-gnu-emacs

>>>>> On Tue, 08 Feb 2022 20:35:38 +0200, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Robert Pluim <rpluim@gmail.com>
    >> Date: Tue, 08 Feb 2022 17:20:41 +0100
    >> Cc: Stefan Monnier <monnier@iro.umontreal.ca>
    >> 
    >> Iʼd rather someone with the requisite skills and tolerance for Windows
    >> changed the Emacs select emulation to remove the limit (perhaps by
    >> using the Gnulib 'select' module).

    Eli> Why do you think the Gnulib module is free from the same (or similar)
    Eli> limitation?  Isn't FD_SETSIZE = 64 in winsock2.h?

It is, but youʼre allowed to set it higher, as far as I know. Maybe it
would be enough to just call winsock `select' from Emacs' select
emulation, but only for sockets.

    Eli> Also, the pselect emulation in Emacs also emulates SIGCHLD from
    Eli> sub-processes, something the Gnulib module does not AFAIR.

Thatʼs only needed for real sub-processes, not sockets, no?

    Eli> And finally, Gnulib dropped Windows 9X support long ago, so its module
    Eli> will not run on those old versions (and actually is unlikely to run
    Eli> even on Windows XP, as they've abandoned that as well).

At some point we have to stop supporting ancient platforms.

Robert
-- 



^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-09  8:43                                     ` Robert Pluim
@ 2022-02-09  9:34                                       ` Po Lu
  2022-02-09  9:45                                         ` Po Lu
  2022-02-09  9:53                                         ` Robert Pluim
  2022-02-09 14:17                                       ` Eli Zaretskii
  1 sibling, 2 replies; 48+ messages in thread
From: Po Lu @ 2022-02-09  9:34 UTC (permalink / raw)
  To: Robert Pluim; +Cc: Eli Zaretskii, help-gnu-emacs

Robert Pluim <rpluim@gmail.com> writes:

>     Eli> And finally, Gnulib dropped Windows 9X support long ago, so its module
>     Eli> will not run on those old versions (and actually is unlikely to run
>     Eli> even on Windows XP, as they've abandoned that as well).

> At some point we have to stop supporting ancient platforms.

Windows XP is hardly ancient, and at my day job we have a machine
running Windows 98 used for legacy government software, and it's
convenient to be able to run a recent version of Emacs there.

Emacs 28 has a bug that prevents pdumper builds from working properly,
but the unexec build works just as well as Emacs 24.5 did.



^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-09  9:34                                       ` Po Lu
@ 2022-02-09  9:45                                         ` Po Lu
  2022-02-09  9:53                                         ` Robert Pluim
  1 sibling, 0 replies; 48+ messages in thread
From: Po Lu @ 2022-02-09  9:45 UTC (permalink / raw)
  To: Robert Pluim; +Cc: Eli Zaretskii, help-gnu-emacs

Po Lu <luangruo@yahoo.com> writes:

> Windows XP is hardly ancient, and at my day job we have a machine
> running Windows 98 used for legacy government software, and it's
> convenient to be able to run a recent version of Emacs there.

It might be useful to note while making this kind of decision that we
also support any X server that understands the X11R6 core protocol,
which was released slightly earlier than Windows 95.

It will probably work with earlier X servers as well with only a small
amount of prodding, but I don't have anything from that period to test
with.

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-09  9:34                                       ` Po Lu
  2022-02-09  9:45                                         ` Po Lu
@ 2022-02-09  9:53                                         ` Robert Pluim
  1 sibling, 0 replies; 48+ messages in thread
From: Robert Pluim @ 2022-02-09  9:53 UTC (permalink / raw)
  To: Po Lu; +Cc: help-gnu-emacs

>>>>> On Wed, 09 Feb 2022 17:34:36 +0800, Po Lu <luangruo@yahoo.com> said:

    Po> Robert Pluim <rpluim@gmail.com> writes:
    Eli> And finally, Gnulib dropped Windows 9X support long ago, so its module
    Eli> will not run on those old versions (and actually is unlikely to run
    Eli> even on Windows XP, as they've abandoned that as well).

    >> At some point we have to stop supporting ancient platforms.

    Po> Windows XP is hardly ancient, and at my day job we have a machine
    Po> running Windows 98 used for legacy government software, and it's
    Po> convenient to be able to run a recent version of Emacs there.

OK, so we canʼt wholesale port the gnulib module, but thereʼs nothing
stopping us from using winsock's 'select' (except that someone needs
to actually do the work)

Robert
-- 



^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-09  8:43                                     ` Robert Pluim
  2022-02-09  9:34                                       ` Po Lu
@ 2022-02-09 14:17                                       ` Eli Zaretskii
  2022-02-10 14:20                                         ` Robert Pluim
  1 sibling, 1 reply; 48+ messages in thread
From: Eli Zaretskii @ 2022-02-09 14:17 UTC (permalink / raw)
  To: help-gnu-emacs

> From: Robert Pluim <rpluim@gmail.com>
> Cc: help-gnu-emacs@gnu.org
> Date: Wed, 09 Feb 2022 09:43:25 +0100
> 
> >>>>> On Tue, 08 Feb 2022 20:35:38 +0200, Eli Zaretskii <eliz@gnu.org> said:
> 
>     >> From: Robert Pluim <rpluim@gmail.com>
>     >> Date: Tue, 08 Feb 2022 17:20:41 +0100
>     >> Cc: Stefan Monnier <monnier@iro.umontreal.ca>
>     >> 
>     >> Iʼd rather someone with the requisite skills and tolerance for Windows
>     >> changed the Emacs select emulation to remove the limit (perhaps by
>     >> using the Gnulib 'select' module).
> 
>     Eli> Why do you think the Gnulib module is free from the same (or similar)
>     Eli> limitation?  Isn't FD_SETSIZE = 64 in winsock2.h?
> 
> It is, but youʼre allowed to set it higher, as far as I know.

Not with the Gnulib emulation, AFAICT, since they call the same API we
call in Emacs, which is limited to waiting on 64 handles.

> Maybe it would be enough to just call winsock `select' from Emacs'
> select emulation, but only for sockets.

That's ugly, complicated (sockets are handles, file descriptors
aren't), and in some use cases will still hit the limit.

These half-measures aren't the kind of a solution I'd consider as good
candidates for replacing the current code.  We should probably use
WSAAsyncSelect for sockets, and/or use several threads, each one
watching 64 handles, to wait on 64*N handles.  _That_ would be worth
it.

>     Eli> Also, the pselect emulation in Emacs also emulates SIGCHLD from
>     Eli> sub-processes, something the Gnulib module does not AFAIR.
> 
> Thatʼs only needed for real sub-processes, not sockets, no?

Yes, but we still need it.

>     Eli> And finally, Gnulib dropped Windows 9X support long ago, so its module
>     Eli> will not run on those old versions (and actually is unlikely to run
>     Eli> even on Windows XP, as they've abandoned that as well).
> 
> At some point we have to stop supporting ancient platforms.

At some point, yes, and the motivation should be more serious than be
able to use internal Windows APIs freely.



^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-09 14:17                                       ` Eli Zaretskii
@ 2022-02-10 14:20                                         ` Robert Pluim
  0 siblings, 0 replies; 48+ messages in thread
From: Robert Pluim @ 2022-02-10 14:20 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: help-gnu-emacs

>>>>> On Wed, 09 Feb 2022 16:17:32 +0200, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Robert Pluim <rpluim@gmail.com>
    Eli> Why do you think the Gnulib module is free from the same (or similar)
    Eli> limitation?  Isn't FD_SETSIZE = 64 in winsock2.h?
    >> 
    >> It is, but youʼre allowed to set it higher, as far as I know.

    Eli> Not with the Gnulib emulation, AFAICT, since they call the same API we
    Eli> call in Emacs, which is limited to waiting on 64 handles.

Youʼre right, Iʼd missed that. Thatʼs fixable, but then we still run
into the issue with supporting old Windows versions.

    >> Maybe it would be enough to just call winsock `select' from Emacs'
    >> select emulation, but only for sockets.

    Eli> That's ugly, complicated (sockets are handles, file descriptors
    Eli> aren't), and in some use cases will still hit the limit.

    Eli> These half-measures aren't the kind of a solution I'd consider as good
    Eli> candidates for replacing the current code.  We should probably use
    Eli> WSAAsyncSelect for sockets, and/or use several threads, each one
    Eli> watching 64 handles, to wait on 64*N handles.  _That_ would be worth
    Eli> it.

Yes, that does sound better, although given the unexpected
side-effects of my experiments with WSAEventSelect, it might be tricky
to get right.

Robert
-- 



^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-07 19:48                           ` Tassilo Horn
                                               ` (3 preceding siblings ...)
  2022-02-07 20:10                             ` Eli Zaretskii
@ 2022-02-07 20:33                             ` Stefan Monnier via Users list for the GNU Emacs text editor
  2022-02-07 20:36                             ` emacsq
  5 siblings, 0 replies; 48+ messages in thread
From: Stefan Monnier via Users list for the GNU Emacs text editor @ 2022-02-07 20:33 UTC (permalink / raw)
  To: help-gnu-emacs

> I've tried it (with just 5 of your urls) on GNU/Linux and see the same.
> The network processes seem to stay in open status for quite some time
> even though I also killed the buffer in the callback function.  It's now

FWIW, the buffer associated to a process is largely irrelevant to the
process: it's only used when running the process filters and sentinels,
so as long as nothing happens on the process side, Emacs won't care of
the buffer is still live or not.


        Stefan




^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-07 19:48                           ` Tassilo Horn
                                               ` (4 preceding siblings ...)
  2022-02-07 20:33                             ` Stefan Monnier via Users list for the GNU Emacs text editor
@ 2022-02-07 20:36                             ` emacsq
  2022-02-08  6:01                               ` emacsq
  5 siblings, 1 reply; 48+ messages in thread
From: emacsq @ 2022-02-07 20:36 UTC (permalink / raw)
  To: Tassilo Horn; +Cc: help-gnu-emacs


>
> I've tried it (with just 5 of your urls) on GNU/Linux and see the same.
> The network processes seem to stay in open status for quite some time
> even though I also killed the buffer in the callback function. It's now
> about 5 minutes since my test and still 4 of initially 8 or 9 network
> processes are listed as open in `list-processes'.

Thanks for checking it and confirming it's not an Windows-issue.

> I have no idea if that's intended or not.
>

I checked url-http.el and it looks like existing open connections
are kept, in case someone wants to talk to the same host:port again
(is connecting so costly?):

https://github.com/emacs-mirror/emacs/blob/3af9e84ff59811734dcbb5d55e04e1fdb7051e77/lisp/url/url-http.el#L174



^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-07 20:36                             ` emacsq
@ 2022-02-08  6:01                               ` emacsq
  2022-02-08  7:10                                 ` emacsq
  0 siblings, 1 reply; 48+ messages in thread
From: emacsq @ 2022-02-08  6:01 UTC (permalink / raw)
  To: Tassilo Horn; +Cc: help-gnu-emacs

> I checked url-http.el and it looks like existing open connections
> are kept, in case someone wants to talk to the same host:port again
>

So going back to the original question: why do I get on windows
"Too many open files" error when increasing the number of parallel connections beyond 1? Sometimes I even get such an error with
parallel = 1, but rarely and only 1 or 2 connections fail. But
with parallel > 1 I get many such errors.

In the context of sockets "Too many open files" means too many
open sockets:

WSAEMFILE 10024
Too many open files.
Too many open sockets. Each implementation may have a maximum number of socket handles available, either globally, per process, or per thread.

url-http having open connections laying around could explain this, but
when I searched for winsock socket limits I found that there is not
really a low limit, you can open thousands of sockets by default,
so having some open connections shouldn't matter.

But the problem is present, so there must be some low limit in
emacs' windows code or elsewhere which prevent fetching urls in
parallel  without errors.

If someone reading this uses emacs on windows, could you run the
test code above, but with parallel connections set to 5, for
example? Do you also get "Too many open files" errors when fetching
the urls?

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-08  6:01                               ` emacsq
@ 2022-02-08  7:10                                 ` emacsq
  2022-02-08 10:37                                   ` Robert Pluim
  0 siblings, 1 reply; 48+ messages in thread
From: emacsq @ 2022-02-08  7:10 UTC (permalink / raw)
  To: Tassilo Horn; +Cc: help-gnu-emacs

> But the problem is present, so there must be some low limit in
> emacs' windows code or elsewhere which prevent fetching urls in
> parallel without errors.

I wrote a simple python script which fetched all the urls in
parallel, in a separate thread each without any waiting, so that's
dozens of urls at the same time and I did not get any "too many"
errors.

So this suggests this too many sockets error is related to
emacs somehow.

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-08  7:10                                 ` emacsq
@ 2022-02-08 10:37                                   ` Robert Pluim
       [not found]                                     ` <IVE50-HMjFHOAVjWY7EGgqGQqfr-ZFf5Tf gkjsYU8KSrVGPgKO4MNzEy=5F7AvZE2oqDAoCj8N6todiJhbx1XeBMIgs96uTgqJmi9jGp9QsOQ=3D@protonmail.com>
                                                       ` (2 more replies)
  0 siblings, 3 replies; 48+ messages in thread
From: Robert Pluim @ 2022-02-08 10:37 UTC (permalink / raw)
  To: emacsq; +Cc: help-gnu-emacs, Tassilo Horn

>>>>> On Tue, 08 Feb 2022 07:10:48 +0000, emacsq <laszlomail@protonmail.com> said:

    >> But the problem is present, so there must be some low limit in
    >> emacs' windows code or elsewhere which prevent fetching urls in
    >> parallel without errors.

    emacsq> I wrote a simple python script which fetched all the urls in
    emacsq> parallel, in a separate thread each without any waiting, so that's
    emacsq> dozens of urls at the same time and I did not get any "too many"
    emacsq> errors.

    emacsq> So this suggests this too many sockets error is related to
    emacsq> emacs somehow.

Didnʼt Eli already explain this? Emacs on windows has a limit of 32
simultaneous sub-processes, and a network connection counts as a
sub-process.

Robert
-- 



^ permalink raw reply	[flat|nested] 48+ messages in thread

[parent not found: <IVE50-HMjFHOAVjWY7EGgqGQqfr-ZFf5Tf gkjsYU8KSrVGPgKO4MNzEy=5F7AvZE2oqDAoCj8N6todiJhbx1XeBMIgs96uTgqJmi9jGp9QsOQ=3D@protonmail.com>]

[parent not found: <aR8DUrM=5FqNL0D1t-H38HOJZjxSN8FuTKFSAWf9JOBl2o1hIf02Br-jh7m6mGKbs0vq8kpBP=5FeYvQ=5FaHuaLzfMTC3308MIPh7JXJB=5Fm2?= =?us-ascii?Q?WlPs=3D@protonmail.com>]

* Re: What does the error "Process <URL> not running" mean?
  2022-02-08 10:37                                   ` Robert Pluim
       [not found]                                     ` <IVE50-HMjFHOAVjWY7EGgqGQqfr-ZFf5Tf gkjsYU8KSrVGPgKO4MNzEy=5F7AvZE2oqDAoCj8N6todiJhbx1XeBMIgs96uTgqJmi9jGp9QsOQ=3D@protonmail.com>
@ 2022-02-08 13:06                                     ` Eli Zaretskii
  2022-02-08 16:22                                     ` emacsq
  2 siblings, 0 replies; 48+ messages in thread
From: Eli Zaretskii @ 2022-02-08 13:06 UTC (permalink / raw)
  To: help-gnu-emacs

> From: Robert Pluim <rpluim@gmail.com>
> Gmane-Reply-To-List: yes
> Date: Tue, 08 Feb 2022 11:37:41 +0100
> Cc: help-gnu-emacs@gnu.org, Tassilo Horn <tsdh@gnu.org>
> 
> >>>>> On Tue, 08 Feb 2022 07:10:48 +0000, emacsq <laszlomail@protonmail.com> said:
> 
>     >> But the problem is present, so there must be some low limit in
>     >> emacs' windows code or elsewhere which prevent fetching urls in
>     >> parallel without errors.
> 
>     emacsq> I wrote a simple python script which fetched all the urls in
>     emacsq> parallel, in a separate thread each without any waiting, so that's
>     emacsq> dozens of urls at the same time and I did not get any "too many"
>     emacsq> errors.
> 
>     emacsq> So this suggests this too many sockets error is related to
>     emacsq> emacs somehow.
> 
> Didnʼt Eli already explain this? Emacs on windows has a limit of 32
> simultaneous sub-processes, and a network connection counts as a
> sub-process.

Indeed.  But given the amount of noise in this (and similar) threads,
I'm not surprised that people miss important information and keep
asking questions that were long answered.



^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-08 10:37                                   ` Robert Pluim
       [not found]                                     ` <IVE50-HMjFHOAVjWY7EGgqGQqfr-ZFf5Tf gkjsYU8KSrVGPgKO4MNzEy=5F7AvZE2oqDAoCj8N6todiJhbx1XeBMIgs96uTgqJmi9jGp9QsOQ=3D@protonmail.com>
  2022-02-08 13:06                                     ` Eli Zaretskii
@ 2022-02-08 16:22                                     ` emacsq
  2022-02-08 18:27                                       ` emacsq
  2 siblings, 1 reply; 48+ messages in thread
From: emacsq @ 2022-02-08 16:22 UTC (permalink / raw)
  To: Robert Pluim; +Cc: Tassilo Horn, help-gnu-emacs

> Didnʼt Eli already explain this? Emacs on windows has a limit of 32
> simultaneous sub-processes, and a network connection counts as a
> sub-process.

I got sidetracked due to other stuff, but if this is the case then
url-http should close network connections immediately (at least on
windows), instead of having a pool of them laying around and
causing this error.



^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-08 16:22                                     ` emacsq
@ 2022-02-08 18:27                                       ` emacsq
  2022-02-09 11:28                                         ` emacsq
  0 siblings, 1 reply; 48+ messages in thread
From: emacsq @ 2022-02-08 18:27 UTC (permalink / raw)
  To: Robert Pluim; +Cc: Tassilo Horn, help-gnu-emacs

> I think both are desirable: I think even under POSIX systems it makes
sense to bound the number of connections we keep open

But it's not just letting connections linger, url-http has a
mechanism to store open connections in a hash and reusing them:

https://github.com/emacs-mirror/emacs/blob/3af9e84ff59811734dcbb5d55e04e1fdb7051e77/lisp/url/url-http.el#L174

Does someone here know what the point of this was? Is quick
repeated http connections to the same host:port such a frequent use
case  that it's worth it to keep open http connections in a hash,
instead of closing and disposing them when the request ends?

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-08 18:27                                       ` emacsq
@ 2022-02-09 11:28                                         ` emacsq
  2022-02-09 12:55                                           ` emacsq
  0 siblings, 1 reply; 48+ messages in thread
From: emacsq @ 2022-02-09 11:28 UTC (permalink / raw)
  To: Robert Pluim; +Cc: Tassilo Horn, help-gnu-emacs

> But it's not just letting connections linger, url-http has a
> mechanism to store open connections in a hash and reusing them:
>
> Does someone here know what the point of this was?

It looks like it's not for later requests to use this stored
connection, but if during the same request a connection is
used in various functions then the connection object is not
always available there. So the author stored the connections
in a hash table, so he can retrieve them from anywhere, it's
a common shared storage for connections.

This is all right. The problem is when the url-retrieve ends then
the connection is left in the hastahble open which is OK for
platforms like Linux with plenty of sockets, because later
they are cleaned up  anyway.

But in case of windows with the 32 process limitation it's
clearly a bug, because the lingering pocesses use up the
available sockets, so while the limit is there, these
processes should be closed when the callback is called.

This is a quick and simple fix which solves the problem
until the limiting code is reenginered by someone  sometime
in the future, and even then this fix can stay, because there
is no point in leaving unsued open connections laying around.

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-09 11:28                                         ` emacsq
@ 2022-02-09 12:55                                           ` emacsq
  0 siblings, 0 replies; 48+ messages in thread
From: emacsq @ 2022-02-09 12:55 UTC (permalink / raw)
  To: Robert Pluim; +Cc: Tassilo Horn, help-gnu-emacs

> But in case of windows with the 32 process limitation it's
> clearly a bug, because the lingering pocesses use up the
> available sockets, so while the limit is there, these
> processes should be closed when the callback is called.

I tried adding process deletion before the callback:

(defun url-http-activate-callback ()

  (if (eq (window-system) 'w32)
      (delete-process url-http-process))

....

which seemed to work when fetching 1 url, it disappeared from
list-processes, but when I fetched a few dozen urls then
lots of processes stayed in the list with open status, despite
deleting the processes before the callback.

I used   (setq url-queue-parallel-processes 1) with url queue
for the test.

^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-03  5:02 ` emacsq
  2022-02-03  5:59   ` Tassilo Horn
@ 2022-02-03  7:34   ` Emanuel Berg via Users list for the GNU Emacs text editor
  2022-02-03 15:27   ` [External] : " Drew Adams
  2 siblings, 0 replies; 48+ messages in thread
From: Emanuel Berg via Users list for the GNU Emacs text editor @ 2022-02-03  7:34 UTC (permalink / raw)
  To: help-gnu-emacs

emacsq wrote:

> I tried finding this "process ... not running" text in the
> lisp code, but I couldn't find it.

src/process.c

-- 
underground experts united
https://dataswamp.org/~incal




^ permalink raw reply	[flat|nested] 48+ messages in thread

* RE: [External] : Re: What does the error "Process <URL> not running" mean?
  2022-02-03  5:02 ` emacsq
  2022-02-03  5:59   ` Tassilo Horn
  2022-02-03  7:34   ` Emanuel Berg via Users list for the GNU Emacs text editor
@ 2022-02-03 15:27   ` Drew Adams
  2022-02-03 16:54     ` emacsq
  2 siblings, 1 reply; 48+ messages in thread
From: Drew Adams @ 2022-02-03 15:27 UTC (permalink / raw)
  To: emacsq, help-gnu-emacs@gnu.org

> > > What does this error mean? I asked emacs to fetch an url,
> > > not start a process
> > Yeah but I think that's what it does ...
> 
> It may do that in the background, I meant from the user
> point of view I told emacs to fetch an url, so it telling
> me some process is not running does not help me as a user
> to know what the problem is.
> 
> > > so why does it tell me this and how can I know from this
> > > what the actual error is?
> I have a document with hundreds of urls in it which needs
> to be checked if they are still  alive and well, so I
> fetched them one by one with a code like this:
> 
> (condition-case errmsg (url-retrieve...

`C-h f url-retrieve' tells you that it invokes
processes.

Granted, you're getting an error msg reported
from a process, not a "higher-level" msg from
`url-retrieve' itself.  But at least that doc
tells you that you're running processes.

^ permalink raw reply	[flat|nested] 48+ messages in thread

* RE: [External] : Re: What does the error "Process <URL> not running" mean?
  2022-02-03 15:27   ` [External] : " Drew Adams
@ 2022-02-03 16:54     ` emacsq
  0 siblings, 0 replies; 48+ messages in thread
From: emacsq @ 2022-02-03 16:54 UTC (permalink / raw)
  To: Drew Adams; +Cc: help-gnu-emacs@gnu.org


> `C-h f url-retrieve' tells you that it invokes processes. Granted,
 you're getting an error msg reported from a process, not a "higher-level" msg from` url-retrieve' itself. But at least that doc tells you that you're running processes.

I understand that that is the underlying implementation, my point is, saying "connection lost unexpectedly" is more informative when emacs fails to fetch an url than saying "process x is not running".




^ permalink raw reply	[flat|nested] 48+ messages in thread

* Re: What does the error "Process <URL> not running" mean?
  2022-02-02 19:52 What does the error "Process <URL> not running" mean? emacsq
  2022-02-03  0:14 ` Emanuel Berg via Users list for the GNU Emacs text editor
  2022-02-03  5:02 ` emacsq
@ 2022-02-03  6:57 ` Eli Zaretskii
  2 siblings, 0 replies; 48+ messages in thread
From: Eli Zaretskii @ 2022-02-03  6:57 UTC (permalink / raw)
  To: help-gnu-emacs

> Date: Wed, 02 Feb 2022 19:52:01 +0000
> From: emacsq <laszlomail@protonmail.com>
> 
> When fetching several urls with url-retrieve, some of them returns with this error:
> 
>     Process <the url> not running
> 
> What does this error mean? I asked emacs to fetch an url, not start a process, so why does it tell me this and how can I know from this what
> the actual error is?

Network connections in Emacs are represented by a special kind of
process object.  See, for example, the doc string of the function
make-network-process.

^ permalink raw reply	[flat|nested] 48+ messages in thread

[parent not found: <BphWlZUu16KoEBDHOcSV5zR7MGU13qm4O-VpG0Vpd5AMTu4mSvvKAZaViOFHQiwbNBLfIMCNPCTYtheBJQBycFHtEPeacyUDSN-rlu=5FUWQw=3D@protonmail.com>]

[parent not found: <BphWlZUu16KoEBDHOcSV5zR7MGU13qm4O-VpG0Vpd5AMTu4mSvvKAZaViOFHQiwbNBLfIMCNPCTYtheBJQBycFHtEPeacyUDSN-rlu=3D5FUWQw=3D3D@protonmail.com>]

[parent not found: <6Ox0QxOSiVddeNsCaACeldkV9F-Nh9dM-rRERWveYqhG8t126cIm2MGmadX7Uy8YL-IQX9-Y=3D5F=3D5FZjAwEKArVB5v81UoZWZ7U4=3D5F1R70ywhZZY=3D3D@protonmail.com>]

[parent not found: <kQ5WkoUp-ir-MeV9eFbiOXYaopRMN0pcwlurXw1SD72UPJ-pcb1MUYYK3T5ucsurROJMoHHQJ0wR9=3D5FVRECOWyuXe95B3PJgZ-phCdgdUpHk=3D3D@protonmail.com>]

[parent not found: <8jy--UTf4wNbxysxHArjzE3ADfF5mB=3D5FZsfnFd7sgKpf=3D5FGM=3D5F9O5YqVK1PH1QbnoizHbb6HonK-BeQEQx0OpCmSRnMSpJzNTcHHXASGOoiD9I=3D3D@protonmail.com>]

end of thread, other threads:[~2022-02-10 14:20 UTC | newest]

Thread overview: 48+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-02-02 19:52 What does the error "Process <URL> not running" mean? emacsq
2022-02-03  0:14 ` Emanuel Berg via Users list for the GNU Emacs text editor
2022-02-03  5:02 ` emacsq
2022-02-03  5:59   ` Tassilo Horn
     [not found]     ` <5Opci81-9PcO6g1ljSQMCeDxh03cTnw4IEQWRzc2QInT4V52Ph2nB2WyyrBkSR11O8A9W4Coh1Wz=5FJ9v-JeqnFC3melWsBOK2-hU9f2tyWQ=3D@protonmail.com>
     [not found]       ` <Ckg2a8v34NSsrKnpfXhAXAOdIOvwsFqlv12BoMp9tr1AiSOzlmoJFCSu7I3hi5c2c-Kv3A3L175kXYQZHG8PVQfp7EpUWK8Vj21kFYx21Mo=3D@protonmail.com>
     [not found]         ` <TalBxoV5qsEqukLpxb9v9C3cjNPPN86=5FH3ruOHNa1PAofx8PLEGFBsC=5FVE8Jye18zcNRLar9mbeAoNguQshYRZ-kjdPWLK=5FTE0uggtyb2Lg=3D@protonmail.com>
2022-02-03  6:18     ` emacsq
2022-02-03  8:08       ` emacsq
2022-02-03  8:49         ` Emanuel Berg via Users list for the GNU Emacs text editor
2022-02-03  9:27         ` Eli Zaretskii
2022-02-03 10:10         ` emacsq
2022-02-03 10:35           ` Eli Zaretskii
2022-02-03 13:32             ` Robert Pluim
2022-02-03 15:05           ` emacsq
2022-02-03 20:07             ` Tassilo Horn
     [not found]               ` <3qySp5xSA2V0n9C8vwql9UbGKia8POa7OZcDnXg6e8jvW59uKuICMg8MMi5o-drq2sIcWWOejQJhal9aBXZaZM09a6oyenNylYnn5Qjp-H8=3D@protonmail.com>
     [not found]               ` <3qySp5xSA2V0n9C8vwql9UbGKia8POa7OZcDnXg6e8jvW59uKuICMg8MMi5o-drq2sIcWWOejQJhal9aBXZaZM09a6oyenNylYn n5Qjp-H8=3D@protonmail.com>
     [not found]                 ` <6Ox0QxOSiVddeNsCaACeldkV9F-Nh9dM-rRERWveYqhG8t126cIm2MGmadX7Uy8YL-IQX9-Y=5F=5FZjAwEKArVB5v81UoZWZ7U4=5F1R70ywhZZY=3D@protonmail.com>
     [not found]                   ` <kQ5WkoUp-ir-MeV9eFbiOXYaopRMN0pcwlurXw1SD72U?= =?us-ascii?Q?PJ-pcb1MUYYK3T5ucsurROJMoHHQJ0wR9=5FVRECOWyuXe95B3PJgZ-phCdgdUpHk=3D@protonmail.com>
     [not found]                     ` <8jy--UTf4wNbxysxHArjzE3ADfF5mB=5FZsfnFd7sgKpf=5FGM=5F9O5YqVK1PH1QbnoizHbb6HonK-BeQEQx0OpCmSRnMSpJzNTcHHXASGOoiD9I=3D@protonmail.com>
     [not found]                       ` <waNRXNl-vCnaC-u0F0hzugc4OjmLVw35PmjluKisu6G9euFh2IXTKrhR=5FVY1CJ0II2LP49ybLHsvIiu-d6TuwIa0vdN rLJ8ULCaBk=5FR38O8=3D@protonmail.com>
     [not found]                       ` <waNRXNl-vCnaC-u0F0hzugc4OjmLVw35PmjluKisu6G9euFh2IXTKrhR=5FVY1CJ0II2LP49ybLHsvIiu-d6TuwIa0vdNrLJ8ULCaBk=5FR38O8=3D@protonmail.com>
2022-02-03 20:34               ` emacsq
2022-02-03 20:47                 ` emacsq
2022-02-06 16:24                   ` emacsq
2022-02-06 16:54                     ` Eli Zaretskii
2022-02-06 18:25                     ` emacsq
2022-02-06 19:20                       ` emacsq
2022-02-07 17:35                         ` emacsq
2022-02-07 19:48                           ` Tassilo Horn
     [not found]                             ` <=5FcUGEWwwd0FeA1-yUQ5OJfKvG6Y2m4lFHC0m42EOzULv02BYrQMv6BYH-YMsVim1q3G8b3ZNnmPMGB0ZgQP7=5F2Ib5vLqAHln3bVDUOqT3eU=3D@protonmail.com>
     [not found]                             ` <=3D5FcUGEWwwd0FeA1-yUQ5OJfKvG6Y2m4lFHC0m42EOzULv02BYrQMv6BY H-YMsVim1q3G8b3ZNnmPMGB0ZgQP7=3D5F2Ib5vLqAHln3bVDUOqT3eU=3D3D@protonmail.com>
     [not found]                             ` <=3D5FcUGEWwwd0FeA1-yUQ5OJfKvG6Y2m4lFHC0m42EOzULv02BYrQMv6BYH-YMsVim1q3G8b3ZNnmPMGB0ZgQP7=3D5F2Ib5vLqAHln3bVDUOqT3eU=3D3D@protonmail.com>
2022-02-07 20:10                             ` Eli Zaretskii
2022-02-08  5:55                               ` Tassilo Horn
2022-02-08 12:33                                 ` Eli Zaretskii
2022-02-08 13:24                               ` Stefan Monnier via Users list for the GNU Emacs text editor
2022-02-08 16:20                                 ` Robert Pluim
2022-02-08 17:49                                   ` Stefan Monnier via Users list for the GNU Emacs text editor
2022-02-08 18:35                                   ` Eli Zaretskii
2022-02-09  8:43                                     ` Robert Pluim
2022-02-09  9:34                                       ` Po Lu
2022-02-09  9:45                                         ` Po Lu
2022-02-09  9:53                                         ` Robert Pluim
2022-02-09 14:17                                       ` Eli Zaretskii
2022-02-10 14:20                                         ` Robert Pluim
2022-02-07 20:33                             ` Stefan Monnier via Users list for the GNU Emacs text editor
2022-02-07 20:36                             ` emacsq
2022-02-08  6:01                               ` emacsq
2022-02-08  7:10                                 ` emacsq
2022-02-08 10:37                                   ` Robert Pluim
     [not found]                                     ` <IVE50-HMjFHOAVjWY7EGgqGQqfr-ZFf5Tf gkjsYU8KSrVGPgKO4MNzEy=5F7AvZE2oqDAoCj8N6todiJhbx1XeBMIgs96uTgqJmi9jGp9QsOQ=3D@protonmail.com>
     [not found]                                       ` <aR8DUrM=5FqNL0D1t-H38HOJZjxSN8FuTKFSAWf9JOBl2o1hIf02Br-jh7m6mGKbs0vq8kpBP=5FeYvQ=5FaHuaLzfMTC3308MIPh7JXJB=5Fm2?= =?us-ascii?Q?WlPs=3D@protonmail.com>
2022-02-08 13:06                                     ` Eli Zaretskii
2022-02-08 16:22                                     ` emacsq
2022-02-08 18:27                                       ` emacsq
2022-02-09 11:28                                         ` emacsq
2022-02-09 12:55                                           ` emacsq
2022-02-03  7:34   ` Emanuel Berg via Users list for the GNU Emacs text editor
2022-02-03 15:27   ` [External] : " Drew Adams
2022-02-03 16:54     ` emacsq
2022-02-03  6:57 ` Eli Zaretskii
     [not found] <BphWlZUu16KoEBDHOcSV5zR7MGU13qm4O-VpG0Vpd5AMTu4mSvvKAZaViOFHQiwbNBLfIMCNPCTYtheBJQBycFHtEPeacyUDSN-rlu=5FUWQw=3D@protonmail.com>
     [not found] <BphWlZUu16KoEBDHOcSV5zR7MGU13qm4O-VpG0Vpd5AMTu4mSvvKAZaViOFHQiwbNBLfIMCNPCTYtheBJQBycFHtEPeacyUDSN-rlu=3D5FUWQw=3D3D@protonmail.com>
     [not found] ` <6Ox0QxOSiVddeNsCaACeldkV9F-Nh9dM-rRERWveYqhG8t126cIm2MGmadX7Uy8YL-IQX9-Y=3D5F=3D5FZjAwEKArVB5v81UoZWZ7U4=3D5F1R70ywhZZY=3D3D@protonmail.com>
     [not found]   ` <kQ5WkoUp-ir-MeV9eFbiOXYaopRMN0pcwlurXw1SD72UPJ-pcb1MUYYK3T5ucsurROJMoHHQJ0wR9=3D5FVRECOWyuXe95B3PJgZ-phCdgdUpHk=3D3D@protonmail.com>
     [not found]     ` <8jy--UTf4wNbxysxHArjzE3ADfF5mB=3D5FZsfnFd7sgKpf=3D5FGM=3D5F9O5YqVK1PH1QbnoizHbb6HonK-BeQEQx0OpCmSRnMSpJzNTcHHXASGOoiD9I=3D3D@protonmail.com>

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.