unofficial mirror of bug-gnu-emacs@gnu.org 
 help / color / mirror / code / Atom feed
* bug#51110: Native-comp leaks fds open on /dev/ptmx
@ 2021-10-09 15:45 Stephen Gildea
  2021-10-09 15:57 ` Eli Zaretskii
  0 siblings, 1 reply; 9+ messages in thread
From: Stephen Gildea @ 2021-10-09 15:45 UTC (permalink / raw)
  To: 51110

Version: 28.0.60
Severity: important

Native-comp leaks fds open on /dev/ptmx if you change 'invocation-name'
and then run a function that loads libraries and causes some compilation.
Emacs seems to compile each file multiple times (16 or more), and it soon
runs out of file descriptors.

To reproduce:

Use a newly built (configure --with-native-compilation) Emacs that has
never (successfully) run MH-E before; if your eln-cache has already been
populated, you will need to delete the cache to repeat the failure:

rm -r ~/.emacs.d/eln-cache

The Emacs must be interactive, and you can reproduce quickly as follows:

src/emacs -Q --eval '(setq invocation-name "emacs-leaks")' --eval '(mh-version)'

In a few seconds the echo area will say: "Creating pipe: Too many open files"

What should appear in the echo area: "nmh 1.7.1 installed as MH variant"
or maybe: "No MH variant found on the system"





^ permalink raw reply	[flat|nested] 9+ messages in thread

* bug#51110: Native-comp leaks fds open on /dev/ptmx
  2021-10-09 15:45 bug#51110: Native-comp leaks fds open on /dev/ptmx Stephen Gildea
@ 2021-10-09 15:57 ` Eli Zaretskii
  2021-10-09 17:01   ` Stephen Gildea
  0 siblings, 1 reply; 9+ messages in thread
From: Eli Zaretskii @ 2021-10-09 15:57 UTC (permalink / raw)
  To: Stephen Gildea, Andrea Corallo; +Cc: 51110

> From: Stephen Gildea <stepheng+emacs@gildea.com>
> Date: Sat, 09 Oct 2021 08:45:25 -0700
> 
> src/emacs -Q --eval '(setq invocation-name "emacs-leaks")' --eval '(mh-version)'

What real-life use case requires you to modify invocation-name?  That
variable is supposed to be read-only: it tells the program name used
to invoke Emacs of the running session.  If you change it, other
features will become confused.

IOW, I don't think we want to support this kind of usage.





^ permalink raw reply	[flat|nested] 9+ messages in thread

* bug#51110: Native-comp leaks fds open on /dev/ptmx
  2021-10-09 15:57 ` Eli Zaretskii
@ 2021-10-09 17:01   ` Stephen Gildea
  2021-10-09 17:15     ` Eli Zaretskii
  0 siblings, 1 reply; 9+ messages in thread
From: Stephen Gildea @ 2021-10-09 17:01 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: Andrea Corallo, 51110

The recipe I gave provides a simple way to show the unbounded resource
use of Emacs while compiling; I don't propose it as an actual use case.

Apparently what's happening is that Emacs forks off a child Emacs to do
the compile and allocates a pty to the child.  These ptys are not
closed until the compile completes.  It seems Emacs needs to better
manage the number of simultaneous open ptys.  Or at least be able to
wait and retry if an open fails.






^ permalink raw reply	[flat|nested] 9+ messages in thread

* bug#51110: Native-comp leaks fds open on /dev/ptmx
  2021-10-09 17:01   ` Stephen Gildea
@ 2021-10-09 17:15     ` Eli Zaretskii
  2021-11-30 20:13       ` Andrea Corallo
  0 siblings, 1 reply; 9+ messages in thread
From: Eli Zaretskii @ 2021-10-09 17:15 UTC (permalink / raw)
  To: Stephen Gildea; +Cc: akrl, 51110

> From: Stephen Gildea <stepheng+emacs@gildea.com>
> cc: Andrea Corallo <akrl@sdf.org>, 51110@debbugs.gnu.org
> Date: Sat, 09 Oct 2021 10:01:43 -0700
> 
> Apparently what's happening is that Emacs forks off a child Emacs to do
> the compile and allocates a pty to the child.  These ptys are not
> closed until the compile completes.  It seems Emacs needs to better
> manage the number of simultaneous open ptys.  Or at least be able to
> wait and retry if an open fails.

Emacs limits the number of these background processes, see
native-comp-async-jobs-number.  (The value zero means use half of the
number of execution units available in the system's processor.)  So we
generally shouldn't have more than that number of open pipes at any
given time.  For the obvious reasons they cannot be closed until the
sub-process exits.





^ permalink raw reply	[flat|nested] 9+ messages in thread

* bug#51110: Native-comp leaks fds open on /dev/ptmx
  2021-10-09 17:15     ` Eli Zaretskii
@ 2021-11-30 20:13       ` Andrea Corallo
  2021-11-30 20:18         ` Eli Zaretskii
  0 siblings, 1 reply; 9+ messages in thread
From: Andrea Corallo @ 2021-11-30 20:13 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: Stephen Gildea, 51110

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stephen Gildea <stepheng+emacs@gildea.com>
>> cc: Andrea Corallo <akrl@sdf.org>, 51110@debbugs.gnu.org
>> Date: Sat, 09 Oct 2021 10:01:43 -0700
>> 
>> Apparently what's happening is that Emacs forks off a child Emacs to do
>> the compile and allocates a pty to the child.  These ptys are not
>> closed until the compile completes.  It seems Emacs needs to better
>> manage the number of simultaneous open ptys.  Or at least be able to
>> wait and retry if an open fails.
>
> Emacs limits the number of these background processes, see
> native-comp-async-jobs-number.  (The value zero means use half of the
> number of execution units available in the system's processor.)  So we
> generally shouldn't have more than that number of open pipes at any
> given time.  For the obvious reasons they cannot be closed until the
> sub-process exits.

Is there any action we need/can take on this bug?

Thanks

  Andrea





^ permalink raw reply	[flat|nested] 9+ messages in thread

* bug#51110: Native-comp leaks fds open on /dev/ptmx
  2021-11-30 20:13       ` Andrea Corallo
@ 2021-11-30 20:18         ` Eli Zaretskii
  2021-11-30 20:24           ` Andrea Corallo
  0 siblings, 1 reply; 9+ messages in thread
From: Eli Zaretskii @ 2021-11-30 20:18 UTC (permalink / raw)
  To: Andrea Corallo; +Cc: stepheng+emacs, 51110

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Stephen Gildea <stepheng+emacs@gildea.com>, 51110@debbugs.gnu.org
> Date: Tue, 30 Nov 2021 20:13:07 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Stephen Gildea <stepheng+emacs@gildea.com>
> >> cc: Andrea Corallo <akrl@sdf.org>, 51110@debbugs.gnu.org
> >> Date: Sat, 09 Oct 2021 10:01:43 -0700
> >> 
> >> Apparently what's happening is that Emacs forks off a child Emacs to do
> >> the compile and allocates a pty to the child.  These ptys are not
> >> closed until the compile completes.  It seems Emacs needs to better
> >> manage the number of simultaneous open ptys.  Or at least be able to
> >> wait and retry if an open fails.
> >
> > Emacs limits the number of these background processes, see
> > native-comp-async-jobs-number.  (The value zero means use half of the
> > number of execution units available in the system's processor.)  So we
> > generally shouldn't have more than that number of open pipes at any
> > given time.  For the obvious reasons they cannot be closed until the
> > sub-process exits.
> 
> Is there any action we need/can take on this bug?

I don't think there's anything we _can_ do here, and I tried to
explain why.





^ permalink raw reply	[flat|nested] 9+ messages in thread

* bug#51110: Native-comp leaks fds open on /dev/ptmx
  2021-11-30 20:18         ` Eli Zaretskii
@ 2021-11-30 20:24           ` Andrea Corallo
  2021-11-30 20:29             ` Eli Zaretskii
  0 siblings, 1 reply; 9+ messages in thread
From: Andrea Corallo @ 2021-11-30 20:24 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: stepheng+emacs, 51110

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: Stephen Gildea <stepheng+emacs@gildea.com>, 51110@debbugs.gnu.org
>> Date: Tue, 30 Nov 2021 20:13:07 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> From: Stephen Gildea <stepheng+emacs@gildea.com>
>> >> cc: Andrea Corallo <akrl@sdf.org>, 51110@debbugs.gnu.org
>> >> Date: Sat, 09 Oct 2021 10:01:43 -0700
>> >> 
>> >> Apparently what's happening is that Emacs forks off a child Emacs to do
>> >> the compile and allocates a pty to the child.  These ptys are not
>> >> closed until the compile completes.  It seems Emacs needs to better
>> >> manage the number of simultaneous open ptys.  Or at least be able to
>> >> wait and retry if an open fails.
>> >
>> > Emacs limits the number of these background processes, see
>> > native-comp-async-jobs-number.  (The value zero means use half of the
>> > number of execution units available in the system's processor.)  So we
>> > generally shouldn't have more than that number of open pipes at any
>> > given time.  For the obvious reasons they cannot be closed until the
>> > sub-process exits.
>> 
>> Is there any action we need/can take on this bug?
>
> I don't think there's anything we _can_ do here, and I tried to
> explain why.

I agree, I don't think we really want support `invocation-name'
modifications at runtime.

Shall we close this bug?

  Andrea





^ permalink raw reply	[flat|nested] 9+ messages in thread

* bug#51110: Native-comp leaks fds open on /dev/ptmx
  2021-11-30 20:24           ` Andrea Corallo
@ 2021-11-30 20:29             ` Eli Zaretskii
  2021-12-01  2:28               ` Stephen Gildea
  0 siblings, 1 reply; 9+ messages in thread
From: Eli Zaretskii @ 2021-11-30 20:29 UTC (permalink / raw)
  To: Andrea Corallo; +Cc: stepheng+emacs, 51110

> From: Andrea Corallo <akrl@sdf.org>
> Cc: stepheng+emacs@gildea.com, 51110@debbugs.gnu.org
> Date: Tue, 30 Nov 2021 20:24:59 +0000
> 
> > I don't think there's anything we _can_ do here, and I tried to
> > explain why.
> 
> I agree, I don't think we really want support `invocation-name'
> modifications at runtime.
> 
> Shall we close this bug?

I think so.  But before we do, I'd like to let Stephen a chance to
convince us otherwise.





^ permalink raw reply	[flat|nested] 9+ messages in thread

* bug#51110: Native-comp leaks fds open on /dev/ptmx
  2021-11-30 20:29             ` Eli Zaretskii
@ 2021-12-01  2:28               ` Stephen Gildea
  0 siblings, 0 replies; 9+ messages in thread
From: Stephen Gildea @ 2021-12-01  2:28 UTC (permalink / raw)
  To: Eli Zaretskii, Andrea Corallo; +Cc: 51110-done

Eli Zaretskii <eliz@gnu.org> wrote:

>   > From: Andrea Corallo <akrl@sdf.org>
>   > Cc: stepheng+emacs@gildea.com, 51110@debbugs.gnu.org
>   > Date: Tue, 30 Nov 2021 20:24:59 +0000
>   > 
>   > > I don't think there's anything we _can_ do here, and I tried to
>   > > explain why.
>   > 
>   > I agree, I don't think we really want support `invocation-name'
>   > modifications at runtime.
>   > 
>   > Shall we close this bug?
>   
>   I think so.  But before we do, I'd like to let Stephen a chance to
>   convince us otherwise.

I'm happy to see this bug closed, and I hereby do so.
Thank you for the discussion.





^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2021-12-01  2:28 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-10-09 15:45 bug#51110: Native-comp leaks fds open on /dev/ptmx Stephen Gildea
2021-10-09 15:57 ` Eli Zaretskii
2021-10-09 17:01   ` Stephen Gildea
2021-10-09 17:15     ` Eli Zaretskii
2021-11-30 20:13       ` Andrea Corallo
2021-11-30 20:18         ` Eli Zaretskii
2021-11-30 20:24           ` Andrea Corallo
2021-11-30 20:29             ` Eli Zaretskii
2021-12-01  2:28               ` Stephen Gildea

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).