From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.lisp.guile.devel Subject: Re: [PATCH] Implement open-process and related functions on MinGW Date: Sun, 23 Feb 2014 19:54:24 +0200 Message-ID: <83r46t1zqn.fsf@gnu.org> References: <834n3x8o7m.fsf@gnu.org> <83y519788a.fsf@gnu.org> <871tz0d5vc.fsf@gnu.org> <83iosc76kz.fsf@gnu.org> <87vbwc72dp.fsf_-_@gnu.org> <8361o74e0k.fsf@gnu.org> <87zjljjg7r.fsf@yeeloong.lan> <83lhx32jhu.fsf@gnu.org> <87a9dijs2v.fsf@yeeloong.lan> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: ger.gmane.org 1393178084 7846 80.91.229.3 (23 Feb 2014 17:54:44 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 23 Feb 2014 17:54:44 +0000 (UTC) Cc: ludo@gnu.org, guile-devel@gnu.org To: Mark H Weaver Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Sun Feb 23 18:54:52 2014 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1WHdGN-0006pB-NM for guile-devel@m.gmane.org; Sun, 23 Feb 2014 18:54:51 +0100 Original-Received: from localhost ([::1]:53774 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WHdGN-00012i-DS for guile-devel@m.gmane.org; Sun, 23 Feb 2014 12:54:51 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:58814) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WHdGG-0000t1-5C for guile-devel@gnu.org; Sun, 23 Feb 2014 12:54:49 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WHdGA-0006XP-ND for guile-devel@gnu.org; Sun, 23 Feb 2014 12:54:44 -0500 Original-Received: from mtaout24.012.net.il ([80.179.55.180]:37924) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WHdGA-0006XG-5M; Sun, 23 Feb 2014 12:54:38 -0500 Original-Received: from conversion-daemon.mtaout24.012.net.il by mtaout24.012.net.il (HyperSendmail v2007.08) id <0N1G00I00MWW1F00@mtaout24.012.net.il>; Sun, 23 Feb 2014 19:53:13 +0200 (IST) Original-Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout24.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0N1G00BNZN0PO880@mtaout24.012.net.il>; Sun, 23 Feb 2014 19:53:13 +0200 (IST) In-reply-to: <87a9dijs2v.fsf@yeeloong.lan> X-012-Sender: halo1@inter.net.il X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 80.179.55.180 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:16926 Archived-At: > From: Mark H Weaver > Cc: ludo@gnu.org, guile-devel@gnu.org > Date: Sun, 23 Feb 2014 00:50:16 -0500 > > Eli Zaretskii writes: > > >> Definitions for these on MinGW are available in Gnulib's 'sys_wait' > >> module. I'll import it soon. > > > > Please don't: that gnulib module is dead wrong for Windows. If we use > > it, we will be unable to report even the simplest error exits, let > > alone programs that crashed due to a fatal signal. > > Hmm. This is rather uncomfortable. Did you discuss these problems with > the Gnulib developers? If so, can you provide a link to the relevant > threads? I'm sorry, but I seem to be unable to efficiently discuss with gnulib developers anything that is related to MinGW and Windows in general. Any such discussion at best drags on for months, and at worst simply dies because there's no response. See a few examples below. And no, I didn't discuss this specific issue with gnulib developers. > Your comment above "We usurp codes above 0xC0000200 for SIGxxx signals" > makes me wonder: How widely used is this convention? What other > software packages use it? I don't think this matters. This convention is used between Guile and itself: in scm_kill we tell the process being killed to terminate with status of 0xC0000200 + SIGNAL, and then we extract the signal number in status:term-sig. So this is just something private to Guile, that's all; its only purpose is to report the signal with which we killed the process. Other applications don't need to know about this. > My reasons for this preference are (1) to keep the Guile code cleaner, > and (2) if some deficiency is found in the wrapper, or if Windows > introduces an improved API some day, the changes have to be made in only > one place: Gnulib. As long as this is practical, I won't argue with you. However, my experience with gnulib is significantly different (see below), and therefore my personal preferences are also different. > > But in general, due to the known problems with getting > > Windows-specific patches to gnulib approved and admitted, I'd advise > > to stay away of gnulib as much as possible, when Windows is concerned. > > Can you provide some links to mailing list threads where you've tried to > get things fixed in Gnulib and encountered resistance? Certainly. My first example is with canonicalize_filename, which started here: http://lists.gnu.org/archive/html/bug-gnulib/2012-01/msg00253.html and then promptly died. My pings here http://lists.gnu.org/archive/html/bug-gnulib/2012-06/msg00222.html http://lists.gnu.org/archive/html/bug-gnulib/2012-07/msg00004.html http://lists.gnu.org/archive/html/bug-gnulib/2012-07/msg00239.html were left without any response whatsoever. Finally, you intervened here: http://lists.gnu.org/archive/html/bug-gnulib/2012-11/msg00021.html and, lo and behold, 3 weeks later the changes were finally committed, more than 10 months after the initial submittal. This is a successful story, mind you. Here's a couple of unsuccessful ones: http://lists.gnu.org/archive/html/bug-gnulib/2013-06/msg00042.html http://lists.gnu.org/archive/html/bug-gnulib/2014-01/msg00072.html Who knows, perhaps a year from now these will also become "successes"? One can hope, can't one? > > E.g., what if tomorrow we would need to make the waitpid emulation > > more sophisticated? > > I suppose I would prefer to get the problem fixed in Gnulib. If there > is truly a problem with the way Gnulib is being maintained w.r.t. to > Windows, perhaps I will make an effort to get that larger problem fixed. See above. > For example, if it is true that we can avoid all the nasty problems with > filename encoding by using the native Windows APIs that use UTF-16 for > filenames, then I'd be in favor of doing that. What nasty problems do you have in mind? I implemented something like this in Emacs not long ago, so perhaps I can help here. > > In general, where the Windows equivalent is to do nothing, my advice > > would be to have a no-op implementation, rather than an unbound > > function. The former approach makes it much easier to write portable > > Guile scripts, instead of spreading boundness checks all over the > > place. > > I think this is a very bad idea. First of all, it's not very common for > Guile programs to query or set the uid/gid, so I don't see much benefit. Unless you remove them from Guile, I don't think this argument stands: as long as the functions exist, some Guile program will certainly use them. For example, those servers where security is important, as you mention. > On the other hand, if a program _does_ try to do one of those things, it > might be important that the job be done right. For example, a program > might be trying to reduce its privileges. Doing nothing while silently > pretending that the operation succeeded is a good way to cause security > flaws to go unnoticed, or to make a programmer waste hours of time > trying to debug a problem that he should have been notified about > immediately with an error message. We are talking about operations that have no equivalent meaning on Windows. they only have meaning in Posix worldview. Why does it make sense to fail an operation that is meaningless? What useful purpose could this possibly serve, except having a subtly broken Guile module? > > FWIW, Emacs takes the former approach with very good results. > > Emacs is primarily an interactive editor Oh, Emacs is much more than that, believe me. > Guile is a general-purpose programming language, and we must > consider the needs of a larger set of users, for example the authors > of server software where security is important. Server software cannot be written portably anyway. I can believe you that Guile has enough infrastructure to support writing a server on Posix platforms, but it certainly does _not_ have the infrastructure for doing that on Windows; using setuid etc., or setting privileges, will be the least of your worries, if you try doing that. So this is a red herring, I think. By contrast, calls to setuid are quite common in modern programs that needs to run on Posix platforms. > >> Can you use the GNU coding conventions for placement of brackets? > > > > Sorry, I don't follow: which part is not according to the GNU coding > > conventions? > > Nevermind, my mistake. Your use of tabs in the indentation combined > with the prefix characters in the email caused some (but not all) lines > to move to the right. I've long been using (setq indent-tabs-mode nil) > globally, for that reason. Using tabs like I did is the Emacs default. Is there a requirement in Guile to use only spaces? If so, I probably missed it. > >> Thanks for working on this, but in a multithreaded program, it's no good > >> to change the file descriptors in the main program temporarily before > >> spawning, and then restore them afterwards. We'll have to find another > >> way of doing this. > > > > Threads don't work in the MinGW build anyway, and no one was able to > > help me understand why (see the discussions last August). As long as > > this limitation exists, this is a non-issue, and certainly better than > > not having this functionality available at all. > > As I wrote elsewhere, madsy on #guile reported having successfully built > a recent Guile snapshot with both threads and posix enabled. > > I know that you asked for more specifics about this, but I don't know > the answers. I asked madsy to reply to you directly when he has some > free time. However, I remember that he cross-built from Ubuntu Cross-building on GNU/Linux is probably the cause of success: the problematic steps run on a Posix platform using Posix APIs, I presume? Otherwise, the issues are subtle and only show when running certain Guile programs that use threading infrastructure, as discussed and described back in August. > and iirc he built libgc (and several other dependencies) from source > code. I've built libgc myself as well (and it passed all the tests), so this cannot be the reason. I also built libgmp. > > The only other way I know of is to bypass Posix-like functions like > > 'open', 'close', and 'dup', and go to the level of Win32 API, where a > > function that creates a child subprocess can accept handles for the > > child's standard I/O. > > This sounds to me like the right approach in this case. OK. > > If you will be comfortable with a lot more Widows specific stuff, I > > can work on that. > > Please do! OK, but please don't block this patch while you wait for me to do that. It is certainly much better to have this functionality with restrictions than not at all. At the very least, we could condition redirection of standard handles on !SCM_USE_PTHREAD_THREADS, and otherwise support only child processes that don't require redirection. This is already much better than what we have today, even if Guile with threads is indeed workable on Windows.