From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.bugs Subject: bug#11519: "Wrong type argument: characterp" building custom-deps while boostrapping Date: Wed, 23 May 2012 16:07:05 -0400 Message-ID: References: <83d360yw48.fsf@gnu.org> <834nrazrtl.fsf@gnu.org> <831umez1p7.fsf@gnu.org> <83vcjpxw18.fsf@gnu.org> <83k404xcpt.fsf@gnu.org> <83hav8xak1.fsf@gnu.org> <83ehqby542.fsf@gnu.org> <838vgiyh4q.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1337803706 22040 80.91.229.3 (23 May 2012 20:08:26 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Wed, 23 May 2012 20:08:26 +0000 (UTC) Cc: schwab@linux-m68k.org, rms@gnu.org, 11519@debbugs.gnu.org, lekktu@gmail.com To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed May 23 22:08:24 2012 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1SXHr3-0000SH-1A for geb-bug-gnu-emacs@m.gmane.org; Wed, 23 May 2012 22:08:21 +0200 Original-Received: from localhost ([::1]:48324 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SXHr2-0006JX-HK for geb-bug-gnu-emacs@m.gmane.org; Wed, 23 May 2012 16:08:20 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:41326) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SXHqz-0006JM-LO for bug-gnu-emacs@gnu.org; Wed, 23 May 2012 16:08:19 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1SXHqu-0000hE-KF for bug-gnu-emacs@gnu.org; Wed, 23 May 2012 16:08:17 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:59712) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SXHqu-0000gl-EV for bug-gnu-emacs@gnu.org; Wed, 23 May 2012 16:08:12 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.72) (envelope-from ) id 1SXHrh-0004y9-Ma for bug-gnu-emacs@gnu.org; Wed, 23 May 2012 16:09:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Stefan Monnier Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 23 May 2012 20:09:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 11519 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 11519-submit@debbugs.gnu.org id=B11519.133780370519057 (code B ref 11519); Wed, 23 May 2012 20:09:01 +0000 Original-Received: (at 11519) by debbugs.gnu.org; 23 May 2012 20:08:25 +0000 Original-Received: from localhost ([127.0.0.1]:41025 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SXHr6-0004xJ-FC for submit@debbugs.gnu.org; Wed, 23 May 2012 16:08:24 -0400 Original-Received: from ironport-out.teksavvy.com ([206.248.143.162]:54290) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SXHqm-0004wV-8F for 11519@debbugs.gnu.org; Wed, 23 May 2012 16:08:23 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av0EAG6Zu09MCpYd/2dsb2JhbABEtBGBCIIVAQEEAVYjBQsLNBIUGA0QAROIHAWrSo4/kEQDozOBWIMF X-IronPort-AV: E=Sophos;i="4.75,637,1330923600"; d="scan'208";a="182045952" Original-Received: from 76-10-150-29.dsl.teksavvy.com (HELO pastel.home) ([76.10.150.29]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 23 May 2012 16:07:07 -0400 Original-Received: by pastel.home (Postfix, from userid 20848) id 581D858C01; Wed, 23 May 2012 16:07:06 -0400 (EDT) In-Reply-To: <838vgiyh4q.fsf@gnu.org> (Eli Zaretskii's message of "Wed, 23 May 2012 19:52:21 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.1.50 (gnu/linux) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:60316 Archived-At: >> > Which other places use C pointers to buffer text and call functions >> > that can allocate memory? >> IIUC any place that uses STRING_CHAR_AND_LENGTH on buffer text is >> vulnerable to the problem. > That's not true. As long as you access buffer text through character > position, you are safe. Right, some of those uses might be safe, indeed. Of course it's not only STRING_CHAR_AND_LENGTH but STRING_CHAR_ADVANCE as well, together with FETCH_* macros which use those, etc... Grepping for those macros shows they're used at *many* places, and I'd be amazed if re_search is the only place where we don't go through the BYTE_POS_ADDR rigmarole. Let's see ...hmmm... yup, set-buffer-multibyte is another example, find_charsets_in_text yet another, and I'm not even trying hard. Just grep for "STRING_CHAR_" and see for yourself. >> If you really want to install your workaround on the emacs-24 branch, go >> for it but let's try to find a real fix for the trunk. > What kind of real fix are you looking for? One that lets us write code without having to worry about such corner cases. E.g. changing STRING_CHAR_ADVANCE so it can't cause relocation. Not using ralloc.c any more would be another good option. Otherwise, changing our macros so they do the BYTE_POS_ADDR internally, discouraging the use of direct pointers into the buffer's content. > Why shouldn't it be the fix in this case, and what better fix can we > invent when we use an essentially externally maintained code (AFAIR, > regex will at some point be re-sync'ed with gnulib) that cannot be > expected to change its code radically so as not to access buffer text > through C pointers? To me, it's clearly a workaround. It's an OK workaround if/when we use a "blackbox" (i.e. externally maintained) regexp engine and keep using ralloc.c. But better would be to eliminate the problem altogether. >> But on other platforms where we use mmap, we do suffer from this >> fragmentation, and yet it doesn't seem to be a real source of problem. > I don't know enough about mmap to answer that. I vaguely recollect > that mmap avoids such fragmentation as an inherent feature, but I may > be wrong. No, fragmentation is a property of the address space, so without relocation you can't avoid it. >> I guess my question turns into "why do we use gmalloc.c instead of >> a malloc library that uses mmap (or some other mechanism that lets it >> return large free chunks to the OS)"? > Use of gmalloc is a different issue. We were talking about ralloc.c. > You could use one, but not the other. Well, still we use ralloc because we don't use mmap, so the question to me is: why don't we use mmap (either via a malloc that does, or directly via USE_MMAP_FOR_BUFFERS) and get rid of ralloc.c? >> AFAIK, Windows is pretty much the only system where we use gmalloc.c and >> ralloc.c nowadays. > My reading of configure is that we use it on more than just Windows > (and MS-DOS). Basically, any platform that uses gmalloc.c (which is > the default, turned off only for GNU/Linux and Darwin) also uses > ralloc.c. To me "all minus GNU/Linux, Mac OS X, and Cygwin (which apparently uses gmalloc but not ralloc)" is pretty close to "just Windows" nowadays. >> Does anyone remember why we don't use the system malloc under >> Windows (and Cygwin)? > I find it hard to believe that going through system malloc on > MS-Windows will let us use buffers as large as 1.5 GB (on a 32-bit > machine). To achieve this today, we reserve a 2GB contiguous chunk of > address space at startup, and then commit and uncommit parts of it as > needed (see w32heap.c). ralloc.c has an important part in this > arrangement. You mean that Windows's system malloc library has a memory that's too fragmented to be able to allocate a single 1.5G chunk? Why? [ I know next to nothing about the w32 API and plead guilty of POSIX-only thinking, so please bear with me. ] Stefan