From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#11519: "Wrong type argument: characterp" building custom-deps while boostrapping Date: Wed, 23 May 2012 19:52:21 +0300 Message-ID: <838vgiyh4q.fsf@gnu.org> References: <83d360yw48.fsf@gnu.org> <834nrazrtl.fsf@gnu.org> <831umez1p7.fsf@gnu.org> <83vcjpxw18.fsf@gnu.org> <83k404xcpt.fsf@gnu.org> <83hav8xak1.fsf@gnu.org> <83ehqby542.fsf@gnu.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: dough.gmane.org 1337792004 21630 80.91.229.3 (23 May 2012 16:53:24 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Wed, 23 May 2012 16:53:24 +0000 (UTC) Cc: schwab@linux-m68k.org, rms@gnu.org, 11519@debbugs.gnu.org, lekktu@gmail.com To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed May 23 18:53:22 2012 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1SXEoK-0001vW-BN for geb-bug-gnu-emacs@m.gmane.org; Wed, 23 May 2012 18:53:20 +0200 Original-Received: from localhost ([::1]:32774 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SXEoK-0002xx-13 for geb-bug-gnu-emacs@m.gmane.org; Wed, 23 May 2012 12:53:20 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:35321) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SXEoG-0002wP-BM for bug-gnu-emacs@gnu.org; Wed, 23 May 2012 12:53:17 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1SXEoC-0006cN-Lr for bug-gnu-emacs@gnu.org; Wed, 23 May 2012 12:53:15 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:59533) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SXEoC-0006bs-J1 for bug-gnu-emacs@gnu.org; Wed, 23 May 2012 12:53:12 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.72) (envelope-from ) id 1SXEoz-0008EJ-SO for bug-gnu-emacs@gnu.org; Wed, 23 May 2012 12:54:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 23 May 2012 16:54:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 11519 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 11519-submit@debbugs.gnu.org id=B11519.133779200831588 (code B ref 11519); Wed, 23 May 2012 16:54:01 +0000 Original-Received: (at 11519) by debbugs.gnu.org; 23 May 2012 16:53:28 +0000 Original-Received: from localhost ([127.0.0.1]:40846 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SXEoR-0008DQ-DS for submit@debbugs.gnu.org; Wed, 23 May 2012 12:53:27 -0400 Original-Received: from mtaout20.012.net.il ([80.179.55.166]:36096) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SXEo6-0008Ca-O6 for 11519@debbugs.gnu.org; Wed, 23 May 2012 12:53:26 -0400 Original-Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0M4H00B00IQYUC00@a-mtaout20.012.net.il> for 11519@debbugs.gnu.org; Wed, 23 May 2012 19:52:09 +0300 (IDT) Original-Received: from HOME-C4E4A596F7 ([87.69.210.75]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0M4H00A3PIUWPZG0@a-mtaout20.012.net.il>; Wed, 23 May 2012 19:52:09 +0300 (IDT) In-reply-to: X-012-Sender: halo1@inter.net.il X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:60312 Archived-At: > From: Stefan Monnier > Cc: Richard Stallman , handa@gnu.org, schwab@linux-m68k.org, lekktu@gmail.com, 11519@debbugs.gnu.org > Date: Wed, 23 May 2012 10:16:17 -0400 > > > Which other places use C pointers to buffer text and call functions > > that can allocate memory? > > IIUC any place that uses STRING_CHAR_AND_LENGTH on buffer text is > vulnerable to the problem. That's not true. As long as you access buffer text through character position, you are safe. The only situation where you are vulnerable is if you store a C pointer to buffer text, e.g., like this: char *text = BEGV_ADDR; or char *text = BYTE_POS_ADDR (current_buffer->pt); then invoke some function that can allocate or reallocate memory, and _then_ access buffer text through that pointer. If you find such a code anywhere, then that's a bug similar to this one. > If you really want to install your workaround on the emacs-24 branch, go > for it but let's try to find a real fix for the trunk. What kind of real fix are you looking for? I agree with Handa-san: being able to suppress relocation in select places is a good feature. Why shouldn't it be the fix in this case, and what better fix can we invent when we use an essentially externally maintained code (AFAIR, regex will at some point be re-sync'ed with gnulib) that cannot be expected to change its code radically so as not to access buffer text through C pointers? > >> >> I wonder: why do we use REL_ALLOC? > >> > AFAIK, we do that only on platforms that don't support mmap for > >> > allocating buffer text. > >> So, IIUC the only reason to use it is so that we can more often return > >> memory to the OS even for the non-mmap case? Is that because returning > >> memory can only be done via sbrk style memory management? > > I don't think this is only about _returning_ memory. It is first and > > foremost about not _asking_ for more memory when we can come up with > > it by reshuffling buffer text. > > So you're saying it's use for fragmentation reasons? Yes. > But on other platforms where we use mmap, we do suffer from this > fragmentation, and yet it doesn't seem to be a real source of problem. I don't know enough about mmap to answer that. I vaguely recollect that mmap avoids such fragmentation as an inherent feature, but I may be wrong. > That's why I think the only real reason is because memory can only be > returned via sbrk-style memory management (i.e. only free memory at the > end of the heap can be returned). Is that right? Yes, AFAIK. > I guess my question turns into "why do we use gmalloc.c instead of > a malloc library that uses mmap (or some other mechanism that lets it > return large free chunks to the OS)"? Use of gmalloc is a different issue. We were talking about ralloc.c. You could use one, but not the other. > AFAIK, Windows is pretty much the only system where we use gmalloc.c and > ralloc.c nowadays. My reading of configure is that we use it on more than just Windows (and MS-DOS). Basically, any platform that uses gmalloc.c (which is the default, turned off only for GNU/Linux and Darwin) also uses ralloc.c. > Does anyone remember why we don't use the system malloc under > Windows (and Cygwin)? I find it hard to believe that going through system malloc on MS-Windows will let us use buffers as large as 1.5 GB (on a 32-bit machine). To achieve this today, we reserve a 2GB contiguous chunk of address space at startup, and then commit and uncommit parts of it as needed (see w32heap.c). ralloc.c has an important part in this arrangement.