From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Pip Cet Newsgroups: gmane.emacs.bugs Subject: bug#36447: 27.0.50; New "Unknown keyword" errors Date: Fri, 5 Jul 2019 18:57:56 +0000 Message-ID: References: <875zon7x0a.fsf@web.de> <8336jqgbhp.fsf@gnu.org> <87h886eoke.fsf@web.de> <87d0iu54d1.fsf@gmail.com> <87k1d14djr.fsf@web.de> <87h884fo0i.fsf@web.de> <85d0is5ry1.fsf@gmail.com> <87lfxdgs1k.fsf@web.de> <83y31capj1.fsf@gnu.org> <83tvc0anwi.fsf@gnu.org> <83r274an61.fsf@gnu.org> <83pnmoacft.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="258412"; mail-complaints-to="usenet@blaine.gmane.org" Cc: michael_heerdegen@web.de, npostavs@gmail.com, 36447@debbugs.gnu.org To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Fri Jul 05 20:59:10 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1hjTQM-00155G-1g for geb-bug-gnu-emacs@m.gmane.org; Fri, 05 Jul 2019 20:59:10 +0200 Original-Received: from localhost ([::1]:55406 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hjTQK-0005zF-Ef for geb-bug-gnu-emacs@m.gmane.org; Fri, 05 Jul 2019 14:59:08 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:46084) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hjTQF-0005yx-DS for bug-gnu-emacs@gnu.org; Fri, 05 Jul 2019 14:59:04 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hjTQE-0006pD-6O for bug-gnu-emacs@gnu.org; Fri, 05 Jul 2019 14:59:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:44927) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hjTQE-0006p8-2d for bug-gnu-emacs@gnu.org; Fri, 05 Jul 2019 14:59:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hjTQE-0003gJ-1d for bug-gnu-emacs@gnu.org; Fri, 05 Jul 2019 14:59:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Pip Cet Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 05 Jul 2019 18:59:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36447 X-GNU-PR-Package: emacs Original-Received: via spool by 36447-submit@debbugs.gnu.org id=B36447.156235312114117 (code B ref 36447); Fri, 05 Jul 2019 18:59:01 +0000 Original-Received: (at 36447) by debbugs.gnu.org; 5 Jul 2019 18:58:41 +0000 Original-Received: from localhost ([127.0.0.1]:53748 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjTPs-0003fc-EN for submit@debbugs.gnu.org; Fri, 05 Jul 2019 14:58:40 -0400 Original-Received: from mail-oi1-f194.google.com ([209.85.167.194]:44759) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjTPq-0003fN-Qd for 36447@debbugs.gnu.org; Fri, 05 Jul 2019 14:58:39 -0400 Original-Received: by mail-oi1-f194.google.com with SMTP id e189so7763829oib.11 for <36447@debbugs.gnu.org>; Fri, 05 Jul 2019 11:58:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=/D2UBDCf53RtAAZJqnZKuBk3aKmHhxyXaFDxdeZPIU8=; b=bEikPid3gltwn31CeCuene66OuRjTBfzNmsGMYOyz2rX6TfkOc+nDXSYgDDzl6wnzj 97/SwYY7puqOx73KvEioa/OeI2MgO3VIAc9ExqpfgS46ZkXvgPiN7LcrloMRRd+HqGp6 cUbTOPjcsi7XQctmzl4hrecObnTaXYAdWETKFBeKMgB65Ct2UJVE8i6oMjefG20/+Wi6 p2yoNo0b0/lA5uBkhR4bKfn066SzlN6aofXCiRViG+S5GeWlWWZ1w5XEDRPl07ageJIc 3+eTvC8USl0omEHSoKzpjIVTLHRj3muR4Eq3PdGlEK8+SQiB+SDGdebUcw1+mFnexwgt 5NXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=/D2UBDCf53RtAAZJqnZKuBk3aKmHhxyXaFDxdeZPIU8=; b=cL2dLNGMzUxOeZsSNuK0OZkb3KIqCf0VDVmhHO10/0hROmF4EFcNl9O6/gVp9IeG9J ZXF4KVO5+Thk83Jw3AUclTaDQ/f+uFt52mUsuiDnF2xJzg9R0vZ/Ko7a/SvJJN2CSto+ /U1vP2M2SAbjudE7VOpL4nQQ1s0+1gDbdneuuJ8Svi4oGWQqLzBmb4F44rB/+dE0txY2 RsD6nPf2pNujm4mnZyjeCrgBV2zKN/HlWGW2JVtxIitdg6NQEt6GX89JGdGgO4xkm4Vw UYu13r+0Oj8v9ZUspohQvUIdJrF9tpo3hZJBXcGdHRt3yHCb2UmTWq32Q71LyGukfk7m btuA== X-Gm-Message-State: APjAAAXn1Iijb7IGOm3dX+1MWUMhSpHMUBzZmYTjhGKDQNx2b3YqWxad fySN22CtzoJ+H8EF5Veyc5oaZR+rbqi9GzxHkOo= X-Google-Smtp-Source: APXvYqx0OKW9N2N/4m02vLtIX3ceyQpAqf0XMOYTM980Qdmcrj8CZ1W+j7q6VqjuA3CE6cqXochK8ce35lXc0Mm7nhg= X-Received: by 2002:aca:aa93:: with SMTP id t141mr2844402oie.128.1562353113041; Fri, 05 Jul 2019 11:58:33 -0700 (PDT) In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:162146 Archived-At: On Fri, Jul 5, 2019 at 6:00 PM Stefan Monnier wr= ote: > > A na=C3=AFve question: wouldn't the problem go away if we modified pure= copy > > not to do the above, i.e. not to merge the next vector of a hash table > > with that of another? > > It might do the trick, yes (the patch below does that, for example, tho > I think Pip's patch is a better approach). > Note that it would mean we still modify data in the purespace, which > ideally shouldn't happen. > > The problem fundamentally is that `purecopy` assumes that the argument > passed to it will never again be modified (it doesn't have to be > immutable before, but it should be afterwards) but if we rehash > afterwards then we break this promise. Surely you mean you're allowed to modify the argument to purecopy, just not the return value of it? > His patch can/should be made slightly more efficient by only doing the > Fcopy_sequence on those hash-tables that are in purespace. How do we test for that? If I'm reading the pdumper code correctly, it'll put all "unstable" hash tables at the end of the dump, far away from the actually pure objects. I'm not sure how difficult it would be, but we could dump the ->index, ->next, ->hash vectors as Qnil (so not include the actual vectors in the dump), which would make the dump slightly smaller and give us a better test than h->count < 0: INLINE bool hash_rehash_needed_p (const struct Lisp_Hash_Table *h) { return NILP (h->index); } (I used h->index because it's in the same cache line as h->count). But that's hard to do without writing a shrink_hash_table function to be used before dumping. I think hash tables should be shrinkable, but that's beyond the scope of this bug. > diff --git a/src/fns.c b/src/fns.c > index 2fc000a7f4..cc4fd7f2d7 100644 > --- a/src/fns.c > +++ b/src/fns.c > @@ -4218,6 +4218,11 @@ maybe_resize_hash_table (struct Lisp_Hash_Table *h= ) > } > } > > +/* Recompute the hashes (and hence also the "next" pointers). And the "index" pointers, if we're listing all of them. > + Normally there's never a need to recompute hashes > + This is done only on first-access to a hash-table loaded from > + the "pdump", because the object's addresses may have changed, thus > + affecting their hash. */ > void > hash_table_rehash (struct Lisp_Hash_Table *h) > { > diff --git a/src/alloc.c b/src/alloc.c > index 64aaa8acdf..b0114a9459 100644 > --- a/src/alloc.c > +++ b/src/alloc.c > @@ -5348,9 +5348,24 @@ purecopy_hash_table (struct Lisp_Hash_Table *table= ) > > pure->header =3D table->header; > pure->weak =3D purecopy (Qnil); > + /* After reloading the pdumped data, objects may ehave changed locatio= n > + in memory, so their hash may have changed. So the `hash`, `next`, = and > + `index` vectors are not immutable and can't safely be hash-cons'ed.= */ > + if (HASH_TABLE_P (Vpurify_flag)) > + { > + Fremhash (table->hash, Vpurify_flag); > + Fremhash (table->next, Vpurify_flag); > + Fremhash (table->index, Vpurify_flag); > + } > pure->hash =3D purecopy (table->hash); > pure->next =3D purecopy (table->next); Slightly contrived problem here: if the single key in a single-entry EQ hash is -7, or an expression that happens to have hash value -1, pure->hash and pure->next would be EQ after these two lines, and updating the hash would corrupt the hash table... > pure->index =3D purecopy (table->index); Same for ->index and ->hash if both are equal to [0]. > + if (HASH_TABLE_P (Vpurify_flag)) > + { > + Fremhash (table->hash, Vpurify_flag); > + Fremhash (table->next, Vpurify_flag); > + Fremhash (table->index, Vpurify_flag); > + }