From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Aris Spathis Newsgroups: gmane.emacs.bugs Subject: bug#69709: `sort` interface improvement and universal ordering predicate Date: Sun, 14 Apr 2024 17:03:11 +0300 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="000000000000c117b406160ef707" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="22876"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 69709@debbugs.gnu.org To: mattias.engdegard@gmail.com Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sun Apr 14 17:21:21 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rw1fV-0005pp-6m for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 14 Apr 2024 17:21:21 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rw1f9-0000hY-8e; Sun, 14 Apr 2024 11:20:59 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rw1f5-0000ga-7J for bug-gnu-emacs@gnu.org; Sun, 14 Apr 2024 11:20:57 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rw1f4-00052m-Tc for bug-gnu-emacs@gnu.org; Sun, 14 Apr 2024 11:20:54 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rw1fG-00072j-0P for bug-gnu-emacs@gnu.org; Sun, 14 Apr 2024 11:21:06 -0400 X-Loop: help-debbugs@gnu.org In-Reply-To: Resent-From: Aris Spathis Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 14 Apr 2024 15:21:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 69709 X-GNU-PR-Package: emacs Original-Received: via spool by 69709-submit@debbugs.gnu.org id=B69709.171310802726643 (code B ref 69709); Sun, 14 Apr 2024 15:21:05 +0000 Original-Received: (at 69709) by debbugs.gnu.org; 14 Apr 2024 15:20:27 +0000 Original-Received: from localhost ([127.0.0.1]:35974 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rw1ea-0006vD-Bz for submit@debbugs.gnu.org; Sun, 14 Apr 2024 11:20:26 -0400 Original-Received: from mail-io1-xd2e.google.com ([2607:f8b0:4864:20::d2e]:55799) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rw0SJ-0001D8-Nl for 69709@debbugs.gnu.org; Sun, 14 Apr 2024 10:03:41 -0400 Original-Received: by mail-io1-xd2e.google.com with SMTP id ca18e2360f4ac-7d6c4c97875so72452839f.3 for <69709@debbugs.gnu.org>; Sun, 14 Apr 2024 07:03:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713103403; x=1713708203; darn=debbugs.gnu.org; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=z4QHApF9fSKwi+LpdR/FGGwZUDePEdLM9vaCw+ed/DQ=; b=GODXobXthbMBCB+LWDmlKIyBGPo1zhz5wVrqAXn+/Rrxkvx/3mR+t+lG65stBovnoG 1fTJvbS/NJIvvJFIj12oOWOJYNw8wuWETQKBr3Dd5npGDHIVzXvM/8bj7id7+RUk2PuB Tt1DAN5OC+DUena0rbjBpklZMcSuHT12bVWcBKn690rdePXlBMmh3v5OVczeZo+oAA1F YY67eblDMU8bTgwfzo3cx0M3tr4fSgxLv02DT4ahB6rIt0KwP31wCRH9ahVFkzpfKkbK xF7wWqVTCx4cXfn2gXqqeUSnb2NntAt/fa3RiAw3Z+e70594gBv+L5lvS9kI0AqzxXXC CZLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713103403; x=1713708203; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=z4QHApF9fSKwi+LpdR/FGGwZUDePEdLM9vaCw+ed/DQ=; b=ne3ktTfg19Q4Y3TmzpPuxjXmBusIPChYgRRcZHpZYTCH2X2kK0w2Gcf4AtDFgfahOb 8EJEWlnkUKS7p2sakbSfpGN3EYqBZ9SQGSS7Xhw6Ftvb5wqLGwwkr81oUd+vJHbcDOEI LEwciQHvqauYChNCjJc3Jl5DPSWI8LAArhlorLwdNQzFgBigkosp/Tlajlv/jxt7faD3 0u/CcWvwimXmF3dAR8m9vbqyagxV2jc8/NC/FHTHRoF9Sky0mz7MsjJE4bu1DH0FFFes aKQKtMGFcqZZa2E2sPG1kRAVqaCp/F1pUPGj3g+JeM0c4hFSavV9Da6ZCAwB+fMrTmNx klNg== X-Gm-Message-State: AOJu0YyQDiRqh6bLCnmb0IB/J1HLkAofbIIyJyPwTXMFJgW08yfDrrLe 8aUuex2vB/5sV4N2m2tMl9jbVgqkypUgbpojrbJY3lx8Isz+wUupEwkUZS2MXWDb7d0wzN8VKg9 auSUFKPTyAQwBwmSkEjrNhR6r1c8= X-Google-Smtp-Source: AGHT+IE14nNPLGR/ue3FDrRfBUL6xxAGGfRBao8nMEYmW9vN/Ezdbu+bjnYZrMDG7RlHwnYFUaghyeHFD73Fi0Mh0pY= X-Received: by 2002:a05:6602:418e:b0:7d5:f591:2cc7 with SMTP id bx14-20020a056602418e00b007d5f5912cc7mr11661398iob.9.1713103402233; Sun, 14 Apr 2024 07:03:22 -0700 (PDT) X-Mailman-Approved-At: Sun, 14 Apr 2024 11:20:21 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:283284 Archived-At: --000000000000c117b406160ef707 Content-Type: multipart/alternative; boundary="000000000000c117b306160ef705" --000000000000c117b306160ef705 Content-Type: text/plain; charset="UTF-8" Thank you for your excellent work on `sort` and related functionality! Unfortunately, the new `sort` implementation occasionally crashes with a segfault. The following code reproduces that in current master: (dotimes (i 500) (sort (make-list 128 42) :key (lambda (n) (make-list i n)))) It happens for inputs of length >= `MERGESTATE_TEMP_SIZE / 2` (= 128 currently) along with a non-NIL `:key` function. In such cases, a `Lisp_Object` array is explicitly heap-allocated to store the keys, which is never marked against GC. This would not be a problem if not for the fact that the `:key` function call may trigger GC. I'm attaching a patch with a proposed solution, consisting of three changes: 1. Allocate with `xzalloc` to have the new array initialized to `Qnil`. This ensures its objects can be marked properly. 2. Mark `ms->allocated_keys` in `merge_markmem`. We don't need to check that `ms->allocated_keys != NULL`, as `merge_register_cleanup` is only called under this exact condition. 3. Move the computation of keys (which may trigger a GC) after `merge_init`, which registers `merge_markmem`. I hope this is useful. Cheers, Aris --000000000000c117b306160ef705 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thank you for your excellent work on `sort` and related fu= nctionality!

Unfortunately, the new `sort` implementation occasional= ly crashes with a
segfault. The following code reproduces that in curren= t master:

(dotimes (i 500)
=C2=A0 (sort (make-list 128 42)
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 :key (lambda (n) (make-list i n))))

It happ= ens for inputs of length >=3D `MERGESTATE_TEMP_SIZE / 2` (=3D 128
cur= rently) along with a non-NIL `:key` function. In such cases, a
`Lisp_Obj= ect` array is explicitly heap-allocated to store the keys, which is
neve= r marked against GC. This would not be a problem if not for the fact thatthe `:key` function call may trigger GC.

I'm attaching a patch= with a proposed solution, consisting of three changes:

1. Allocate = with `xzalloc` to have the new array initialized to `Qnil`. This
=C2=A0 = =C2=A0ensures its objects can be marked properly.

2. Mark `ms->al= located_keys` in `merge_markmem`. We don't need to check that
=C2=A0= =C2=A0`ms->allocated_keys !=3D NULL`, as `merge_register_cleanup` is on= ly called
=C2=A0 =C2=A0under this exact condition.

3. Move the co= mputation of keys (which may trigger a GC) after `merge_init`,
=C2=A0 = =C2=A0which registers `merge_markmem`.

I hope this is useful.
Cheers,
Aris

--000000000000c117b306160ef705-- --000000000000c117b406160ef707 Content-Type: text/x-patch; charset="US-ASCII"; name="sort-segfault-fix.patch" Content-Disposition: attachment; filename="sort-segfault-fix.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_luzl9p7d0 ZGlmZiAtLWdpdCBhL3NyYy9zb3J0LmMgYi9zcmMvc29ydC5jCmluZGV4IDUyN2Q1NTUwMzQyLi41 NjJmODhlZGUzYyAxMDA2NDQKLS0tIGEvc3JjL3NvcnQuYworKysgYi9zcmMvc29ydC5jCkBAIC01 MzIsNiArNTMyLDggQEAgbWVyZ2VfbWFya21lbSAodm9pZCAqYXJnKQogICBtZXJnZV9zdGF0ZSAq bXMgPSBhcmc7CiAgIGVhc3N1bWUgKG1zICE9IE5VTEwpOwogCisgIG1hcmtfb2JqZWN0cyAobXMt PmFsbG9jYXRlZF9rZXlzLCBtcy0+bGlzdGxlbik7CisKICAgaWYgKG1zLT5yZWxvYy5zaXplICE9 IE5VTEwgJiYgKm1zLT5yZWxvYy5zaXplID4gMCkKICAgICB7CiAgICAgICBMaXNwX09iamVjdCAq c3JjID0gKG1zLT5yZWxvYy5zcmMtPnZhbHVlcwpAQCAtMTEwNywyMSArMTEwOSwyOSBAQCB0aW1f c29ydCAoTGlzcF9PYmplY3QgcHJlZGljYXRlLCBMaXNwX09iamVjdCBrZXlmdW5jLAogICAgICAg aWYgKGxlbmd0aCA8IE1FUkdFU1RBVEVfVEVNUF9TSVpFIC8gMikKIAlrZXlzID0gJm1zLnRlbXBh cnJheVtsZW5ndGggKyAxXTsKICAgICAgIGVsc2UKLQlrZXlzID0gYWxsb2NhdGVkX2tleXMgPSB4 bWFsbG9jIChsZW5ndGggKiB3b3JkX3NpemUpOwotCi0gICAgICBmb3IgKHB0cmRpZmZfdCBpID0g MDsgaSA8IGxlbmd0aDsgaSsrKQotCWtleXNbaV0gPSBjYWxsMSAoa2V5ZnVuYywgc2VxW2ldKTsK Kwl7CisJICAvKiBBbGxvY2F0ZSB3aXRoIHh6YWxsb2MgdG8gb2J0YWluIGFuIGFycmF5IG9mIHZh bGlkCisJICAgICBMaXNwX09iamVjdHMgKFFuaWxzKSwgc28gdGhhdCB0aGV5IGNhbiBiZSBtYXJr ZWQuICovCisJICB2ZXJpZnkgKE5JTF9JU19aRVJPKTsKKwkgIGtleXMgPSBhbGxvY2F0ZWRfa2V5 cyA9IHh6YWxsb2MgKGxlbmd0aCAqIHdvcmRfc2l6ZSk7CisJfQogCiAgICAgICBsby5rZXlzID0g a2V5czsKICAgICAgIGxvLnZhbHVlcyA9IHNlcTsKICAgICB9CiAKKyAgbWVyZ2VfaW5pdCAoJm1z LCBsZW5ndGgsIGFsbG9jYXRlZF9rZXlzLCAmbG8sIHByZWRpY2F0ZSk7CisKKyAgLyogQ29tcHV0 ZSBrZXlzIGFmdGVyIG1lcmdlX21hcmttZW0gaGFzIGJlZW4gcmVnaXN0ZXJlZCBieSBtZXJnZV9p bml0CisgICAgIChhbnkgY2FsbCB0byBrZXlmdW5jIG1pZ2h0IHRyaWdnZXIgYSBHQykuICovCisg IGlmICghTklMUCAoa2V5ZnVuYykpCisgICAgZm9yIChwdHJkaWZmX3QgaSA9IDA7IGkgPCBsZW5n dGg7IGkrKykKKyAgICAgIGtleXNbaV0gPSBjYWxsMSAoa2V5ZnVuYywgc2VxW2ldKTsKKwogICAv KiBGSVhNRTogVGhpcyBpcyB3aGVyZSB3ZSB3b3VsZCBjaGVjayB0aGUga2V5cyBmb3IgaW50ZXJl c3RpbmcKICAgICAgcHJvcGVydGllcyBmb3IgbW9yZSBvcHRpbWlzZWQgY29tcGFyaXNvbiAoc3Vj aCBhcyBhbGwgYmVpbmcgZml4bnVtcwogICAgICBldGMpLiAgKi8KIAotICBtZXJnZV9pbml0ICgm bXMsIGxlbmd0aCwgYWxsb2NhdGVkX2tleXMsICZsbywgcHJlZGljYXRlKTsKLQogICAvKiBNYXJj aCBvdmVyIHRoZSBhcnJheSBvbmNlLCBsZWZ0IHRvIHJpZ2h0LCBmaW5kaW5nIG5hdHVyYWwgcnVu cywKICAgICAgYW5kIGV4dGVuZGluZyBzaG9ydCBuYXR1cmFsIHJ1bnMgdG8gbWlucnVuIGVsZW1l bnRzLiAgKi8KICAgY29uc3QgcHRyZGlmZl90IG1pbnJ1biA9IG1lcmdlX2NvbXB1dGVfbWlucnVu IChsZW5ndGgpOwo= --000000000000c117b406160ef707--