Re: mark_object crash in 22.1 and latest CVS (as of tonight)

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Richard Stallman]

    Adding a subcase of Lisp_Misc_Free inside the
    (in mark_object) which calls break (i.e. ignores it) causes my crash
    to go away.

Lisp_Misc_Free should only be found in objects on the free list.
To have a pointer to such an object inside a live Lisp object
indicates a bug.  The code in mark_object does the right thing
when it detects such a bug: it calls abort.

    Since I have a reproducible test case I'd be happy to track down
    where these are coming from, but I need some help (in the form
    of information) to know what I'm chasing.

Thanks -- this is exactly what is needed for this bug.

The first questions are, what object contains the bad pointer?
What data type is it?  What data structure is it part of?

Once you answer those, you can try to figure out how it happened
that the data structure ended up with a bad pointer.
Maybe GC failed to mark that pointer, so the misc object got freed
even though it was still in use.


DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta;
	h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type;
	bh=L0zqqT/Q601vgQwBA+d4VFKM+JG92nL2YU2ykBchtks=;
	b=L3+Nm92uC/xFXmTN3dv7g4z6wBaqQ6Fqle2Hf1ehnL8Hu/eRmo7cM1kpUEBJvLnE4f4dXU7t3slAx2eQJWnFmOwNhpjGXC0e2T+jz64ZSAwR2NnEvITXAIWH/CxAbY2pfqLNyBIt+lyyekszBoBI3FCNDfaWT1WDhpqQovW8GnU=
Date: Thu, 8 Nov 2007 22:55:46 -0500
From: "Kalman Reti" <kalman.reti@gmail.com>
To: bug-gnu-emacs@gnu.org
MIME-Version: 1.0
Content-Type: multipart/mixed; 
	boundary="----=_Part_6215_19104712.1194580546565"
Subject: mark_object crash in 22.1 and latest CVS (as of tonight)

------=_Part_6215_19104712.1194580546565
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

See attached file for gdb session of garbage collector crash in
a [GNU/]linux emacs built from sources checked out tonight.

  Kalman

------=_Part_6215_19104712.1194580546565
Content-Type: text/plain; name=emacscrash.text
Content-Transfer-Encoding: base64
X-Attachment-Id: f_f8s6d7do0
Content-Disposition: attachment; filename=emacscrash.text

VGhpcyBpcyBhIGdhcmJhZ2UtY29sbGVjdCBjcmFzaCBpbiBhIGJ1aWx0LWZyb20tQ1ZTIGVtYWNz
IHRyZWUNCmNoZWNrZWQgb3V0IHRvbmlnaHQgKE5vdiA4LCAyMDA3KS4NCg0KSSBoYWQgb3JpZ2lu
YWxseSBleHBlcmllbmNlZCB0aGlzIGNyYXNoIGluIDIyLjEsIGJvdGggb24gV2luZG93cw0KYW5k
IExpbnV4LCBidXQgd2FudGVkIHRvIG1ha2Ugc3VyZSB0aGUgYnVnIGV4aXN0ZWQgaW4gdGhlIGxh
dGVzdA0KdmVyc2lvbiBiZWZvcmUgcmVwb3J0aW5nIGl0Lg0KDQpJJ3ZlIHdyaXR0ZW4gc29tZSBm
dW5jdGlvbnMgd2hpY2ggaXNzdWUgU2hlbGwgQ29tbWFuZHMgdG8gaW50ZXJhY3QNCndpdGggb3Vy
IHBlcmZvcmNlIHNlcnZlciBhdCB3b3JrOyB0aGVzZSBjb21tYW5kcyBwYXJzZSB0aGUgKlNoZWxs
IE91dHB1dA0KQnVmZmVyKiB0byBwaWNrIHVwIGJpdHMgb2YgaW5mb3JtYXRpb24uICBUaGVzZSBo
YXZlIGJlZW4gd29ya2luZyB2ZXJ5DQp3ZWxsIGZvciBtZSwgYnV0IHRvZGF5IEkgZ290IGEgcmVw
cm9kdWNpYmxlIGNhc2UgdGhhdCBjcmFzaGVzIEVtYWNzLg0KVW5mb3J0dW5hdGVseSwgaXQgaXMg
b25seSByZXByb2R1Y2libGUgYWZ0ZXIgaXNzdWluZyBtYW55IGNvbW1hbmRzDQphZ2FpbnN0IG91
ciBwZXJmb3JjZSBzZXJ2ZXIuDQoNClNvIEkgYnVpbHQgZnJvbSBzb3VyY2VzLCByYW4gd2l0aCBn
ZGIsIGFuZCBjYXB0dXJlZCB0aGUgZm9sbG93aW5nIGluZm9ybWF0aW9uLg0KVGhlIG9iamVjdCBp
dCB0cmlwcyBvdmVyIGlzIGFsd2F5cyBhIG1pc2MgZnJlZSBjZWxsIGFuZCBpdCBhbHdheXMNCmhp
dHMgdGhlIGRlZmF1bHQgbGVnIG9mIHRoZSBjYXNlIHN0YXRlbWVudCBpbiBtYXJrX29iamVjdC4N
Cg0KTGV0IG1lIGtub3cgaWYgeW91IG5lZWQgbWUgdG8gY29sbGVjdCBtb3JlIGluZm9ybWF0aW9u
Lg0KDQokIGdkYiAuL2VtYWNzDQpnZGIgLi9lbWFjcw0KR05VIGdkYiBSZWQgSGF0IExpbnV4ICg1
LjMuOTAtMC4yMDAzMDcxMC40MS4yLjFyaCkNCkNvcHlyaWdodCAyMDAzIEZyZWUgU29mdHdhcmUg
Rm91bmRhdGlvbiwgSW5jLg0KR0RCIGlzIGZyZWUgc29mdHdhcmUsIGNvdmVyZWQgYnkgdGhlIEdO
VSBHZW5lcmFsIFB1YmxpYyBMaWNlbnNlLCBhbmQgeW91IGFyZQ0Kd2VsY29tZSB0byBjaGFuZ2Ug
aXQgYW5kL29yIGRpc3RyaWJ1dGUgY29waWVzIG9mIGl0IHVuZGVyIGNlcnRhaW4gY29uZGl0aW9u
cy4NClR5cGUgInNob3cgY29weWluZyIgdG8gc2VlIHRoZSBjb25kaXRpb25zLg0KVGhlcmUgaXMg
YWJzb2x1dGVseSBubyB3YXJyYW50eSBmb3IgR0RCLiAgVHlwZSAic2hvdyB3YXJyYW50eSIgZm9y
IGRldGFpbHMuDQpUaGlzIEdEQiB3YXMgY29uZmlndXJlZCBhcyAiaTM4Ni1yZWRoYXQtbGludXgi
Li4uVXNpbmcgaG9zdCBsaWJ0aHJlYWRfZGIgbGlicmFyeSAiL2xpYi9saWJ0aHJlYWRfZGIuc28u
MSIuDQoNCkRJU1BMQVkgPSA6MS4wDQpURVJNID0gZHVtYg0KQnJlYWtwb2ludCAxIGF0IDB4ODBl
MDM5YTogZmlsZSBlbWFjcy5jLCBsaW5lIDQzMS4NCkJyZWFrcG9pbnQgMiBhdCAweDgwZjcxNDU6
IGZpbGUgc3lzZGVwLmMsIGxpbmUgMTQzNS4NCihnZGIpIHJ1bg0KU3RhcnRpbmcgcHJvZ3JhbTog
L3Uva3JldGkvZ251ZW1hY3MtbGludXgxMS9lbWFjcy9zcmMvZW1hY3MgLWdlb21ldHJ5IDgweDQw
KzArMA0KDQpCcmVha3BvaW50IDEsIGFib3J0ICgpIGF0IGVtYWNzLmM6NDMxDQo0MzEJICBraWxs
IChnZXRwaWQgKCksIFNJR0FCUlQpOw0KKGdkYikgd2hlcmUNCiMwICBhYm9ydCAoKSBhdCBlbWFj
cy5jOjQzMQ0KIzEgIDB4MDgxMmIxNzkgaW4gbWFya19vYmplY3QgKGFyZz0xNDcyMTEwNTApIGF0
IGFsbG9jLmM6NTczNA0KIzIgIDB4MDgxMmIxZGEgaW4gbWFya19vYmplY3QgKGFyZz0xNDE1Mzc0
ODUpIGF0IGFsbG9jLmM6NTc1MQ0KIzMgIDB4MDgxMmIxZGEgaW4gbWFya19vYmplY3QgKGFyZz0x
NDE1Mzc0MzcpIGF0IGFsbG9jLmM6NTc1MQ0KIzQgIDB4MDgxMmIyYjIgaW4gbWFya19idWZmZXIg
KGJ1Zj0xNDY5MzY0MjgpIGF0IGFsbG9jLmM6NTgwOA0KIzUgIDB4MDgxMmFlNDggaW4gbWFya19v
YmplY3QgKGFyZz0xNDY5MzY0MjgpIGF0IGFsbG9jLmM6NTU1OA0KIzYgIDB4MDgxMmIwZWMgaW4g
bWFya19vYmplY3QgKGFyZz0xMzgyODM0NTgpIGF0IGFsbG9jLmM6NTY3OQ0KIzcgIDB4MDgxMmIw
MjYgaW4gbWFya19vYmplY3QgKGFyZz0xMzc1NTg5MDUpIGF0IGFsbG9jLmM6NTYzOQ0KIzggIDB4
MDgxMmIxZGEgaW4gbWFya19vYmplY3QgKGFyZz0xMzc4NTk1NDkpIGF0IGFsbG9jLmM6NTc1MQ0K
IzkgIDB4MDgxMmIxZGEgaW4gbWFya19vYmplY3QgKGFyZz0xMzc4NTk4NjEpIGF0IGFsbG9jLmM6
NTc1MQ0KIzEwIDB4MDgxMmIwZTMgaW4gbWFya19vYmplY3QgKGFyZz0xMzc2NDA5MjIpIGF0IGFs
bG9jLmM6NTY3OA0KIzExIDB4MDgxMmIwMjYgaW4gbWFya19vYmplY3QgKGFyZz0xMzc3Mjg5Njkp
IGF0IGFsbG9jLmM6NTYzOQ0KIzEyIDB4MDgxMmIxZGEgaW4gbWFya19vYmplY3QgKGFyZz0xMzc4
NTQ5ODEpIGF0IGFsbG9jLmM6NTc1MQ0KIzEzIDB4MDgxMmIxZGEgaW4gbWFya19vYmplY3QgKGFy
Zz0xMzc0MDAyNTMpIGF0IGFsbG9jLmM6NTc1MQ0KIzE0IDB4MDgxMmIwMzggaW4gbWFya19vYmpl
Y3QgKGFyZz0xNDEzODAzMzMpIGF0IGFsbG9jLmM6NTY0MQ0KIzE1IDB4MDgxMmFlYzMgaW4gbWFy
a19vYmplY3QgKGFyZz0xNDEzOTExNTYpIGF0IGFsbG9jLmM6NTU4MQ0KIzE2IDB4MDgxMmIwMmYg
aW4gbWFya19vYmplY3QgKGFyZz0xMzc4MjY5NjEpIGF0IGFsbG9jLmM6NTY0MA0KIzE3IDB4MDgx
MmIxZGEgaW4gbWFya19vYmplY3QgKGFyZz0xNDEzODAyMzcpIGF0IGFsbG9jLmM6NTc1MQ0KIzE4
IDB4MDgxMmIwMzggaW4gbWFya19vYmplY3QgKGFyZz0xMzc0NTkzNDUpIGF0IGFsbG9jLmM6NTY0
MQ0KIzE5IDB4MDgxMmIxZGEgaW4gbWFya19vYmplY3QgKGFyZz0xMzkyOTcxODEpIGF0IGFsbG9j
LmM6NTc1MQ0KIzIwIDB4MDgxMmIwMzggaW4gbWFya19vYmplY3QgKGFyZz0xMzkyOTcxMzMpIGF0
IGFsbG9jLmM6NTY0MQ0KIzIxIDB4MDgxMmIxZGEgaW4gbWFya19vYmplY3QgKGFyZz0xMzc4NjAy
NjEpIGF0IGFsbG9jLmM6NTc1MQ0KIzIyIDB4MDgxMmIwMzggaW4gbWFya19vYmplY3QgKGFyZz0x
Mzc2Nzg0MjUpIGF0IGFsbG9jLmM6NTY0MQ0KIzIzIDB4MDgxMmIxZGEgaW4gbWFya19vYmplY3Qg
KGFyZz0xNDE0NzMyMjkpIGF0IGFsbG9jLmM6NTc1MQ0KIzI0IDB4MDgxMmFlYzMgaW4gbWFya19v
YmplY3QgKGFyZz0xNDE2ODgxNTYpIGF0IGFsbG9jLmM6NTU4MQ0KIzI1IDB4MDgxMmIwMmYgaW4g
bWFya19vYmplY3QgKGFyZz0xNDQ1MjQ3NTMpIGF0IGFsbG9jLmM6NTY0MA0KIzI2IDB4MDgxMmIx
ZGEgaW4gbWFya19vYmplY3QgKGFyZz0xNDQ0OTkyMDUpIGF0IGFsbG9jLmM6NTc1MQ0KIzI3IDB4
MDgxMmIxZGEgaW4gbWFya19vYmplY3QgKGFyZz0xNDQ0OTk0MzcpIGF0IGFsbG9jLmM6NTc1MQ0K
IzI4IDB4MDgxMmIwMmYgaW4gbWFya19vYmplY3QgKGFyZz0xNDQ1MjQ3MjkpIGF0IGFsbG9jLmM6
NTY0MA0KIzI5IDB4MDgxMmFkNmYgaW4gbWFya192ZWN0b3JsaWtlIChwdHI9MHg4MzBjOTY4KSBh
dCBhbGxvYy5jOjU0NTYNCiMzMCAweDA4MTJiMDA0IGluIG1hcmtfb2JqZWN0IChhcmc9MTM3NDE1
MDIwKSBhdCBhbGxvYy5jOjU2MjgNCiMzMSAweDA4MTJhNzg2IGluIEZnYXJiYWdlX2NvbGxlY3Qg
KCkgYXQgYWxsb2MuYzo1MTQxDQojMzIgMHgwODEzZGY1YSBpbiBGZnVuY2FsbCAobmFyZ3M9MSwg
YXJncz0weGJmZmVjNDIwKSBhdCBldmFsLmM6MzAyMQ0KIzMzIDB4MDgxNjE5YjQgaW4gRmJ5dGVf
Y29kZSAoYnl0ZXN0cj0xNDQ2NTg3ODcsIHZlY3Rvcj0xNDQ2NjMxNDgsIG1heGRlcHRoPTU2KQ0K
ICAgIGF0IGJ5dGVjb2RlLmM6Njc5DQojMzQgMHgwODEzZTQ2YSBpbiBmdW5jYWxsX2xhbWJkYSAo
ZnVuPTE0NDY2MzM1NiwgbmFyZ3M9MywgDQogICAgYXJnX3ZlY3Rvcj0weGJmZmVjNGUwKSBhdCBl
dmFsLmM6MzIxMQ0KIzM1IDB4MDgxM2UxYjYgaW4gYXBwbHlfbGFtYmRhIChmdW49MTQ0NjYzMzU2
LCBhcmdzPTE0Njg4NTkxNywgZXZhbF9mbGFnPTEpDQogICAgYXQgZXZhbC5jOjMxMzUNCiMzNiAw
eDA4MTNkNzAzIGluIEZldmFsIChmb3JtPTE0Njg4NTkwOSkgYXQgZXZhbC5jOjI0MTUNCiMzNyAw
eDA4MTNiMDg5IGluIEZzZXRxIChhcmdzPTE0Njg4NTkwMSkgYXQgZXZhbC5jOjU1Mg0KIzM4IDB4
MDgxM2Q0M2EgaW4gRmV2YWwgKGZvcm09MTQ2ODg1ODkzKSBhdCBldmFsLmM6MjMwMg0KIzM5IDB4
MDgxM2Q1MGQgaW4gRmV2YWwgKGZvcm09MTQ2ODg1ODg1KSBhdCBldmFsLmM6MjM0MA0KIzQwIDB4
MDgxM2RmNmYgaW4gRmZ1bmNhbGwgKG5hcmdzPTIsIGFyZ3M9MHhiZmZlYzgzNCkgYXQgZXZhbC5j
OjMwMjQNCiM0MSAweDA4MTYxOWI0IGluIEZieXRlX2NvZGUgKGJ5dGVzdHI9MTM2NTI0NDU5LCB2
ZWN0b3I9MTM2NTI0NDc2LCBtYXhkZXB0aD0yNCkNCiAgICBhdCBieXRlY29kZS5jOjY3OQ0KIzQy
IDB4MDgxM2U0NmEgaW4gZnVuY2FsbF9sYW1iZGEgKGZ1bj0xMzY1MjQ0MjAsIG5hcmdzPTEsIA0K
ICAgIGFyZ192ZWN0b3I9MHhiZmZlYzk0NCkgYXQgZXZhbC5jOjMyMTENCiM0MyAweDA4MTNlMDg5
IGluIEZmdW5jYWxsIChuYXJncz0yLCBhcmdzPTB4YmZmZWM5NDApIGF0IGV2YWwuYzozMDgxDQoj
NDQgMHgwODE2MTliNCBpbiBGYnl0ZV9jb2RlIChieXRlc3RyPTEzNjUyNDcwNywgdmVjdG9yPTEz
NjUyNDcyNCwgbWF4ZGVwdGg9MjQpDQogICAgYXQgYnl0ZWNvZGUuYzo2NzkNCiM0NSAweDA4MTNl
NDZhIGluIGZ1bmNhbGxfbGFtYmRhIChmdW49MTM2NTI0NjY4LCBuYXJncz0xLCANCiAgICBhcmdf
dmVjdG9yPTB4YmZmZWNhNTQpIGF0IGV2YWwuYzozMjExDQojNDYgMHgwODEzZTA4OSBpbiBGZnVu
Y2FsbCAobmFyZ3M9MiwgYXJncz0weGJmZmVjYTUwKSBhdCBldmFsLmM6MzA4MQ0KIzQ3IDB4MDgx
NjE5YjQgaW4gRmJ5dGVfY29kZSAoYnl0ZXN0cj0xMzY1MjI5MDcsIHZlY3Rvcj0xMzY1MjI5MjQs
IG1heGRlcHRoPTE2KQ0KICAgIGF0IGJ5dGVjb2RlLmM6Njc5DQojNDggMHgwODEzZTQ2YSBpbiBm
dW5jYWxsX2xhbWJkYSAoZnVuPTEzNjUyMjg3NiwgbmFyZ3M9MCwgDQogICAgYXJnX3ZlY3Rvcj0w
eGJmZmVjYjg0KSBhdCBldmFsLmM6MzIxMQ0KIzQ5IDB4MDgxM2UwODkgaW4gRmZ1bmNhbGwgKG5h
cmdzPTEsIGFyZ3M9MHhiZmZlY2I4MCkgYXQgZXZhbC5jOjMwODENCiM1MCAweDA4MTNkYzM0IGlu
IGFwcGx5MSAoZm49MTM4MzA3MTA1LCBhcmc9MTM3NDEzOTY5KSBhdCBldmFsLmM6Mjc2NQ0KIzUx
IDB4MDgxMzk4ZmMgaW4gRmNhbGxfaW50ZXJhY3RpdmVseSAoZnVuY3Rpb249MTM4MzA3MTA1LCAN
CiAgICByZWNvcmRfZmxhZz0xMzc0MTM5NjksIGtleXM9MTM3NDYyMjQ0KSBhdCBjYWxsaW50LmM6
Mzg1DQojNTIgMHgwODBlZGIxNSBpbiBGY29tbWFuZF9leGVjdXRlIChjbWQ9MTM4MzA3MTA1LCBy
ZWNvcmRfZmxhZz0xMzc0MTM5NjksIA0KICAgIGtleXM9MTM3NDEzOTY5LCBzcGVjaWFsPTEzNzQx
Mzk2OSkgYXQga2V5Ym9hcmQuYzoxMDM2Mw0KIzUzIDB4MDgwZTNjNjUgaW4gY29tbWFuZF9sb29w
XzEgKCkgYXQga2V5Ym9hcmQuYzoxOTM5DQojNTQgMHgwODEzYzQyMiBpbiBpbnRlcm5hbF9jb25k
aXRpb25fY2FzZSAoYmZ1bj0weDgwZTJmNzAgPGNvbW1hbmRfbG9vcF8xPiwgDQogICAgaGFuZGxl
cnM9MTM3NDgwNjA5LCBoZnVuPTB4ODBlMmEzYyA8Y21kX2Vycm9yPikgYXQgZXZhbC5jOjE0OTMN
CiM1NSAweDA4MGUyZDBlIGluIGNvbW1hbmRfbG9vcF8yICgpIGF0IGtleWJvYXJkLmM6MTM5Ng0K
IzU2IDB4MDgxM2JmOTMgaW4gaW50ZXJuYWxfY2F0Y2ggKHRhZz0xMzc0NjI5MDUsIA0KICAgIGZ1
bmM9MHg4MGUyY2YwIDxjb21tYW5kX2xvb3BfMj4sIGFyZz0xMzc0MTM5NjkpIGF0IGV2YWwuYzox
MjI5DQojNTcgMHgwODBlMmM5YyBpbiBjb21tYW5kX2xvb3AgKCkgYXQga2V5Ym9hcmQuYzoxMzc1
DQojNTggMHgwODBlMjZjMCBpbiByZWN1cnNpdmVfZWRpdF8xICgpIGF0IGtleWJvYXJkLmM6OTg0
DQojNTkgMHgwODBlMjgwMCBpbiBGcmVjdXJzaXZlX2VkaXQgKCkgYXQga2V5Ym9hcmQuYzoxMDQ2
DQojNjAgMHgwODBlMTY5NSBpbiBtYWluIChhcmdjPTMsIGFyZ3Y9MHhiZmZlZDMzNCkgYXQgZW1h
Y3MuYzoxNzc3DQoNCkxpc3AgQmFja3RyYWNlOg0KImdhcmJhZ2UtY29sbGVjdCIgKDB4YmZmZWM0
MjQpDQoiY2hhbmdlc2V0cy1iZXR3ZWVuIiAoMHhiZmZlYzRlMCkNCiJzZXRxIiAoMHhiZmZlYzY2
OCkNCiJsZW5ndGgiICgweGJmZmVjNzI4KQ0KImV2YWwiICgweGJmZmVjODM4KQ0KImV2YWwtbGFz
dC1zZXhwLTEiICgweGJmZmVjOTQ0KQ0KImV2YWwtbGFzdC1zZXhwIiAoMHhiZmZlY2E1NCkNCiJl
dmFsLXByaW50LWxhc3Qtc2V4cCIgKDB4YmZmZWNiODQpDQoiY2FsbC1pbnRlcmFjdGl2ZWx5IiAo
MHhiZmZlY2QzMCkNCihnZGIpIHByaW50IDE0NjkzNjQyOA0KJDEgPSAxNDY5MzY0MjgNCihnZGIp
IHByDQojPGJ1ZmZlciA+DQooZ2RiKSBwcmludCAxNDE1Mzc0MzcNCiQyID0gMTQxNTM3NDM3DQoo
Z2RiKSBwcg0KKCgxIC4gNzMpICgiLy9kZXBvdC9yZWxlYXNlLTEzLTMwL3NyYy9NYWtlZmlsZSM0
MiAtIGVkaXQgY2hhbmdlIDIyNzIwNCAodGV4dCkNCiIgLiAxKSAoIzxtaXNjIGZyZWUgY2VsbD4g
LiAtNTgpICgjPG1pc2MgZnJlZSBjZWxsPiAuIC02NSkgKCM8bWFya2VyIGluIG5vIGJ1ZmZlcj4g
LiAtNTgpICgjPG1hcmtlciBpbiBubyBidWZmZXI+IC4gLTY0KSAoMSAuIDczKSAoIi8vZGVwb3Qv
VjEzLTMwLXBhdGNoL3NyYy9NYWtlZmlsZQ0KLi4uICMxIGNoYW5nZSAyMjc3NTYgYnJhbmNoIG9u
IDIwMDcvMTEvMDIgYnkgbWFqb3JtYWpvckBtYWpvcm1ham9yLXA0YnJhbmNoLWF1dG81MjEgKHRl
eHQpICdDcmVhdGUnDQouLi4gLi4uIGJyYW5jaCBmcm9tIC8vZGVwb3QvcmVsZWFzZS0xMy0zMC9z
cmMvTWFrZWZpbGUjMSwjNDINCiIgLiAxKSAoIzxtYXJrZXIgaW4gbm8gYnVmZmVyPiAuIC0xNDMp
ICgjPG1hcmtlciBpbiBubyBidWZmZXI+IC4gLTIwMikgKCM8bWlzYyBmcmVlIGNlbGw+IC4gLTE2
NCkgKCM8bWlzYyBmcmVlIGNlbGw+IC4gLTE3NikgKCM8bWFya2VyIGluIG5vIGJ1ZmZlcj4gLiAt
MjAwKSAoIzxtYXJrZXIgaW4gbm8gYnVmZmVyPiAuIC0yMDIpICgxIC4gMjA0KSAoIi8vZGVwb3Qv
VjEzLTMwLXBhdGNoL3NyYy9NYWtlZmlsZSMxIC0gYnJhbmNoIGNoYW5nZSAyMjc3NTYgKHRleHQp
DQoiIC4gMSkgKCM8bWFya2VyIGluIG5vIGJ1ZmZlcj4gLiAtNTgpICgjPG1hcmtlciBpbiBubyBi
dWZmZXI+IC4gLTY1KSAoIzxtYXJrZXIgaW4gbm8gYnVmZmVyPiAuIC01OCkgKCM8bWFya2VyIGlu
IG5vIGJ1ZmZlcj4gLiAtNjQpICgxIC4gNzMpKQ0KKGdkYikgcHJpbnQgMTQxNTM3NDg1DQokMyA9
IDE0MTUzNzQ4NQ0KKGdkYikgcHINCigjPG1pc2MgZnJlZSBjZWxsPiAuIC01OCkNCihnZGIpIHBy
aW50IDE0NzIxMTA1MA0KJDQgPSAxNDcyMTEwNTANCihnZGIpIHByDQojPG1pc2MgZnJlZSBjZWxs
Pg0KKGdkYikgeG1pc2NmcmVlIDE0NzIxMTA1MA0KJDUgPSAoc3RydWN0IExpc3BfRnJlZSAqKSAw
eDhjNjQzMjgNCihnZGIpIHByDQoxODQwMTM4MQ0K
------=_Part_6215_19104712.1194580546565--

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Kalman Reti]

On Nov 11, 2007 12:22 AM, Richard Stallman <rms@gnu.org> wrote:

> The first questions are, what object contains the bad pointer?
> What data type is it?  What data structure is it part of?

The gdb pr output near the end of the attachment in my first message
shows it is part of a list, which, in turn, is part of a buffer.  I assumed
someone would recognize WHAT part of a buffer from the contents of the,
list, a mixture of conses with marker-in-no-buffer in the car of some and
Lisp_Misc_Free  in the car of others, the cdr's being negative numbers
of pretty small absolute magnitude.  If it isn't recognizable from its contents,
I'll have to wait till I'm next at work to find out exactly which slot
in the buffer
this list comes from using gdb.

The code I'm running is pretty simple, it executes a shell command (i.e.
a perforce command) and then uses search-forward-regexp to find
relevant lines in the output, capturing things like revision number or
branch using match-string after the regexp matches.  The searching
is done within a save-excursion which switches to  the *Shell Command
Output*  buffer.  I suspect one could reproduce the bug without issuing
perforce commands,  I'll give that a stab tonight.

>
> Once you answer those, you can try to figure out how it happened
> that the data structure ended up with a bad pointer.
> Maybe GC failed to mark that pointer, so the misc object got freed
> even though it was still in use.

Are there any tools to help with this, e.g. an allocation trace or GC trace?
I'm afraid this is the first time I've looked at the Emacs src code.

[rest of message elided]

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Stefan Monnier]

>> The first questions are, what object contains the bad pointer?
>> What data type is it?  What data structure is it part of?

> The gdb pr output near the end of the attachment in my first message
> shows it is part of a list, which, in turn, is part of a buffer.  I assumed
> someone would recognize WHAT part of a buffer from the contents of the,
> list, a mixture of conses with marker-in-no-buffer in the car of some and
> Lisp_Misc_Free  in the car of others, the cdr's being negative numbers
> of pretty small absolute magnitude.  If it isn't recognizable from its contents,
> I'll have to wait till I'm next at work to find out exactly which slot
> in the buffer
> this list comes from using gdb.

Sounds like the contents of the buffer-undo-list.  Especially since this
variable is GC'd specially and getting it right is tricky.


        Stefan

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Kalman Reti]

I looked at the code, and there are comments saying both that
the undo_list should be before the name slot and that it should
come after.  In the CVS code, it definitely comes after which looks
to me like it will get marked twice, once in the normal loop which
starts at name and marks all the following objects and then again
at the special code for marking the undo list.  This is contrary to
what the comments say should be happening, but I don't know which
of the comments or the code is right.

On Nov 12, 2007 5:03 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> >> The first questions are, what object contains the bad pointer?
> >> What data type is it?  What data structure is it part of?
>
> > The gdb pr output near the end of the attachment in my first message
> > shows it is part of a list, which, in turn, is part of a buffer.  I assumed
> > someone would recognize WHAT part of a buffer from the contents of the,
> > list, a mixture of conses with marker-in-no-buffer in the car of some and
> > Lisp_Misc_Free  in the car of others, the cdr's being negative numbers
> > of pretty small absolute magnitude.  If it isn't recognizable from its contents,
> > I'll have to wait till I'm next at work to find out exactly which slot
> > in the buffer
> > this list comes from using gdb.
>
> Sounds like the contents of the buffer-undo-list.  Especially since this
> variable is GC'd specially and getting it right is tricky.
>
>
>         Stefan
>

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Richard Stallman]

    > I'll have to wait till I'm next at work to find out exactly which slot
    > in the buffer
    > this list comes from using gdb.

    Sounds like the contents of the buffer-undo-list.  Especially since this
    variable is GC'd specially and getting it right is tricky.

It should be easy to verify that guess by examining the undo list slot
in the buffer object.

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Kalman Reti]

On Nov 13, 2007 3:03 PM, Richard Stallman <rms@gnu.org> wrote:
>     > I'll have to wait till I'm next at work to find out exactly which slot
>     > in the buffer
>     > this list comes from using gdb.
>
>     Sounds like the contents of the buffer-undo-list.  Especially since this
>     variable is GC'd specially and getting it right is tricky.
>
> It should be easy to verify that guess by examining the undo list slot
> in the buffer object.
>

By moving up the stack in gdb at the time of the abort, I was able to see
that the top-level mark_object call is from the undo list processing
in Fgarbage_collect.

The undo list is for the *Shell Command Output* buffer, and is very long
since that buffer gets used over and over again for the many shell commands
the elisp code issues.

Looking harder at the code, I'm convinced that the undo_list should come before
the name entry in the buffer structure, so I moved it there.  However,
I still get
the crash.

My first experiment of putting a proceeding breakpoint in the
undo_list processing
which printed out the list failed to result in an obvious correlation
between elements
of the undo_list the last time it was processed and the time which
resulted in the
abort. I suspect that the Lisp_Misc_Free cells were markers which
should have been
removed but for some as yet unknown reason, weren't.  I'll have to
craft a more thorough
experiment next time.

Anyone know what the elements of the undo_list mean?  Some are conses
with a marker
in their CAR and a number in their CDR, some are just conses of two
numbers and some
are conses of a string and a number.

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Stefan Monnier]

> Anyone know what the elements of the undo_list mean?  Some are conses
> with a marker in their CAR and a number in their CDR, some are just
> conses of two numbers and some are conses of a string and a number.

It's documented in the docstring of buffer-undo-list.


        Stefan

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Kalman Reti]

On Nov 14, 2007 1:51 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> > Anyone know what the elements of the undo_list mean?  Some are conses
> > with a marker in their CAR and a number in their CDR, some are just
> > conses of two numbers and some are conses of a string and a number.
>
> It's documented in the docstring of buffer-undo-list.

Thanks for the pointer.

I've done some more experiments; it occurred to me that if the marker in the
undo list was gc-marked already when we got to the special processing, then
it would be skipped.  I verified this by splitting out the last of the
three-legged-and
conditions into its own if.  Presumably this means that the marker is
shared in some
other structure which got marked previously.  Could the last match data and the
undo list perhaps share a marker?  Where is the last match data kept?
If it isn't
there, any suggestions on how to go about finding out where another pointer to
this marker is stored?

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Richard Stallman]

    I've done some more experiments; it occurred to me that if the marker in the
    undo list was gc-marked already when we got to the special processing, then
    it would be skipped.

I looked to see what you mean, and I see that some elements do get
removed from the undo list.  I hadn't remembered that -- sorry.

Is this the special processing you mean?

	/* If a buffer's undo list is Qt, that means that undo is
	   turned off in that buffer.  Calling truncate_undo_list on
	   Qt tends to return NULL, which effectively turns undo back on.
	   So don't call truncate_undo_list if undo_list is Qt.  */
	if (! EQ (nextb->undo_list, Qt))
	  {
	  ...

If so, it is supposed to delete elements for markers
that weren't already marked by GC.  And then it marks the undo
list in the normal way.

Does it look like that code failed to remove an element
which was supposed to update a marker?

Was the marker already corrupted (replaced with Lisp_Misc_Free)
before the start of the loop?

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Kalman Reti]

On Nov 15, 2007 12:09 PM, Richard Stallman <rms@gnu.org> wrote:
>     I've done some more experiments; it occurred to me that if the marker in the
>     undo list was gc-marked already when we got to the special processing, then
>     it would be skipped.
>
> I looked to see what you mean, and I see that some elements do get
> removed from the undo list.  I hadn't remembered that -- sorry.
>
> Is this the special processing you mean?
>
>         /* If a buffer's undo list is Qt, that means that undo is
>            turned off in that buffer.  Calling truncate_undo_list on
>            Qt tends to return NULL, which effectively turns undo back on.
>            So don't call truncate_undo_list if undo_list is Qt.  */
>         if (! EQ (nextb->undo_list, Qt))
>           {
>           ...
>

Yes.

> If so, it is supposed to delete elements for markers
> that weren't already marked by GC.  And then it marks the undo
> list in the normal way.

I believe it works to do this if you move the undo_list before name.
Otherwise, everything on the list is already marked by the normal
"start at the name offset and mark until you've reached the buffer
struct size" mechanism.

>
> Does it look like that code failed to remove an element
> which was supposed to update a marker?

No, it looks like a marker in the list is already marked; this
marker gets turned into the Lisp_Misc_Free cell.

>
> Was the marker already corrupted (replaced with Lisp_Misc_Free)
> before the start of the loop?

I believe so.  I think the culprit is the free_marker call in Fset_match_data.
I think this because I added a checking routine which, given a marker, looped
over all the cells in all the undo lists of all the buffers to see if
that marker
was in the caar of one of them, calling a dummy routine (krabort, on which
I could set a breakpoint) if so.  I added a call to this checking routine in
free_misc, fired up my test case and almost immediately got a hit.  (The
backtrace below can't be the whole story, since this happens much earlier
than the crash.  A gdb session which is automatically capturing a
backtrace at this
point and continuing, so I can show you the latest stack trace before the
crash, has run overnight now without reaching the crash.  Presumably
there is some mechanism which removes the Lisp_Misc_Free cell created
here before the GC trips over it and that something else [much] later on
is causing that mechanism to fail to work in the runnup to the crash.)

The early stack trace is at the end of this message.  One thing that isn't
clear to me is exactly who is calling set-match-data with the reseat
argument set to evaporate inside of the shell-command function.  This is
happening somewhere inside of the shell-command function which my
code calls.

(gdb) where
#0  krabort () at alloc.c:3364
#1  0x08129319 in check_for_problem (marker=147919074) at alloc.c:3380
#2  0x0812934c in free_misc (misc=147919074) at alloc.c:3394
#3  0x0811c354 in Fset_match_data (list=146951973, reseat=137508953)
    at search.c:3057
#4  0x0813e252 in Ffuncall (nargs=3, args=0xbffea3f0) at eval.c:3027
#5  0x08161c84 in Fbyte_code (bytestr=136239067, vector=136239092, maxdepth=24)
    at bytecode.c:679
#6  0x0813d87e in Feval (form=136239053) at eval.c:2361
#7  0x0813b22f in Fprogn (args=136239045) at eval.c:450
#8  0x0813eb33 in unbind_to (count=25, value=137414769) at eval.c:3378
#9  0x08162361 in Fbyte_code (bytestr=136238739, vector=136238756, maxdepth=64)
    at bytecode.c:890
#10 0x0813e73a in funcall_lambda (fun=136238676, nargs=1,
    arg_vector=0xbffea6e4) at eval.c:3211
#11 0x0813e359 in Ffuncall (nargs=2, args=0xbffea6e0) at eval.c:3081
#12 0x08161c84 in Fbyte_code (bytestr=144608715, vector=144609972, maxdepth=64)
    at bytecode.c:679
#13 0x0813e73a in funcall_lambda (fun=144610268, nargs=5,
    arg_vector=0xbffea804) at eval.c:3211
#14 0x0813e359 in Ffuncall (nargs=6, args=0xbffea800) at eval.c:3081
#15 0x08161c84 in Fbyte_code (bytestr=144597347, vector=144598732, maxdepth=48)
    at bytecode.c:679
#16 0x0813e73a in funcall_lambda (fun=144598884, nargs=3,
    arg_vector=0xbffea924) at eval.c:3211
#17 0x0813e359 in Ffuncall (nargs=4, args=0xbffea920) at eval.c:3081
#18 0x08161c84 in Fbyte_code (bytestr=144645315, vector=144646532, maxdepth=56)
    at bytecode.c:679
#19 0x0813e73a in funcall_lambda (fun=144646748, nargs=3,
    arg_vector=0xbffea9e0) at eval.c:3211
#20 0x0813e486 in apply_lambda (fun=144646748, args=146894853, eval_flag=1)
    at eval.c:3135
#21 0x0813d9d3 in Feval (form=146896869) at eval.c:2415
#22 0x0813b359 in Fsetq (args=146896861) at eval.c:552
#23 0x0813d70a in Feval (form=146896853) at eval.c:2302
#24 0x0813d7dd in Feval (form=146896845) at eval.c:2340
#25 0x0813e23f in Ffuncall (nargs=2, args=0xbffead34) at eval.c:3024
#26 0x08161c84 in Fbyte_code (bytestr=136525275, vector=136525292, maxdepth=24)
    at bytecode.c:679
#27 0x0813e73a in funcall_lambda (fun=136525236, nargs=1,
    arg_vector=0xbffeae44) at eval.c:3211
#28 0x0813e359 in Ffuncall (nargs=2, args=0xbffeae40) at eval.c:3081
#29 0x08161c84 in Fbyte_code (bytestr=136525523, vector=136525540, maxdepth=24)
    at bytecode.c:679
#30 0x0813e73a in funcall_lambda (fun=136525484, nargs=1,
    arg_vector=0xbffeaf54) at eval.c:3211
#31 0x0813e359 in Ffuncall (nargs=2, args=0xbffeaf50) at eval.c:3081
#32 0x08161c84 in Fbyte_code (bytestr=136523723, vector=136523740, maxdepth=16)
    at bytecode.c:679
#33 0x0813e73a in funcall_lambda (fun=136523692, nargs=0,
    arg_vector=0xbffeb084) at eval.c:3211
#34 0x0813e359 in Ffuncall (nargs=1, args=0xbffeb080) at eval.c:3081
#35 0x0813df04 in apply1 (fn=137580545, arg=137414769) at eval.c:2765
#36 0x08139bcc in Fcall_interactively (function=137580545,
    record_flag=137414769, keys=137463044) at callint.c:385
#37 0x080edd6d in Fcommand_execute (cmd=137580545, record_flag=137414769,
    keys=137414769, special=137414769) at keyboard.c:10435
#38 0x080e3e99 in command_loop_1 () at keyboard.c:1939
#39 0x0813c6f2 in internal_condition_case (bfun=0x80e31a4 <command_loop_1>,
    handlers=137472161, hfun=0x80e2c70 <cmd_error>) at eval.c:1493
#40 0x080e2f42 in command_loop_2 () at keyboard.c:1396
#41 0x0813c263 in internal_catch (tag=137463729,
    func=0x80e2f24 <command_loop_2>, arg=137414769) at eval.c:1229
#42 0x080e2ed0 in command_loop () at keyboard.c:1375
#43 0x080e28f4 in recursive_edit_1 () at keyboard.c:984
#44 0x080e2a34 in Frecursive_edit () at keyboard.c:1046
#45 0x080e18c9 in main (argc=3, argv=0xbffeb834) at emacs.c:1777

Lisp Backtrace:
"set-match-data" (0xbffea3f4)
"byte-code" (0xbffea480)
"shell-command" (0xbffea6e4)
"diffs-between-depot-and-client-different-branches" (0xbffea804)
"diffs-between" (0xbffea924)
"changesets-between" (0xbffea9e0)
"setq" (0xbffeab68)
"length" (0xbffeac28)
"eval" (0xbffead38)
"eval-last-sexp-1" (0xbffeae44)
"eval-last-sexp" (0xbffeaf54)
"eval-print-last-sexp" (0xbffeb084)
"call-interactively" (0xbffeb230)
(gdb)

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Kalman Reti]

On Nov 16, 2007 7:05 AM, Kalman Reti <kalman.reti@gmail.com> wrote:

>                                                                              One thing that isn't
> clear to me is exactly who is calling set-match-data with the reseat
> argument set to evaporate inside of the shell-command function.  This is
> happening somewhere inside of the shell-command function which my
> code calls.
>

I just figured this part out.  The save-match-data macro generates an
unwind-protect call to set-match-data with 'evaporate as a second argument.

What I haven't figured out is why these are mostly OK.  Perhaps it is just
a garbage collection being kicked of at an inconvenient time?

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[martin rudalics]

Do you mean in code wrapped in `save-match-data' you delete some region
of text containing a marker of the saved match-data.  Thus
record_marker_adjustment puts an entry on `buffer-undo-list' referencing
that marker.  The unwindforms of `save-match-data' call `set-match-data'
with evaporate/reseat non-nil, which calls free_marker and subsequently
free_misc.  mark_object - operating from `buffer-undo-list' - detects
that the object is already free and aborts.

If I understand correctly, this means that either markers used for
saving match-data should not go to `buffer-undo-list' or the "evaporate"
option set by `save-match-data' is inherently broken.

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Kalman Reti]

On Nov 16, 2007 12:28 PM, martin rudalics <rudalics@gmx.at> wrote:
> Do you mean in code wrapped in `save-match-data' you delete some region
> of text containing a marker of the saved match-data.

It isn't in my code, it is in the shell-command function in simple.el, but
essentially this is correct.

Most of the guts of calling the subprocess to generate the output is
inside save-match-data; I don't know exactly what path results in the
markers' getting on the undo list, but if I create a new macro
save-match-data-noevaporate that is identical to the original
minus the 'evaporate argument to set-match-data and use that
inside of shell-command instead of the original, my crash goes away.

>                                                                                 Thus
> record_marker_adjustment puts an entry on `buffer-undo-list' referencing
> that marker.  The unwindforms of `save-match-data' call `set-match-data'
> with evaporate/reseat non-nil, which calls free_marker and subsequently
> free_misc.  mark_object - operating from `buffer-undo-list' - detects
> that the object is already free and aborts.

There is something which causes this not to happen all the time which
I have not yet found.  If you are lucky and this "something" happens before the
next GC, all is well.  I'd been doing exactly the same sorts of shell operations
in elisp functions for years before encountering one big enough to have a 100%
chance of being unlucky.  It does many hundreds of shell operations (perhaps
even thousands, I haven't counted them) taking many minutes.

>
> If I understand correctly, this means that either markers used for
> saving match-data should not go to `buffer-undo-list' or the "evaporate"
> option set by `save-match-data' is inherently broken.
>

My suspicion is that the save-match-data was intended to be wrapped around
very short local uses of markers, not the collection of arbitrary amounts of
shell stdout output...

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Richard Stallman]

    My suspicion is that the save-match-data was intended to be wrapped around
    very short local uses of markers, not the collection of arbitrary amounts of
    shell stdout output...

That's true, but `save-match-data' should work correctly regardless
of what goes on in its body.  This is a real bug.

Thanks for tracking it down.

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Kalman Reti]

On Nov 16, 2007 11:54 PM, Richard Stallman <rms@gnu.org> wrote:
>     My suspicion is that the save-match-data was intended to be wrapped around
>     very short local uses of markers, not the collection of arbitrary amounts of
>     shell stdout output...
>
> That's true, but `save-match-data' should work correctly regardless
> of what goes on in its body.  This is a real bug.
>
> Thanks for tracking it down.
>

You're quite welcome.  BTW, I applied Stefan's search.c diff to a fresh
copy of the CVS sources and successfully ran my test case.

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Stefan Monnier]

> If I understand correctly, this means that either markers used for
> saving match-data should not go to `buffer-undo-list' or the "evaporate"
> option set by `save-match-data' is inherently broken.

The `evaporate' option is inherently dangerous since it reclaims the
marker object forcefully without checking that nobody else is holding on
to it, even tho it comes from a plain normal argument and the caller may
very well have kept another ref to it somewhere.

I suggest we kill it,


        Stefan


--- orig/src/search.c
+++ mod/src/search.c
@@ -3023,10 +3023,7 @@
 
 	    if (!NILP (reseat) && MARKERP (m))
 	      {
-		if (EQ (reseat, Qevaporate))
-		  free_marker (m);
-		else
-		  unchain_marker (XMARKER (m));
+		unchain_marker (XMARKER (m));
 		XSETCAR (list, Qnil);
 	      }
 
@@ -3044,10 +3041,7 @@
 
 	    if (!NILP (reseat) && MARKERP (m))
 	      {
-		if (EQ (reseat, Qevaporate))
-		  free_marker (m);
-		else
-		  unchain_marker (XMARKER (m));
+		unchain_marker (XMARKER (m));
 		XSETCAR (list, Qnil);
 	      }
 	  }
@@ -3111,8 +3105,7 @@
 unwind_set_match_data (list)
      Lisp_Object list;
 {
-  /* It is safe to free (evaporate) the markers immediately.  */
-  return Fset_match_data (list, Qevaporate);
+  return Fset_match_data (list, Qt);
 }
 
 /* Called to unwind protect the match data.  */

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[martin rudalics]

 > The `evaporate' option is inherently dangerous since it reclaims the
 > marker object forcefully without checking that nobody else is holding on
 > to it, even tho it comes from a plain normal argument and the caller may
 > very well have kept another ref to it somewhere.

Indeed.  I faintly recall a discussion about restricting the access to
match data.  Maybe that option was an ill-fated attempt based on the
assumption that no one was allowed to copy a reference to match data or
any of its components.

 > I suggest we kill it,

Don't forget to kill it in emacs_base too.

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Stefan Monnier]

>> The `evaporate' option is inherently dangerous since it reclaims the
>> marker object forcefully without checking that nobody else is holding on
>> to it, even tho it comes from a plain normal argument and the caller may
>> very well have kept another ref to it somewhere.

> Indeed.  I faintly recall a discussion about restricting the access to
> match data.

Using an uninterned Qevaporate would kind of do that, but as we're now
seeing it's not sufficient since those markers may end up in the
undo-log anyway.

> Maybe that option was an ill-fated attempt based on the assumption
> that no one was allowed to copy a reference to match data or any of
> its components.

Sounds like it.

>> I suggest we kill it,

> Don't forget to kill it in emacs_base too.

Indeed,


        Stefan

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Stefan Monnier]

> Don't forget to kill it in emacs_base too.

Done,


        Stefan

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Kalman Reti]

On Nov 16, 2007 5:16 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> > Don't forget to kill it in emacs_base too.
>
> Done,
>
>
>         Stefan
>

I may have been misunderstanding what you meant by "done", but
when I just updated from CVS I didn't get a change to search.c and,
indeed, the code shown as removed in the diff is still there.

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Stefan Monnier]

>> > Don't forget to kill it in emacs_base too.
>> 
>> Done,
>> 
>> 
>> Stefan
>> 

> I may have been misunderstanding what you meant by "done", but
> when I just updated from CVS I didn't get a change to search.c and,
> indeed, the code shown as removed in the diff is still there.

It's only installed in the 22 branch for now.  It'll get merged into the
trunk at some point in the future.


        Stefan

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Richard Stallman]

    The `evaporate' option is inherently dangerous since it reclaims the
    marker object forcefully without checking that nobody else is holding on
    to it, even tho it comes from a plain normal argument and the caller may
    very well have kept another ref to it somewhere.

There are low-level constructs in Emacs that can crash Emacs if misused.
This mere fact is not enough reason to get rid of it.

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Stefan Monnier]

>     The `evaporate' option is inherently dangerous since it reclaims the
>     marker object forcefully without checking that nobody else is holding on
>     to it, even tho it comes from a plain normal argument and the caller may
>     very well have kept another ref to it somewhere.

> There are low-level constructs in Emacs that can crash Emacs if misused.
> This mere fact is not enough reason to get rid of it.

It's an "optimization" and nothing more.  In my book, if an optimization
is unsafe, it had better make a good case for itself.  As it stands
I see no evidence that this optimization is ever useful.  As long as
nobody can show us numbers that demonstrate a measurable performance
impact, I think we're better off without this optimization.


        Stefan

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Richard Stallman]

    It's an "optimization" and nothing more.  In my book, if an optimization
    is unsafe, it had better make a good case for itself.  As it stands
    I see no evidence that this optimization is ever useful.  As long as
    nobody can show us numbers that demonstrate a measurable performance
    impact, I think we're better off without this optimization.

Yes, that's the crucial question.  It should be easy to get some numbers
by running an interactive application that often uses save-match-data
and compare the memory usage and amount of GC of the two versions.

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     It's an "optimization" and nothing more.  In my book, if an
>     optimization is unsafe, it had better make a good case for itself.
>     As it stands I see no evidence that this optimization is ever
>     useful.  As long as nobody can show us numbers that demonstrate a
>     measurable performance impact, I think we're better off without
>     this optimization.
>
> Yes, that's the crucial question.  It should be easy to get some
> numbers by running an interactive application that often uses
> save-match-data and compare the memory usage and amount of GC of the
> two versions.

The problem is not the memory usage: garbage collection will set in
anyway when the memory is tight.

The problem is that editing becomes awfully slow in a buffer with many
markers.  And temporary markers created with save-match-data will only
be unseated from the buffer once they get collected.

Perhaps it would be a useful idea to have the "evaporate" argument only
unseat the markers from the buffer (the equivalent of (move-marker
marker nil)) without garbage-collecting it.

I think that should meet the main performance objective without
introducing any spurious "free" potential.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Stefan Monnier]

>> It's an "optimization" and nothing more.  In my book, if an
>> optimization is unsafe, it had better make a good case for itself.
>> As it stands I see no evidence that this optimization is ever
>> useful.  As long as nobody can show us numbers that demonstrate a
>> measurable performance impact, I think we're better off without
>> this optimization.
>> 
>> Yes, that's the crucial question.  It should be easy to get some
>> numbers by running an interactive application that often uses
>> save-match-data and compare the memory usage and amount of GC of the
>> two versions.

Someone needs to write the "optimized" but correct version first.

> The problem is not the memory usage: garbage collection will set in
> anyway when the memory is tight.

> The problem is that editing becomes awfully slow in a buffer with many
> markers.  And temporary markers created with save-match-data will only
> be unseated from the buffer once they get collected.

No: the current code (where I removed the `evaporate' optimization)
still unseats the markers right away.  It just doesn't free them.

> Perhaps it would be a useful idea to have the "evaporate" argument only
> unseat the markers from the buffer (the equivalent of (move-marker
> marker nil)) without garbage-collecting it.

That's already what the `unseat' argument does (which used to accept
a special `evaporate' value to imply that it shouldn't just unseat but
also free).


        Stefan

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Richard Stallman]

    > Yes, that's the crucial question.  It should be easy to get some
    > numbers by running an interactive application that often uses
    > save-match-data and compare the memory usage and amount of GC of the
    > two versions.

    The problem is not the memory usage: garbage collection will set in
    anyway when the memory is tight.

In Emacs, GC doesn't occur when memory is scarce; Emacs doesn't
measure that.  GC occurs based on the amount of consing done.  If the
markers are not freed, Emacs will allocate more space for markers.

    The problem is that editing becomes awfully slow in a buffer with many
    markers.  And temporary markers created with save-match-data will only
    be unseated from the buffer once they get collected.

That is also a factor, indeed.

    Perhaps it would be a useful idea to have the "evaporate" argument only
    unseat the markers from the buffer (the equivalent of (move-marker
    marker nil)) without garbage-collecting it.

That would be the basic RESEAT functionality in `set-match-data'.
Perhaps it is the right thing to do, but I still wonder how much
extra space the extra GCs will cause.

It occurs to me that free_marker could un-count the consing of that
marker, so that GC would also happen less often.

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Richard Stallman]

    If I understand correctly, this means that either markers used for
    saving match-data should not go to `buffer-undo-list' or the "evaporate"
    option set by `save-match-data' is inherently broken.

I think `evaporate' is broken.

We could arrange a way to avoid putting these markers in
`buffer-undo-list', but then they would not be relocated correctly if
the body does some undoing.  So that is not a solution.  And once the
marker is in `buffer-undo-list', the Lisp code can get ahold of it.

The simplest solution is to make `evaporate' not do any thing special.

Here's a second solution.  Define a flag bit in markers, and set that
bit when relocation of the marker puts it into `buffer-undo-list'.
`set-match-data' can free the marker if the bit is not set.

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Richard Stallman]

Nothing gets "removed" from the undo list in normal use.  It gets
truncated, which drops off elements at the end, but other than that
all that normally happens is that editing operations add elements.

Markers in the list should not become free, because the undo list
itself should preserve them from GC.

If this is reproducible, can you put a breakpoint at Fgarbage_collect
and examine the data just before the GC which gets this crash?
Examine that list using the x... commands, and see if that marker
is already free.

    Looking harder at the code, I'm convinced that the undo_list should come before
    the name entry in the buffer structure,

Definitely not.  It needs to be AFTER `name' so that it will be marked
by GC.

    Anyone know what the elements of the undo_list mean?  Some are conses
    with a marker
    in their CAR and a number in their CDR, some are just conses of two
    numbers and some
    are conses of a string and a number.

The Lisp Manual documents these.  Node `Undo'.

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Kalman Reti]

On Nov 14, 2007 10:08 PM, Richard Stallman <rms@gnu.org> wrote:
> Nothing gets "removed" from the undo list in normal use.  It gets
> truncated, which drops off elements at the end, but other than that
> all that normally happens is that editing operations add elements.
>
> Markers in the list should not become free, because the undo list
> itself should preserve them from GC.
>
> If this is reproducible, can you put a breakpoint at Fgarbage_collect
> and examine the data just before the GC which gets this crash?
> Examine that list using the x... commands, and see if that marker
> is already free.
>
>     Looking harder at the code, I'm convinced that the undo_list should come before
>     the name entry in the buffer structure,
>
> Definitely not.  It needs to be AFTER `name' so that it will be marked
> by GC.

There is special code at the end of Fgarbage_collect (just before the
call to gc_sweep) which seems like it would have no point if this were
true.  It removes elements referring to unmarked markers and then
explicitly marks the undo_list slot afterwards.  The comment there reads:

	/* Now that we have stripped the elements that need not be in the
	   undo_list any more, we can finally mark the list.  */
	mark_object (nextb->undo_list);

It seems to me that if the undo_list were after name, then all the markers
in the list would have already been marked and this code would be an
elaborate no-op, no?

>
>     Anyone know what the elements of the undo_list mean?  Some are conses
>     with a marker
>     in their CAR and a number in their CDR, some are just conses of two
>     numbers and some
>     are conses of a string and a number.
>
> The Lisp Manual documents these.  Node `Undo'.

Thanks.  Someone already pointed me at the documentation string for
buffer-undo-list.

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Kalman Reti]

On Nov 15, 2007 3:38 AM, Kalman Reti <kalman.reti@gmail.com> wrote:
> On Nov 14, 2007 10:08 PM, Richard Stallman <rms@gnu.org> wrote:
> >
> > Definitely not.  It needs to be AFTER `name' so that it will be marked
> > by GC.

I've performed the experiment of building code straight from CVS and
putting a breakpoint in the special code for handling un-gc-marked-markers.

In my (long running previously resulting in a crash) test case,
this breakpoint NEVER is reached.

When I move the undo_list to before name and redo the experiment, I hit
the breakpoint many many times.

So either the special undo_list handling code should be removed or the
undo_list moved before name in buffer.h.

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Stefan Monnier]

> When I move the undo_list to before name and redo the experiment, I hit
> the breakpoint many many times.

> So either the special undo_list handling code should be removed or the
> undo_list moved before name in buffer.h.

Agreed.  The field was moved by Richard on 14-Oct-2002 but the change
log doesn't say why this was done, so I just undid it.


        Stefan

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[martin rudalics]

> Agreed.  The field was moved by Richard on 14-Oct-2002 but the change
> log doesn't say why this was done, so I just undid it.

Does this mean those cells always survived the current cycle?
Then we now have a chance to test whether the "remove unmarked
markers from the undo list" stuff really works in one and the
same collection cycle.  Interesting.

Stefan, unless you have already done so, could you please fix
those identic

"If a buffer's undo list is Qt, ..."

comments in alloc.c too?  The second mentions truncate_undo_list
which hardly makes sense.

buffer-undo-list (was: mark_object crash in 22.1 and latest CVS (as of tonight))

[Johan Bockgård]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> When I move the undo_list to before name and redo the experiment, I hit
>> the breakpoint many many times.
>
>> So either the special undo_list handling code should be removed or the
>> undo_list moved before name in buffer.h.
>
> Agreed.  The field was moved by Richard on 14-Oct-2002 but the change
> log doesn't say why this was done, so I just undid it.

But now

  (assq 'buffer-undo-list (buffer-local-variables))
    => nil

  (buffer-local-value 'buffer-undo-list (get-buffer "*scratch*"))
    => nil (global value)

-- 
Johan Bockgård

Re: buffer-undo-list

[Stefan Monnier]

>>> When I move the undo_list to before name and redo the experiment, I hit
>>> the breakpoint many many times.
>> 
>>> So either the special undo_list handling code should be removed or the
>>> undo_list moved before name in buffer.h.
>> 
>> Agreed.  The field was moved by Richard on 14-Oct-2002 but the change
>> log doesn't say why this was done, so I just undid it.

> But now

>   (assq 'buffer-undo-list (buffer-local-variables))
>     => nil

>   (buffer-local-value 'buffer-undo-list (get-buffer "*scratch*"))
>     => nil (global value)

That makes sense, thanks.  I'll try and come up with a proper fix.


        Stefan

Re: buffer-undo-list

[Stefan Monnier]

>>>>> "Johan" == Johan Bockgård <bojohan> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>> When I move the undo_list to before name and redo the experiment, I hit
>>> the breakpoint many many times.
>> 
>>> So either the special undo_list handling code should be removed or the
>>> undo_list moved before name in buffer.h.
>> 
>> Agreed.  The field was moved by Richard on 14-Oct-2002 but the change
>> log doesn't say why this was done, so I just undid it.

> But now

>   (assq 'buffer-undo-list (buffer-local-variables))
>     => nil

>   (buffer-local-value 'buffer-undo-list (get-buffer "*scratch*"))
>     => nil (global value)

I installed the patch below which should fix these problems.


        Stefan


Index: src/buffer.c
===================================================================
RCS file: /sources/emacs/emacs/src/buffer.c,v
retrieving revision 1.550
diff -u -r1.550 buffer.c
--- src/buffer.c	10 Feb 2008 02:14:00 -0000	1.550
+++ src/buffer.c	11 Feb 2008 17:54:48 -0000
@@ -496,7 +496,9 @@
 
   XSETBUFFER (to_buffer, to);
 
-  for (offset = PER_BUFFER_VAR_OFFSET (name) + sizeof (Lisp_Object);
+  /* buffer-local Lisp variables start at `undo_list',
+     tho only the ones from `name' on are GC'd normally.  */
+  for (offset = PER_BUFFER_VAR_OFFSET (undo_list) + sizeof (Lisp_Object);
        offset < sizeof *to;
        offset += sizeof (Lisp_Object))
     {
@@ -808,7 +810,9 @@
   /* For each slot that has a default value,
      copy that into the slot.  */
 
-  for (offset = PER_BUFFER_VAR_OFFSET (name);
+  /* buffer-local Lisp variables start at `undo_list',
+     tho only the ones from `name' on are GC'd normally.  */
+  for (offset = PER_BUFFER_VAR_OFFSET (undo_list);
        offset < sizeof *b;
        offset += sizeof (Lisp_Object))
     {
@@ -940,7 +944,9 @@
       int found = 0;
 
       /* Look in special slots */
-      for (offset = PER_BUFFER_VAR_OFFSET (name);
+      /* buffer-local Lisp variables start at `undo_list',
+	 tho only the ones from `name' on are GC'd normally.  */
+      for (offset = PER_BUFFER_VAR_OFFSET (undo_list);
 	   offset < sizeof (struct buffer);
 	   /* sizeof EMACS_INT == sizeof Lisp_Object */
 	   offset += (sizeof (EMACS_INT)))
@@ -1051,7 +1057,9 @@
   {
     int offset, idx;
 
-    for (offset = PER_BUFFER_VAR_OFFSET (name);
+    /* buffer-local Lisp variables start at `undo_list',
+       tho only the ones from `name' on are GC'd normally.  */
+    for (offset = PER_BUFFER_VAR_OFFSET (undo_list);
 	 offset < sizeof (struct buffer);
 	 /* sizeof EMACS_INT == sizeof Lisp_Object */
 	 offset += (sizeof (EMACS_INT)))

Re: mark_object crash in 22.1 and latest CVS (as of tonight)

[Richard Stallman]

      I assumed
    someone would recognize WHAT part of a buffer from the contents of the,
    list, a mixture of conses with marker-in-no-buffer in the car of some and
    Lisp_Misc_Free  in the car of others, the cdr's being negative numbers
    of pretty small absolute magnitude.

I didn't see that when I looked at the other message.  Can anyone
guess what data this is?

    > Once you answer those, you can try to figure out how it happened
    > that the data structure ended up with a bad pointer.
    > Maybe GC failed to mark that pointer, so the misc object got freed
    > even though it was still in use.

    Are there any tools to help with this, e.g. an allocation trace or GC trace?
    I'm afraid this is the first time I've looked at the Emacs src code.

The x... GDB commands in .gdbinit are useful for examining data
structures during GC.

`last_marked' and `last_marked_index' keep track of the sequence of
data objects that were marked.  You can use that to determine precisely
how the bad data was reached.