From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Linas Vepstas Newsgroups: gmane.lisp.guile.devel Subject: Re: Locks and threads Date: Wed, 11 Feb 2009 18:18:39 -0600 Message-ID: <3ae3aa420902111618t2d483198ked8defaf3e36039f@mail.gmail.com> References: <87mycsd2rj.fsf@arudy.ossau.uklinux.net> <3ae3aa420902111530k671cd331i308915e0efaf75bb@mail.gmail.com> <87eiy4cyz0.fsf@arudy.ossau.uklinux.net> Reply-To: linasvepstas@gmail.com NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary=0016e645ba527c98ad0462ada940 X-Trace: ger.gmane.org 1234397939 26480 80.91.229.12 (12 Feb 2009 00:18:59 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 12 Feb 2009 00:18:59 +0000 (UTC) Cc: Guile Development To: Neil Jerram Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Thu Feb 12 01:20:14 2009 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1LXPJN-0000AO-5E for guile-devel@m.gmane.org; Thu, 12 Feb 2009 01:20:13 +0100 Original-Received: from localhost ([127.0.0.1]:36707 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1LXPI3-0003OE-IR for guile-devel@m.gmane.org; Wed, 11 Feb 2009 19:18:51 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1LXPHu-0003LL-QZ for guile-devel@gnu.org; Wed, 11 Feb 2009 19:18:42 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1LXPHt-0003Ip-Es for guile-devel@gnu.org; Wed, 11 Feb 2009 19:18:42 -0500 Original-Received: from [199.232.76.173] (port=34357 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1LXPHt-0003Ib-AS for guile-devel@gnu.org; Wed, 11 Feb 2009 19:18:41 -0500 Original-Received: from yw-out-1718.google.com ([74.125.46.156]:12901) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1LXPHs-0006o3-HI for guile-devel@gnu.org; Wed, 11 Feb 2009 19:18:40 -0500 Original-Received: by yw-out-1718.google.com with SMTP id 6so283816ywa.66 for ; Wed, 11 Feb 2009 16:18:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:reply-to:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=SRGmHWnhXnUOHt4EYTQ2/fO6XbGJI3sM+8tV/9CM+MM=; b=q/5XYeRFf0177fucZNSyjdY1pnlt+LuIuFhrBEKz6XVH5uSbPaIklc4mWDIvreYnx3 B7z0tLoZbutEy5OvC5Y7I0EVJyW+XCoLo1nI3HWjo47SmHt3VMTLKNuynapyYvfp5hGl Kn3kPAguP5HnkbHOgdi0GxtImY2x+oxxnVGuY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; b=B8PeEOcBILkHZGwwqOYjq8W7P2NqSRH8sKDjHDCGEEIZrckTZDBf6ZlED0wLOe4aTC 2vC9FrWyC8LerNGI50avlSEqrcQcXKHoUoDgDjrwpllPzhmlcGaOfHTWYF0XMv6YlGzC HQojgF1Ua+kg/3+nk8usmFngY0tI+jRTriNC4= Original-Received: by 10.100.93.19 with SMTP id q19mr379764anb.156.1234397919221; Wed, 11 Feb 2009 16:18:39 -0800 (PST) In-Reply-To: <87eiy4cyz0.fsf@arudy.ossau.uklinux.net> X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6 (newer, 2) X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:8157 Archived-At: --0016e645ba527c98ad0462ada940 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit 2009/2/11 Neil Jerram : > Linas Vepstas writes: > >> Err, sort of, yes, unless I misunderstand. Guile 1.8 makes >> a certain basic assumption that is splattered throughout >> the code; it rather intentionally re-orders the order in which >> one of the locks is taken. If I remember correctly, its the >> "in guile mode" lock. So if you just go looking for locks >> that are released out-of-order, you'll find lots of these. > > Yes, I think I understand this now (having seen it myself). The > pattern is > > - thread holding its heap_mutex - which is the normal state in guile Yes, that's the one. > That in itself doesn't actually cause an ordering problem, but then > the thread releases the other mutex without releasing the heap mutex > first - which is perceived (by helgrind at least) as a problem. > > (Is something like this actually _ever_ a problem? If locks are > always _acquired_ in the right order, how can the order of _releasing_ > ever cause a problem?) Yes, it can be a problem; I don't want to dream up some particular scenario (this stuff destroys brain cells) ... but I do vaguely remember skimming some wikipedia article on locking that discussed this. I think the scenario involves three locks, though. This is why helgrind checks lock order -- its one of the locking problems it can actually detect. However, for the case of guile, the heap mutex is not visible to anything that isn't guile, and thus, its safe in this particular case. If there were outside users, things would be different. > The other thing to bear in mind is that 99% of this will just > evaporate if we move to BDW-GC for 2.0.x; so - assuming we do end up > doing that - it makes sense to take a slightly more pragmatic approach > than normal for 1.8.x. Sure. As a reminder ... the only real remaining problem that remained with a race to update some hash table, when define was being used from several threads. *thats* the bug that needs attention (but is hard to fix). > [1] I am only running a basic startup test, though: "valgrind > --tool=helgrind guile -q < more complex than that? I had written some simple test case, which I think sprouted a bunch of threads, and then did simple scheme things in each .. e.g. just adding numbers, or whatever. I'm attaching some kind of simple test case to this email however, it is very hacked, so I don't know if it actually will find bugs, and its probably doesn't do what it claims to do. I provide it only as a short-cut for creating a new test case. ... --linas --0016e645ba527c98ad0462ada940 Content-Type: text/x-csrc; charset=US-ASCII; name="thread-exp.c" Content-Disposition: attachment; filename="thread-exp.c" Content-Transfer-Encoding: base64 X-Attachment-Id: f_fr2oljgi0 LyoqCiAqIEd1aWxlIHRocmVhZGluZyBidWcvdW5leHBlY3RlZCBiZWhhdmlvdXIuCiAqCiAqIFR3 byBwb3NpeCB0aHJlYWRzIGFyZSBjcmVhdGVkLiAgVGhlIGZpcnN0IHRocmVhZCBydW5zLCBhbmQg ZGVmaW5lcwogKiBzb21ldGhpbmcuIFRoZSBzZWNvbmQgdGhyZWFkIHRyaWVzIHRvIGFjY2VzcyB0 aGUgdGhpbmcgZGVmaW5lZCBieSAKICogdGhlIGZpcnN0LiBCdXQsIGR1ZSB0byBzb21lICJ1bmV4 cGVjdGVkIiB0aHJlYWRpbmcgYmVoYXZpb3VyLCB0aGUKICogdGhpbmdzIGRlZmluZWQgaW4gdGhl IGZpcnN0IHRocmVhZCBhcmUgbm90IHZpc2libGUgdG8gdGhlIGZpcnN0LgogKiBJJ20gcHJldHR5 IGNvbnZpbmNlZCB0aGlzIGlzIGEgYnVnLCBhbmQgYW0gaHVudGluZyBpdCBkb3duIG5vdy4KICoK ICogVGhlIHByaW50ZWQgb3V0cHV0IG9mIHRoaXMgcHJvZ3JhbSwgZm9yIG1lLCB1bmRlciBndWls ZTEuOC41LCBpcwogKgogKiBIRWxsbywgdGhyZWFkIG9uZSBpcyBydW5uaW5nCiAqIEdvb2RieWUs IHRocmVhZCBvbmUgaXMgZXhpdGluZwogKiBIRWxsbywgdGhyZWFkIHR3byBpcyBydW5uaW5nCiAq IEVSUk9SOiBVbmJvdW5kIHZhcmlhYmxlOiB4CiAqIEdvb2RieWUsIHRocmVhZCB0d28gaXMgZXhp dGluZwogKiBNYWluIGlzIGV4aXRpbmcKICoKICogVG8gZ2V0IGEgYmV0dGVyIGlkZWEgb2Ygd2hh dCdzIGdvaW5nIG9uLCBpdCBzZWVtcyB0byBoYXZlIHNvbWV0aGluZwogKiB0byBkbyB3aXRoIHRo ZSBjdXJyZW50IG1vZHVsZS4gQnkgdHVybmluZyBvbiBwcmludCBkZWJ1Z2dpbmcsIHRoZSAKICog b3V0cHV0LCBmb3IgbWUsIGJlY29tZXM6CiAqIAogKiBIRWxsbywgdGhyZWFkIG9uZSBpcyBydW5u aW5nCiAqIHRocmVhZCBvbmUgaGFkIG91dHB1dDoKICogdGhlIHJvb3QgbW9kdWxlIGlzICM8bW9k dWxlIChndWlsZSkgZjczZjllODA+CiAqIHRoZSBjdXJyZW50IG1vZHVsZSBpcyAjPGRpcmVjdG9y eSAoZ3VpbGUtdXNlcikgZjczZmM2MDA+CiAqIAogKiBHb29kYnllLCB0aHJlYWQgb25lIGlzIGV4 aXRpbmcKICogSEVsbG8sIHRocmVhZCB0d28gaXMgcnVubmluZwogKiB0aHJlYWQgdHdvIGhhZCBv dXRwdXQ6CiAqIHRoZSByb290IG1vZHVsZSBpcyAjPG1vZHVsZSAoZ3VpbGUpIGY3M2Y5ZTgwPgog KiB0aGUgY3VycmVudCBtb2R1bGUgaXMgIzxtb2R1bGUgKGd1aWxlKSBmNzNmOWU4MD4KICogCiAq IEVSUk9SOiBVbmJvdW5kIHZhcmlhYmxlOiB4CiAqIEdvb2RieWUsIHRocmVhZCB0d28gaXMgZXhp dGluZwogKiBNYWluIGlzIGV4aXRpbmcKICovCgoKI2luY2x1ZGUgPHB0aHJlYWQuaD4KI2luY2x1 ZGUgPHN0ZGlvLmg+CiNpbmNsdWRlIDxzdGRsaWIuaD4KI2luY2x1ZGUgPGxpYmd1aWxlLmg+CgpT Q00gb3V0cG9ydDsKCnZvaWQgcHJ0ZGJnKGNvbnN0IGNoYXIgKiBpZCkKewojZGVmaW5lIFNIT1df U1RVRkYgMQojaWZkZWYgU0hPV19TVFVGRgoKCWNoYXIgYnVmZlsxMDBdOwoJc3ByaW50ZihidWZm LCAiKGRpc3BsYXkgXCJkdXV1ZGUgaWQgJXNcXG5cIilcbiIsIGlkKTsKCXNjbV9jX2V2YWxfc3Ry aW5nIChidWZmKTsKCXNjbV9jX2V2YWxfc3RyaW5nICgiKGRpc3BsYXkgXCJ0aGUgcm9vdCBtb2R1 bGUgaXMgXCIpXG4iKTsKCXNjbV9jX2V2YWxfc3RyaW5nICgiKGRpc3BsYXkgdGhlLXJvb3QtbW9k dWxlKVxuIik7CglzY21fY19ldmFsX3N0cmluZyAoIihuZXdsaW5lKVxuIik7CglzY21fY19ldmFs X3N0cmluZyAoIihkaXNwbGF5IFwidGhlIGN1cnJlbnQgbW9kdWxlIGlzIFwiKVxuIik7CglzY21f Y19ldmFsX3N0cmluZyAoIihkaXNwbGF5IChjdXJyZW50LW1vZHVsZSkpXG4iKTsKCXNjbV9jX2V2 YWxfc3RyaW5nICgiKG5ld2xpbmUpXG4iKTsKCiAgIFNDTSBvdXQgPSBzY21fZ2V0X291dHB1dF9z dHJpbmcob3V0cG9ydCk7CiAgIGNoYXIgKiBzdHIgPSBzY21fdG9fbG9jYWxlX3N0cmluZyhvdXQp OwogICBzY21fdHJ1bmNhdGVfZmlsZShvdXRwb3J0LCBzY21fZnJvbV91aW50MTYoMCkpOwogCiAg IHByaW50ZigiJXMgaGFkIG91dHB1dDpcbiVzXG4iLCBpZCwgc3RyKTsKI2VuZGlmCn0KCnZvaWQg KiBzY21fb25lICh2b2lkICpwKQp7CglvdXRwb3J0ID0gc2NtX29wZW5fb3V0cHV0X3N0cmluZygp OwoJc2NtX3NldF9jdXJyZW50X291dHB1dF9wb3J0KG91dHBvcnQpOwoJcHJ0ZGJnKCJ0aHJlYWQg b25lIik7CgkvLyBzY21fY19ldmFsX3N0cmluZyAoIihzZXQtY3VycmVudC1tb2R1bGUgdGhlLXJv b3QtbW9kdWxlKVxuIik7CgkvLyBwcnRkYmcoInRocmVhZCBvbmUgYWdhaW4iKTsKCXNjbV9jX2V2 YWxfc3RyaW5nICgiKGRlZmluZSB4IFwiYXNkZGZcIilcbiIpOwp9Cgp2b2lkICogc2NtX3R3byAo dm9pZCAqcCkKewoJc2NtX3NldF9jdXJyZW50X291dHB1dF9wb3J0KG91dHBvcnQpOwoJcHJ0ZGJn KCJ0aHJlYWQgdHdvIik7CgkvLyBzY21fY19ldmFsX3N0cmluZyAoIihkaXNwbGF5IHgpXG4iKTsK CXNjbV9jX2V2YWxfc3RyaW5nICgiKGRpc3BsYXkgXCJkdXV1dXV1dXV1dXV1YXV1dXV1dXV1dXV1 dWRlXCIpXG4iKTsKCXNjbV9jX2V2YWxfc3RyaW5nICgiKGRlZmluZSAoZmFjdCBuKSAoaWYgKD0g biAxKSAxICgqIG4gKGZhY3QgKC0gbiAxKSkpKSlcbiIKCQkiKGRpc3BsYXkgXCJ3YWhhenp6dXBw cFxcblwiKVxuIgoJCSIoZGlzcGxheSAoZmFjdCAzNjkpKVxuIgoJCSIoZGlzcGxheSBcInllYWhc XG5cIilcbiIpOwp9Cgp2b2lkICogc2NtX3RocmVlICh2b2lkICpwKQp7CglzY21fc2V0X2N1cnJl bnRfb3V0cHV0X3BvcnQob3V0cG9ydCk7CglwcnRkYmcoInRocmVhZCB0aHJlZSIpOwoJc2NtX2Nf ZXZhbF9zdHJpbmcgKCIoZGlzcGxheSB4KVxuIik7Cn0KCnZvaWQgKiBzY21fZm91ciAodm9pZCAq cCkKewoJcHJ0ZGJnKCJ0aHJlYWQgZm91ciIpOwoJc2NtX2NfZXZhbF9zdHJpbmcgKCIoZGlzcGxh eSB4KVxuIik7Cn0KCnN0YXRpYyB2b2lkICogdGhyZWFkX3plcm8gKHZvaWQgKiBhcmcpCnsKCXBy aW50ZigiVGhyZWFkIHplcm9vb29vbyBcbiIpOwoJc2NtX3dpdGhfZ3VpbGUoc2NtX29uZSwgTlVM TCk7CglwcmludGYoIlRocmVhZCBleGl0IFxuIik7CglyZXR1cm4gTlVMTDsKfQoKc3RhdGljIHZv aWQgKiB0aHJlYWRfb25lICh2b2lkICogYXJnKQp7CglwcmludGYoIkhFbGxvLCB0aHJlYWQgb25l IGlzIHJ1bm5pbmdcbiIpOwoKCXB0aHJlYWRfYXR0cl90IGF0dHI7CglwdGhyZWFkX3QgdDA7Cglw dGhyZWFkX2F0dHJfaW5pdCgmYXR0cik7CgoJLy8gcHRocmVhZF9jcmVhdGUoJnQwLCAmYXR0ciwg dGhyZWFkX3plcm8sIE5VTEwpOwoJc2NtX3dpdGhfZ3VpbGUoc2NtX29uZSwgTlVMTCk7CgoJcHJp bnRmKCJHb29kYnllLCB0aHJlYWQgb25lIGlzIGV4aXRpbmdcbiIpOwoJcmV0dXJuIE5VTEw7Cn0K CnN0YXRpYyB2b2lkICogdGhyZWFkX3R3byAodm9pZCAqIGFyZykKewoJcHJpbnRmKCJIRWxsbywg dGhyZWFkIHR3byBpcyBydW5uaW5nXG4iKTsKCXNjbV93aXRoX2d1aWxlKHNjbV90d28sIE5VTEwp OwoJcHJpbnRmKCJHb29kYnllLCB0aHJlYWQgdHdvIGlzIGV4aXRpbmdcbiIpOwoJcmV0dXJuIE5V TEw7Cn0KCnN0YXRpYyB2b2lkICogdGhyZWFkX3RocmVlICh2b2lkICogYXJnKQp7CglwcmludGYo IkhFbGxvLCB0aHJlYWQgdGhyZWUgaXMgcnVubmluZ1xuIik7CglzY21fd2l0aF9ndWlsZShzY21f dGhyZWUsIE5VTEwpOwoJcHJpbnRmKCJHb29kYnllLCB0aHJlYWQgdGhyZWUgaXMgZXhpdGluZ1xu Iik7CglyZXR1cm4gTlVMTDsKfQoKc3RhdGljIHZvaWQgKiB0aHJlYWRfZm91ciAodm9pZCAqIGFy ZykKewoJcHJpbnRmKCJIRWxsbywgdGhyZWFkIGZvdXIgaXMgcnVubmluZ1xuIik7CglzY21fd2l0 aF9ndWlsZShzY21fZm91ciwgTlVMTCk7CglwcmludGYoIkdvb2RieWUsIHRocmVhZCBmb3VyIGlz IGV4aXRpbmdcbiIpOwoJcmV0dXJuIE5VTEw7Cn0KCm1haW4oKQp7CglpbnQgcmM7CglwdGhyZWFk X2F0dHJfdCBhdHRyOwoJcHRocmVhZF90IHQxLCB0MiwgdDMsIHQ0OwoKCXB0aHJlYWRfYXR0cl9p bml0KCZhdHRyKTsKCglyYyA9IHB0aHJlYWRfY3JlYXRlKCZ0MSwgJmF0dHIsIHRocmVhZF9vbmUs IE5VTEwpOwoJaWYocmMpCgl7CgkJZnByaW50ZihzdGRlcnIsICJGYXRhbCBlcnJvcjogY2FuJ3Qg Y3JlYXRlIHRocmVhZCBvbmVcbiIpOwoJCWV4aXQoMSk7Cgl9CglzbGVlcCgxKTsKCXJjID0gcHRo cmVhZF9jcmVhdGUoJnQyLCAmYXR0ciwgdGhyZWFkX3R3bywgTlVMTCk7CglpZihyYykKCXsKCQlm cHJpbnRmKHN0ZGVyciwgIkZhdGFsIGVycm9yOiBjYW4ndCBjcmVhdGUgdGhyZWFkIHR3b1xuIik7 CgkJZXhpdCgxKTsKCX0KCglzbGVlcCgxKTsKCXJjID0gcHRocmVhZF9jcmVhdGUoJnQzLCAmYXR0 ciwgdGhyZWFkX3RocmVlLCBOVUxMKTsKCWlmKHJjKQoJewoJCWZwcmludGYoc3RkZXJyLCAiRmF0 YWwgZXJyb3I6IGNhbid0IGNyZWF0ZSB0aHJlYWQgdGhyZWVcbiIpOwoJCWV4aXQoMSk7Cgl9Cglz bGVlcCgxKTsKCXJjID0gcHRocmVhZF9jcmVhdGUoJnQ0LCAmYXR0ciwgdGhyZWFkX2ZvdXIsIE5V TEwpOwoKCXNsZWVwKDUpOwoJcHJpbnRmKCJNYWluIGlzIGV4aXRpbmdcbiIpOwp9Cg== --0016e645ba527c98ad0462ada940--