* bug#19180: guile bug#19180: vacuum_weak_hash_table error [not found] <CAHrUA3554+87CVKsW2OT+-roVeyK4sE+xyDPaGf4ar5fAWQ2+Q@mail.gmail.com> @ 2014-12-08 7:34 ` Anand Mohanadoss 2014-12-08 9:34 ` Ludovic Courtès 0 siblings, 1 reply; 9+ messages in thread From: Anand Mohanadoss @ 2014-12-08 7:34 UTC (permalink / raw) To: linasvepstas, Ludovic Courtès, 19180 [-- Attachment #1: Type: text/plain, Size: 1133 bytes --] Dear Linas, Thank you so much! I did indeed miss your earlier responses and I appreciate your reaching out to me! I downloaded the current guile code in git and will give it a try. But, given that this is not a stable release, I am not sure if we can really use it for our purposes. Do you or Ludovic know when a stable 2.2 version will be released? From stability point of you, would it be better to use a patched version of 2.0.11 that handles the case we noticed (e.g. don't attempt to remove items from hash table if len = 0)? Thanks, Anand On Sun, Dec 7, 2014 at 7:22 AM, Linas Vepstas <linasvepstas@gmail.com> wrote: > emailing you directly, in case you did not see it: > > I've seen the same bug; I tried to apply Ludo's patch, and discovered that > the current guile code, in git, no longer even has the > scm_fixup_weak_alist or the vacuum_weak_hash_table calls anywhere in > them. Also .. my version of the bug goes away. > > The code in git seems to be guile-2.2, which has not yet been released -- > so your mileage may vary.. but you certainly won't get that error print > message any more. > > -- Linas > [-- Attachment #2: Type: text/html, Size: 1676 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#19180: guile bug#19180: vacuum_weak_hash_table error 2014-12-08 7:34 ` bug#19180: guile bug#19180: vacuum_weak_hash_table error Anand Mohanadoss @ 2014-12-08 9:34 ` Ludovic Courtès 2014-12-15 6:36 ` Anand Mohanadoss 0 siblings, 1 reply; 9+ messages in thread From: Ludovic Courtès @ 2014-12-08 9:34 UTC (permalink / raw) To: Anand Mohanadoss; +Cc: 19180, linasvepstas Anand Mohanadoss <anand108@gmail.com> skribis: > I downloaded the current guile code in git and will give it a try. But, > given that this is not a stable release, I am not sure if we can really use > it for our purposes. > > Do you or Ludovic know when a stable 2.2 version will be released? From > stability point of you, would it be better to use a patched version of > 2.0.11 that handles the case we noticed (e.g. don't attempt to remove items > from hash table if len = 0)? There is not ETA for 2.2. For now, I would recommend using the ‘stable-2.0’ branch, which corresponds to what will (hopefully soon) become 2.0.12. Thanks, Ludo’. ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#19180: guile bug#19180: vacuum_weak_hash_table error 2014-12-08 9:34 ` Ludovic Courtès @ 2014-12-15 6:36 ` Anand Mohanadoss 2016-06-22 14:55 ` Andy Wingo 0 siblings, 1 reply; 9+ messages in thread From: Anand Mohanadoss @ 2014-12-15 6:36 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 19180, Linas Vepstas [-- Attachment #1: Type: text/plain, Size: 6752 bytes --] Dear Ludovic, Given your recommendation to use the 'stable-2.0' branch, we tried to create a fix for the issue with our limited knowledge (assuming that the size of the hash is getting corrupted by multiple threads). The process ran to completion, but, we got different results compared to what we get with threading disabled. So, it didn't really help and our assumption appears to be incorrect. Is there something else we can try, to help you with creating a proper patch for this issue? Here is what we changed in hashtab.c - 130a131 > size_t orig_len = len; 137,138c138,144 < assert (removed <= len); < len -= removed; --- > if (removed <= len) > len -= removed; > else > { > printf ("Vacuum weak hash table assert Table=%p len=%zi removed=%zi orig_len=%zi n_items=%zi\n", table, len, removed, orig_len, SCM_HASHTABLE_N_ITEMS (table)); > len = 0; > } With this change, we got lines similar to the following printed periodically - Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=2 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=2 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=3 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=2 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=4 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=2 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2321 n_items=2321 ...... Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=205 n_items=205 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=205 n_items=205 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=205 n_items=205 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=205 n_items=205 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=205 n_items=205 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=205 n_items=205 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=205 n_items=205 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=205 n_items=205 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=205 n_items=205 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=205 n_items=205 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=205 n_items=205 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=205 n_items=205 ....... Vacuum weak hash table assert Table=0x9bdb840 len=1 removed=2 orig_len=2274 n_items=2274 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2274 n_items=2274 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2274 n_items=2274 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=2 orig_len=2274 n_items=2274 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2274 n_items=2274 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2274 n_items=2274 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2274 n_items=2274 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2274 n_items=2274 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2274 n_items=2274 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2274 n_items=2274 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2274 n_items=2274 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2274 n_items=2274 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2274 n_items=2274 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2274 n_items=2274 Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 orig_len=2274 n_items=2274 Thanks, Anand On Mon, Dec 8, 2014 at 3:04 PM, Ludovic Courtès <ludo@gnu.org> wrote: > Anand Mohanadoss <anand108@gmail.com> skribis: > > > I downloaded the current guile code in git and will give it a try. But, > > given that this is not a stable release, I am not sure if we can really > use > > it for our purposes. > > > > Do you or Ludovic know when a stable 2.2 version will be released? From > > stability point of you, would it be better to use a patched version of > > 2.0.11 that handles the case we noticed (e.g. don't attempt to remove > items > > from hash table if len = 0)? > > There is not ETA for 2.2. > > For now, I would recommend using the ‘stable-2.0’ branch, which > corresponds to what will (hopefully soon) become 2.0.12. > > Thanks, > Ludo’. > [-- Attachment #2: Type: text/html, Size: 7471 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#19180: guile bug#19180: vacuum_weak_hash_table error 2014-12-15 6:36 ` Anand Mohanadoss @ 2016-06-22 14:55 ` Andy Wingo 2016-06-22 15:43 ` Anand Mohanadoss 2016-07-12 7:32 ` Andy Wingo 0 siblings, 2 replies; 9+ messages in thread From: Andy Wingo @ 2016-06-22 14:55 UTC (permalink / raw) To: Anand Mohanadoss; +Cc: 19180, Ludovic Courtès, Linas Vepstas Hi :) On Mon 15 Dec 2014 07:36, Anand Mohanadoss <anand108@gmail.com> writes: > Here is what we changed in hashtab.c - > > 130a131 >> size_t orig_len = len; > 137,138c138,144 > < assert (removed <= len); > < len -= removed; > --- >> if (removed <= len) >> len -= removed; >> else >> { >> printf ("Vacuum weak hash table assert Table=%p len=%zi removed=%zi > orig_len=%zi n_items=%zi\n", table, len, removed, orig_len, > SCM_HASHTABLE_N_ITEMS (table)); >> len = 0; >> } > > With this change, we got lines similar to the following printed > periodically - > > Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 > orig_len=2321 n_items=2321 I guess printing a warning is not worse than crashing. I was unable to make this table work in a reliable way in 2.0 without rewriting it, so in 2.2 there's a new implementation with hopefully no bug in this regard. Ludovic what do you thing, should we just be sloppy in 2.0 and remove the assertion? I don't think it's fixable. The other option I see is to close as WONTFIX. Andy ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#19180: guile bug#19180: vacuum_weak_hash_table error 2016-06-22 14:55 ` Andy Wingo @ 2016-06-22 15:43 ` Anand Mohanadoss 2016-06-22 21:11 ` Linas Vepstas 2016-06-22 21:33 ` Andy Wingo 2016-07-12 7:32 ` Andy Wingo 1 sibling, 2 replies; 9+ messages in thread From: Anand Mohanadoss @ 2016-06-22 15:43 UTC (permalink / raw) To: Andy Wingo; +Cc: 19180, Ludovic Courtès, Linas Vepstas [-- Attachment #1: Type: text/plain, Size: 1376 bytes --] Hi Andy, Thanks a lot for looking into this and your response! Any idea when we will have a stable 2.2 release that we can move to given that 2.1 has been out for a few months. Thanks, Anand On Wed, Jun 22, 2016 at 8:25 PM, Andy Wingo <wingo@pobox.com> wrote: > Hi :) > > On Mon 15 Dec 2014 07:36, Anand Mohanadoss <anand108@gmail.com> writes: > > > Here is what we changed in hashtab.c - > > > > 130a131 > >> size_t orig_len = len; > > 137,138c138,144 > > < assert (removed <= len); > > < len -= removed; > > --- > >> if (removed <= len) > >> len -= removed; > >> else > >> { > >> printf ("Vacuum weak hash table assert Table=%p len=%zi removed=%zi > > orig_len=%zi n_items=%zi\n", table, len, removed, orig_len, > > SCM_HASHTABLE_N_ITEMS (table)); > >> len = 0; > >> } > > > > With this change, we got lines similar to the following printed > > periodically - > > > > Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 > > orig_len=2321 n_items=2321 > > I guess printing a warning is not worse than crashing. I was unable to > make this table work in a reliable way in 2.0 without rewriting it, so > in 2.2 there's a new implementation with hopefully no bug in this > regard. > > Ludovic what do you thing, should we just be sloppy in 2.0 and remove > the assertion? I don't think it's fixable. The other option I see is > to close as WONTFIX. > > Andy > [-- Attachment #2: Type: text/html, Size: 2069 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#19180: guile bug#19180: vacuum_weak_hash_table error 2016-06-22 15:43 ` Anand Mohanadoss @ 2016-06-22 21:11 ` Linas Vepstas 2016-06-22 21:33 ` Andy Wingo 1 sibling, 0 replies; 9+ messages in thread From: Linas Vepstas @ 2016-06-22 21:11 UTC (permalink / raw) To: Anand Mohanadoss; +Cc: 19180, Ludovic Courtès [-- Attachment #1: Type: text/plain, Size: 2394 bytes --] Anand, I've been using guile, straight from git for maybe almost 2 years(??) in a semi-harsh environment (lots of threads, lots of C++ smob jiggery-pokery, entering and exiting guile, redirecting ports, using fluids at the scheme/c++ boundary, catching and throwing exceptions to/from C++, interleaving all this with python, too, -- and medium cpu burn over many days/week) and it seems to work fine. Try it -- fix a tag, run system test, I bet it will work for you. I think Ludo and Andy have done a good job. The only recent glitch is that the setvbuf API changed. An old quasi-issue is that garbage collection seems to not be aggressive enough. After several layers of C calling guile calling C and so on, mem usage seems to bloat (a lot -- many many GB's) unless I forcibly run GC every 50th time I re-enter guile. But I think it does that in guile-2.0 too. --linas On Wed, Jun 22, 2016 at 10:43 AM, Anand Mohanadoss <anand108@gmail.com> wrote: > Hi Andy, > > Thanks a lot for looking into this and your response! Any idea when we > will have a stable 2.2 release that we can move to given that 2.1 has been > out for a few months. > > Thanks, > Anand > > On Wed, Jun 22, 2016 at 8:25 PM, Andy Wingo <wingo@pobox.com> wrote: > >> Hi :) >> >> On Mon 15 Dec 2014 07:36, Anand Mohanadoss <anand108@gmail.com> writes: >> >> > Here is what we changed in hashtab.c - >> > >> > 130a131 >> >> size_t orig_len = len; >> > 137,138c138,144 >> > < assert (removed <= len); >> > < len -= removed; >> > --- >> >> if (removed <= len) >> >> len -= removed; >> >> else >> >> { >> >> printf ("Vacuum weak hash table assert Table=%p len=%zi removed=%zi >> > orig_len=%zi n_items=%zi\n", table, len, removed, orig_len, >> > SCM_HASHTABLE_N_ITEMS (table)); >> >> len = 0; >> >> } >> > >> > With this change, we got lines similar to the following printed >> > periodically - >> > >> > Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 >> > orig_len=2321 n_items=2321 >> >> I guess printing a warning is not worse than crashing. I was unable to >> make this table work in a reliable way in 2.0 without rewriting it, so >> in 2.2 there's a new implementation with hopefully no bug in this >> regard. >> >> Ludovic what do you thing, should we just be sloppy in 2.0 and remove >> the assertion? I don't think it's fixable. The other option I see is >> to close as WONTFIX. >> >> Andy >> > > [-- Attachment #2: Type: text/html, Size: 3428 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#19180: guile bug#19180: vacuum_weak_hash_table error 2016-06-22 15:43 ` Anand Mohanadoss 2016-06-22 21:11 ` Linas Vepstas @ 2016-06-22 21:33 ` Andy Wingo 2016-06-23 4:24 ` Anand Mohanadoss 1 sibling, 1 reply; 9+ messages in thread From: Andy Wingo @ 2016-06-22 21:33 UTC (permalink / raw) To: Anand Mohanadoss; +Cc: 19180, Ludovic Courtès, Linas Vepstas On Wed 22 Jun 2016 17:43, Anand Mohanadoss <anand108@gmail.com> writes: > Any idea when we will have a stable 2.2 release that we can move to > given that 2.1 has been out for a few months. I hope for 2.2 within the next couple months, but we'll see :) Andy ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#19180: guile bug#19180: vacuum_weak_hash_table error 2016-06-22 21:33 ` Andy Wingo @ 2016-06-23 4:24 ` Anand Mohanadoss 0 siblings, 0 replies; 9+ messages in thread From: Anand Mohanadoss @ 2016-06-23 4:24 UTC (permalink / raw) To: Andy Wingo; +Cc: 19180, Ludovic Courtès, Linas Vepstas [-- Attachment #1: Type: text/plain, Size: 414 bytes --] Hi Andy and Linas, Thank you very much for your responses! Anand On Thu, Jun 23, 2016 at 3:03 AM, Andy Wingo <wingo@pobox.com> wrote: > On Wed 22 Jun 2016 17:43, Anand Mohanadoss <anand108@gmail.com> writes: > > > Any idea when we will have a stable 2.2 release that we can move to > > given that 2.1 has been out for a few months. > > I hope for 2.2 within the next couple months, but we'll see :) > > Andy > [-- Attachment #2: Type: text/html, Size: 902 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#19180: guile bug#19180: vacuum_weak_hash_table error 2016-06-22 14:55 ` Andy Wingo 2016-06-22 15:43 ` Anand Mohanadoss @ 2016-07-12 7:32 ` Andy Wingo 1 sibling, 0 replies; 9+ messages in thread From: Andy Wingo @ 2016-07-12 7:32 UTC (permalink / raw) To: Anand Mohanadoss; +Cc: 19180-done, Ludovic Courtès, Linas Vepstas Hi, On Wed 22 Jun 2016 16:55, Andy Wingo <wingo@pobox.com> writes: > On Mon 15 Dec 2014 07:36, Anand Mohanadoss <anand108@gmail.com> writes: > >> Vacuum weak hash table assert Table=0x9bdb840 len=0 removed=1 >> orig_len=2321 n_items=2321 > > I guess printing a warning is not worse than crashing. I was unable to > make this table work in a reliable way in 2.0 without rewriting it, so > in 2.2 there's a new implementation with hopefully no bug in this > regard. I changed this assert to a warning and added a comment like this: /* The move to BDW-GC with Guile 2.0 introduced some bugs related to weak hash tables, threads, memory usage, and the alloc lock. We were unable to fix these issues satisfactorily in 2.0 but have addressed them via a rewrite in 2.2. If you see this message often, you probably want to upgrade to 2.2. */ Andy ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2016-07-12 7:32 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <CAHrUA3554+87CVKsW2OT+-roVeyK4sE+xyDPaGf4ar5fAWQ2+Q@mail.gmail.com>
2014-12-08 7:34 ` bug#19180: guile bug#19180: vacuum_weak_hash_table error Anand Mohanadoss
2014-12-08 9:34 ` Ludovic Courtès
2014-12-15 6:36 ` Anand Mohanadoss
2016-06-22 14:55 ` Andy Wingo
2016-06-22 15:43 ` Anand Mohanadoss
2016-06-22 21:11 ` Linas Vepstas
2016-06-22 21:33 ` Andy Wingo
2016-06-23 4:24 ` Anand Mohanadoss
2016-07-12 7:32 ` Andy Wingo
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).