unofficial mirror of guile-devel@gnu.org 
 help / color / mirror / Atom feed
* [PATCH] Fixed `scm_i_take_stringbufn ()'
@ 2006-02-13 16:38 Ludovic Courtès
  2006-02-13 21:38 ` Kevin Ryde
  2006-02-15  0:48 ` Kevin Ryde
  0 siblings, 2 replies; 10+ messages in thread
From: Ludovic Courtès @ 2006-02-13 16:38 UTC (permalink / raw)


Hi,

The patch below fixes `scm_i_take_stringbufn ()' so that it registers
the right amount of collectable memory.  Failing to do so, an underflow
was triggered in `decrease_mtrigger ()' (when decreasing
SCM_MALLOCATED), resulting in a weird behavior (typically: fast increase
in memory consumption quickly followed by the memory overflow error in
`increase_mtrigger ()').

The problem can be reproduced using the program found at the end of this
post.  It could hardly be turned into a test case, however.

Note: I'm not sure what to do with `assert ()'.  Perhaps we should
define our own macro somewhere?

Thanks,
Ludovic.


2006-02-13  Ludovic Courtès  <ludovic.courtes@laas.fr>

	*strings.c (scm_i_take_stringbufn): Register LEN + 1 octets
	instead of LEN.  Without this, too much collectable memory gets
	unregistered, which results in an underflow of SCM_MALLOCATED in
	`decrease_mtrigger ()'.

	* gc-malloc.c (decrease_mtrigger): Make sure SIZE is lower than
	or equal to SCM_MALLOCATED.


--- orig/libguile/gc-malloc.c
+++ mod/libguile/gc-malloc.c
@@ -64,6 +64,10 @@
 #include <unistd.h>
 #endif
 
+#ifdef HAVE_ASSERT_H
+#include <assert.h>
+#endif
+
 /*
   INIT_MALLOC_LIMIT is the initial amount of malloc usage which will
   trigger a GC.
@@ -184,6 +188,14 @@
 decrease_mtrigger (size_t size, const char * what)
 {
   scm_i_pthread_mutex_lock (&scm_i_gc_admin_mutex);
+
+#ifdef HAVE_ASSERT_H
+  assert (size <= scm_mallocated);
+#else
+  if (size > scm_mallocated)
+    abort ();
+#endif
+
   scm_mallocated -= size;
   scm_gc_malloc_collected += size;
   scm_i_pthread_mutex_unlock (&scm_i_gc_admin_mutex);


--- orig/libguile/strings.c
+++ mod/libguile/strings.c
@@ -122,12 +122,12 @@
     }
 }
 
-/* Return a new stringbuf whose underlying storage consists of the LEN octets
-   pointed to by STR.  */
+/* Return a new stringbuf whose underlying storage consists of the LEN+1
+   octets pointed to by STR (the last octet is zero).  */
 SCM_C_INLINE SCM
 scm_i_take_stringbufn (char *str, size_t len)
 {
-  scm_gc_register_collectable_memory (str, len, "stringbuf");
+  scm_gc_register_collectable_memory (str, len + 1, "stringbuf");
 
   return scm_double_cell (STRINGBUF_TAG, (scm_t_bits) str,
 			  (scm_t_bits) len, (scm_t_bits) 0);



\f
/* Copyright (C) 1996,1997,2000,2001 Free Software Foundation, Inc.
 * 
 * This library is free software; you can redistribute it and/or
 * modify it under the terms of the GNU Lesser General Public
 * License as published by the Free Software Foundation; either
 * version 2.1 of the License, or (at your option) any later version.
 *
 * This library is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 * Lesser General Public License for more details.
 *
 * You should have received a copy of the GNU Lesser General Public
 * License along with this library; if not, write to the Free Software
 * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
 */

/* This is the 'main' function for the `guile' executable.  It is not
   included in libguile.a.

   Eventually, we hope this file will be automatically generated,
   based on the list of installed, statically linked libraries on the
   system.  For now, please don't put interesting code in here.  */

#if HAVE_CONFIG_H
#  include <config.h>
#endif

#include <libguile.h>

#ifdef HAVE_CONFIG_H
#include <libguile/scmconfig.h>
#endif

#include <string.h>
#include <stdio.h>
#include <signal.h>

static void
inner_main (void *closure SCM_UNUSED, int argc, char **argv)
{
  static unsigned long long i = 0;

  while (1)
    {
      SCM str;

      i++;
      if ((i && !(i % 1000)) || (i > 82100))
        /* At some point you will witness the underflow of
           SCM_MALLOCATED.  */
	printf ("i=%llu mallocated: %lu\n", i, scm_mallocated);

#if 0
      if (i == 82112)
        /* Stop just before the underflow to attach GDB.  */
	kill (getpid (), SIGTSTP);
#endif

      str = scm_take_locale_string (strdup ("csdfsdfsfsdfsldfkslkdfnlskdf"));
    }
}

int
main (int argc, char **argv)
{
  scm_boot_guile (argc, argv, inner_main, 0);
  return 0; /* never reached */
}



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] Fixed `scm_i_take_stringbufn ()'
  2006-02-13 16:38 [PATCH] Fixed `scm_i_take_stringbufn ()' Ludovic Courtès
@ 2006-02-13 21:38 ` Kevin Ryde
  2006-02-14  9:47   ` Ludovic Courtès
  2006-02-15  0:48 ` Kevin Ryde
  1 sibling, 1 reply; 10+ messages in thread
From: Kevin Ryde @ 2006-02-13 21:38 UTC (permalink / raw)


ludovic.courtes@laas.fr (Ludovic Courtès) writes:
>
> Note: I'm not sure what to do with `assert ()'.

"Don't do that", to coin a phrase.

> Perhaps we should define our own macro somewhere?

The `if' you put, with an fprintf stderr, should be fine.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] Fixed `scm_i_take_stringbufn ()'
  2006-02-13 21:38 ` Kevin Ryde
@ 2006-02-14  9:47   ` Ludovic Courtès
  2006-02-15  0:38     ` Kevin Ryde
  0 siblings, 1 reply; 10+ messages in thread
From: Ludovic Courtès @ 2006-02-14  9:47 UTC (permalink / raw)


Hi,

Kevin Ryde <user42@zip.com.au> writes:

> The `if' you put, with an fprintf stderr, should be fine.

I think you're right.  Here's the updated patch.

Thanks,
Ludovic.


2006-02-14  Ludovic Courtès  <ludovic.courtes@laas.fr>

	*strings.c (scm_i_take_stringbufn): Register LEN + 1 octets
	instead of LEN.  Without this, too much collectable memory gets
	unregistered, which results in an underflow of SCM_MALLOCATED in
	`decrease_mtrigger ()'.

	* gc-malloc.c (decrease_mtrigger): Make sure SIZE is lower than
	or equal to SCM_MALLOCATED.


--- orig/libguile/gc-malloc.c
+++ mod/libguile/gc-malloc.c
@@ -64,6 +64,7 @@
 #include <unistd.h>
 #endif
 
+
 /*
   INIT_MALLOC_LIMIT is the initial amount of malloc usage which will
   trigger a GC.
@@ -184,6 +185,16 @@
 decrease_mtrigger (size_t size, const char * what)
 {
   scm_i_pthread_mutex_lock (&scm_i_gc_admin_mutex);
+
+  if (size > scm_mallocated)
+    {
+      fprintf (stderr, "`scm_mallocated' underflow.  This means that more "
+	       "memory was unregistered\n"
+	       "via `scm_gc_unregister_collectable_memory ()' than "
+	       "registered.\n");
+      abort ();
+    }
+
   scm_mallocated -= size;
   scm_gc_malloc_collected += size;
   scm_i_pthread_mutex_unlock (&scm_i_gc_admin_mutex);


--- orig/libguile/strings.c
+++ mod/libguile/strings.c
@@ -122,12 +122,12 @@
     }
 }
 
-/* Return a new stringbuf whose underlying storage consists of the LEN octets
-   pointed to by STR.  */
+/* Return a new stringbuf whose underlying storage consists of the LEN+1
+   octets pointed to by STR (the last octet is zero).  */
 SCM_C_INLINE SCM
 scm_i_take_stringbufn (char *str, size_t len)
 {
-  scm_gc_register_collectable_memory (str, len, "stringbuf");
+  scm_gc_register_collectable_memory (str, len + 1, "stringbuf");
 
   return scm_double_cell (STRINGBUF_TAG, (scm_t_bits) str,
 			  (scm_t_bits) len, (scm_t_bits) 0);



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] Fixed `scm_i_take_stringbufn ()'
  2006-02-14  9:47   ` Ludovic Courtès
@ 2006-02-15  0:38     ` Kevin Ryde
  2006-02-15  7:58       ` Ludovic Courtès
  0 siblings, 1 reply; 10+ messages in thread
From: Kevin Ryde @ 2006-02-15  0:38 UTC (permalink / raw)


ludovic.courtes@laas.fr (Ludovic Courtès) writes:
>
> Here's the updated patch.

Thanks, I checked it in.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] Fixed `scm_i_take_stringbufn ()'
  2006-02-13 16:38 [PATCH] Fixed `scm_i_take_stringbufn ()' Ludovic Courtès
  2006-02-13 21:38 ` Kevin Ryde
@ 2006-02-15  0:48 ` Kevin Ryde
  2006-02-15  8:01   ` Ludovic Courtès
  1 sibling, 1 reply; 10+ messages in thread
From: Kevin Ryde @ 2006-02-15  0:48 UTC (permalink / raw)


ludovic.courtes@laas.fr (Ludovic Courtès) writes:
>
> The problem can be reproduced using the program found at the end of this
> post.  It could hardly be turned into a test case, however.

I made some memory leak checks for myself by demanding that
"(assoc-ref (gc-stats) 'bytes-malloced)" was steady across some
repetitions of a test.  That's actually a different number though (is
it?), but it should in theory be possible to exercise some of the
malloc/free counting or balancing (post 1.8 that is :-).


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] Fixed `scm_i_take_stringbufn ()'
  2006-02-15  0:38     ` Kevin Ryde
@ 2006-02-15  7:58       ` Ludovic Courtès
  2006-02-15 22:03         ` Kevin Ryde
  0 siblings, 1 reply; 10+ messages in thread
From: Ludovic Courtès @ 2006-02-15  7:58 UTC (permalink / raw)


Kevin Ryde <user42@zip.com.au> writes:

> Thanks, I checked it in.

That is, in both branches?  :-)

Thanks,
Ludovic.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] Fixed `scm_i_take_stringbufn ()'
  2006-02-15  0:48 ` Kevin Ryde
@ 2006-02-15  8:01   ` Ludovic Courtès
  2006-02-15 22:15     ` Kevin Ryde
  0 siblings, 1 reply; 10+ messages in thread
From: Ludovic Courtès @ 2006-02-15  8:01 UTC (permalink / raw)


Kevin Ryde <user42@zip.com.au> writes:

> I made some memory leak checks for myself by demanding that
> "(assoc-ref (gc-stats) 'bytes-malloced)" was steady across some
> repetitions of a test.  That's actually a different number though (is
> it?), but it should in theory be possible to exercise some of the
> malloc/free counting or balancing (post 1.8 that is :-).

I'm not sure doing so would be that easy because `bytes-malloced'
follows a periodic distribution whose period is unknown (i.e., each call
to `scm_take_locale_stringn ()' makes it slightly increase, and each
mark/sweep sequence -- which is occurs roughly periodically -- changes
it back to a much lower value).

Thanks,
Ludovic.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] Fixed `scm_i_take_stringbufn ()'
  2006-02-15  7:58       ` Ludovic Courtès
@ 2006-02-15 22:03         ` Kevin Ryde
  2006-02-21  8:37           ` Ludovic Courtès
  0 siblings, 1 reply; 10+ messages in thread
From: Kevin Ryde @ 2006-02-15 22:03 UTC (permalink / raw)


ludovic.courtes@laas.fr (Ludovic Courtès) writes:
>
> That is, in both branches?  :-)

No, in 1.8.  Someone can do merges into the head (there isn't anything
in 1.8 which isn't also meant to be in the head, yet).


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] Fixed `scm_i_take_stringbufn ()'
  2006-02-15  8:01   ` Ludovic Courtès
@ 2006-02-15 22:15     ` Kevin Ryde
  0 siblings, 0 replies; 10+ messages in thread
From: Kevin Ryde @ 2006-02-15 22:15 UTC (permalink / raw)


ludovic.courtes@laas.fr (Ludovic Courtès) writes:
>
> a periodic distribution whose period is unknown

The idea is to force gc's before and after, expecting mem counters to
be unchanged by the test code in the middle.  I had to setup for
multiple attempts, I guess memoizing or other freaky stuff happens on
the first couple of goes.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] Fixed `scm_i_take_stringbufn ()'
  2006-02-15 22:03         ` Kevin Ryde
@ 2006-02-21  8:37           ` Ludovic Courtès
  0 siblings, 0 replies; 10+ messages in thread
From: Ludovic Courtès @ 2006-02-21  8:37 UTC (permalink / raw)


Hello,

Kevin Ryde <user42@zip.com.au> writes:

> No, in 1.8.  Someone can do merges into the head (there isn't anything
> in 1.8 which isn't also meant to be in the head, yet).

Did "someone" do that merge?  :-)

I'm afraid we're going to forget some stuff in one or another branch...

Thanks,
Ludovic.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2006-02-21  8:37 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-02-13 16:38 [PATCH] Fixed `scm_i_take_stringbufn ()' Ludovic Courtès
2006-02-13 21:38 ` Kevin Ryde
2006-02-14  9:47   ` Ludovic Courtès
2006-02-15  0:38     ` Kevin Ryde
2006-02-15  7:58       ` Ludovic Courtès
2006-02-15 22:03         ` Kevin Ryde
2006-02-21  8:37           ` Ludovic Courtès
2006-02-15  0:48 ` Kevin Ryde
2006-02-15  8:01   ` Ludovic Courtès
2006-02-15 22:15     ` Kevin Ryde

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).