cell heap usage in 1.8 vs 1.6

cell heap usage in 1.8 vs 1.6

[Kevin Ryde]

I'm having trouble in my charting program with the amount of heap space
allocated for cells in 1.8.  It ends up allocating more and more heap
(as reported by gc-stats 'cell-heap-segments and confirmed by
mallinfo()), apparently without bound.  I've got between 150k and 200k
objects according to gc-live-object-stats, which should be about 5Mb of
cells, but the heap keeps growing to as much as 60Mb.

The same code seems ok in 1.6, it lives happily in about 5Mb of heap,
though it does seem to do a few more gc's than 1.8.  A backport of
gc-live-object-stats to 1.6 says the objects in use are about the same
there.

I stuck in lots of printfs, which haven't told me much so far, except
that the "collected" counts seem to be lower than I'd expect, and cause
min_yield to be increased, and the heap then added to (every few gc's).

Should the collected counts end up basically as "heapsize - liveobjects"
every time?  They seem to be smaller than that, but I don't know where
to look for how or why.  scm_i_sweep_segment() looks slightly doubtful.
Does it deliberately not count the balance of a lazy sweep towards the
collected counts?  I wondered if a gc is provoked by the double cells
being exhausted but only a little of the single cells having been
collected, leaving a small collected count in the latter (or vice
versa).

Alas I don't have a simple program to provoke the problem.  Perhaps some
other big or long run program users could look at their heap consumption
vs live objects, to see if it's just me.

(define (total-heap)
  (apply + (map (lambda (elem)
                  (abs (- (cdr elem) (car elem))))
                (assoc-ref (gc-stats) 'cell-heap-segments))))

(define (count-alive)
  (apply + (map cdr (gc-live-object-stats))))


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: cell heap usage in 1.8 vs 1.6

[Ludovic Courtès]

Hi Kevin,

Kevin Ryde <user42@zip.com.au> writes:

> I'm having trouble in my charting program with the amount of heap space
> allocated for cells in 1.8.  It ends up allocating more and more heap
> (as reported by gc-stats 'cell-heap-segments and confirmed by
> mallinfo()), apparently without bound.  I've got between 150k and 200k
> objects according to gc-live-object-stats, which should be about 5Mb of
> cells, but the heap keeps growing to as much as 60Mb.

Did you try running it with HEAD?  There were small GC-stats "cleanups"
committed there that are not in 1.8, so it may be worth trying.

Besides, is your Guile compiled with `-O2'?  If not, it could be the
case that large amounts of stack space are used, but not always
overwritten, which erroneously leaves references to otherwise
unreferenced objects on the stack.  (That is unlikely to explain the
whole phenomenon, though.)

> Should the collected counts end up basically as "heapsize - liveobjects"
> every time?

I think so.

> They seem to be smaller than that, but I don't know where
> to look for how or why.  scm_i_sweep_segment() looks slightly doubtful.
> Does it deliberately not count the balance of a lazy sweep towards the
> collected counts?

Sorry, I don't understand what you mean here.

> I wondered if a gc is provoked by the double cells
> being exhausted but only a little of the single cells having been
> collected, leaving a small collected count in the latter (or vice
> versa).

I dunno.  At any rate, `(gc)' calls `scm_i_sweep all_segments ()', so
this should allow you to test this hypothesis.

Also, did you try fiddling with the `GUILE_MIN_YIELD_{1,2}' variables?

Thanks,
Ludovic.



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: cell heap usage in 1.8 vs 1.6

[Kevin Ryde]

ludo@gnu.org (Ludovic Courtès) writes:
>
> Did you try running it with HEAD?  There were small GC-stats "cleanups"
> committed there that are not in 1.8, so it may be worth trying.

Ah yes, it's better there.  What bit is the operative fix?  Both the
per-freelist min_yield adjust and the counting in
scm_i_sweep_all_segments look likely.  (Though it still all looks
complicated, and sprayed around a dozen functions :-(

> Also, did you try fiddling with the `GUILE_MIN_YIELD_{1,2}' variables?

No.  I agree with the yield fractions at the point they're applied, I
just think the supposed collected cells (ie. unused space) in the
calculation is too small, making the calculation think the heap should
be grown, when in fact there's plenty of heap already.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: cell heap usage in 1.8 vs 1.6

[Ludovic Courtès]

Hey,

Kevin Ryde <user42@zip.com.au> writes:

> Ah yes, it's better there.

As good as 1.6?

> What bit is the operative fix?

I couldn't find exactly.  The ChangeLog entry on 2006-01-04 gives some
hints about functions to look at.  There's also this post:
http://lists.gnu.org/archive/html/guile-devel/2005-12/msg00095.html .

> Both the per-freelist min_yield adjust and the counting in
> scm_i_sweep_all_segments look likely.  (Though it still all looks
> complicated, and sprayed around a dozen functions :-(

Hopefully removal of globals made it slightly simpler to follow...

Thanks,
Ludovic.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: cell heap usage in 1.8 vs 1.6

[Ludovic Courtès]

Hi,

Kevin Ryde <user42@zip.com.au> writes:

> ludovic.courtes@laas.fr (Ludovic Courtès) writes:

>> Hopefully removal of globals made it slightly simpler to follow...
>
> No, that's made it harder to see what actually changed.

I was referring to the code, not to the diff.

You can still diff 1.8 against 1.6, since 1.8 doesn't have the change.

Thanks,
Ludovic.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: cell heap usage in 1.8 vs 1.6

[Ludovic Courtès]

Hi,

Kevin Ryde <user42@zip.com.au> writes:

> I'm having trouble in my charting program with the amount of heap space
> allocated for cells in 1.8.  It ends up allocating more and more heap
> (as reported by gc-stats 'cell-heap-segments and confirmed by
> mallinfo()), apparently without bound.  I've got between 150k and 200k
> objects according to gc-live-object-stats, which should be about 5Mb of
> cells, but the heap keeps growing to as much as 60Mb.

OK, I *think* I hit it with a long-running program with HEAD (this has
become visible now that the thing has been running for more than 72
hours...).

> (define (total-heap)
>   (apply + (map (lambda (elem)
>                   (abs (- (cdr elem) (car elem))))
>                 (assoc-ref (gc-stats) 'cell-heap-segments))))
>
> (define (count-alive)
>   (apply + (map cdr (gc-live-object-stats))))

Note that `count-alive' returns a number of objects, while `total-heap'
(which really is "total *cell* heap") returns a number of bytes, right?
Thus, the result of `count-alive' should be multiplied by 8 (regular
cells) or 16 (double cells).

Nevertheless, even when multiplying `count-alive' by 16, I get a
`total-heap' an order of magnitude larger, which seems to confirm your
hypothesis (that more heap is being allocated despite the availability
of free cells).

To be continued...

Ludo'.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: cell heap usage in 1.8 vs 1.6

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 1006 bytes --]

Hi,

I hopefully managed to characterize the problem.  I instrumented HEAD
with the attached patch that exposes the current "min yield" of both
freelists.  Then, I ran the attached script that does the following:

  * at regular intervals, plot the total cell heap, alive cell heap
    (using your computation method), yield of the single-cell freelist,
    yield of the double-cell freelist;

  * every 1000 iteration, perform a large memory allocation (single
    cells), and then discard the allocated cells.

The resulting file for around 5000 iterations is attached.  It can be
plotted with GNUplot using:

  plot ',,head.sporadic.gp' \
       using 1:($2/$3) with lines title "total/alive", \
    '' using 1:4 with lines axes x1y2 title "yield1",  \
    '' using 1:5 with lines axes x1y2 title "yield2"

This shows the oscillation of the total to alive cell heap ratio, as
well as the variations of the "min yield" of the freelists (x axis
represents the iteration number):


[-- Attachment #2: The plot --]
[-- Type: image/png, Size: 4897 bytes --]

[-- Attachment #3: Type: text/plain, Size: 809 bytes --]


The following observations can be made:

  * the heap gets more and more underutilized, although it seems to
    eventually stabilize somewhere around 5% (!) utilization (i.e.,
    total/alive = 20);

  * the "min yield" of the single-cell freelist grows way too fast as a
    reaction to the first memory allocation (at iteration 1000), which
    consequently leads to the increase of the total heap size.

Keep in mind that currently heap is never reclaimed, so its total size
can only increase.  Likewise, "min yield" can only increase, as its
typically a fraction (40% by default) of the total heap size.

Strangely enough, with 1.8, `alive-cell-heap' remains exactly constant
until iteration 1000.  I suspect something wrong with
`gc-live-object-stats' there...

Thanks,
Ludovic.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: The patch --]
[-- Type: text/x-patch, Size: 748 bytes --]

--- orig/libguile/gc.c
+++ mod/libguile/gc.c
@@ -408,6 +408,28 @@
 }
 #undef FUNC_NAME
 
+SCM scm_freelist1_yield (void);
+SCM scm_freelist2_yield (void);
+
+SCM_DEFINE (scm_freelist1_yield, "freelist1-yield", 0, 0, 0,
+	    (void),
+	    "")
+#define FUNC_NAME s_scm_freelist1_yield
+{
+  return (scm_from_long (scm_i_master_freelist.min_yield));
+}
+#undef FUNC_NAME
+
+SCM_DEFINE (scm_freelist2_yield, "freelist2-yield", 0, 0, 0,
+	    (void),
+	    "")
+#define FUNC_NAME s_scm_freelist2_yield
+{
+  return (scm_from_long (scm_i_master_freelist2.min_yield));
+}
+#undef FUNC_NAME
+
+
 /* Update the global sweeping/collection statistics by adding SWEEP_STATS to
    SCM_I_GC_SWEEP_STATS and updating related variables.  */
 static inline void


[-- Attachment #5: The script --]
[-- Type: application/octet-stream, Size: 1265 bytes --]

(use-modules (srfi srfi-1)
             (srfi srfi-14))

(define (total-cell-heap)
  (apply + (map (lambda (elem)
                  (abs (- (cdr elem) (car elem))))
                (assoc-ref (gc-stats) 'cell-heap-segments))))

(define (alive-cell-heap)
  (* 16 (apply + (map cdr (gc-live-object-stats)))))

\f
(setvbuf (current-output-port) _IOLBF)
(format #t "### plot '-' using 1:($2/$3) with lines title \"total/alive\"~%")

(let loop ((iteration 0))
  (if #f ;;(> iteration 10000)
      #t
      (begin
        (let ((lst (list-tabulate 1000
                                  (lambda (i)
                                    (char-set #\.)  ;; double-cell
                                    (make-list 100)))))

          (if (and (= (modulo iteration 1000) 0)
                   (> iteration 0))
              ;; sporadic heap-intensive job
              (make-list 1000000)))

        (if (= 0 (modulo iteration 10))
            (let ((total  (total-cell-heap))
                  (alive  (alive-cell-heap))
                  (yield1 (freelist1-yield))
                  (yield2 (freelist2-yield)))
              (format #t "~a ~a ~a ~a ~a~%" iteration
                      total alive
                      yield1 yield2)))

        (loop (1+ iteration)))))

[-- Attachment #6: The GNUplot file --]
[-- Type: application/octet-stream, Size: 16842 bytes --]

### plot '-' using 1:($2/$3) with lines title "total/alive"
0 1931264 1929088 159432 16624
10 4057088 1783120 285215 16624
20 4057088 789344 285215 16624
30 4057088 1411744 285215 16624
40 4057088 939568 487276 16624
50 4057088 1562128 487276 16624
60 4057088 2183888 487276 16624
70 6762496 1190048 487276 16624
80 6762496 1813296 487276 16624
90 6762496 1681712 487276 16624
100 6762496 689024 487276 16624
110 6762496 1309632 487276 16624
120 6762496 1931792 487276 16624
130 6762496 939504 487276 16624
140 6762496 1560528 487276 16624
150 6762496 2182272 487276 16624
160 6762496 1188432 487276 16624
170 6762496 1811680 487276 16624
180 6762496 1680096 487276 16624
190 6762496 687408 487276 16624
200 6762496 1308016 487276 16624
210 6762496 1930176 487276 16624
220 6762496 937888 487276 16624
230 6762496 1558912 487276 16624
240 6762496 2180656 487276 16624
250 6762496 1186816 487276 16624
260 6762496 1810064 487276 16624
270 6762496 1678480 487276 16624
280 6762496 685792 487276 16624
290 6762496 1306400 487276 16624
300 6762496 1928560 487276 16624
310 6762496 936272 487276 16624
320 6762496 1557296 487276 16624
330 6762496 2179040 487276 16624
340 6762496 1185200 487276 16624
350 6762496 1808448 487276 16624
360 6762496 1676864 487276 16624
370 6762496 684176 487276 16624
380 6762496 1304784 487276 16624
390 6762496 1926944 487276 16624
400 6762496 934656 487276 16624
410 6762496 1555680 487276 16624
420 6762496 2177424 487276 16624
430 6762496 1183584 487276 16624
440 6762496 1806832 487276 16624
450 6762496 1675248 487276 16624
460 6762496 682560 487276 16624
470 6762496 1303168 487276 16624
480 6762496 1925328 487276 16624
490 6762496 933040 487276 16624
500 6762496 1554064 487276 16624
510 6762496 2175808 487276 16624
520 6762496 1181968 487276 16624
530 6762496 1805216 487276 16624
540 6762496 1673632 487276 16624
550 6762496 680944 487276 16624
560 6762496 1301552 487276 16624
570 6762496 1923712 487276 16624
580 6762496 931424 487276 16624
590 6762496 1552448 487276 16624
600 6762496 2174192 487276 16624
610 6762496 1180352 487276 16624
620 6762496 1803600 487276 16624
630 6762496 1672016 487276 16624
640 6762496 679328 487276 16624
650 6762496 1299936 487276 16624
660 6762496 1922096 487276 16624
670 6762496 929808 487276 16624
680 6762496 1550832 487276 16624
690 6762496 2172576 487276 16624
700 6762496 1178736 487276 16624
710 6762496 1801984 487276 16624
720 6762496 1670400 487276 16624
730 6762496 677712 487276 16624
740 6762496 1298320 487276 16624
750 6762496 1920480 487276 16624
760 6762496 928192 487276 16624
770 6762496 1549216 487276 16624
780 6762496 2170960 487276 16624
790 6762496 1177120 487276 16624
800 6762496 1800368 487276 16624
810 6762496 1668784 487276 16624
820 6762496 676096 487276 16624
830 6762496 1296704 487276 16624
840 6762496 1918864 487276 16624
850 6762496 926576 487276 16624
860 6762496 1547600 487276 16624
870 6762496 2169344 487276 16624
880 6762496 1175504 487276 16624
890 6762496 1798752 487276 16624
900 6762496 1667168 487276 16624
910 6762496 674480 487276 16624
920 6762496 1295088 487276 16624
930 6762496 1917248 487276 16624
940 6762496 924960 487276 16624
950 6762496 1545984 487276 16624
960 6762496 2167728 487276 16624
970 6762496 1173888 487276 16624
980 6762496 1797136 487276 16624
990 6762496 1665552 487276 16624
1000 11272192 672864 487276 16624
1010 11272192 1635536 1048683 16624
1020 11272192 642160 1048683 16624
1030 11272192 1262768 1048683 16624
1040 11272192 1886016 1048683 16624
1050 11272192 892640 1048683 16624
1060 11272192 1513568 1048683 16624
1070 11272192 2135408 1048683 16624
1080 11272192 1141568 1048683 16624
1090 11272192 1764816 1048683 16624
1100 11272192 1633328 1048683 16624
1110 11272192 640544 1048683 16624
1120 11272192 1261152 1048683 16624
1130 11272192 1884400 1048683 16624
1140 11272192 891024 1048683 16624
1150 11272192 1511952 1048683 16624
1160 11272192 2133792 1048683 16624
1170 11272192 1139952 1048683 16624
1180 11272192 1763200 1048683 16624
1190 11272192 1631712 1048683 16624
1200 11272192 638928 1048683 16624
1210 11272192 1259488 1048683 16624
1220 11272192 1882784 1048683 16624
1230 11272192 889408 1048683 16624
1240 11272192 1510336 1048683 16624
1250 11272192 2132176 1048683 16624
1260 11272192 1138336 1048683 16624
1270 11272192 1761584 1048683 16624
1280 11272192 1630096 1048683 16624
1290 11272192 637312 1048683 16624
1300 11272192 1257920 1048683 16624
1310 11272192 1881168 1048683 16624
1320 11272192 887792 1048683 16624
1330 11272192 1508720 1048683 16624
1340 11272192 2130560 1048683 16624
1350 11272192 1136720 1048683 16624
1360 11272192 1759968 1048683 16624
1370 11272192 1628480 1048683 16624
1380 11272192 635696 1048683 16624
1390 11272192 1256304 1048683 16624
1400 11272192 1879552 1048683 16624
1410 11272192 886176 1048683 16624
1420 11272192 1507104 1048683 16624
1430 11272192 2128944 1048683 16624
1440 11272192 1135104 1048683 16624
1450 11272192 1758352 1048683 16624
1460 11272192 1626864 1048683 16624
1470 11272192 634080 1048683 16624
1480 11272192 1254688 1048683 16624
1490 11272192 1877936 1048683 16624
1500 11272192 884560 1048683 16624
1510 11272192 1505488 1048683 16624
1520 11272192 2127328 1048683 16624
1530 11272192 1133488 1048683 16624
1540 11272192 1756736 1048683 16624
1550 11272192 1625248 1048683 16624
1560 11272192 632464 1048683 16624
1570 11272192 1253072 1048683 16624
1580 11272192 1876336 1048683 16624
1590 11272192 882944 1048683 16624
1600 11272192 1503872 1048683 16624
1610 11272192 2125712 1048683 16624
1620 11272192 1131872 1048683 16624
1630 11272192 1755120 1048683 16624
1640 11272192 1623632 1048683 16624
1650 11272192 630848 1048683 16624
1660 11272192 1251456 1048683 16624
1670 11272192 1874704 1048683 16624
1680 11272192 881328 1048683 16624
1690 11272192 1502256 1048683 16624
1700 11272192 2124096 1048683 16624
1710 11272192 1130256 1048683 16624
1720 11272192 1753504 1048683 16624
1730 11272192 1622016 1048683 16624
1740 11272192 629232 1048683 16624
1750 11272192 1249840 1048683 16624
1760 11272192 1873088 1048683 16624
1770 11272192 879712 1048683 16624
1780 11272192 1500640 1048683 16624
1790 11272192 2122480 1048683 16624
1800 11272192 1128640 1048683 16624
1810 11272192 1751888 1048683 16624
1820 11272192 1620400 1048683 16624
1830 11272192 627616 1048683 16624
1840 11272192 1248224 1048683 16624
1850 11272192 1871472 1048683 16624
1860 11272192 878096 1048683 16624
1870 11272192 1499024 1048683 16624
1880 11272192 2120880 1048683 16624
1890 11272192 1127024 1048683 16624
1900 11272192 1750272 1048683 16624
1910 11272192 1618784 1048683 16624
1920 11272192 626000 1048683 16624
1930 11272192 1246608 1048683 16624
1940 11272192 1869856 1048683 16624
1950 11272192 876480 1048683 16624
1960 11272192 1497408 1048683 16624
1970 11272192 2119248 1048683 16624
1980 11272192 1125408 1048683 16624
1990 11272192 1748656 1048683 16624
2000 18788352 1617168 1048683 16624
2010 18788352 624384 1048683 16624
2020 18788352 1244992 1048683 16624
2030 18788352 1868256 1048683 16624
2040 18788352 874864 1048683 16624
2050 18788352 1495696 1048683 16624
2060 18788352 2117632 1048683 16624
2070 18788352 1123792 1048683 16624
2080 18788352 1747040 1048683 16624
2090 18788352 1615648 1048683 16624
2100 18788352 622768 1048683 16624
2110 18788352 1243376 1048683 16624
2120 18788352 1866640 1048683 16624
2130 18788352 873248 1048683 16624
2140 18788352 1494080 1048683 16624
2150 18788352 2116016 1048683 16624
2160 18788352 1122144 1048683 16624
2170 18788352 1745424 1048683 16624
2180 18788352 1614032 1048683 16624
2190 18788352 2235600 1048683 16624
2200 18788352 1241760 1048683 16624
2210 18788352 1865024 1048683 16624
2220 18788352 871632 1048683 16624
2230 18788352 1492464 1048683 16624
2240 18788352 2114400 1048683 16624
2250 18788352 1120560 1048683 16624
2260 18788352 1743808 1048683 16624
2270 18788352 1612416 1048683 16624
2280 18788352 2233984 1048683 16624
2290 18788352 1240144 1048683 16624
2300 18788352 1863408 1048683 16624
2310 18788352 870016 1048683 16624
2320 18788352 1490848 1048683 16624
2330 18788352 2112784 1048683 16624
2340 18788352 1118944 1048683 16624
2350 18788352 1742192 1048683 16624
2360 18788352 1610800 1048683 16624
2370 18788352 2232368 1048683 16624
2380 18788352 1238528 1048683 16624
2390 18788352 1861792 1048683 16624
2400 18788352 868400 1048683 16624
2410 18788352 1489232 1048683 16624
2420 18788352 2111168 1048683 16624
2430 18788352 1117328 1048683 16624
2440 18788352 1740576 1048683 16624
2450 18788352 1609184 1048683 16624
2460 18788352 2230752 1048683 16624
2470 18788352 1236912 1048683 16624
2480 18788352 1860176 1048683 16624
2490 18788352 866784 1048683 16624
2500 18788352 1487616 1048683 16624
2510 18788352 2109552 1048683 16624
2520 18788352 1115712 1048683 16624
2530 18788352 1738960 1048683 16624
2540 18788352 1607568 1048683 16624
2550 18788352 2229136 1048683 16624
2560 18788352 1235296 1048683 16624
2570 18788352 1858560 1048683 16624
2580 18788352 865168 1048683 16624
2590 18788352 1486000 1048683 16624
2600 18788352 2107936 1048683 16624
2610 18788352 1114096 1048683 16624
2620 18788352 1737344 1048683 16624
2630 18788352 1605952 1048683 16624
2640 18788352 2227520 1048683 16624
2650 18788352 1233680 1048683 16624
2660 18788352 1856944 1048683 16624
2670 18788352 863552 1048683 16624
2680 18788352 1484384 1048683 16624
2690 18788352 2106320 1048683 16624
2700 18788352 1112480 1048683 16624
2710 18788352 1735728 1048683 16624
2720 18788352 1604336 1048683 16624
2730 18788352 2225904 1048683 16624
2740 18788352 1232064 1048683 16624
2750 18788352 1855328 1048683 16624
2760 18788352 861936 1048683 16624
2770 18788352 1482768 1048683 16624
2780 18788352 2104704 1048683 16624
2790 18788352 1110864 1048683 16624
2800 18788352 1734112 1048683 16624
2810 18788352 1602720 1048683 16624
2820 18788352 2224288 1048683 16624
2830 18788352 1230448 1048683 16624
2840 18788352 1853712 1048683 16624
2850 18788352 860320 1048683 16624
2860 18788352 1481152 1048683 16624
2870 18788352 2103088 1048683 16624
2880 18788352 1109248 1048683 16624
2890 18788352 1732496 1048683 16624
2900 18788352 1601104 1048683 16624
2910 18788352 2222672 1048683 16624
2920 18788352 1228832 1048683 16624
2930 18788352 1852096 1048683 16624
2940 18788352 858704 1048683 16624
2950 18788352 1479536 1048683 16624
2960 18788352 2101472 1048683 16624
2970 18788352 1107632 1048683 16624
2980 18788352 1730880 1048683 16624
2990 18788352 1599488 1048683 16624
3000 18788352 2221056 1048683 16624
3010 18788352 1227216 1048683 16624
3020 18788352 1850480 1048683 16624
3030 18788352 857088 1048683 16624
3040 18788352 1477920 1048683 16624
3050 18788352 2099856 1048683 16624
3060 18788352 1106016 1048683 16624
3070 18788352 1729264 1048683 16624
3080 18788352 1597872 1048683 16624
3090 18788352 2219440 1048683 16624
3100 18788352 1225552 1048683 16624
3110 18788352 1848864 1048683 16624
3120 18788352 855472 1048683 16624
3130 18788352 1476304 1048683 16624
3140 18788352 2098240 1048683 16624
3150 18788352 1104400 1048683 16624
3160 18788352 1727648 1048683 16624
3170 18788352 1596256 1048683 16624
3180 18788352 2217824 1048683 16624
3190 18788352 1223984 1048683 16624
3200 18788352 1847248 1048683 16624
3210 18788352 853856 1048683 16624
3220 18788352 1474688 1048683 16624
3230 18788352 2096624 1048683 16624
3240 18788352 1102800 1048683 16624
3250 18788352 1726032 1048683 16624
3260 18788352 1594640 1048683 16624
3270 18788352 2216208 1048683 16624
3280 18788352 1222368 1048683 16624
3290 18788352 1845632 1048683 16624
3300 18788352 852240 1048683 16624
3310 18788352 1473072 1048683 16624
3320 18788352 2095008 1048683 16624
3330 18788352 1101168 1048683 16624
3340 18788352 1724416 1048683 16624
3350 18788352 1593024 1048683 16624
3360 18788352 2214592 1048683 16624
3370 18788352 1220752 1048683 16624
3380 18788352 1844016 1048683 16624
3390 18788352 850624 1048683 16624
3400 18788352 1471456 1048683 16624
3410 18788352 2093392 1048683 16624
3420 18788352 1099552 1048683 16624
3430 18788352 1722800 1048683 16624
3440 18788352 1591408 1048683 16624
3450 18788352 2212976 1048683 16624
3460 18788352 1219136 1048683 16624
3470 18788352 1842400 1048683 16624
3480 18788352 847808 1048683 16624
3490 18788352 1471040 1048683 16624
3500 18788352 2091776 1048683 16624
3510 18788352 1097936 1048683 16624
3520 18788352 1721184 1048683 16624
3530 18788352 1589792 1048683 16624
3540 18788352 2211360 1048683 16624
3550 18788352 1217520 1048683 16624
3560 18788352 1840784 1048683 16624
3570 18788352 846192 1048683 16624
3580 18788352 1469424 1048683 16624
3590 18788352 2090160 1048683 16624
3600 18788352 1096320 1048683 16624
3610 18788352 1719568 1048683 16624
3620 18788352 1588176 1048683 16624
3630 18788352 2209744 1048683 16624
3640 18788352 1215904 1048683 16624
3650 18788352 1839168 1048683 16624
3660 18788352 844576 1048683 16624
3670 18788352 1467808 1048683 16624
3680 18788352 2088544 1048683 16624
3690 18788352 1094704 1048683 16624
3700 18788352 1717952 1048683 16624
3710 18788352 1586560 1048683 16624
3720 18788352 2208128 1048683 16624
3730 18788352 1214288 1048683 16624
3740 18788352 1837552 1048683 16624
3750 18788352 842960 1048683 16624
3760 18788352 1466192 1048683 16624
3770 18788352 2086928 1048683 16624
3780 18788352 1093088 1048683 16624
3790 18788352 1715360 1048683 16624
3800 18788352 1585664 1048683 16624
3810 18788352 2206512 1048683 16624
3820 18788352 1212672 1048683 16624
3830 18788352 1835936 1048683 16624
3840 18788352 841344 1048683 16624
3850 18788352 1464576 1048683 16624
3860 18788352 2085312 1048683 16624
3870 18788352 1091472 1048683 16624
3880 18788352 1713744 1048683 16624
3890 18788352 1584048 1048683 16624
3900 18788352 2204896 1048683 16624
3910 18788352 1211056 1048683 16624
3920 18788352 1834320 1048683 16624
3930 18788352 839728 1048683 16624
3940 18788352 1462960 1048683 16624
3950 18788352 2083696 1048683 16624
3960 18788352 1089856 1048683 16624
3970 18788352 1712128 1048683 16624
3980 18788352 1582432 1048683 16624
3990 18788352 2203280 1048683 16624
4000 18788352 1209440 1048683 16624
4010 18788352 1831680 1048683 16624
4020 18788352 839120 1048683 16624
4030 18788352 1460336 1048683 16624
4040 18788352 2082080 1048683 16624
4050 18788352 1088192 1048683 16624
4060 18788352 1710512 1048683 16624
4070 18788352 1580816 1048683 16624
4080 18788352 2201664 1048683 16624
4090 18788352 1207824 1048683 16624
4100 18788352 1831088 1048683 16624
4110 18788352 836496 1048683 16624
4120 18788352 1459728 1048683 16624
4130 18788352 2080464 1048683 16624
4140 18788352 1086624 1048683 16624
4150 18788352 1708896 1048683 16624
4160 18788352 1579200 1048683 16624
4170 18788352 2200048 1048683 16624
4180 18788352 1206208 1048683 16624
4190 18788352 1829472 1048683 16624
4200 18788352 834880 1048683 16624
4210 18788352 1458112 1048683 16624
4220 18788352 2078848 1048683 16624
4230 18788352 1085008 1048683 16624
4240 18788352 1707280 1048683 16624
4250 18788352 1577584 1048683 16624
4260 18788352 2198432 1048683 16624
4270 18788352 1204592 1048683 16624
4280 18788352 1827856 1048683 16624
4290 18788352 833264 1048683 16624
4300 18788352 1456496 1048683 16624
4310 18788352 2077232 1048683 16624
4320 18788352 1083392 1048683 16624
4330 18788352 1705664 1048683 16624
4340 18788352 1575968 1048683 16624
4350 18788352 2196816 1048683 16624
4360 18788352 1202976 1048683 16624
4370 18788352 1826240 1048683 16624
4380 18788352 831648 1048683 16624
4390 18788352 1454880 1048683 16624
4400 18788352 2075616 1048683 16624
4410 18788352 1081792 1048683 16624
4420 18788352 1704048 1048683 16624
4430 18788352 1574352 1048683 16624
4440 18788352 2195200 1048683 16624
4450 18788352 1201360 1048683 16624
4460 18788352 1824624 1048683 16624
4470 18788352 830032 1048683 16624
4480 18788352 1453264 1048683 16624
4490 18788352 2074000 1048683 16624
4500 18788352 1080160 1048683 16624
4510 18788352 1702432 1048683 16624
4520 18788352 1572736 1048683 16624
4530 18788352 2193584 1048683 16624
4540 18788352 1199744 1048683 16624
4550 18788352 1823008 1048683 16624
4560 18788352 828416 1048683 16624
4570 18788352 1451648 1048683 16624
4580 18788352 2072384 1048683 16624
4590 18788352 1078544 1048683 16624
4600 18788352 1700816 1048683 16624
4610 18788352 1571120 1048683 16624
4620 18788352 2191968 1048683 16624
4630 18788352 1198128 1048683 16624
4640 18788352 1821392 1048683 16624
4650 18788352 826800 1048683 16624
4660 18788352 1450032 1048683 16624
4670 18788352 2070768 1048683 16624
4680 18788352 1076928 1048683 16624
4690 18788352 1699200 1048683 16624
4700 18788352 1569504 1048683 16624
4710 18788352 2190352 1048683 16624
4720 18788352 1196512 1048683 16624
4730 18788352 1819776 1048683 16624
4740 18788352 825184 1048683 16624
4750 18788352 1448416 1048683 16624
4760 18788352 2069152 1048683 16624

[-- Attachment #7: Type: text/plain, Size: 143 bytes --]

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: cell heap usage in 1.8 vs 1.6

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 888 bytes --]

Hi,

ludovic.courtes@laas.fr (Ludovic Courtès) writes:

>   * the heap gets more and more underutilized, although it seems to
>     eventually stabilize somewhere around 5% (!) utilization (i.e.,
>     total/alive = 20);

The main reason appears to be that the size of individual segments grows
exponentially (it's computed in `scm_i_get_new_heap_segment ()').  This
can be seen by regularly dumping the result of the following expression:

  (map (lambda (elem)
         (abs (- (cdr elem) (car elem))))
       (assoc-ref (gc-stats) 'cell-heap-segments))

The default value of `scm_max_segment_size' is
`SCM_DEFAULT_MAX_SEGMENT_SIZE' and is as large as 20MiB (!).  Setting it
to 64KiB, using the `GUILE_MAX_SEGMENT_SIZE' environment variable,
reduces the total heap size (not shown here) by a factor of 2, and
significantly improves the total-to-alive heap ratio:


[-- Attachment #2: The thing --]
[-- Type: image/png, Size: 6511 bytes --]

[-- Attachment #3: Type: text/plain, Size: 604 bytes --]


A side effect of using a smaller maximum segment size is that
`scm_i_adjust_min_yield ()' gets called more frequently (from
`scm_gc_for_newcell ()'), which explains the "smoother" increases of the
min yield.

Using the script I posted earlier, I get the following execution times
for 2500 iterations:

  * max-segment-size = 64 KiB

    real    3m1.167s
    user    2m56.200s
    sys     0m2.220s

  * max-segment-size = 20 MiB

    real    3m29.047s
    user    3m22.392s
    sys     0m4.764s

Thus it would be sensible to change `SCM_DEFAULT_MAX_SEGMENT_SIZE' to
some smaller value.

Thanks,
Ludovic.

[-- Attachment #4: Type: text/plain, Size: 143 bytes --]

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel