From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Andy Wingo Newsgroups: gmane.lisp.guile.bugs Subject: bug#20079: Fwd: Memory leak from seek/ftell with files larger than 2GB Date: Thu, 23 Jun 2016 15:01:57 +0200 Message-ID: <87shw4njne.fsf@pobox.com> References: NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1466687011 18923 80.91.229.3 (23 Jun 2016 13:03:31 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 23 Jun 2016 13:03:31 +0000 (UTC) Cc: 20079-done@debbugs.gnu.org To: Anand Mohanadoss Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Thu Jun 23 15:03:21 2016 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bG4Hv-0006Ag-Pg for guile-bugs@m.gmane.org; Thu, 23 Jun 2016 15:03:20 +0200 Original-Received: from localhost ([::1]:36683 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bG4Hv-0006p3-4z for guile-bugs@m.gmane.org; Thu, 23 Jun 2016 09:03:19 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:35116) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bG4Hk-0006fs-LG for bug-guile@gnu.org; Thu, 23 Jun 2016 09:03:12 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bG4He-0002RC-VC for bug-guile@gnu.org; Thu, 23 Jun 2016 09:03:08 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:39578) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bG4He-0002Qm-Qt for bug-guile@gnu.org; Thu, 23 Jun 2016 09:03:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1bG4He-0000wy-GG for bug-guile@gnu.org; Thu, 23 Jun 2016 09:03:02 -0400 Resent-From: Andy Wingo Original-Sender: "Debbugs-submit" Resent-To: bug-guile@gnu.org Resent-Date: Thu, 23 Jun 2016 13:03:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: cc-closed 20079 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Mail-Followup-To: 20079@debbugs.gnu.org, wingo@pobox.com, anand108@gmail.com Original-Received: via spool by 20079-done@debbugs.gnu.org id=D20079.14666869343595 (code D ref 20079); Thu, 23 Jun 2016 13:03:02 +0000 Original-Received: (at 20079-done) by debbugs.gnu.org; 23 Jun 2016 13:02:14 +0000 Original-Received: from localhost ([127.0.0.1]:51915 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bG4Go-0000vs-SE for submit@debbugs.gnu.org; Thu, 23 Jun 2016 09:02:14 -0400 Original-Received: from pb-sasl1.pobox.com ([64.147.108.66]:65234 helo=sasl.smtp.pobox.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bG4Gk-0000sR-3L for 20079-done@debbugs.gnu.org; Thu, 23 Jun 2016 09:02:09 -0400 Original-Received: from sasl.smtp.pobox.com (unknown [127.0.0.1]) by pb-sasl1.pobox.com (Postfix) with ESMTP id AC1F721EF8; Thu, 23 Jun 2016 09:02:05 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; s=sasl; bh=+5dJZCKBitAFSrfAaK+mX2ZfDK8=; b=Q5wt51 l82FsGZYewehD1cynCL1FctkWDZTbh6URnzweLhgqY+gVgfqWDd/pI7E54ZKDpJo iNkVg2LGd3m/JdBSZXdpXWATsfM8UeL41lH79ZqP7N+E9LrnF0ApBrQjFTmKZPlP sbaM5qCEmJgGuwwWEqOqd7BkReCsfftpYEKX8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; q=dns; s=sasl; b=kAIuhh29qw9HTyMf8oT8DzAxPrUuuFuG GxedX2XBdUlYp1w2bFQjN6n7FU57fuAF8MEehh502QJoeycPPnFi4Z8IrgWqFIan xhEDT8t8lgk17nXjywxQCh/v2j7KwArPXE453ayOeQcPIcWZpHmxwRXWqDufTYt8 BUlraAci3dY= Original-Received: from pb-sasl1.nyi.icgroup.com (unknown [127.0.0.1]) by pb-sasl1.pobox.com (Postfix) with ESMTP id A212A21EF6; Thu, 23 Jun 2016 09:02:05 -0400 (EDT) Original-Received: from clucks (unknown [88.160.190.192]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pb-sasl1.pobox.com (Postfix) with ESMTPSA id BF45C21EF5; Thu, 23 Jun 2016 09:02:04 -0400 (EDT) In-Reply-To: (Anand Mohanadoss's message of "Wed, 11 Mar 2015 18:08:18 +0530") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) X-Pobox-Relay-ID: AF269310-3942-11E6-9342-C1836462E9F6-02397024!pb-sasl1.pobox.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.org gmane.lisp.guile.bugs:8155 Archived-At: Hi, Thank you very much for this one! Turns out we had an incredibly embarrassing bug in which we forgot to attach finalizers for bignums created by scm_from_{uint64,int64} on 32-bit platforms. Fixed in master and stable-2.0. Cheers, Andy On Wed 11 Mar 2015 13:38, Anand Mohanadoss writes: > Hi, > > I had sent the following to the user forum and did not receive any > comments. I am reposting it in the bug forum with the hope that one of > the experts may be able to comment... > > Thanks, > Anand > > ---------- Forwarded message ---------- > From: Anand Mohanadoss > Date: Wed, Feb 25, 2015 at 9:35 PM > Subject: Memory leak from seek/ftell with files larger than 2GB > To: guile-user@gnu.org > > Hi, > > We are seeing an issue with seek and ftell leaking memory with files > larger than 2GB. > > We are using 2.0.11 guile built as a 32-bit application with large > file support enabled (guile was built using gcc 4.4.0 for Linux with > flags _FILE_OFFSET_BITS=64, _LARGEFILE_SOURCE and _ > LARGEFILE64_SOURCE). The issue also appears to happen with guile 2.2. > > The memory leaks start only after the offset exceeds maximum positive > value for a 32-bit signed integer. ftell and seek do work as expected > (given how lseek should work with large file support). > > Appended is a program that illustrates the problem. The first seek > simply skips the part of the file where you won't see a memory leak. > If you comment out ftell and the second seek lines and un-comment the > lines that follow them, there is no memory leak. > > Is this a bug in guile or should we be doing things differently? If > this is a known issue, is there a recommended work around? > > Thanks, > Anand > > (define MAX_SIGNED_INT 2147483647) > (define BYTES_TO_READ 10) > > (define file "/tmp/test.pcap") ;sample file greater than 2.5GB > > (define (traverse file) > (let* ((port (open-input-file file #:binary #t)) > (file-sz (stat:size (stat port))) > (ua (make-bytevector BYTES_TO_READ 0)) > (cur-offset 0)) > (seek port (- MAX_UNSIGNED_INT 1000) SEEK_CUR) > (while (< (ftell port) (- file-sz BYTES_TO_READ)) > ;(while (< cur-offset (- file-sz BYTES_TO_READ)) > (seek port BYTES_TO_READ SEEK_CUR) > ;(get-bytevector-n! port ua 0 BYTES_TO_READ) > (set! cur-offset (+ BYTES_TO_READ cur-offset))) > (close-port port))) > > (traverse file)