From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Andy Wingo Newsgroups: gmane.lisp.guile.bugs Subject: bug#20079: Fwd: Memory leak from seek/ftell with files larger than 2GB Date: Thu, 23 Jun 2016 14:22:33 +0200 Message-ID: <87y45wnlh2.fsf@pobox.com> References: NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: ger.gmane.org 1466684611 12597 80.91.229.3 (23 Jun 2016 12:23:31 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 23 Jun 2016 12:23:31 +0000 (UTC) Cc: 20079@debbugs.gnu.org To: Anand Mohanadoss Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Thu Jun 23 14:23:17 2016 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bG3fB-0005zf-AS for guile-bugs@m.gmane.org; Thu, 23 Jun 2016 14:23:17 +0200 Original-Received: from localhost ([::1]:36457 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bG3fA-0001i5-KP for guile-bugs@m.gmane.org; Thu, 23 Jun 2016 08:23:16 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:52462) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bG3f2-0001g9-El for bug-guile@gnu.org; Thu, 23 Jun 2016 08:23:09 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bG3ew-0000aR-Gb for bug-guile@gnu.org; Thu, 23 Jun 2016 08:23:07 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:39560) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bG3ew-0000aD-B2 for bug-guile@gnu.org; Thu, 23 Jun 2016 08:23:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1bG3ew-0006pK-5C for bug-guile@gnu.org; Thu, 23 Jun 2016 08:23:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Andy Wingo Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Thu, 23 Jun 2016 12:23:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20079 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 20079-submit@debbugs.gnu.org id=B20079.146668456526201 (code B ref 20079); Thu, 23 Jun 2016 12:23:02 +0000 Original-Received: (at 20079) by debbugs.gnu.org; 23 Jun 2016 12:22:45 +0000 Original-Received: from localhost ([127.0.0.1]:51897 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bG3ee-0006oW-U7 for submit@debbugs.gnu.org; Thu, 23 Jun 2016 08:22:45 -0400 Original-Received: from pb-sasl1.pobox.com ([64.147.108.66]:54641 helo=sasl.smtp.pobox.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bG3ed-0006oP-30 for 20079@debbugs.gnu.org; Thu, 23 Jun 2016 08:22:43 -0400 Original-Received: from sasl.smtp.pobox.com (unknown [127.0.0.1]) by pb-sasl1.pobox.com (Postfix) with ESMTP id 7B30921A99; Thu, 23 Jun 2016 08:22:41 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; s=sasl; bh=L7Nx0gZQ9xNCcnvQFgBotxJr4NY=; b=nD2Wmd /n15Zs5AUuUNqvuxSEzhotJsuhRY9/bL5RztWHsXfHQjf1XYP30GsoTwpFrvVZW7 6iTKW9phKssIdNRYY/8apyIjSaQUUYEyB1DnppC1Ko7zwRgeB0/WBtWYSRCQKV+4 7j9SXJNDiLYZK72JYbQaTliRmUPuZ5GuQSCn8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; q=dns; s=sasl; b=Y6Sp0oGHRZwL9q9Fc7Knb89agJ/UHg6u Lapw7ubG/rPrMisCKFq/CUvXkJtO09wknUyJSV7wPbFZj0OQ2MGQvIM580TNFnmQ jaqOGLt1gSitR/46RQPDA5jB3KHQgTpQ+cEVKNb64PwfKFToQUccK9t+s+vVAm6a BcOQvYLi10Q= Original-Received: from pb-sasl1.nyi.icgroup.com (unknown [127.0.0.1]) by pb-sasl1.pobox.com (Postfix) with ESMTP id 7247E21A98; Thu, 23 Jun 2016 08:22:41 -0400 (EDT) Original-Received: from clucks (unknown [88.160.190.192]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pb-sasl1.pobox.com (Postfix) with ESMTPSA id A70D121A97; Thu, 23 Jun 2016 08:22:40 -0400 (EDT) In-Reply-To: (Anand Mohanadoss's message of "Wed, 11 Mar 2015 18:08:18 +0530") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) X-Pobox-Relay-ID: 2DF36444-393D-11E6-96DB-C1836462E9F6-02397024!pb-sasl1.pobox.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.org gmane.lisp.guile.bugs:8154 Archived-At: --=-=-= Content-Type: text/plain Very strange bug! I can reproduce it with this file: --=-=-= Content-Type: text/plain Content-Disposition: inline; filename=foo.scm Content-Description: run as "guile foo.scm" (use-modules (rnrs bytevectors) (ice-9 binary-ports)) (define MAX_SIGNED_INT 2147483647) (define BYTES_TO_READ 10) (define (traverse port) (let* ((file-sz (stat:size (stat port))) (ua (make-bytevector BYTES_TO_READ 0)) (cur-offset 0)) (let lp ((cur-offset (seek port (- MAX_SIGNED_INT 1000) SEEK_CUR))) (when (< cur-offset (- file-sz BYTES_TO_READ)) (lp (seek port BYTES_TO_READ SEEK_CUR)))) (close-port port))) (define port (mkstemp! (string-copy "/tmp/big-file-XXXXXX"))) (truncate-file port #e2.5e9) (traverse port) --=-=-= Content-Type: text/plain I wonder what it could be! Andy On Wed 11 Mar 2015 13:38, Anand Mohanadoss writes: > Hi, > > I had sent the following to the user forum and did not receive any > comments. I am reposting it in the bug forum with the hope that one of > the experts may be able to comment... > > Thanks, > Anand > > ---------- Forwarded message ---------- > From: Anand Mohanadoss > Date: Wed, Feb 25, 2015 at 9:35 PM > Subject: Memory leak from seek/ftell with files larger than 2GB > To: guile-user@gnu.org > > Hi, > > We are seeing an issue with seek and ftell leaking memory with files > larger than 2GB. > > We are using 2.0.11 guile built as a 32-bit application with large > file support enabled (guile was built using gcc 4.4.0 for Linux with > flags _FILE_OFFSET_BITS=64, _LARGEFILE_SOURCE and _ > LARGEFILE64_SOURCE). The issue also appears to happen with guile 2.2. > > The memory leaks start only after the offset exceeds maximum positive > value for a 32-bit signed integer. ftell and seek do work as expected > (given how lseek should work with large file support). > > Appended is a program that illustrates the problem. The first seek > simply skips the part of the file where you won't see a memory leak. > If you comment out ftell and the second seek lines and un-comment the > lines that follow them, there is no memory leak. > > Is this a bug in guile or should we be doing things differently? If > this is a known issue, is there a recommended work around? > > Thanks, > Anand > > (define MAX_SIGNED_INT 2147483647) > (define BYTES_TO_READ 10) > > (define file "/tmp/test.pcap") ;sample file greater than 2.5GB > > (define (traverse file) > (let* ((port (open-input-file file #:binary #t)) > (file-sz (stat:size (stat port))) > (ua (make-bytevector BYTES_TO_READ 0)) > (cur-offset 0)) > (seek port (- MAX_UNSIGNED_INT 1000) SEEK_CUR) > (while (< (ftell port) (- file-sz BYTES_TO_READ)) > ;(while (< cur-offset (- file-sz BYTES_TO_READ)) > (seek port BYTES_TO_READ SEEK_CUR) > ;(get-bytevector-n! port ua 0 BYTES_TO_READ) > (set! cur-offset (+ BYTES_TO_READ cur-offset))) > (close-port port))) > > (traverse file) --=-=-=--