From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Anand Mohanadoss Newsgroups: gmane.lisp.guile.bugs Subject: bug#20079: Fwd: Memory leak from seek/ftell with files larger than 2GB Date: Wed, 11 Mar 2015 18:08:18 +0530 Message-ID: References: NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=089e01537f62b320b20511028a18 X-Trace: ger.gmane.org 1426077560 19979 80.91.229.3 (11 Mar 2015 12:39:20 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 11 Mar 2015 12:39:20 +0000 (UTC) To: 20079@debbugs.gnu.org Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Wed Mar 11 13:39:12 2015 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1YVfup-0002Nj-Dg for guile-bugs@m.gmane.org; Wed, 11 Mar 2015 13:39:11 +0100 Original-Received: from localhost ([::1]:54399 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YVfuo-0006IN-K6 for guile-bugs@m.gmane.org; Wed, 11 Mar 2015 08:39:10 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:39413) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YVfuj-0006I6-GH for bug-guile@gnu.org; Wed, 11 Mar 2015 08:39:06 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YVfuh-0008Ls-26 for bug-guile@gnu.org; Wed, 11 Mar 2015 08:39:05 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:43788) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YVfug-0008Lh-VC for bug-guile@gnu.org; Wed, 11 Mar 2015 08:39:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1YVfug-0005OJ-Lm for bug-guile@gnu.org; Wed, 11 Mar 2015 08:39:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Anand Mohanadoss Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Wed, 11 Mar 2015 12:39:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 20079 X-GNU-PR-Package: guile X-GNU-PR-Keywords: X-Debbugs-Original-To: bug-guile@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.142607751220680 (code B ref -1); Wed, 11 Mar 2015 12:39:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 11 Mar 2015 12:38:32 +0000 Original-Received: from localhost ([127.0.0.1]:42356 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YVfuB-0005NT-N2 for submit@debbugs.gnu.org; Wed, 11 Mar 2015 08:38:32 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:37602) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YVfu9-0005N9-1D for submit@debbugs.gnu.org; Wed, 11 Mar 2015 08:38:29 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YVfu2-0007sx-9o for submit@debbugs.gnu.org; Wed, 11 Mar 2015 08:38:23 -0400 Original-Received: from lists.gnu.org ([2001:4830:134:3::11]:56954) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YVfu2-0007st-6B for submit@debbugs.gnu.org; Wed, 11 Mar 2015 08:38:22 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:39319) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YVfu0-0006Hc-Tk for bug-guile@gnu.org; Wed, 11 Mar 2015 08:38:22 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YVftz-0007rD-KD for bug-guile@gnu.org; Wed, 11 Mar 2015 08:38:20 -0400 Original-Received: from mail-ob0-x22f.google.com ([2607:f8b0:4003:c01::22f]:35398) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YVftz-0007qM-AB for bug-guile@gnu.org; Wed, 11 Mar 2015 08:38:19 -0400 Original-Received: by obcwo20 with SMTP id wo20so8496517obc.2 for ; Wed, 11 Mar 2015 05:38:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=+hjBN9N/jJuZ9BPtMndgDn02v5in5pXYkqJcFIrxp20=; b=I0iRrsmWqcRd8ibESmwiSRyNRm85xYJ959j3eiNjNL1RpCEnxm/oUVc0kCOmdADlC2 O/iVRuhi4+ZMoR99n6PNbL28Mv6xhhgl6GDoNzr5zDKYd5/ih/z3EG9qBaxiufxAN7BV NzD+kahOkPUoQByf/WpsVhEMui8ADoVrVFz36OMKuauouVmY29r7N1BUqMm07Kwqda9c 6QhIdVg93gi3YOPk4GPs8NYSgl2nB9f4r8cU6gtqr72VwcC0c5tdVXdpNUJo+FAfFCTz tH+gtJ+EXwP90LPbgzvaOknehCw3voPxMCeGUZ0am74zrGr6xjbJMBedjF15y2UAhcr+ xE0w== X-Received: by 10.182.65.71 with SMTP id v7mr29738780obs.42.1426077498152; Wed, 11 Mar 2015 05:38:18 -0700 (PDT) Original-Received: by 10.202.195.9 with HTTP; Wed, 11 Mar 2015 05:38:18 -0700 (PDT) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 140.186.70.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.bugs:7746 Archived-At: --089e01537f62b320b20511028a18 Content-Type: text/plain; charset=UTF-8 Hi, I had sent the following to the user forum and did not receive any comments. I am reposting it in the bug forum with the hope that one of the experts may be able to comment... Thanks, Anand ---------- Forwarded message ---------- From: Anand Mohanadoss Date: Wed, Feb 25, 2015 at 9:35 PM Subject: Memory leak from seek/ftell with files larger than 2GB To: guile-user@gnu.org Hi, We are seeing an issue with seek and ftell leaking memory with files larger than 2GB. We are using 2.0.11 guile built as a 32-bit application with large file support enabled (guile was built using gcc 4.4.0 for Linux with flags _FILE_OFFSET_BITS=64, _LARGEFILE_SOURCE and _LARGEFILE64_SOURCE). The issue also appears to happen with guile 2.2. The memory leaks start only after the offset exceeds maximum positive value for a 32-bit signed integer. ftell and seek do work as expected (given how lseek should work with large file support). Appended is a program that illustrates the problem. The first seek simply skips the part of the file where you won't see a memory leak. If you comment out ftell and the second seek lines and un-comment the lines that follow them, there is no memory leak. Is this a bug in guile or should we be doing things differently? If this is a known issue, is there a recommended work around? Thanks, Anand (define MAX_SIGNED_INT 2147483647) (define BYTES_TO_READ 10) (define file "/tmp/test.pcap") ;sample file greater than 2.5GB (define (traverse file) (let* ((port (open-input-file file #:binary #t)) (file-sz (stat:size (stat port))) (ua (make-bytevector BYTES_TO_READ 0)) (cur-offset 0)) (seek port (- MAX_UNSIGNED_INT 1000) SEEK_CUR) (while (< (ftell port) (- file-sz BYTES_TO_READ)) ;(while (< cur-offset (- file-sz BYTES_TO_READ)) (seek port BYTES_TO_READ SEEK_CUR) ;(get-bytevector-n! port ua 0 BYTES_TO_READ) (set! cur-offset (+ BYTES_TO_READ cur-offset))) (close-port port))) (traverse file) --089e01537f62b320b20511028a18 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi,

I had sent the following t= o the user forum and did not receive any comments.=C2=A0 I am reposting it = in the bug forum with the hope that one of the experts may be able to comme= nt...

Thanks,
Anand

---------- Forwarded message ----------
From: Anand Mohanadoss <anand108@gmail.com>
Date: W= ed, Feb 25, 2015 at 9:35 PM
Subject: Memory leak from seek/ftell with fi= les larger than 2GB
To: guile-user= @gnu.org


Hi,<= br>
We are seeing an issue with seek and ftell leaking memory with= files larger than 2GB.

We are using 2.0.11 guile built as a 3= 2-bit application with large file support enabled (guile was built using gc= c 4.4.0 for Linux with flags _FILE_OFFSET_BITS=3D64, _LARGEFILE_SOURCE and = _LARGEFILE64_SOURCE).=C2=A0 The issue also appears to happen with guile 2.2= .

The memory leaks start only after the offset exceeds ma= ximum positive value for a 32-bit signed integer. ftell and seek do work as= expected (given how lseek should work with large file support).

Appended is a program that illustrates the problem.=C2=A0 The first seek= simply skips the part of the file where you won't see a memory leak. I= f you comment out ftell and the second seek lines and un-comment the lines = that follow them, there is no memory leak.=C2=A0

Is this a bu= g in guile or should we be doing things differently?=C2=A0 If this is a kno= wn issue, is there a recommended work around?

Thanks,
Anand
=C2=A0
(define MAX_SIGNED_= INT 214= 7483647)
(define BYTES_TO_READ 10)

(define file "/tmp/te= st.pcap")=C2=A0 ;sample file greater than 2.5GB

(define (traver= se file)
=C2=A0(let* ((port (open-input-file file #:binary #t))
=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (file-sz (stat:size (stat port)))=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (ua (make-bytevector BYTES_TO_= READ 0))
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (cur-offset 0))
= =C2=A0=C2=A0 (seek port (- MAX_UNSIGNED_INT 1000) SEEK_CUR)
=C2=A0=C2=A0= (while (< (ftell port) (- file-sz BYTES_TO_READ))
=C2=A0=C2=A0 ;(whi= le (< cur-offset (- file-sz BYTES_TO_READ))
=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 (seek=C2=A0 port BYTES_TO_READ SEEK_CUR)
=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 ;(get-bytevector-n! port ua 0 BYTES_TO_READ)
=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (set! cur-offset (+ BYTES_TO_READ cur-offset= )))
=C2=A0=C2=A0 (close-port port)))

(traverse file)

--089e01537f62b320b20511028a18--