From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Carlos O'Donell Newsgroups: gmane.emacs.bugs Subject: bug#43389: 28.0.50; Emacs memory leaks using hard disk all time Date: Wed, 25 Nov 2020 12:45:04 -0500 Organization: Red Hat Message-ID: <522e3cc0-c563-3308-7264-1b09cd5e264b@redhat.com> References: <86y2j2brg2.fsf@protected.rcdrun.com> <83blfxth7c.fsf@gnu.org> <83y2j0qb2v.fsf@gnu.org> <831rgppg3w.fsf@gnu.org> <83zh3czbvz.fsf@gnu.org> <83blfovzxz.fsf@gnu.org> <87o8jnu5f2.fsf@mail.trevorbentley.com> <83o8jmu49z.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="26590"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.4.0 Cc: fweimer@redhat.com, 43389@debbugs.gnu.org, bugs@gnu.support, dj@redhat.com, michael_heerdegen@web.de To: Eli Zaretskii , Trevor Bentley Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Nov 25 18:46:16 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1khyrv-0006lU-5U for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 25 Nov 2020 18:46:15 +0100 Original-Received: from localhost ([::1]:55456 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1khyru-0004MQ-7E for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 25 Nov 2020 12:46:14 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:51688) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1khyri-0004MC-HZ for bug-gnu-emacs@gnu.org; Wed, 25 Nov 2020 12:46:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:53547) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1khyri-0004B1-8J for bug-gnu-emacs@gnu.org; Wed, 25 Nov 2020 12:46:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1khyri-0004UD-6T for bug-gnu-emacs@gnu.org; Wed, 25 Nov 2020 12:46:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Carlos O'Donell Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 25 Nov 2020 17:46:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43389 X-GNU-PR-Package: emacs Original-Received: via spool by 43389-submit@debbugs.gnu.org id=B43389.160632631417186 (code B ref 43389); Wed, 25 Nov 2020 17:46:02 +0000 Original-Received: (at 43389) by debbugs.gnu.org; 25 Nov 2020 17:45:14 +0000 Original-Received: from localhost ([127.0.0.1]:36860 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1khyqw-0004T7-FW for submit@debbugs.gnu.org; Wed, 25 Nov 2020 12:45:14 -0500 Original-Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:52448) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1khyqt-0004Sy-Rx for 43389@debbugs.gnu.org; Wed, 25 Nov 2020 12:45:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1606326311; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2h4Rj/qP+2HOPw4ugkTWm5Do9UVpuWTGL+C+BRxiyp8=; b=NsJ3nY1t3gwMVwo3CiDFuXq3vHUzY1YrcEmCK36wHVtxr2o0fGhj8EYPdinVWbN+Tm1aQP fGAWpaMq9hWGv/zOMZ+m9as7+UT92lMIY7gb90tZGwzBPTHus0GbNmKXjP17wgyPOcfsIt UVWpeISapfwL2yo3rPx1RywcaKts4yc= Original-Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-189-bQvFgBUfMjeL4YZ7gf_8sg-1; Wed, 25 Nov 2020 12:45:08 -0500 X-MC-Unique: bQvFgBUfMjeL4YZ7gf_8sg-1 Original-Received: by mail-qt1-f198.google.com with SMTP id r29so2995691qtu.21 for <43389@debbugs.gnu.org>; Wed, 25 Nov 2020 09:45:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=2h4Rj/qP+2HOPw4ugkTWm5Do9UVpuWTGL+C+BRxiyp8=; b=JehC5ifdyCEc8+tu1yL+nFzZbBhg89q81a9EYOe2AutR8EYQ0vdtorMzUm/bdHJ+u+ M0yYS8c7OJWJ1UHcos0GVaf7rD3i/QEcP8nnK00LGD0sCKDVVLE++KKPzifVLBdaZSFV 2Bv8SOva8Rzr25dcUIFkCe3G4H9htkDO5m7S9eqPPeri2Ok6fzGEt+69p/WduhX1XQ/s K5IMfltMGZhcXJgp4MJCamaOWY+FcA9Z4RLabhEOir6xEwyN5sa1PYTBHkqyiIPzlTsL rTjGhoT7nBgGAnSb1kPxEfwhklHzr1bvlCuf/082EqkrGnm9ZP0hikiyasxxkCvLnm+5 Tf+g== X-Gm-Message-State: AOAM532ywQbXBy5pR8Jent5mXll7LFI9SzfDaiuHwCmUJkLfRVbBB/vN aM4u/WzEicpDH8Lkx3QvOYmpJO3K8yOx61ZJfDyH3eOadJ4VR4gZJ2zYyWjO+BFmdMGiBdpqy25 kK5HcOFTF9mrbfho= X-Received: by 2002:ad4:4721:: with SMTP id l1mr4666231qvz.30.1606326307877; Wed, 25 Nov 2020 09:45:07 -0800 (PST) X-Google-Smtp-Source: ABdhPJxrmLNaFgXompfA8yf4j9JCh7H26mNprKE1tP3doB84wyVy4SppMajVxh/jnt7lFaMXThPyvQ== X-Received: by 2002:ad4:4721:: with SMTP id l1mr4666200qvz.30.1606326307636; Wed, 25 Nov 2020 09:45:07 -0800 (PST) Original-Received: from [192.168.1.16] (198-84-214-74.cpe.teksavvy.com. [198.84.214.74]) by smtp.gmail.com with ESMTPSA id q32sm3130610qtb.71.2020.11.25.09.45.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 25 Nov 2020 09:45:06 -0800 (PST) In-Reply-To: <83o8jmu49z.fsf@gnu.org> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=carlos@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:194211 Archived-At: On 11/24/20 11:07 AM, Eli Zaretskii wrote: > Look at the large chunks in the tail of this. Together, they do > account for ~2GB. > > Carlos, are these chunks in use (i.e. allocated and not freed), or are > they the free chunks that are available for allocation, but not > released to the OS? If the former, then it sounds like this session > does have around 2GB of allocated heap data, so either there's some > allocated memory we don't account for, or there is indeed a memory > leak in Emacs. If these are the free chunks, then the way glibc > manages free'd memory is indeed an issue. These chunks are all free and mapped for use by the algorithm to satisfy a request by the application. Looking at the last malloc_info (annotated): https://trevorbentley.com/emacs_malloc_info.log =============================================== ;; malloc-info (malloc-info) => No fast bins. => 1 unused bin. => In total we have only 112KiB in 1 unused chunk free'd on the stack. => The rest of the stack is in use by the application. => It looks like the application usage goes down to zero and then up again? => Currently at 4.2GiB in arena 0 (kernel assigned heap). => The application is using that sbrk'd memory. => This indicates *real* API usage of 4.2GiB. => This is arena 1, which is a thread heap, and uses mmap to create heaps. => Pretty small, 912 bytes in fastbins, and 42MiB in cached chunks. =============================================== This shows the application is USING memory on the main system heap. It might not be "leaked" memory since the application might be using it. You want visibility into what is USING that memory. With glibc-malloc-trace-utils you can try to do that with: LD_PRELOAD=libmtrace.so \ MTRACE_CTL_FILE=/home/user/app.mtr \ MTRACE_CTL_BACKTRACE=1 \ ./app This will use libgcc's unwinder to get a copy of the malloc caller address and then we'll have to decode that based on a /proc/self/maps. Next steps: - Get a glibc-malloc-trace-utils trace of the application ratcheting. - Get a copy of /proc/$PID/maps for the application (shorter version of smaps). Then we might be able to correlate where all the kernel heap data went? -- Cheers, Carlos.