From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: libnettle/libhogweed WIP Date: Fri, 21 Apr 2017 22:15:16 +0300 Message-ID: <837f2da7y3.fsf@gnu.org> References: <87lgr6yakj.fsf@lifelogs.com> <87wpamww9k.fsf@lifelogs.com> <8337daggnj.fsf@gnu.org> <87d1cdwxt6.fsf@lifelogs.com> <83tw5pg1q3.fsf@gnu.org> <87zifhulc2.fsf@lifelogs.com> <83h91og80k.fsf@gnu.org> <87pogbuhoe.fsf@lifelogs.com> <834lxndmd9.fsf@gnu.org> <87efwrug6z.fsf@lifelogs.com> <87r30qu5av.fsf@lifelogs.com> <874lxmtxyy.fsf@lifelogs.com> <87r30prvwt.fsf@lifelogs.com> <8337d4csez.fsf@gnu.org> <87r30nq9el.fsf@lifelogs.com> <83inlyc1k2.fsf@gnu.org> <87inlyrfni.fsf@lifelogs.com> <878tmurbhj.fsf@lifelogs.com> Reply-To: Eli Zaretskii NNTP-Posting-Host: blaine.gmane.org X-Trace: blaine.gmane.org 1492802101 27575 195.159.176.226 (21 Apr 2017 19:15:01 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 21 Apr 2017 19:15:01 +0000 (UTC) Cc: emacs-devel@gnu.org To: Lars Ingebrigtsen Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Fri Apr 21 21:14:52 2017 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1d1e16-0006sT-Ad for ged-emacs-devel@m.gmane.org; Fri, 21 Apr 2017 21:14:52 +0200 Original-Received: from localhost ([::1]:32851 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d1e1A-0007DJ-28 for ged-emacs-devel@m.gmane.org; Fri, 21 Apr 2017 15:14:56 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:50200) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d1e0u-000794-28 for emacs-devel@gnu.org; Fri, 21 Apr 2017 15:14:44 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d1e0o-0002nL-QL for emacs-devel@gnu.org; Fri, 21 Apr 2017 15:14:40 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:39717) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d1e0o-0002nH-Mc; Fri, 21 Apr 2017 15:14:34 -0400 Original-Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:2419 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1d1e0n-0005pv-Vi; Fri, 21 Apr 2017 15:14:34 -0400 In-reply-to: (message from Lars Ingebrigtsen on Fri, 21 Apr 2017 20:45:58 +0200) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:214197 Archived-At: > From: Lars Ingebrigtsen > Date: Fri, 21 Apr 2017 20:45:58 +0200 > > If I understand the code correctly (and I may definitely not be doing > that; I've just skimmed it very, very briefly), you may be able to point > the encryption code at the Emacs buffer contents directly without > copying it anywhere beforehand, and then (since the results are usually > of very similar length) back to the same Emacs buffer afterwards. > > 4GB Emacs buffer -> encrypted to 4GB GnuTLS buffer -> 4GB Emacs buffer But since (AFAIU) the size of the encrypted text is not always known in advance, the encryption might be significantly slower than just reading in the original file, because of the need to enlarge the gap (and thus reallocate text) several times. Similar things happen when you visit a compressed file, which triggers its on-the-fly decompression. > (with-temp-buffer > (set-buffer-multibyte nil) <<<<<<<<<<<<<<< this is not needed > (let ((coding-system-for-read 'binary) > (coding-system-for-write 'binary)) > (unwind-protect > (progn > (insert-file-contents "My DVD.iso") > (gnutls-encrypt ... ... (current-buffer)) > (write-region ...)) > (clear-buffer (current-buffer))))) ;; New function that runs memset > ;; over the buffer area > > Or something. We have to look at what buffers write-region creates and > stuff, but in the 'binary case, I don't think it creates copies of the > Emacs buffer anywhere. The data will always leave traces, because doing the above involves reallocation of memory, so you are likely to leave traces in the page file and in memory. But I don't think you can avoid that, whatever you do: as long as data needs to be read into memory to process it, it will always leave traces.