From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Petteri Hintsanen Newsgroups: gmane.emacs.help Subject: Re: [External] : Passing buffers to function in elisp Date: Fri, 24 Feb 2023 22:08:11 +0200 Message-ID: <87bklihln8.fsf@iki.fi> References: <87mt56hg4e.fsf@iki.fi> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="4115"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) To: help-gnu-emacs@gnu.org Cancel-Lock: sha1:FVwmwEr6EfzbgIEy4hrrlLvkq0I= Original-X-From: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane-mx.org@gnu.org Fri Feb 24 21:09:02 2023 Return-path: Envelope-to: geh-help-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pVeNK-0000mp-7d for geh-help-gnu-emacs@m.gmane-mx.org; Fri, 24 Feb 2023 21:09:02 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVeMk-0003Ah-79; Fri, 24 Feb 2023 15:08:26 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVeMi-0003AV-Mi for help-gnu-emacs@gnu.org; Fri, 24 Feb 2023 15:08:24 -0500 Original-Received: from ciao.gmane.io ([116.202.254.214]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVeMg-0007X5-QA for help-gnu-emacs@gnu.org; Fri, 24 Feb 2023 15:08:24 -0500 Original-Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1pVeMe-000ALt-7F for help-gnu-emacs@gnu.org; Fri, 24 Feb 2023 21:08:20 +0100 X-Injected-Via-Gmane: http://gmane.org/ Received-SPF: pass client-ip=116.202.254.214; envelope-from=geh-help-gnu-emacs@m.gmane-mx.org; helo=ciao.gmane.io X-Spam_score_int: -15 X-Spam_score: -1.6 X-Spam_bar: - X-Spam_report: (-1.6 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: help-gnu-emacs@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Users list for the GNU Emacs text editor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.help:142830 Archived-At: writes: > On Tue, Feb 21, 2023 at 11:21:47PM +0000, Drew Adams wrote: >> What is it that you're really trying to do? > > That's exactly the point, yes. Specifics, as usual, are somewhat messy. But I try to summarize below. I'm working with Ogg audio files, specifically Vorbis and Opus. I need to extract certain metadata from such files. This code is part of EMMS (see https://git.savannah.gnu.org/cgit/emms.git/tree/emms-info-native.el if you're curious, but please note that it's not the same version I'm working on.) Ogg file is basically a sequence of logical "pages", and each page has zero or more logical "packets". I need to read and decode the first two packets and the last page from a given file. Page size is bounded by 65307 bytes, while packets can be of any size (they can span multiple pages). My code extracts the first two packets by repeatedly reading and decoding a single page along its packet data ("payload"), until I have assembled two complete packets. These packets contain most of the metadata I'm interested in. Each page is read and decoded in somewhat wasteful manner by reading 65307 bytes worth of data from a certain offset (a page boundary) into a temporary buffer. So "func1" in my original posting is actually this: (defun emms-info-native--read-and-decode-ogg-page (filename offset) (with-temp-buffer (set-buffer-multibyte nil) (insert-file-contents-literally filename nil offset (+ offset emms-info-native--ogg-page-size)) (emms-info-native--decode-ogg-page (buffer-string)))) The function emms-info-native--decode-ogg-page uses bindat to do the actual decoding, and packs the results into a plist, which is then returned to the caller. I'm using separate function here because it is easy to test -- just supply fixed byte vectors for it and check that you get correct results. Calling code looks like this: (defun emms-info-native--decode-ogg-packets (filename packets) (let ((num-packets 0) (offset 0) (stream (vector))) (while (< num-packets packets) (let ((page (emms-info-native--read-and-decode-ogg-page filename offset))) (cl-incf num-packets (or (plist-get page :num-packets) 0)) (cl-incf offset (plist-get page :num-bytes)) (setq stream (vconcat stream (plist-get page :stream))) stream)) This function calls emms-info-native--read-and-decode-ogg-page in a loop until the desired number of packets has been extracted. So by evaluating (emms-info-native--decode-ogg-packets filename 2) I get what I need. All data is read-only in the sense it is read from the disk and then just copied around to alists, plists, vectors and so on. ----- I added a counter for tracking the number of temp buffers and ran a benchmark against some 3000+ Ogg files. This was done on primed cache so disk I/O should have had minimal effect. There were 12538 temp buffers created (= 12538 pages decoded). Benchmark function output was "Elapsed time: 23.806966s (18.743661s in 373 GCs)" So this means that ~78% of the time was spent on garbage collection? If so, I think my design sucks. ----- I am well aware that (preliminary) optimization is best avoided. Also, "when in doubt, use brute force." And even the current performance is good enough. My problem here is of more fundamental sort: I don't know what are the right data structures and calling conventions. I am still learning (emacs) lisp, and it shows. In C or C++ it is "easier": just pass pointers or references and you're good. With Lisp and especially Emacs Lisp things are more convoluted -- at least until you learn the necessary idioms. Thanks, Petteri