From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Philip Kaludercic Newsgroups: gmane.emacs.devel Subject: Re: [NonGNU ELPA] Add package gptel Date: Mon, 29 Apr 2024 18:21:16 +0000 Message-ID: <87r0eokp2b.fsf@posteo.net> References: <877cgi9m4w.fsf@gmail.com> <87ttjlsxro.fsf@posteo.net> <871q6pa0ts.fsf@gmail.com> <87le4wn12q.fsf@posteo.net> <87le4w8xun.fsf@gmail.com> <87h6fkmxse.fsf@posteo.net> <87frv484p8.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="34143"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org To: Karthik Chikmagalur Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Apr 29 20:22:11 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1s1Vdj-0008de-QF for ged-emacs-devel@m.gmane-mx.org; Mon, 29 Apr 2024 20:22:11 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s1Vd8-00043V-IM; Mon, 29 Apr 2024 14:21:35 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s1Vd2-00043A-Pm for emacs-devel@gnu.org; Mon, 29 Apr 2024 14:21:29 -0400 Original-Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s1Vcw-0003dT-OE for emacs-devel@gnu.org; Mon, 29 Apr 2024 14:21:27 -0400 Original-Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 9408F240103 for ; Mon, 29 Apr 2024 20:21:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1714414877; bh=XF4rtUOt4rrUEHrgRRDWyVGtBDA31Tp9DQUMGsZdxn8=; h=From:To:Cc:Subject:OpenPGP:Date:Message-ID:MIME-Version: Content-Type:From; b=Q3abtLi4CaYnI/8qOjMSYUfDZX6AEnOxm289QBhJmeNZZ2fBIhXLF7s00ejmquskI 5k/HbKcDbYgZJr2pSPARphpCkyvN5FLrq52rT9Cns5sWUcqpYa8FVmoVtkhSwwUCxR fssxaOVGOKTwODu15KjPvEf2R1cKSJi9nL8hwcYZvo8eHLvJjYCct6HqiiJ9RMy32F BSIr3MzaKEsTZDO52+94nh/5jF/SX2sLodH+jsvjE1+LipI/WyM/T0ufpnl8yGVgLX waUpAOLtEvBr3YJ0wEr5+oAYGIx+RXMhyZuFGQLoN3nY6aFENykmR42CWgioGUfcoy jYCtAzTC70Y1g== Original-Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4VSs8s0sxFz6trs; Mon, 29 Apr 2024 20:21:17 +0200 (CEST) In-Reply-To: <87frv484p8.fsf@gmail.com> (Karthik Chikmagalur's message of "Mon, 29 Apr 2024 10:21:55 -0700") OpenPGP: id=7126E1DE2F0CE35C770BED01F2C3CC513DB89F66; url="https://keys.openpgp.org/vks/v1/by-fingerprint/7126E1DE2F0CE35C770BED01F2C3CC513DB89F66"; preference=signencrypt Received-SPF: pass client-ip=185.67.36.66; envelope-from=philipk@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:318372 Archived-At: Karthik Chikmagalur writes: >>> This is no more dangerous than having that line of text at the top of >>> the buffer and sending the buffer contents as a query. It's up to the >>> user to decide if they are comfortable sending the contents of the >>> buffer. >> >> What do you mean by the top of the buffer? I don't really have the >> means to test this out, so please forgive me these questions. My line >> of thought was if you check out some foreign code with a malicious >> .dir-locals.el, you wouldn't realise that it could change this option. >> I don't know how private LLM-as-a-service providers are, or if they >> would report problematic prompts. > > It is essentially prepended to the buffer text in the query payload. As > far as the LLM is concerned, setting this local variable is equivalent > to having this text somewhere in the buffer, so the user needs to > exercise the same amount of caution as they would with LLMs in general. > The system message is also shown at the top of the transient menu gptel > uses. > > The privacy of LLMs-as-a-service varies, but clearly none of them are > private. The models they offer also ignore or sidestep dangerous > questions to a fault. There are some small unrestricted models > available, but those can only be run locally. OK, I didn't understand that the system messages are also displayed. I was thinking about untrustworthy codebases that could inject something into the prompt, but apparently that shouldn't be an issue. >>>> Does this have any significance? I am not familiar with the timeline. >>> >>> Only in that I expect many more users are familiar with gptel as a >>> result. >> >> Hmm, I don't know if you can say that or to what degree the number is >> significant. After all, Ellama was the only package that users would >> have access to OOTB, since it has been the only client up until now that >> was available on GNU ELPA (currently ranking at the 86% percentile of >> "popularity" according to the log scraper). > > Okay. > >>> Ollama, GPT4All and Llama.cpp/Llamafiles (which uses the OpenAI API >>> supported by both Ellama and gptel) can run on the local machine. >> >> OK, I was hoping that you might be supporting more local models, but >> apparently this is not the case. > > These are the only local options with HTTP APIs available right now. > There are several more local web applications with bespoke interfaces > but no API. > > When there are more I'll add support for them to gptel. So just to clarify, you do not intend to use the llm package as a dependency going forward? >> I recently uploaded a video to https://spectra.video/ and it was easy. >> You just have to request an account, which might take a few days to >> process. > > I'll take a look at available instances. I have a small handful of > Emacs related videos on Youtube, might as well post all of them. 1+ >> But no, none of this is blocking. I am just trying to help improve the >> package before we add it. The only blocking issue would be if it broke >> the NonGNU ELPA rules, e.g. by having a hard dependency on non-free >> software or SaaSS. > > Okay. > > Karthik Can you just add a .elpaignore file to your repository that would exclude the test/ directory? And would you be OK with us using the Commentary section in gptel.el for the package description generated by M-x describe-package? I feel it would be more readable than if we convert the README.org file to plain text. -- Philip Kaludercic on peregrine