From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Daniel Fleischer Newsgroups: gmane.emacs.devel Subject: Re: [NonGNU ELPA] New package: llm Date: Sat, 19 Aug 2023 11:15:37 +0300 Message-ID: References: <54c21d90-8bd6-8723-9e33-d69179b37bd0@gmail.com> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="27570"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: emacs-devel@gnu.org To: Richard Stallman Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Aug 19 10:16:34 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qXH8M-0006zn-MG for ged-emacs-devel@m.gmane-mx.org; Sat, 19 Aug 2023 10:16:34 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXH7b-0004Zi-34; Sat, 19 Aug 2023 04:15:47 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXH7Z-0004Z5-Jo for emacs-devel@gnu.org; Sat, 19 Aug 2023 04:15:45 -0400 Original-Received: from mail-lj1-x22f.google.com ([2a00:1450:4864:20::22f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXH7W-0003Pw-Kq; Sat, 19 Aug 2023 04:15:44 -0400 Original-Received: by mail-lj1-x22f.google.com with SMTP id 38308e7fff4ca-2b9bb097c1bso25532241fa.0; Sat, 19 Aug 2023 01:15:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692432940; x=1693037740; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=Oc0AHMwvYySHECyKzngoeZ2xX57+iUs13YB91XD0Ua0=; b=FVbfLizyzodA3OHQ5/IXwBTilMqm4M6nvLPIYauPPC1geYsub9Fb0mcFwSdimBFa5A XSO4+oLcbwg95sORvplwHiye6w9j3sJfXD0yYtaZjQPKoj0NfkvRpV5gLODhTBx0hCyy cdbJxVW1psUOONgb5oylK9pca90W0EPmYzld41tHgaRp+9y71Lr62c43NpTj7CTUISL5 kSrDO8XGpbKIgSFat+l8DR7DVzikmKfJdzCDvxq2mU7kRBgFtO5p1Gu5qSR0J+przktg qIbcaftOK9rHfrb1V/+UHeZ6typ4NfwrxdE0D6Y9xlmi0jZm2F5ssQ3PiYOP/O75Dxnn 6vew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692432940; x=1693037740; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Oc0AHMwvYySHECyKzngoeZ2xX57+iUs13YB91XD0Ua0=; b=AJG2FvOu45osXhO4m7ro1QZIMNLu6dGAOgRPpIWSrtCdR/X+Sfkn20JA+2ZwpmhPVr A6QtoJogJdHIDIcLo4zseETDZgOEXN/ZTcGLuqA19pshVpF0W55UkAlRGbUFMfGdkUeE 82O5P1rmtVsXnDbM4GX33W0HvjZRRwIodOxYb42dtnl2Cbpnv0HiwEtbsf0mnV/VSiQq A4lZFY3d81dQW8rlyb1kt7trxihIMmrQhFdbGUTxwLpyF0YXHqd/kdHOl+gdYji7HjGP +6RQWQ+2nObNh2DmMWqh2ghI+ldiaPR3Db5EICUVmfPJVhrCU0o7QV37EsTal0JIFVez xbqA== X-Gm-Message-State: AOJu0YyJqZX0ys/xKt4Rw7d7bXsrA4ido0CnAPXrWbFio5kit9uqR4iC Kv4f36ZSk8NWU8uxIbG+YTxIeV43n+w= X-Google-Smtp-Source: AGHT+IEbKlz+kQpYcQ2IIWxlvmn2QSKry1QY4MPplXsTIW23y3UjNNq1MT8YQvyEwIWyiJNk+vnbmQ== X-Received: by 2002:a2e:6e0e:0:b0:2bb:8eea:7558 with SMTP id j14-20020a2e6e0e000000b002bb8eea7558mr940839ljc.13.1692432939724; Sat, 19 Aug 2023 01:15:39 -0700 (PDT) Original-Received: from Home-Mac ([141.226.169.23]) by smtp.gmail.com with ESMTPSA id y26-20020a7bcd9a000000b003fed7fa6c00sm3321173wmj.7.2023.08.19.01.15.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Aug 2023 01:15:39 -0700 (PDT) In-Reply-To: (Richard Stallman's message of "Fri, 18 Aug 2023 21:49:40 -0400") Received-SPF: pass client-ip=2a00:1450:4864:20::22f; envelope-from=danflscr@gmail.com; helo=mail-lj1-x22f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:308917 Archived-At: Local LLMs usually run using the python `transformers' library; in order to interact with them using a REST API, some glue code is needed, for example: https://github.com/go-skynet/LocalAI The API is based on OpenAI.com which is what others are following and thus are relevant for the API access the llm package is going to offer. Richard Stallman writes: > We are slightly miscommunicating. Yes there are models that could run > locally on your machine, but all the ones I know of were released > under a nonfree license. > Could you confirm that this is a language model itself, not the > program that runs the language model? The most popular software framework for running LLMs is called `transformers' (named after the models' architecture): https://github.com/huggingface/transformers (Apache 2) Huggingface also offers free hosting for models and data sets. There are several families of free models: - XGEN https://huggingface.co/Salesforce/xgen-7b-8k-base - MPT https://huggingface.co/mosaicml/mpt-7b - Falcon https://huggingface.co/tiiuae/falcon-7b These are git project, e.g. see https://huggingface.co/tiiuae/falcon-7b/tree/main. These models are released under Apache 2. The models contains the weights (compressed numerical matrices) and possibly some Python code files needed and they explicitly depend on the `transformers' library and the `pytorch' neural networks library (BSD-3). -- Daniel Fleischer