From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Jim Porter Newsgroups: gmane.emacs.devel Subject: Re: [NonGNU ELPA] New package: llm Date: Sun, 20 Aug 2023 21:48:06 -0700 Message-ID: <705ab838-142a-b3cc-8cc8-6f4d143c4341@gmail.com> References: <54c21d90-8bd6-8723-9e33-d69179b37bd0@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="7504"; mail-complaints-to="usenet@ciao.gmane.io" Cc: ahyatt@gmail.com, emacs-devel@gnu.org To: Daniel Fleischer , Richard Stallman Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Aug 21 06:48:32 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qXwq8-0001o2-J9 for ged-emacs-devel@m.gmane-mx.org; Mon, 21 Aug 2023 06:48:32 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXwpq-0002AB-TO; Mon, 21 Aug 2023 00:48:14 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXwpp-00029z-Bi for emacs-devel@gnu.org; Mon, 21 Aug 2023 00:48:13 -0400 Original-Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXwpm-0005fl-8o; Mon, 21 Aug 2023 00:48:12 -0400 Original-Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1bf1935f6c2so18826205ad.1; Sun, 20 Aug 2023 21:48:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692593287; x=1693198087; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:mime-version:date:message-id:from:to:cc :subject:date:message-id:reply-to; bh=aylp4PQPJHM5J1fM6+2M19PaGba0mUp4hhqrqGj8BAk=; b=BX0BoKQTFqVRu43AOz/GyTzYNwoXAuJxiDhhS8ND8eitbrVf9GT3HcPvSC3v0dbL+n u92gk1HMMZC5FKvWmuXq6EgXDYA0RFP7EQvFZ21TCx3/c7sZDWkUX1JmUJoYR9SbNgbw X3xAkTajNQgriMYmHFo1+f1tlh+hEBIXmkVdLfOIPZ3u/nxz69X+MDl74BzwR/lg+3OG xbqwq2HNot9JoK452KuhPCuqXt5S4JLj/llZkW/nqoVjwi2nNddMu+Qc3tqRNbndch4S MR2dwGid0oB5jLdxIvtMiWQGY5mnNjhzaxooqJi4x55EHcPtA/+tsO8292tRINMMju6b MRkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692593287; x=1693198087; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=aylp4PQPJHM5J1fM6+2M19PaGba0mUp4hhqrqGj8BAk=; b=jY12f4p3SKYb8FW3iWw3NV6Vptcxc+Y97MUjonZXeipUmzdYHT7CLKZ3rPWil4i4nE Qud+ESC5yxjnnUeqLfWQTzJMAAB71zFOxmHFshTqDwqM4E1wQLAmYQHS0IMKotwxhEjP XnLKMyawoxSWvIib/wjk8WKN+8/qnhhut4v0V3meJ8hDk9KFaEVSxAjLAbh+LCK+FY6U 9Q9QooLhfb24o1g0r4J50WBnNrnN6qzfXmyXxS+t2iHc0zJEULqqE6npzPdQulzh/gvu XN9zXu4UUYgEHk2u9ILbczYHPh200ULTm5sIUu1/Z+b7GhpLo+HtIqYev6vSamrF3vDt E59g== X-Gm-Message-State: AOJu0YyjN1YGw+IfhzDO84kiR6EfcUeZxx82KdV/okHuQ8jp83n2pOW6 b3BjxwRwDoyOyiTGSNr3moc= X-Google-Smtp-Source: AGHT+IFUDXNbG9PDTRepJfUzsdU2V3cwhgGqOuLHwaSGqiRF3nBKNBbFZIWRVX0HjrHs4qF6w6Xj6g== X-Received: by 2002:a17:902:ced0:b0:1b8:8702:1e7c with SMTP id d16-20020a170902ced000b001b887021e7cmr9118211plg.33.1692593287324; Sun, 20 Aug 2023 21:48:07 -0700 (PDT) Original-Received: from [192.168.1.2] (cpe-76-168-148-233.socal.res.rr.com. [76.168.148.233]) by smtp.googlemail.com with ESMTPSA id y13-20020a1709027c8d00b001bdbe6c86a9sm5949829pll.225.2023.08.20.21.48.06 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 20 Aug 2023 21:48:06 -0700 (PDT) Content-Language: en-US In-Reply-To: Received-SPF: pass client-ip=2607:f8b0:4864:20::633; envelope-from=jporterbugs@gmail.com; helo=mail-pl1-x633.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:309019 Archived-At: On 8/17/2023 10:08 AM, Daniel Fleischer wrote: > That is not accurate; LLMs can definitely run locally on your machine. > Models can be downloaded and ran using Python. Here is an LLM released > under Apache 2 license [0]. There are "black-box" models, served in the > cloud, but the revolution we're is precisely because many models are > released freely and can be ran (and trained) locally, even on a laptop. > > [0] https://huggingface.co/mosaicml/mpt-7b The link says that this model has been pretrained, which is certainly useful for the average person who doesn't want (or doesn't have the resources) to perform the training themselves, but from the documentation, it's not clear how I *would* perform the training myself if I were so inclined. (I've only toyed with LLMs, so I'm not an expert at more "advanced" cases like this.) I do see that the documentation mentions the training datasets used, but it also says that "great efforts have been taken to clean the pretraining data". Am I able to access the cleaned datasets? I looked over their blog post[1], but I didn't see anything describing this in detail. While I certainly appreciate the effort people are making to produce LLMs that are more open than OpenAI (a low bar), I'm not sure if providing several gigabytes of model weights in binary format is really providing the *source*. It's true that you can still edit these models in a sense by fine-tuning them, but you could say the same thing about a project that only provided the generated output from GNU Bison, instead of the original input to Bison. (Just to be clear, I don't mean any of the above to be leading questions. I really don't know the answers, and using analogies to previous cases like Bison can only get us so far. I truly hope there *is* a freedom-respecting way to interface with LLMs, but I also think it's worth taking some extra care at the beginning so we can choose the right path forward.) [1] https://www.mosaicml.com/blog/mpt-7b