Re: [NonGNU ELPA] New package: llm

[Jim Porter <jporterbugs@gmail.com>]

On 8/17/2023 10:08 AM, Daniel Fleischer wrote:
> That is not accurate; LLMs can definitely run locally on your machine.
> Models can be downloaded and ran using Python. Here is an LLM released
> under Apache 2 license [0]. There are "black-box" models, served in the
> cloud, but the revolution we're is precisely because many models are
> released freely and can be ran (and trained) locally, even on a laptop.
> 
> [0] https://huggingface.co/mosaicml/mpt-7b

The link says that this model has been pretrained, which is certainly 
useful for the average person who doesn't want (or doesn't have the 
resources) to perform the training themselves, but from the 
documentation, it's not clear how I *would* perform the training myself 
if I were so inclined. (I've only toyed with LLMs, so I'm not an expert 
at more "advanced" cases like this.)

I do see that the documentation mentions the training datasets used, but 
it also says that "great efforts have been taken to clean the 
pretraining data". Am I able to access the cleaned datasets? I looked 
over their blog post[1], but I didn't see anything describing this in 
detail.

While I certainly appreciate the effort people are making to produce 
LLMs that are more open than OpenAI (a low bar), I'm not sure if 
providing several gigabytes of model weights in binary format is really 
providing the *source*. It's true that you can still edit these models 
in a sense by fine-tuning them, but you could say the same thing about a 
project that only provided the generated output from GNU Bison, instead 
of the original input to Bison.

(Just to be clear, I don't mean any of the above to be leading 
questions. I really don't know the answers, and using analogies to 
previous cases like Bison can only get us so far. I truly hope there 
*is* a freedom-respecting way to interface with LLMs, but I also think 
it's worth taking some extra care at the beginning so we can choose the 
right path forward.)

[1] https://www.mosaicml.com/blog/mpt-7b