Re: Help building Pen.el (GPT for emacs)

[Shane Mulligan <mullikine@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4706 bytes --]

* Thank you all for your attention
I will move my conversations into emacs-
tangents@gnu.org until asked otherwise to
return.

** Response to Juri Linkov
> Can it help to write git commit messages from diffs?
> Has anyone tried to train a model on the existing
> git commit logs?  This would be a killer feature.

Now you are catching on. I already have a task
for myself to do this when I have to time. I
am focusing on core features though.

** Response to Richard Stallman

*** What does pen.el do?
Pen.el stands for Prompt Engineering in emacs.
Prompt Engineering is the art of describing what you would
like a language model (transformer) to do. It is a new type of programming,
example oriented;
like literate programming, but manifested automatically.

A transformer takes some text (called a
prompt) and continues it. However, the
continuation is the superset of all NLP tasks,
as the generation can also be a
classification, for instance. Those NLP tasks
extend beyond world languages and into
programming languages (whatever has been
'indexed' or 'learned') from these large LMs.

Pen.el is an editing environment for designing
'prompts' to LMs. It is better than anything
that exists, even at OpenAI or at Microsoft. I
have been working on it and preparing for this
for a long time.

These prompts are example- based tasks. There
are a number of design patterns which Pen.el
is seeking to encode into a domain-specific
language called 'examplary' for example-
oriented programming.

Pen.el creates functions 1:1 for a prompt to
an emacs lisp function.

Emacs is Grammarly, Google Translate, Copilot,
Stackoveflow and infinitely many other
services all rolled into one and allows you to
have a private parallel to all these services
that is completely private and open source --
that is if you have downloaded the EleutherAI
model locally.

*** What is its relation to GPT-3?
Pen.el is GPL and completely separate from
GPT-3 but currently GPT-3 is the only
standardised service in which to model the
the prompt-engineering workflow towards. No
such API or standard exists yet and so I am
designing my own interface and prompt format standard.

*** "Perhaps that is a good path, but we need to know more."
I encourage GNU to look into EleutherAI. I
will continue to work with EleutherAI and make
it a priority to bring this technology to
emacs.

However, I have absolutely no support by
anyone and this project is too big for me
alone. I'm focusing on core Pen.el features
right now and will seek help from the
EleutherAI community to build the open-source
component and host the free GPT.

** Response to Jean Louis
- And I do not think it should be in GNU ELPA due to above reasons.

I am glad I have forewarned you guys. This is
my current goal. Help in my project would be
appreciated. I cannot do it alone and I cannot
convince all of you.

> Why don't you simply make an Emacs package as .tar as described in Emacs
Lisp manual?
Thank you for taking a look at my emacs
package. It's not ready net for Melpa merge. I
hope that I will be able to find some help in
order to prepare it, but the rules are very
strict and this may not happen.

> How does that solves the licensing problems?
The current EleutherAI model which competes with GPT-3 is GPT-Neo.
It is MIT licensed.
Also the data it has been trained on is MIT licensed.

The current EleutherAI model which competes with Codex is GPT-j.
It is licensed with Apache-2.0 License

Both models are trained on The Pile, which is licensed MIT.

https://github.com/EleutherAI/the-pile

> "AI-related developments out there, and who purchased whom and for how
much, is not appropriate."
Understood.

** Thank you all for your time
Best of luck and contact me any time.

Sincerely,
Shane

Shane Mulligan

How to contact me:
🇦🇺 00 61 421 641 250
🇳🇿 00 64 21 1462 759 <+64-21-1462-759>
mullikine@gmail.com


On Sun, Jul 18, 2021 at 7:03 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Sun, 18 Jul 2021 08:38:52 +0300
> > From: Jean Louis <bugs@gnu.support>
> > Cc: Eli Zaretskii <eliz@gnu.org>, Stefan Kangas <stefan@marxist.se>,
> >   rms@gnu.org, Emacs developers <emacs-devel@gnu.org>
> >
> > * Shane Mulligan <mullikine@gmail.com> [2021-07-18 00:03]:
> > > Microsft GPT is an attack on the innermost workings of emacs -- the
> text
> > > stream. So embracing the OpenSource alternatives from EleutherAI is
> > > crucial.
> >
> > How does that solves the licensing problems?
>
> Please take discussions of the GPT and OpenAI licensing to
> emacs-tangents.  It isn't relevant to this list.
>

[-- Attachment #2: Type: text/html, Size: 7249 bytes --]