From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Shane Mulligan Newsgroups: gmane.emacs.devel Subject: Re: Help building Pen.el (GPT for emacs) Date: Sun, 18 Jul 2021 20:00:58 +1200 Message-ID: References: <83im1948mj.fsf@gnu.org> <83lf642jeh.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="0000000000003f4b4805c76139b5" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="13649"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Stefan Kangas , rms@gnu.org, Jean Louis , Emacs developers To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Jul 18 10:01:54 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1m51kH-0003Nw-IC for ged-emacs-devel@m.gmane-mx.org; Sun, 18 Jul 2021 10:01:53 +0200 Original-Received: from localhost ([::1]:37964 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m51kG-0000Sm-Jo for ged-emacs-devel@m.gmane-mx.org; Sun, 18 Jul 2021 04:01:52 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:41546) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m51jf-0008D3-RU for emacs-devel@gnu.org; Sun, 18 Jul 2021 04:01:15 -0400 Original-Received: from mail-yb1-xb36.google.com ([2607:f8b0:4864:20::b36]:35456) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m51jc-0003Et-Bh; Sun, 18 Jul 2021 04:01:15 -0400 Original-Received: by mail-yb1-xb36.google.com with SMTP id t186so22372258ybf.2; Sun, 18 Jul 2021 01:01:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=eW/wg1QqXmDM9tN8ypLFH6J3A9KeVbi15w6bCWptsD0=; b=U06CuG8WaiuTGKJYsyeEwqn6OB8XlDwmdQyNqDAQHnZ3SJtmpFY0E6Q1cFlKkZKqz8 O6uJqIjQ6g7FERYVZWyDzCzGjiBZp30N2BSbtRfjRo4gDrvneBkrbwLNn1G69bSSd9VS vN8tzB3SJLAqFccPlqJ5/Q2IIlIRP3wwSyXVcWEHuWvrDA6DgRCSUfx6/z+IBFo/6Bir y014GxXpIR6SW8zxz0NbeAhuO5UaWl+m7PZbkyQdbDjLi5Xv9Z67KkwzJeYfPZGOVuBX UqBcOxVlaoH2rHTny9o5jH1xtvsYAPDWx8z9/99lOiaCUSKDyMM6HRwznjSduKr1nitf UoCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=eW/wg1QqXmDM9tN8ypLFH6J3A9KeVbi15w6bCWptsD0=; b=OWTefCycQRyQjr99ABAxLkjNFEmNAuYCEkSJtVEfIXNCdwaXy9ONAOMnJ5dh245T7H 9wfcyBbjZYpne6Dqb48JCiTy0ujyKZNaHL/bRxLNwxCcqXytfAFtDNhX9wq7BtBDyFpJ 9NZRPfKzUZyqM3EsknKhEx3lkamLKCRok8fSQ4KEPP25SwyO/hSWsB2MmuSjQqaIYJhu PtF2weC5XxlYFjBrwotCWfQD71z1miPQPHUmG3fXBMnxatgCqe8bUUBYQXkS0X5fN5lC bAMKEqUIY7cdDTuH5d1LjPZby60GCxD6+TyTlhB3dxqm8pXtavDmAOzi5xNTx+qejakj qoNg== X-Gm-Message-State: AOAM530lZHDtO7A8EvtaYgI3VE8PSOfW7uvYwtgzXc3m//ms0CA50Dno Ubt1z4UzwmQuUJCyf88sH2UPzC0w63VT3d1B2tXR6yGh1ZwK96LXOQ== X-Google-Smtp-Source: ABdhPJzOQ0Dzc1zzEY24cdN5/pZVJEeD1o7SphY5ZixXUe3JBDTT/doKevmNrswxLNSiF3YApSawvoLNQnGMKRcniBU= X-Received: by 2002:a25:ed01:: with SMTP id k1mr23498997ybh.74.1626595269756; Sun, 18 Jul 2021 01:01:09 -0700 (PDT) In-Reply-To: <83lf642jeh.fsf@gnu.org> Received-SPF: pass client-ip=2607:f8b0:4864:20::b36; envelope-from=mullikine@gmail.com; helo=mail-yb1-xb36.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:271355 Archived-At: --0000000000003f4b4805c76139b5 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable * Thank you all for your attention I will move my conversations into emacs- tangents@gnu.org until asked otherwise to return. ** Response to Juri Linkov > Can it help to write git commit messages from diffs? > Has anyone tried to train a model on the existing > git commit logs? This would be a killer feature. Now you are catching on. I already have a task for myself to do this when I have to time. I am focusing on core features though. ** Response to Richard Stallman *** What does pen.el do? Pen.el stands for Prompt Engineering in emacs. Prompt Engineering is the art of describing what you would like a language model (transformer) to do. It is a new type of programming, example oriented; like literate programming, but manifested automatically. A transformer takes some text (called a prompt) and continues it. However, the continuation is the superset of all NLP tasks, as the generation can also be a classification, for instance. Those NLP tasks extend beyond world languages and into programming languages (whatever has been 'indexed' or 'learned') from these large LMs. Pen.el is an editing environment for designing 'prompts' to LMs. It is better than anything that exists, even at OpenAI or at Microsoft. I have been working on it and preparing for this for a long time. These prompts are example- based tasks. There are a number of design patterns which Pen.el is seeking to encode into a domain-specific language called 'examplary' for example- oriented programming. Pen.el creates functions 1:1 for a prompt to an emacs lisp function. Emacs is Grammarly, Google Translate, Copilot, Stackoveflow and infinitely many other services all rolled into one and allows you to have a private parallel to all these services that is completely private and open source -- that is if you have downloaded the EleutherAI model locally. *** What is its relation to GPT-3? Pen.el is GPL and completely separate from GPT-3 but currently GPT-3 is the only standardised service in which to model the the prompt-engineering workflow towards. No such API or standard exists yet and so I am designing my own interface and prompt format standard. *** "Perhaps that is a good path, but we need to know more." I encourage GNU to look into EleutherAI. I will continue to work with EleutherAI and make it a priority to bring this technology to emacs. However, I have absolutely no support by anyone and this project is too big for me alone. I'm focusing on core Pen.el features right now and will seek help from the EleutherAI community to build the open-source component and host the free GPT. ** Response to Jean Louis - And I do not think it should be in GNU ELPA due to above reasons. I am glad I have forewarned you guys. This is my current goal. Help in my project would be appreciated. I cannot do it alone and I cannot convince all of you. > Why don't you simply make an Emacs package as .tar as described in Emacs Lisp manual? Thank you for taking a look at my emacs package. It's not ready net for Melpa merge. I hope that I will be able to find some help in order to prepare it, but the rules are very strict and this may not happen. > How does that solves the licensing problems? The current EleutherAI model which competes with GPT-3 is GPT-Neo. It is MIT licensed. Also the data it has been trained on is MIT licensed. The current EleutherAI model which competes with Codex is GPT-j. It is licensed with Apache-2.0 License Both models are trained on The Pile, which is licensed MIT. https://github.com/EleutherAI/the-pile > "AI-related developments out there, and who purchased whom and for how much, is not appropriate." Understood. ** Thank you all for your time Best of luck and contact me any time. Sincerely, Shane Shane Mulligan How to contact me: =F0=9F=87=A6=F0=9F=87=BA 00 61 421 641 250 =F0=9F=87=B3=F0=9F=87=BF 00 64 21 1462 759 <+64-21-1462-759> mullikine@gmail.com On Sun, Jul 18, 2021 at 7:03 PM Eli Zaretskii wrote: > > Date: Sun, 18 Jul 2021 08:38:52 +0300 > > From: Jean Louis > > Cc: Eli Zaretskii , Stefan Kangas , > > rms@gnu.org, Emacs developers > > > > * Shane Mulligan [2021-07-18 00:03]: > > > Microsft GPT is an attack on the innermost workings of emacs -- the > text > > > stream. So embracing the OpenSource alternatives from EleutherAI is > > > crucial. > > > > How does that solves the licensing problems? > > Please take discussions of the GPT and OpenAI licensing to > emacs-tangents. It isn't relevant to this list. > --0000000000003f4b4805c76139b5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
* Thank you all for your attention
I will move my conve= rsations into emacs-
tangents@gnu.or= g until asked otherwise to
return.

** Response to Juri Linkov=
> Can it help to write git commit messages from diffs?
> Has a= nyone tried to train a model on the existing
> git commit logs?=C2=A0= This would be a killer feature.

Now you are catching on. I already = have a task
for myself to do this when I have to time. I
am focusing = on core features though.

** Response to Richard Stallman

***= What does pen.el do?
Pen.el stands for Prompt Engineering in emacs.
= Prompt Engineering is the art of describing what you would
like a= language model (transformer) to do. It is a new type of programming, examp= le oriented;=C2=A0
like literate programming, but manifested auto= matically.

A transformer takes some text (called a
prompt) and co= ntinues it. However, the
continuation is the superset of all NLP tasks,<= br>as the generation can also be a
classification, for instance. Those N= LP tasks
extend beyond world languages and into
programming languages= (whatever has been
'indexed' or 'learned') from these l= arge LMs.

Pen.el is an editing environment for designing
'pro= mpts' to LMs. It is better than anything
that exists, even at OpenAI= or at Microsoft. I
have been working on it and preparing for this
fo= r a long time.

These prompts are example- based tasks. There
are = a number of design patterns which Pen.el
is seeking to encode into a dom= ain-specific
language called 'examplary' for example-
oriente= d programming.

Pen.el creates functions 1:1 for a prompt to
an em= acs lisp function.

Emacs is Grammarly, Google Translate, Copilot,Stackoveflow and infinitely many other
services all rolled into one and= allows you to
have a private parallel to all these services
that is = completely private and open source --
that is if you have downloaded the= EleutherAI
model locally.

*** What is its relation to GPT-3?
= Pen.el is GPL and completely separate from
GPT-3 but currently GPT-3 is = the only
standardised service in which to model the
the prompt-engin= eering workflow towards. No
such API or standard exists yet and so I am<= br>designing my own interface and prompt format standard.

*** "= Perhaps that is a good path, but we need to know more."
I encourage= GNU to look into EleutherAI. I
will continue to work with EleutherAI an= d make
it a priority to bring this technology to
emacs.

Howeve= r, I have absolutely no support by
anyone and this project is too big fo= r me
alone. I'm focusing on core Pen.el features
right now and wi= ll seek help from the
EleutherAI community to build the open-source
c= omponent and host the free GPT.

** Response to Jean Louis
- And I= do not think it should be in GNU ELPA due to above reasons.

I am gl= ad I have forewarned you guys. This is
my current goal. Help in my proje= ct would be
appreciated. I cannot do it alone and I cannot
convince a= ll of you.

> Why don't you simply make an Emacs package as .t= ar as described in Emacs Lisp manual?
Thank you for taking a look at my = emacs
package. It's not ready net for Melpa merge. I
hope that I = will be able to find some help in
order to prepare it, but the rules are= very
strict and this may not happen.

> How does that solves t= he licensing problems?
The current EleutherAI model which competes with = GPT-3 is GPT-Neo.
It is MIT licensed.
Also the data it has been train= ed on is MIT licensed.

The current EleutherAI model which competes w= ith Codex is GPT-j.
It is licensed with Apache-2.0 License

Both m= odels are trained on The Pile, which is licensed MIT.

https://github.com/EleutherAI/the-pile=

> "AI-related developments out there, and who purchased= whom and for how much, is not appropriate."
Understood.

** = Thank you all for your time
Best of luck and contact me any time.
Sincerely,
Shane

Shane Mulligan

=
How to contact me:
3D""
=F0=9F=87=A6=F0=9F=87=BA00 61 421 641 250
=F0=9F= =87=B3=F0=9F=87=BF00 6= 4 21 1462 759
mullikine@gmail.com



On Sun, Jul 18, 2021 at= 7:03 PM Eli Zaretskii <eliz@gnu.org= > wrote:
>= Date: Sun, 18 Jul 2021 08:38:52 +0300
> From: Jean Louis <bugs@gnu.support>
> Cc: Eli Zaretskii <eliz@gnu.org>, Stefan Kangas <stefan@marxist.se>,
>=C2=A0 =C2=A0rms@gnu.o= rg, Emacs developers <emacs-devel@gnu.org>
>
> * Shane Mulligan <mullikine@gmail.com> [2021-07-18 00:03]:
> > Microsft GPT is an attack on the innermost workings of emacs -- t= he text
> > stream. So embracing the OpenSource alternatives from EleutherAI = is
> > crucial.
>
> How does that solves the licensing problems?

Please take discussions of the GPT and OpenAI licensing to
emacs-tangents.=C2=A0 It isn't relevant to this list.
--0000000000003f4b4805c76139b5--