Re: GPT-4 knows Guile! :)

[Mikael Djurfeldt <mikael@djurfeldt.com>]

[-- Attachment #1: Type: text/plain, Size: 1157 bytes --]

On Sat, Mar 18, 2023 at 9:46 AM <tomas@tuxteam.de> wrote:

> On Sat, Mar 18, 2023 at 09:41:37AM +0100, Mikael Djurfeldt wrote:
> > On Sat, Mar 18, 2023 at 9:36 AM <tomas@tuxteam.de> wrote:
>
> [...]
>
> > > Perhaps you didn't know, but you are training the model :-)
> > >
> >
> > Unfortunately not. I'm prompting it within its 32000 token (GPT-4)
> > attention span. Next conversation the model is back to exactly the same
> > state again. Then, of course, it is possible that OpenAI chooses to
> filter
> > out something from the dialogs it has had.
>
> You don't think that those coversations end up as raw data for the
> next model? I'd be surprised, but you know definitely more than me.
>

I know very little apart from knowing what deep learning is and having
skimmed the "Attention is all you need"-paper. I only meant that you are
not training the model during and between sessions. It is certainly
possible that OpenAI filters out things from the dialogs to use as part of
training for the next version. They warn you that they may take data from
the dialogs. If and how they do that I don't know.

Cheers,
Mikael

[-- Attachment #2: Type: text/html, Size: 1604 bytes --]