From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Mikael Djurfeldt Newsgroups: gmane.lisp.guile.devel Subject: Re: GPT-4 knows Guile! :) Date: Sat, 18 Mar 2023 09:58:28 +0100 Message-ID: References: Reply-To: mikael@djurfeldt.com Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000636d0505f728e622" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="5784"; mail-complaints-to="usenet@ciao.gmane.io" Cc: guile-devel@gnu.org To: tomas@tuxteam.de Original-X-From: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Sat Mar 18 09:59:00 2023 Return-path: Envelope-to: guile-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pdSOx-0001Kg-Qm for guile-devel@m.gmane-mx.org; Sat, 18 Mar 2023 09:58:59 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pdSOl-000329-HX; Sat, 18 Mar 2023 04:58:47 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pdSOi-00031m-R9 for guile-devel@gnu.org; Sat, 18 Mar 2023 04:58:44 -0400 Original-Received: from mail-ed1-f53.google.com ([209.85.208.53]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pdSOf-0002ZE-9r for guile-devel@gnu.org; Sat, 18 Mar 2023 04:58:43 -0400 Original-Received: by mail-ed1-f53.google.com with SMTP id er8so17156760edb.0 for ; Sat, 18 Mar 2023 01:58:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679129919; h=cc:to:subject:message-id:date:from:reply-to:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=nN2MM59FxQTut2jTBTmEB/eYjK2Yasihkh9htbSlREM=; b=HORlM60dy3CEDCKNhy/X0TWO8xKHPGIsg9Y77S2k3/xpvMXWuu5mQR2B3HZ02VBjOV PfszET6XPrbenoGyB28w1/WrLhkMvlIjs6j8z92We5IJYcHsHWoyU40VXCCNEA9CCdgL HoyzfDW45wxPWplDupUwSNIK2wnakmugudsBO2RIvyUXEo5bdWiVAp/4lmkdZRs3mMNd EOtsHbA75x+qxJI+z2lFDghY4wNRUAl8ty3T1y66R1x4j3x0MKD5Eo6jPvClrn+uk8o0 MFBgy9EKz08nvitr+aVFiiEireewvdIMNGRHKS6+UUi3BVmgzZa43Njpw4GHXEsLeXQK Fv4A== X-Gm-Message-State: AO0yUKVNxCF/eWI2pGTyTyZssHo8bN84xJ0FHOqslZHABuqurpeCsLFM S3C/vUm6P/aPiw9pesnfe752oGuJMbYZLbuvPUo= X-Google-Smtp-Source: AK7set84rHjVgyE4rDShSBqwmwctTJwivhE2/yg2AoJos9vzWX4OtFlSPVueD17xD59+DXLQtFKHsMv8sZ7o0mCLnW0= X-Received: by 2002:a50:d687:0:b0:4fc:6494:81c3 with SMTP id r7-20020a50d687000000b004fc649481c3mr3006038edi.1.1679129919580; Sat, 18 Mar 2023 01:58:39 -0700 (PDT) In-Reply-To: Received-SPF: pass client-ip=209.85.208.53; envelope-from=mdjurfeldt@gmail.com; helo=mail-ed1-f53.google.com X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.25, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.lisp.guile.devel:21792 Archived-At: --000000000000636d0505f728e622 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Mar 18, 2023 at 9:46=E2=80=AFAM wrote: > On Sat, Mar 18, 2023 at 09:41:37AM +0100, Mikael Djurfeldt wrote: > > On Sat, Mar 18, 2023 at 9:36=E2=80=AFAM wrote: > > [...] > > > > Perhaps you didn't know, but you are training the model :-) > > > > > > > Unfortunately not. I'm prompting it within its 32000 token (GPT-4) > > attention span. Next conversation the model is back to exactly the same > > state again. Then, of course, it is possible that OpenAI chooses to > filter > > out something from the dialogs it has had. > > You don't think that those coversations end up as raw data for the > next model? I'd be surprised, but you know definitely more than me. > I know very little apart from knowing what deep learning is and having skimmed the "Attention is all you need"-paper. I only meant that you are not training the model during and between sessions. It is certainly possible that OpenAI filters out things from the dialogs to use as part of training for the next version. They warn you that they may take data from the dialogs. If and how they do that I don't know. Cheers, Mikael --000000000000636d0505f728e622 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Sat, Mar 18, 2023 at 9:46=E2=80=AFAM &= lt;tomas@tuxteam.de> wrote:
On Sat, Mar 18, 2023 at 09:41:37AM +0100, Mikael Djurfeldt wrote:
> On Sat, Mar 18, 2023 at 9:36=E2=80=AFAM <tomas@tuxteam.de> wrote:

[...]

> > Perhaps you didn't know, but you are training the model :-) > >
>
> Unfortunately not. I'm prompting it within its 32000 token (GPT-4)=
> attention span. Next conversation the model is back to exactly the sam= e
> state again. Then, of course, it is possible that OpenAI chooses to fi= lter
> out something from the dialogs it has had.

You don't think that those coversations end up as raw data for the
next model? I'd be surprised, but you know definitely more than me.
=

I know very little apart from knowing what= deep learning is and having skimmed the "Attention is all you need&qu= ot;-paper. I only meant that you are not training the model during and betw= een sessions. It is certainly possible that OpenAI filters out things from = the dialogs to use as part of training for the next version. They warn you = that they may take data from the dialogs. If and how they do that I don'= ;t know.

Cheers,
Mikael
<= /div> --000000000000636d0505f728e622--