From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Lars Ingebrigtsen Newsgroups: gmane.emacs.devel Subject: Re: sqlite3 Date: Mon, 06 Dec 2021 20:51:30 +0100 Message-ID: <87pmq9jyv1.fsf@gnus.org> References: <87tufmjyai.fsf@gnus.org> <87sfv5ljxn.fsf@gnus.org> <8735n5leza.fsf@gnus.org> <835ys11pxy.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="24381"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) Cc: emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Dec 06 20:53:24 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1muK39-000650-3D for ged-emacs-devel@m.gmane-mx.org; Mon, 06 Dec 2021 20:53:23 +0100 Original-Received: from localhost ([::1]:51200 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1muK37-0007Pg-JO for ged-emacs-devel@m.gmane-mx.org; Mon, 06 Dec 2021 14:53:21 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:55148) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1muK1Y-0004nQ-Sf for emacs-devel@gnu.org; Mon, 06 Dec 2021 14:51:45 -0500 Original-Received: from [2a01:4f9:2b:f0f::2] (port=58244 helo=quimby.gnus.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1muK1T-0003xM-EH; Mon, 06 Dec 2021 14:51:44 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID :In-Reply-To:Date:References:Subject:Cc:To:From:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=PA+QWpgTpY6vSHHN5vkxQZva/D6ZEdOHFsIqC5vl+c0=; b=oX2s1AOtPCT8F0EkLaL3LogAML jfW90N/Sbe38d+g8/I3S+hXKIyqzg5eCV7nX17dOe/CGkJ6w3YZ2is/Yb7BGqtSD6zVPdzQNCnNvT D1/YXE+WHjDhEXIH60TLOuXEAGB+VEo1wuOd2wmHQX4PmEEqCGHuN2Kj5fOy6OSr5Npw=; Original-Received: from [84.212.220.105] (helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1muK1L-0008Py-I2; Mon, 06 Dec 2021 20:51:35 +0100 Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwAgMAAAAqbBEUAAAABGdBTUEAALGPC/xhBQAAACBj SFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAADFBMVEXjza7QuqCogWD/ //+BhT4TAAAAAWJLR0QDEQxM8gAAAAd0SU1FB+UMBhMPFgfqJvsAAAGoSURBVCjPLVFNi9swEB0t 9kJvCcSB7LmU9l/IsAm0Jwn0RO1bIQm1f8eS8zZUhua0Kbax5ld2pO2A7XnWvA9JRHuqCkWPRhlH eKZV8aGiKpiCXFXWzkDVAZqg96pU2ihQRQ2l0vKoE61Tr/LLkk1/qwy8KJMI5NmGQELPvRKfQj4Z iV5TpOaBmeuJhrXOIJ5rR77OMxVtZCyUrwkUmfMH4b3PAnCkTHYFbesfh1dTSrwHAXBl78qkGCg8 fa/6YJE5QFf2k0HqyOJs+2iM2EhQzOi5hlYyajyHnltgb2Tj236SlEBdg1rfJjC4lbpT5zkVIqnu HbR8WJKPH2bmR74ctfpJCFMCg8SNku2t50u8CifQo1jy3/EaNQWqTi334/gpajURTrj55bhd0tiu w22Inw+LE+kvDQaOnVqMrDSo2lv8qkYjHGtw4/mbnbTcAoxreb4YAWmzo+fh+vKWQRMVhunlrtVA eOLS+tF5XQfCmW98Xo61Vr9oB56DX/7fHOQwpJ5XctYB0dtm4HbuQEXl7R2DP7W/11QCuwiPI/SG XDAfsXNkrd78A4oupznw7ZKaAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDIxLTEyLTA2VDE5OjE1OjIy KzAwOjAwsXG/LwAAACV0RVh0ZGF0ZTptb2RpZnkAMjAyMS0xMi0wNlQxOToxNToyMiswMDowMMAs B5MAAAAASUVORK5CYII= X-Now-Playing: Yukihiro Takahashi's _Neuromantic_: "Something in the Air" In-Reply-To: <835ys11pxy.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 06 Dec 2021 21:41:29 +0200") X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a01:4f9:2b:f0f::2 (failed) Received-SPF: pass client-ip=2a01:4f9:2b:f0f::2; envelope-from=larsi@gnus.org; helo=quimby.gnus.org X-Spam_score_int: -35 X-Spam_score: -3.6 X-Spam_bar: --- X-Spam_report: (-3.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:281160 Archived-At: Eli Zaretskii writes: >>>From what I see there, you completely ignore the text encoding issue. > SQLite needs UTF-8, but our internal representation is a superset of > UTF-8, so we need to decide what to do with the stuff that's beyond > UTF-8. Does SQLite check this and return an error indication if the > byte sequence we feed it is not a valid UTF-8? If it does check that, > we can rely on it, but we need to signal an error with a clear > explanation when it detects something like that. Alternatively, we > should check this ourselves before calling SQLite, and signal an error > on our own. Good catch. =F0=9F=98=80 I was scratching my head about that for about ten minutes -- did I forget to put a "POSSIBLY FIXME" in there? SQLite does expect UTF-8, but as far as I can tell, it doesn't really check anything. So the current code puts our private binary utf-8-private-plane-encoded things into SQLite -- and gets it back as is. (See the sqlite-char test for this stuff.) That is,=20 "insert into test2 (col1, col2) values ('f=C3=B3\x82o', 3)" and "insert into test2 (col1, col2) values ('f\x82o', 3)" both round-trip without any problems as is. Perhaps SQlite just sees it as a byte string? So I wasn't quite sure what to do here. > One other comment is the use of fixnums for integer values. Since > SQLite can use the full 64-bit width, I guess we need to use bignums > when the value exceeds a fixnum? Yup. --=20 (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no