From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#54591: 29.0.50; sqlite-select returns blob result as multibyte string Date: Sat, 02 Apr 2022 09:52:31 +0300 Message-ID: <83lewo3rkg.fsf@gnu.org> References: <83h77jaof6.fsf@gnu.org> <87lewsakng.fsf@gnus.org> <83o81o93ak.fsf@gnu.org> <87a6d672xr.fsf@gnus.org> <878rso7iuu.fsf@flokut.localdomain> <83r16g3vav.fsf@gnu.org> <87ilrsro30.fsf@flokut.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="35975"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 54591@debbugs.gnu.org To: Johannes =?UTF-8?Q?Gr=C3=B8dem?= Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Apr 02 08:53:39 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1naXdi-0009Dp-UX for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 02 Apr 2022 08:53:39 +0200 Original-Received: from localhost ([::1]:51218 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1naXdh-0003O7-Fp for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 02 Apr 2022 02:53:37 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:36152) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1naXdB-0003Nw-3e for bug-gnu-emacs@gnu.org; Sat, 02 Apr 2022 02:53:06 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:50576) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1naXd7-0002ya-Rr for bug-gnu-emacs@gnu.org; Sat, 02 Apr 2022 02:53:04 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1naXd7-00078m-Oq for bug-gnu-emacs@gnu.org; Sat, 02 Apr 2022 02:53:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 02 Apr 2022 06:53:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 54591 X-GNU-PR-Package: emacs Original-Received: via spool by 54591-submit@debbugs.gnu.org id=B54591.164888234627405 (code B ref 54591); Sat, 02 Apr 2022 06:53:01 +0000 Original-Received: (at 54591) by debbugs.gnu.org; 2 Apr 2022 06:52:26 +0000 Original-Received: from localhost ([127.0.0.1]:44473 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1naXcY-00077w-Gh for submit@debbugs.gnu.org; Sat, 02 Apr 2022 02:52:26 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:48378) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1naXcW-00077k-Ed for 54591@debbugs.gnu.org; Sat, 02 Apr 2022 02:52:25 -0400 Original-Received: from [2001:470:142:3::e] (port=38414 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1naXcQ-0002vk-R2; Sat, 02 Apr 2022 02:52:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=G9N/3+nOxDaV31SPg8N46FBUHH6Q2ak7ka8E42U8SCQ=; b=orO4PrazA1Zw3HHIc2oT wQsVW2OxPq6I6z+Pnhl4nsUi9kfAtzgj2OChJVKkawn43Ec4ctnZmsuf2A9oe3GmVuueecReW4mjA 1zEE9C42g8+StJfzMizmGBQuy3boyLV3R0WeQsMzTT7q91JaqmwExH+KGte2Iw2Bg/lz+5O6BqdGX F0EexqrIg9VbFPr1QzzWtuzVA5yYB1HzHFbVb+5Z7xhlEfHfUWF/s0lFAeF8N3lGLW6rVQ2Nq22+j RN/9vkhiqtgb65crAnFR/GQsWLZut/mWsxkFRp0qv/ycpsRcGjxVYQV5IS1xPGJmpTpTvK8pgOVPz nvGJ3z8QQu7PgQ==; Original-Received: from [87.69.77.57] (port=3877 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1naXcP-000407-IS; Sat, 02 Apr 2022 02:52:18 -0400 In-Reply-To: <87ilrsro30.fsf@flokut.localdomain> (message from Johannes =?UTF-8?Q?Gr=C3=B8dem?= on Sat, 02 Apr 2022 08:33:55 +0200) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:229255 Archived-At: > From: Johannes Grødem > Date: Sat, 02 Apr 2022 08:33:55 +0200 > > > Does SQLite TEXT allow the superset of UTF-8 encoding Emacs uses > > internally to store characters that are not in Unicode? If it does, we > > could indeed assume that any BLOB is binary data and not attempt > > encoding/decoding it. > > SQLite documentation says this... > > TEXT. The value is a text string, stored using the database encoding > (UTF-8, UTF-16BE or UTF-16LE). > > ...but it's still possible to store byte sequences that are not legal > Unicode in there. This breaks the mentioned Python SQLite3 API, and > possibly others, so maybe not great if someone wants to read tables from > something else than Emacs. This probably means we should reject text with raw bytes or characters whose codepoints are beyond #x10FFFF, and document that those should be encoded manually and stored as BLOBs. Thanks.