From: Eli Zaretskii <eliz@gnu.org>
To: Stefan Kangas <stefankangas@gmail.com>, Yuan Fu <casouri@gmail.com>
Cc: luangruo@yahoo.com, emacs-devel@gnu.org
Subject: Re: master d995429e7bc: Use SBYTES instead of strlen in treesit.c
Date: Mon, 22 Jul 2024 14:30:59 +0300 [thread overview]
Message-ID: <8634o1br4c.fsf@gnu.org> (raw)
In-Reply-To: <CADwFkmmNvWU=797PcjZhQJ1qBu22dfWu0Bjark=AdQrv4_2PLg@mail.gmail.com> (message from Stefan Kangas on Mon, 22 Jul 2024 04:06:30 -0700)
> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Mon, 22 Jul 2024 04:06:30 -0700
>
> Po Lu <luangruo@yahoo.com> writes:
>
> > Have you verified that these functions accept strings holding '\0'?
>
> AFAIK, SBYTES returns the string length excluding '\0', same as strlen.
That's not the issue here. The issue is that Emacs Lisp strings can
include embedded null bytes, which strlen will exclude, but SBYTES
will not.
There's perhaps a more general issue here: since tree-sitter accepts
UTF-8 encoded strings, we should encode the Lisp strings before we
pass them to tree-sitter.
Yuan, can you please look into this?
Btw, where does the tree-sitter docs say that all strings are supposed
to be in UTF-8 and that their length is supposed to be passed as
byte-counts, not character-counts?
next prev parent reply other threads:[~2024-07-22 11:30 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <172164369582.30827.14373383262408294645@vcs2.savannah.gnu.org>
[not found] ` <20240722102136.6C9D6C3534A@vcs2.savannah.gnu.org>
2024-07-22 10:27 ` master d995429e7bc: Use SBYTES instead of strlen in treesit.c Po Lu
2024-07-22 11:06 ` Stefan Kangas
2024-07-22 11:30 ` Eli Zaretskii [this message]
2024-07-22 11:55 ` Stefan Kangas
2024-07-22 12:22 ` Eli Zaretskii
2024-07-23 20:42 ` Stefan Kangas
2024-07-24 9:09 ` Mattias Engdegård
2024-07-24 11:33 ` Stefan Kangas
2024-07-24 12:02 ` Mattias Engdegård
2024-07-23 17:09 ` Yuan Fu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/emacs/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8634o1br4c.fsf@gnu.org \
--to=eliz@gnu.org \
--cc=casouri@gmail.com \
--cc=emacs-devel@gnu.org \
--cc=luangruo@yahoo.com \
--cc=stefankangas@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).