> Is this the right behavior?

Yes, I think so.  Let me try to remember how I reasoned about
this when I implemented this around 2014.

In modes whose syntax table defines different types of paired
delimiters (braces and parenthesis), such as here, you would
want to be notified first and foremost that the closing brace is
unmatched. Which is what happens here.  As soon as you give
the closing brace an opening brace to match against, the
opening and closing parenthesis resume the expected matching.

But what if the closing brace isn't really an element of the
language you're editing, as seems to be the case in your
.adoc files?  Well, in that case, the major mode has to know
about it.  So I suspect that if you change the syntax table to
define the backtick as a string delimiter, things will magically
start working as you intend them.  Let me try:

   (modify-syntax-entry ?` "\"")

Yep. Try that in the buffer.  It should fix it.  Of course such
a line should ideally be in a major mode definition for your
files.  Some:

(define-derived-mode asciidoctor-mode prog-mode "Adoc"
  (modify-syntax-entry ?` "\"")
  ... likely more stuff ...)

Notice that the brace is still unmatched inside the string.
That is also by design.  Because matching delimiters insider
strings and comments follow largely the same logic (but
isolated for that domain).  If the brace really has no special
meaning in your language, the major mode must also
know about that (but by default braces have meaning).

HTH,
João