To summarize:

I have been confused by a misleading comment in ispell.el - frame 
switches should actually exit isearch.  Others have been confused as 
well, see https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41338#59.

Attached is a patch that changes the comment to avoid future confusion. 
I'm still in the process of FSF copyright assignment, but the patch 
probably classifies as paperwork-exempt, anyway.

If you feel that this all not worth the hassle, you can also just close 
this bug without patching anything.