all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* bug#34525: replace-regexp missing some matches
@ 2019-02-18  8:28 Daniel Lopez
       [not found] ` <handler.34525.B.15504786524313.ack@debbugs.gnu.org>
                   ` (2 more replies)
  0 siblings, 3 replies; 47+ messages in thread
From: Daniel Lopez @ 2019-02-18  8:28 UTC (permalink / raw)
  To: 34525

[-- Attachment #1: Type: text/plain, Size: 3324 bytes --]

Reproduce:

- Start "emacs -Q" and open the file BitmapFontFace.h
- Evaluate the expression (replace-regexp "\\<Bitmap\\>" "SharedBitmap")
- The text "Replaced 8 occurrences" appears in the echo area.

Problem:

There were actually 12 occurrences (ie. of the word "Bitmap" surrounded 
by word boundaries) in the file that should have been replaced. If I now 
move point back to the start of the buffer and evaluate the expression 
again, it says "Replaced 4 occurrences".

The exact number of incorrect replacements perhaps varies over time. 
That is, I can test it five times in a row and get 8 initial replacments 
each time, but after trying some other search terms, messing with the 
file, restarting Emacs etc, I try my initial test again and then maybe 
it consistently replaces 10 the first time, for a while. So your exact 
numbers may vary.

I debugged the Lisp as far as I could and it appears to be wrong answers 
coming out of the re-search-forward C call that is in 
isearch-search-fun-default.

The bug filters up to a number of string replacement user actions - I 
first noticed it when trying to do this replacement interactively with 
query-replace on word boundaries (C-u M-%), entering "Bitmap" as search 
string, then "SharedBitmap" as replacement string. Trying now, as I 
press space repeatedly about once a second to confirm each one, I see 
the pink highlight skip valid matches to ask me about one that is 
further down even while I see the skipped one highlighted in blue a few 
lines above, and in the end it may have replaced only 6-8 of the 
occurrences. Though, if I press 'n' instead of space to skip without 
making any replacements, it does visit all of the occurrences.

I see from the Lisp that plain (non-regexp) query-replace on word 
boundaries gets preprocessed into the equivalent regexp search as in my 
initial example. I don't think there are any problems with plain string 
search and replacement.

Some more experimental observations:

- The replacement text can be any string instead of "SharedBitmap", eg. 
"qwertyasdfgh", "qwer", etc, and the bug still happens. The number of 
matches seems to be related to the length of the replacement string. 
Currently 12 character replacement strings are causing replace-regexp to 
make 8 replacements on the first call for me, while 4 character strings 
cause 7 replacements. 6 character replacement strings - ie. same length 
as "Bitmap" - always work, replacing all 12 occurrences.

- The bug doesn't happen in fundamental-mode, nor c-mode, js-mode, 
text-mode or any other major modes I tried.

- I've seen this happen in other of my C++ files where I was making the 
same replacement, so the problem's not precisely unique to this one. 
I've been trying to simplify this one but haven't found anything much 
more revealing so far. For example if I delete all the comments and 
blank lines, then the first replacement finds 9 occurrences out of 10. 
If I cut the file in half by deleting line 140 onwards, the first 
replacement finds 3 occurrences out of 6. But if I do something very 
simple like just pasting "Bitmap<PixelType>" on 100 consecutive lines, 
it's not fooled and it replaces them all.

I've tried this in GNU Emacs 26.1 on Arch Linux and 25.2.1 on Windows 7 
and am seeing the same behaviour in both.

Thanks,
Daniel

[-- Attachment #2: BitmapFontFace.h --]
[-- Type: text/x-chdr, Size: 10515 bytes --]

// BitmapFontFace: A class to store a font in which the glyphs are stored in bitmaps.

#pragma once

// This namespace
#include "FontFaceMetrics.h"

// Dan std
#include <dan/Bitmap.h>
#include <dan/math/Vector2.h>
#include <dan/math/Rect2.h>
#include <dan/Error.h>
using dan::Error;

// Boost
#include <boost/scoped_array.hpp>
using boost::scoped_array;
#include <boost/shared_ptr.hpp>

// C++ std
#include <vector>
#include <map>


namespace dan {


struct BitmapFontGlyph
// Where to find the graphic for a single character
//
// (This is only used by BitmapFontFace but isn't declared within it as it doesn't need to
// use BitmapFontFace's template parameters)
{
    unsigned int bitmapNo;
    // Which bitmap of the BitmapFontFace contains the glyph of this character
    dan::math::Rect2I rect;
    // The position and size on the bitmap of the character's glyph

    dan::math::Vector2I bearing;
    // Distance from the drawing pen position to where the top-left of the glyph should be

    dan::math::Vector2I advance;
    // The number of pixels that the drawing pen position should be moved
    // after printing this character, to be ready for the next one

    BitmapFontGlyph(unsigned int i_bitmapNo, const dan::math::Rect2I & i_rect, const dan::math::Vector2I & i_bearing, const dan::math::Vector2I & i_advance)
        : bitmapNo(i_bitmapNo), rect(i_rect), bearing(i_bearing), advance(i_advance)
    {}
};


typedef std::map<unsigned int, BitmapFontGlyph> BitmapFontCharmap;
// Represents a character map,
// ie. a full alphabet of mappings from character code numbers to glyphs.
// This collection of mappings is also known as a character encoding.
//
// (This is only used by BitmapFontFace but isn't declared within it as it doesn't need to
// use BitmapFontFace's template parameters)


template<typename PixelType>
class BitmapFontFace
{
    // + Shared part {{{

protected:
    struct Shared
    {
        std::vector< Bitmap<PixelType> > m_bitmaps;
        std::vector<BitmapFontCharmap> m_charmaps;
        float m_emHeight;
        FontFaceMetrics m_metrics;

        unsigned int m_replacementForMissingCharCode = 0;

        // + Construction {{{

        Shared(float i_emHeight = 1,
               const FontFaceMetrics & i_metrics = FontFaceMetrics())
            : m_emHeight(i_emHeight)
            , m_metrics(i_metrics)
        // Construct with no bitmaps or charmaps
        {}

        Shared(const std::vector< Bitmap<PixelType> > & i_bitmaps,
               const std::vector<BitmapFontCharmap> & i_charmaps,
               float i_emHeight = 1,
               const FontFaceMetrics & i_metrics = FontFaceMetrics())
            : m_emHeight(i_emHeight)
            , m_metrics(i_metrics)
        // Construct with multiple bitmaps and multiple charmaps
        {
            m_bitmaps.assign(i_bitmaps.begin(), i_bitmaps.end());
            m_charmaps.assign(i_charmaps.begin(), i_charmaps.end());
        }

        Shared(const Bitmap<PixelType> & i_bitmap,
               const std::vector<BitmapFontCharmap> & i_charmaps,
               float i_emHeight = 1,
               const FontFaceMetrics & i_metrics = FontFaceMetrics())
            : m_emHeight(i_emHeight)
            , m_metrics(i_metrics)
        // Construct with single bitmap and multiple charmaps
        // (bitmap specified in Bitmap object)
        {
            m_bitmaps.push_back(i_bitmap);
            m_charmaps.assign(i_charmaps.begin(), i_charmaps.end());
        }

        Shared(PixelType * i_srcBitmap_pixels, unsigned int i_srcBitmap_width, unsigned int i_srcBitmap_height,
               const std::vector<BitmapFontCharmap> & i_charmaps,
               float i_emHeight = 1,
               const FontFaceMetrics & i_metrics = FontFaceMetrics())
            : m_emHeight(i_emHeight)
            , m_metrics(i_metrics)
        // Construct with single bitmap and multiple charmaps
        // (bitmap specified as buffer, width and height)
        {
            m_bitmaps.push_back(Bitmap<PixelType>(i_srcBitmap_pixels, i_srcBitmap_width, i_srcBitmap_height));
            m_charmaps.assign(i_charmaps.begin(), i_charmaps.end());
        }

        Shared(const std::vector<BitmapFontCharmap> & i_charmaps,
               float i_emHeight = 1,
               const FontFaceMetrics & i_metrics = FontFaceMetrics())
            : m_emHeight(i_emHeight)
            , m_metrics(i_metrics)
        // Construct without any bitmaps stored (class stores font metadata only)
        {
            m_charmaps.assign(i_charmaps.begin(), i_charmaps.end());
        }

        // + }}}
    };

    boost::shared_ptr<Shared> m_shared;

    // + }}}

    // + Construction {{{

public:
    BitmapFontFace(float i_emHeight = 1,
                   const FontFaceMetrics & i_metrics = FontFaceMetrics())
        : m_shared(new Shared(i_emHeight, i_metrics))
    // Construct with no bitmaps or charmaps
    {}

    BitmapFontFace(const std::vector< Bitmap<PixelType> > & i_bitmaps,
                   const std::vector<BitmapFontCharmap> & i_charmaps,
                   float i_emHeight = 1,
                   const FontFaceMetrics & i_metrics = FontFaceMetrics())
        : m_shared(new Shared(i_bitmaps, i_charmaps, i_emHeight, i_metrics))
    // Construct with multiple bitmaps and multiple charmaps
    {}

    BitmapFontFace(const Bitmap<PixelType> & i_bitmap,
                   const std::vector<BitmapFontCharmap> & i_charmaps,
                   float i_emHeight = 1,
                   const FontFaceMetrics & i_metrics = FontFaceMetrics())
        : m_shared(new Shared(i_bitmap, i_charmaps, i_emHeight, i_metrics))
    // Construct with single bitmap and multiple charmaps
    // (bitmap specified in Bitmap object)
    {}

    BitmapFontFace(PixelType * i_srcBitmap_pixels, unsigned int i_srcBitmap_width, unsigned int i_srcBitmap_height,
                   const std::vector<BitmapFontCharmap> & i_charmaps,
                   float i_emHeight = 1,
                   const FontFaceMetrics & i_metrics = FontFaceMetrics())
        : m_shared(new Shared(i_srcBitmap_pixels, i_srcBitmap_width, i_srcBitmap_height, i_charmaps, i_emHeight, i_metrics))
    // Construct with single bitmap and multiple charmaps
    // (bitmap specified as buffer, width and height)
    {}

    BitmapFontFace(const std::vector<BitmapFontCharmap> & i_charmaps,
                   float i_emHeight = 1,
                   const FontFaceMetrics & i_metrics = FontFaceMetrics())
        : m_shared(new Shared(i_charmaps, i_emHeight, i_metrics))
    // Construct without any bitmaps stored (class stores font metadata only)
    {}

    // + }}}

    // + Bitmaps {{{

    const std::vector< Bitmap<PixelType> > & bitmaps() const
    {
        return m_shared->m_bitmaps;
    }
    std::vector< Bitmap<PixelType> > & bitmaps()
    {
        return m_shared->m_bitmaps;
    }

    //void releaseBitmaps()
    //{
    //    m_bitmaps.clear();
    //}

    // + }}}

    // + Charmaps {{{

    const std::vector<BitmapFontCharmap> & charmaps() const
    {
        return m_shared->m_charmaps;
    }

    unsigned int charmap_count() const { return m_shared->m_charmaps.size(); }
    unsigned int charmap_charCount() const { return m_shared->m_charmaps.front().size(); }

    // + }}}

    // + Metrics {{{

    unsigned int emHeight() const { return m_shared->m_emHeight; }
    FontFaceMetrics & metrics() const { return m_shared->m_metrics; }

    // + }}}

    // + Get details of a single char {{{

    unsigned int replacementForMissingCharCode() const
    {
        return m_shared->m_replacementForMissingCharCode;
    }
    void replacementForMissingCharCode(unsigned int i_charCode) const
    {
        m_shared->m_replacementForMissingCharCode = i_charCode;
    }

    const BitmapFontGlyph & getGlyph(unsigned int i_charmapNo, unsigned int i_charCode) const
    {
        if (i_charmapNo >= m_shared->m_charmaps.size())
            throw( Error(0, "in BitmapFontFace::getGlyph, i_charmapNo out of range") );
        BitmapFontCharmap & charmap = m_shared->m_charmaps[i_charmapNo];

        BitmapFontCharmap::const_iterator glyphItr = charmap.find(i_charCode);
        if (glyphItr == charmap.end())
        {
            if (m_shared->m_replacementForMissingCharCode == 0)
                throw( Error(0, "in BitmapFontFace::getGlyph, no glyph in charmap for i_charCode") );

            glyphItr = charmap.find(m_shared->m_replacementForMissingCharCode);
            if (glyphItr == charmap.end())
                throw( Error(0, "in BitmapFontFace::getGlyph, no glyph in charmap for i_charCode or its replacement code") );
        }

        return glyphItr->second;
    }

    const Bitmap<PixelType> & getGlyphBitmap(unsigned int i_charmapNo, unsigned int i_charCode) const
    {
        return m_shared->m_bitmaps[getGlyph(i_charmapNo, i_charCode).bitmapNo];
    }

    const dan::math::Rect2I & getGlyphRect(unsigned int i_charmapNo, unsigned int i_charCode) const
    {
        return getGlyph(i_charmapNo, i_charCode).rect;
    }

    // + }}}

    dan::math::Vector2I charAdvanceMax() const
    {
        dan::math::Vector2I maxValue = 0;
        for (unsigned int charmapNo = 0; charmapNo < m_shared->m_charmaps.size(); ++charmapNo)
        {
            const BitmapFontCharmap & charmap = m_shared->m_charmaps[charmapNo];
            for (BitmapFontCharmap::const_iterator glyphItr = charmap.begin(); glyphItr != charmap.end(); ++glyphItr)
            {
                if (glyphItr->second.advance[0] > maxValue[0])
                    maxValue[0] = glyphItr->second.advance[0];
                if (glyphItr->second.advance[1] > maxValue[1])
                    maxValue[1] = glyphItr->second.advance[1];
            }
        }
        return maxValue;
    }

    dan::math::Vector2I charRectSizeMax() const
    {
        dan::math::Vector2I maxValue = 0;
        for (unsigned int charmapNo = 0; charmapNo < m_shared->m_charmaps.size(); ++charmapNo)
        {
            const BitmapFontCharmap & charmap = m_shared->m_charmaps[charmapNo];
            for (BitmapFontCharmap::const_iterator glyphItr = charmap.begin(); glyphItr != charmap.end(); ++glyphItr)
            {
                if (glyphItr->second.rect.width() > maxValue[0])
                    maxValue[0] = glyphItr->second.rect.width();
                if (glyphItr->second.rect.height() > maxValue[1])
                    maxValue[1] = glyphItr->second.rect.height();
            }
        }
        return maxValue;
    }
};


}

^ permalink raw reply	[flat|nested] 47+ messages in thread

end of thread, other threads:[~2019-03-01 17:58 UTC | newest]

Thread overview: 47+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-02-18  8:28 bug#34525: replace-regexp missing some matches Daniel Lopez
     [not found] ` <handler.34525.B.15504786524313.ack@debbugs.gnu.org>
2019-02-18  8:37   ` bug#34525: Acknowledgement (replace-regexp missing some matches) Daniel Lopez
2019-02-18 15:50 ` bug#34525: replace-regexp missing some matches Eli Zaretskii
2019-02-18 16:46   ` Alan Mackenzie
2019-02-18 21:10   ` Alan Mackenzie
2019-02-20 17:07   ` Alan Mackenzie
     [not found]   ` <20190220170722.GA9655@ACM>
2019-02-20 18:02     ` Eli Zaretskii
2019-02-20 18:58       ` Alan Mackenzie
2019-02-20 19:27         ` Eli Zaretskii
2019-02-20 21:30           ` Alan Mackenzie
     [not found]           ` <20190220213003.GC9655@ACM>
2019-02-21  3:40             ` Eli Zaretskii
2019-02-24 17:37               ` Alan Mackenzie
2019-02-24 17:56                 ` Eli Zaretskii
2019-02-24 21:00                   ` Alan Mackenzie
2019-02-25 20:11                     ` Eli Zaretskii
2019-02-25 20:48                       ` Alan Mackenzie
2019-02-26 13:50                       ` Alan Mackenzie
     [not found]                       ` <20190226135048.GA19653@ACM>
2019-02-26 15:00                         ` Alan Mackenzie
2019-02-26 15:39                           ` Eli Zaretskii
2019-02-26 16:11                             ` Alan Mackenzie
2019-02-26 16:42                               ` Eli Zaretskii
2019-02-26 16:55                                 ` Alan Mackenzie
     [not found]                                 ` <20190226165505.GD19653@ACM>
2019-02-26 17:20                                   ` Eli Zaretskii
2019-02-26 17:23                                     ` Alan Mackenzie
2019-02-26 15:36                         ` Eli Zaretskii
2019-02-26 20:09                         ` Stefan Monnier
     [not found]                         ` <jwv8sy2z5yc.fsf-monnier+emacsbugs@gnu.org>
2019-02-26 21:45                           ` Alan Mackenzie
2019-02-26 22:09                             ` Stefan Monnier
2019-02-27 14:22                           ` Alan Mackenzie
     [not found]                           ` <20190227142251.GB4772@ACM>
2019-02-27 15:08                             ` Alan Mackenzie
     [not found]                             ` <20190227150849.GC4772@ACM>
2019-02-27 15:40                               ` Stefan Monnier
2019-02-27 17:10                                 ` Alan Mackenzie
2019-02-27 16:39                             ` Eli Zaretskii
2019-02-27 17:31                               ` Alan Mackenzie
2019-02-27 17:41                               ` Stefan Monnier
     [not found]                               ` <20190227173132.GG4772@ACM>
2019-02-27 18:07                                 ` Eli Zaretskii
2019-02-28 10:50                                   ` Alan Mackenzie
2019-02-28 17:41                                     ` Eli Zaretskii
2019-02-28 21:54                                       ` Alan Mackenzie
     [not found]                               ` <jwvpnrdb0xj.fsf-monnier+emacsbugs@gnu.org>
2019-02-27 18:48                                 ` Eli Zaretskii
2019-02-27 20:43                                   ` Alan Mackenzie
2019-02-26 23:00                         ` Stefan Monnier
2019-02-20 21:25         ` Daniel Lopez
2019-02-22 16:26           ` Alan Mackenzie
2019-03-01 14:34           ` Alan Mackenzie
     [not found]           ` <20190301143414.GD5674@ACM>
2019-03-01 17:58             ` Daniel Lopez
2019-03-01 17:42 ` Alan Mackenzie

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.