1 Topic by reevev

  • reevev
  • Member
  • Offline
  • Registered: 2009-07-10
  • Posts: 10

Topic: Slow Search (ViEmu 3/VS2010)

I have had an issue with the speed of searches in ViEmu within Visual Studio 2010 for a while now.  I've just downloaded ViEmu 3 and the problem is still present.

Using / to search larger files is very slow.  I've tried turning off incsearch and the problem remains.  This only appears to be an issue in larger files.

Interestingly, searching for some terms in the files that I notice a slowdown is instant, while other searches have a noticeable delay.  After some investigation it seems the less matches that are present for a search, the longer the delay when searching, or pressing n to get to the next result.

I get that it is probably scanning the whole file each time, however the same search using Visual Studio search is instant and this is what I would expect from ViEmu. I've also opened the same files in vim and the searches are also instant (probably even faster than Visual Studio).

This has got to the stage that I am avoiding the vim style searches in favour of Visual Studio because the searching is just way too slow.

Are there plans to speed this up, or is there any settings I can change to make this faster?  I know a faster search is possible as Visual Studio can perform the same searches instantly, but obviously I want the full capabilities of the vim search including regex.

2 Reply by Jon_Symnum

  • Jon_Symnum
  • Administrator
  • Offline
  • Registered: 2006-01-20
  • Posts: 1,658

Re: Slow Search (ViEmu 3/VS2010)

Reevev, we'll have a look at this as we can find a bit of time. Indeed, I think search has become quite a bit slower going from pure native ViEmu/VS "Classic" to the new WPF-based, .NET-based managed, ViEmu/VS2010 and beyond. I'm not certain what the reason is, it might be with the text-buffer access APIs, or the regex code compiled as .NET C++ instead of native C++. We have to research this somewhat and see what can be done. I'll post here.

Apologies for having to live with this slower search for now, and thanks for your patience!

   -- Jon

3 Reply by MogulManX

  • MogulManX
  • Member
  • Offline
  • Registered: 2012-06-28
  • Posts: 5

Re: Slow Search (ViEmu 3/VS2010)

I just wanted to add that I've had this problem as well for quite some time (though I just recently discovered the ViEmu forums!). Unfortunately, I regularly (daily) need to edit a few files with as many as 30,000 lines of code, so this hits me quite frequently. It's particularly bad if I make a typo in the middle of a search string, such that there are no matches, as I'll wait about 10 seconds for each additional character to fail to match.

If you can fix this, I'd greatly appreciate it!

4 Reply by Isaac_Symnum

  • Isaac_Symnum
  • Administrator
  • Offline
  • Registered: 2010-07-13
  • Posts: 75

Re: Slow Search (ViEmu 3/VS2010)

Searches should be faster in 3.0.6, probably still not good for 30,000 line files, but it should be quite noticeable. Can you please confirm it?
There's still work to do in this area, and we have some ideas for optimization that hopefully will bring more speed in future releases. Just wanted you to know that we are aware of the search speed problem and still trying to find ways to improve it.

5 Reply by MogulManX

  • MogulManX
  • Member
  • Offline
  • Registered: 2012-06-28
  • Posts: 5

Re: Slow Search (ViEmu 3/VS2010)

I installed 3.0.6, and a spot check (on my 30,000 line file) shows that searches are noticeably faster. Cool!

There are still pauses, though. Let me suggest a few optimizations:

(1) if you've typed "/xyz" and there are no matches in the file, there's no need to search the file at all for any subsequent characters typed. That is, if "xyz" doesn't match anything, then "xyzabc" won't, either. Currently, every character you type after the point where there are no matches still incurs a big pause, seemingly because the entire file is searched again. It seems like it should be possible to not do any searching once you know there's no match, thus making subsequent character typing instantaneous. This is important because if the searching does contain pauses due to a large file, I'll often be typing ahead of the search. If I make a typo, I need to wait until the editor catches up, waiting a few seconds for each character after the typo, before I can see what mistake I made and back up.

(2) if I type "/xyz" and the search cursor (?) scrolls to a "current match" on a string "xyzabc", then I type "abc", it seems like it should be fast. After all, it already scrolled to the first instance of the "xyz" match, and the first instance of the "xyzabc" match has to be the same one. However, it appears to re-search the whole file to find the match, incurring a pause.

Btw, it appears one way to test this pausing is to set your cursor to the top of a (very large) file, then search for something where the first occurrence is at the end of the file.

6 Reply by Jon_Symnum

  • Jon_Symnum
  • Administrator
  • Offline
  • Registered: 2006-01-20
  • Posts: 1,658

Re: Slow Search (ViEmu 3/VS2010)

Mogul, great to hear it's better now. Indeed, we have thought about those optimizations. The issue is that this should be done in a "smart-ish" way: given that the search term is always treated as a regular expression, adding characters doesn't necessarily mean making the search more restrictive (that only happens with literals). For example, /[ab shouldn't find anything (it's an invalid regex), while after typing a closing square bracket, /[ab] should match any a or b. Likewise with adding "\c" anywhere in a search, which turns it case-insensitive.

Although we plan to do so at some point, we haven't been able to find the time to invest in optimizing this more heavily.
As always, you can approach this type of optimization from at least two different angles:

One of them, trying to improve the user-experience side of things without touching the underlying regex engine, which are the type of optimizations you suggest above. I usually mostly favor this approach, as it often gets you the most bang for the buck.

On the other hand, we could try to really dig deeper and try to improve the raw performance of the regex engine. Keep in mind that we wrote our own custom regex engine for ViEmu, including the regex parser, compiler, and VM to evaluate searches. I don't think the regex engine per-se is the bottleneck, I'm not sure whether it's more related to VS text buffer access, to how the VM evaluates regex-matching, or to the regex engine itself. We would need to do some evaluation here before getting into the action of actually optimizing the code. Depending on where this is, the effects could also help other areas apart from searches (if the slowness is due to VS text buffer access, and we improve that with more aggressive caching or something similar, then it could help with many other non-search operations).

We'll definitely try to improve this area in the future, thanks for the suggestions, and apologies for the hassles meanwhile!

7 Reply by reevev

  • reevev
  • Member
  • Offline
  • Registered: 2009-07-10
  • Posts: 10

Re: Slow Search (ViEmu 3/VS2010)

I can confirm the 3.0.6 version has vastly improved performance.  We don't have 30,000 line files but a couple in the 10,000 range in which the viemu search was previously not usable, but with 3.0.6 this is much better.  Still not as fast as using the native search but much but I'm very happy with this update.