Hello Praful,
I have reviewed this in detail and remembered the reason. Sorry for the delay in answering.
It turns out Word's extensibility is very, very poor. ViEmu has to go to great contortions in order to intercept and process keys "the vi way". In some cases, there are problems, and it has to take trade-offs.
One such case is insert mode - if ViEmu processes keypresses in insert mode the same as in in normal mode, with interception, then Word's autostyling and autocorrection stop to work. I haven't ever understood very well why this happens, but since this is all a very undocumented area, it's not difficult to admit. Given that autostyling and autocorrection are so important, I decided to implement a workaround: "pass-through insert mode". In this mode, ViEmu just lets all input pass through in insert mode. On one hand, this allows insert mode to work almost exactly like non-ViEmu-intercepted Word, including autostyling and autocorrection. On the other hand, the drawback is that vi/vim-functionality in insert mode is not active: :imap's are not applied, Ctrl-H doesn't erase (as you describe), etc... most people don't miss the functionality that much, and having a "high-quality" insert mode is more important.
There is a setting in the settings window which controls this behavior: "Implement insert mode passing all input to Word". It is turned on by default, with the functionality limitations you describe, but you can turn it off. If you do, you will get insert-mode Ctrl-H back, but you will lose autostyling and autocorrection. Sorry about that, but nothing else can be done due to Word's design.
Regards,
-- Jon