This issue is a bit random.
Occasionally, when I max-zoomed-out in character mode; then I switch to map mode (tab), then I cannot fully zoom in. When that happen, I can zoom in a bit, but scrolling further would immediately set zoom to max-zoom-out, before slowly zoom in a bit, then jump back to max-zoom-out.
Another clue is that when that happens, I notice the cut-over point happens immediately after the world is rendered (ie. I can see assembler as assembler), at that point, if I scroll the mouse, it jumps right to assembler-as-a-square view.
When that happens, tabbing out of map mode then tab back in does not fix the problem. The workaround I found is that tab out to character mode, zoom in a bit, then tab in to map mode. Now I can fully zoom in, and zooming in stay zoomed in (not switching in-out-in)