Issues with magic mouse

I noticed that when I try to scroll left/right or up/down using my mac magic mouse, it activates the zoom feature and quickly zooms all of the way out. So if I accidentally move my finger across the mouse, the only solution seems to be to hit the zoom-in button a bunch of times, then try my best to avoid a repeat.