I used to get this happening with a wired optical mouse. It happened on all surfaces although some were worse than others. Since changing to a wireless optical mouse it never happens. I suspect it was related to the mouse driver but can't be certain.