That's so fucking messed up. Being homeschooled as I was, I was mostly sheltered from this shit. I've been hearing more and more about the fucked up public school system and it's pissing me off so much.
The things we have to do to stay safe. Though I think you should verbally demand he remove his hand first before you physically remove it. Hurting someone is so much more satisfying if you warn them first and they continue to make a poor decision.