I think that the shape of your hole is OK, and I don't think you want to grind it.
I haven't see one without the bumper in it, and didn't want to unscrew mine, but I think that the area just to the left (in the picture) directly toward the larger hole, has been pulled up and needs to be bent back down to re-form the shape of the screw thread.
It definitely looks like that area took a hit of some kind.
Sky NA 2007 Midnight Blue
Sky RL 2008 Carbon Flash - Naked