Running without the governor should not be an issue... unless you are in the the habit of free revving the engine.
Free revving means something like running it wide open without a load.
Your automobile has no governor. You wouldn't free rev your automobile engine would you?
It would not last very long if you did.
I've been running ungoverned Lawn-Boy engines on a '68 Fox Doodle Bug and 2 gokarts since 2007. Of the gokarts one is running a single, the other twins.
All will go over 50 mph.