It should not be pulling against the PTO spring.
When the PTO is in the off position the PTO spring should be quite slack.
If not then the PTO cable is wrong or something is causing it to hang.
When you disengage the PTO, the tension on the belt should push the tension arm back till the pulley just touched the belt and the return spring ( which may also be the brake spring ) pulls the tension lever further back making the belt very slack.