Have you replaced the spindles? Maybe it's the angle of the camera, but it looks like the spindle pulleys are too low for the idlers. If the deck were bent enough for that much misalignment, it would be very noticeable. I ordered one set of spindles (sans pulleys) that had spacers to go under under the pulleys on the splines. Another had the spacer machined into the spindle shaft. Another was just too tall and the belt wouldn't stay on at all. There's a lot of variations on these spindles, and just because the seller claims it will fit our mower (I have the exact same model as you) that isn't always true. I've returned one set and marveled at the difference in a set that I bought twice off the same seller. Many sellers have absolutely no clue which garage in China they're coming from or just how (in)compatible they really are. There certainly isn't any quality control.
It ain't rocket science, but the axial centers of the idlers and spindle pulleys should be pretty close to being in the same plane. Doesn't have to be exact, but exacter than what I'm looking at. I'd wager that the deck is bent under the tensioner arm or the tensioner arm is twisted. That's too much tilt on the tensioner idler, which is why the paint is worn off across the whole pully face. Then again, the belt is being pulled on the LH side and slack thrown to the RH side to drive the mower. The tensioner bounces a LOT and the pulses from the engine make that belt something to behold when it's running. I don't know how it stays on at all. That too could be the source of the wear on the tensioner idler. The stationary idler looks like the belt is riding a bit low on it, but it's a lot better than the tensioner idler. There's no sign the back of the belt is rubbing the flange of the idler. I don't think I'd worry about that one though it could be a little flatter. The belt is coming off that spindle on the right (which is actually the LH spindle with it mounted on the mower) going toward the engine. That is where the most tension is because the engine is pulling it on that side.
The tension arm is on the RH spindle side. A small rock dent next to the mounting point would make a pretty big difference on that tension arm. These decks are pretty thin, and I've beat out my share of rock dents that affect the alignment. Mine is currently twisted slightly causing some minor cut quality issues. But, I bought it to cut a rough lawn, not a golf green. Up until last year, I had well over 20 trees in the yard which had some negative effects on having a lawn, to say the least. I have almost no topsoil on top of rock filled clay that also appears to have been the city dump at one time, so I'm no stranger to broken belts, bent blades, and broken spindles. I've overhauled my deck at least 5 times.
A half inch of deflection on the deck belt is hooey. What do you think that spring on the actuator cable is for? That's to allow the tensioner arm to bounce while the blades are accelerating to let the belt slip during that time, and also to keep a rock or root from bending the crankshaft of the engine. That belt's a LOT stronger than some think. That belt flaps nearly 2 inches because it's nearly 30 inches from the engine crankshaft to the idlers with little more than some smooth hooks to keep it from falling off when the blades are disengaged.
One problem you MIGHT be having is the worn out brakes. WHEN is the belt breaking? If it's when you engage the blades, it may already be off because last time you disengaged them, the inertia of the blades will throw slack in the belt. If it comes off in the right place, it will jam and likely break next time you try to engage the blades. The brakes never have and never will do much for safety stopping the deck, but they're brakes and all that is required by OPC. Their biggest contribution is preventing the inertia of the blades from throwing the belt off. Try this little trick that I used until I replaced the brakes on mine. Drop the engine speed to idle before engaging/disengaging the blades. That knocks down the inertia that throws the belt off. If it's breaking while mowing, chances are the belt is hitting something sharp under the machine that's cutting it. Gonna have to lift it with the deck off and have a close look at the entire belt path and anything 2 inches either side of that path. It doesn't take much to snag and break that belt, though the machine will normally make some hair-raising noises when that happens.
Belt slippage can also be oil on the crankshaft pulley. It is not uncommon for the sump gasket, oil filter or crankshaft seal on the B&S engines to start drooling oil in places you really don't want oil. I've had the oil filter come loose, I've replaced the sump gasket (when the governor gear grenaded), and now I need a crankshaft seal.
Sorry for the length, but I tried to cover all the issues you described. Like I said, I have the same equipment with nearly twice the hours on it, and have repaired the deck many times when others would have just gone out and bought a new mower of a different brand that came out of the same outsource factory to think they were replacing junk. I'm pretty happy it's still going after 11 years of abuse by me and my yard.