Well, maybe. The cam that operates the shoes isn't exactly symmetrical. You can't see it with the naked eye but the front edge of the cam that's closer to the shoe pivot point is ground down a little so it moves it's shoe slightly less. This results in both shoes being moved the same amount and being applied with the same force. You'll be pulling the brake arm and rotating the cam in the opposite direction so you won't be utilizing this little known design feature. Poor braking may be the result.
Having thought about this, I just don't buy what your suggesting. It would mean that when the brake is off, the cam end of one brake shoe would have to be further away from the drum than the other one. So having done that you then make an asymmetric cam to compensate . Why would you do that?
Also, I think that the splined cam shaft is dimpled, (not all are, mine isn't) because if you fit it the wrong way round, the plate the cam sits on, I've arrowed in the picture below, interferes with the brake drum even though you can make the wheel fit, and it's not immediately obvious until you fit the wheel. The wheel turns, but the brake plate goes with it. I know this, I tried.
However, as I'm in my eighth decade, I'm smart enough to know I don't know everything, but I'll take some convincing on this one!
Regarding the brake reaction arm, as others have pointed out it's a hollow tube, and like most things on an XS it's magnificently over engineered, probably designed and made by the Japanese equivalent of John Deere tractors. Assuming the arm is in good condition, I'd imagine brake lock up will be apparent long before the tube starts to bend.