Replace the brushes, near wear limits is not good enough. The XS650 uses up brushes, it's part of normal maintenance to replace them.
You should have a ground wire from the headlight back to the frame. Otherwise you are sending electricity through your steering stem bearings. Yamaha figured out that wasn't such a good idea about 1975. Brushes are the number one cause of charging system problems on XS650s, closely followed by or aggravated by, owners continuing to use bad batteries. Using a bad battery puts excessive strain on the charging system as it works endlessly trying to bring the battery up to proper voltage, which the battery can no longer achieve. Result? the charging system craps out under the abuse. A $40 battery replacement just became a $200 + troubleshooting nightmare.
To REALLY trouble shoot you will need to cut the connectors and measure the stator.