“Well, I don’t think bypassing the safety relay will affect the charging system, but that all depends on what you mean by bypass. Was there an issue with the safety relay? Problems in that area are more likely to affect your ability to start the bike. For simplicity sake and to set your mind at ease, you may want to consider hooking it back up and undoing what ever bypass you did.
I guess this might be a good time to ask if there are any other wiring changes? Were there any changes done to the regulator/rectifier?
In terms of troubleshooting the charging system... I like to start with a good basic diagram. Not the detailed one, but a simplified one such as in the Clymer manual page 179. It shows the basic elements of the system, and will help you focus on what you need to know. The basic elements are: (1) AC generator(rotor, stator, brushes), (2)rectifier/regulator, (3) battery, (4)main switch and fuse, and most important (5) wiring to connect it all together.
I believe you checked the rotor, stator, brushes, and battery. There are procedures for checking the rectifier with a meter. I am not sure about the regulator portion...? The switch and fuse are pretty easy to check. My concern is that something may be wonky with the wiring. Maybe a corroded or broken connection or something bypassed or wired incorrectly? You may have to check all the wiring in the circuit. Pay special attention to the ground connections. They are often assumed to be okay and overlooked.
I helped a neighbour one time who was bobbing his xj650. He disconnected a lot of wiring, literally chopping off things he didn’t think he needed and then couldn’t get it started afterward. Incidentally, he had to reconnect the safety relay. His major problem was that he disconnected the ground connections. But that was different. His bike was basically inert. Yours just has a little charging issue.
You said that you had voltage at the brown wire at the generator. That brown wire goes a few other places. Check that.