drum roller assembly.

I am just rebuilding my XS650 bottom end and have come across a potential problem. I have re-fitted the roller assembly that holds the gear selector drum in place and the drum rotates freely but when I tighten down on the bolt that passes through the roller assembly plate it gets much harder...