With the pivot bolt torqued to spec, 36 to 58 ft-lbs, there should not be any play.
If you get play then shims might help, where to get them or how thick I don't know.
The 81 Special I picked up last week has a shimmy at about 35 mph. I checked the swing arm bushings, ok. I checked and adjusted the head bearings, ok. Still the shimmy.
I swapped the front wheel and tire from the 75, shimmy gone. Put the 81 front on the 75 no shimmy. Put the front wheel with a very checked tire on the 81, just a bit of shimmy.
I think it may be a combo of both the front and rear tires causing my shimmy. The front tire doesn't show any checking. The rear tire shows some in the side wall. The parts bike tire is checked very much in the tread. The parts bike tire was made in 2007, the tires that came on the bike are both 2005 tires. They all have like new tread.
This shows that even with good tread a tire can still be bad.