There appears to be quite a bit of wear on the end of the pushrod where it contacts the ball. Check the other end too. It may be one of those aftermarket rods from MikesXS/XS650Direct. On some, they skipped the hardening process and they wear quite quickly. Here's what mine looked like after about 10K miles. I'll only use originals now .....
View attachment 207322
If you still have the two piece rod, the older type long one piece rod is a nice upgrade. You can still find N.O.S. originals on eBay pretty easily. I see you also have the long arm worm gear. That will give easier clutch pull but at the expense of less travel and less clutch plate separation. You may experience hard-to-find-neutal-when-hot issues. An upgrade for that would be the "1L9" XS400 worm.