Hey, dps650rider!
Yup, might be some problems in there.
To start, do a forum search on "shiftdrum 1st". Maybe a dozen threads.
The screw in the starwheel also holds the 2 parts of the shiftdrum together. That's more important, a loose starwheel shouldn't be the problem.
Yes, the drum stopper needs to be in good shape. Also, the "C" shaped stopper plate.
Those could be the origin of your problem.
Then there's the aftereffects.
Possibly bent fork, rounded gear dogs.
In one of those threads I describe a method to help determine how much rounding and bending.
In another 1st gear thread, we discuss in excruciating detail the positioning, spacing, and bushing of the large 1st wheel gear.
Sorry so brief. On stoopidphone...