Looking at that rocker shaft, there might be enough wear to justify replacement. And fully agree with Jim's point at #858.
But to answer your general question, visible marks are not enough to justify replacing a part. The technical answer would be to measure accurately and see whether it's within limits. The practical answer is can you feel anything when you rub a finger nail along the surface? If not, what you see is usually just surface rubbing and not enough to worry about. My 2C.
Is the visible scoring something I should even worry about? Not needing new shafts will save me some change. . . Seems to me that the wear marks could just be the surface that the rocker rides on.
But to answer your general question, visible marks are not enough to justify replacing a part. The technical answer would be to measure accurately and see whether it's within limits. The practical answer is can you feel anything when you rub a finger nail along the surface? If not, what you see is usually just surface rubbing and not enough to worry about. My 2C.