Even if the shafts neck down to TJ diameter, the seal probably rides on the fat part, and thats why they it needs to be changed.
I've changed the seals a lot, but never took the time to note what part of the shaft rides in it. But, I dont imagine the writes would tell you to swap seals if you didnt need to.
Or you can just try it. If it psses on you, you know you need to change the seals.