Any sort of competition or challenge outside of our daily regiment constitutes mental "toughness" training on some level, and I personally think those are absolutely important. Those things develop a well of confidence that we draw from later, and having a deep well is definitely part of mental resiliency. I think for most people, taking on one challenge makes it more likely that you'll be willing to take on the next. But we typically subject ourselves to those kind of things voluntarily because we enjoy them at some level. Do they really built "toughness" if we enjoy them? Maybe the question is whether we should subject ourselves to things that we do not enjoy and will not look back on with any particular sense of pride afterwards - actual hardships. Like most folks, I would be inclined to answer "hell no, life is too short for that," and I consider myself fortunate that I don't have to.
So, bottom line, I think that the occasional challenge session is a good thing, but the purpose needs to be sound. The idea should not be to build your tolerance for abuse or misery, it should be to reinforce that the work you are putting in is building toward something you want. You should still enjoy the challenge.
And I can see, for the goalless athlete, challenge sessions probably make no sense.