I've seen people lose tremendous amounts of weight cutting out carbs with an end goal such as a wedding, a cruise or whatever. Within a short time of stopping the diet their weight usually ballooned back up and then some!! Sustainability seems more sensible to me.
Yo-Yo Weight Loss and Regain
This is here to stay and is not going to change.
"The U.S. weight loss market is now worth a record $72 billion,..."
Source:
The $72 Billion Weight Loss & Diet Control Market in the United States, 2019-2023 - Why Meal Replacements are Still Booming, but Not OTC Diet Pills - ResearchAndMarkets.com
The Weight Loss Business is going to continue to do well.
Successful Vs Unsuccessful Individual
As someone once said, Successful Individual are willing to do what Unsuccessful Individual won't.
That is one of the primary reason the majority of individual who go on a diet go off of it. Then gain the weight back, plus more.
Other Factors
1) Poor Genetics. Some are predisposted to being over weight.
These individual can lose weight but most likely will never be thin. They will need to stay focused on their "Eating Plan".
2) Metabolic Conditions
Hypothyroidism is one of the reason many women and some men gain weight and cannot lose it.
No diet will work with these individual without until their thyroid condition is take care of.
Sustainable Diet
All diets initially work.
Mark Haub, MS Nutrtionist, Kansas State proved eating junk food in a calorie deficit was effective; losing 27 lbs in three months.
The keys, as you essentially noted, is to find a diet that you can live with/that you like.
Restricitive Diets
The issue with diets that are highly restrictive is that are harder to mantain/stick with. That applies to everything in life.
The harder you make something the less like someone will stick with it.
Even so, the majority of individual who go on a diet, eventually fall off of it; they lack enough motivation to stick with it.
Dr. Jason Curtis' Sports Phyhologist Research
At the 2011 Albuquerque Strength Clinic, Curtis provided some interesting research on a local Weight Loss Challange Contest. It was similar to the "Body For Life Weight Loss Challange" by Bill Phillips.
Clients were assigned a Personal Training to assist them with their training program and provide diet assistance.
Curtis research basically determined that, metaphorically speaking, most didn't have the "Motivation Gene"; they lacked the drive to stick with the weight loss program.
One of the most interesting part of Curtis' research was the connection/influence between some individuals on the Weight Loss Challage to their Personal Trainers.
While the individual still lacked motivation, a strong relationship between some individual in the Weight Loss Challange and their their Personal Trainer, encouged them enough to obtain some good results and stick with their program.
Summary
As they saying goes, "You can lead a horse to water but you can make it drink."
Until someone is motivated enough, they are not going to succeed.