Mediocre hardware specs and Mediocre hardware quality
Hardware is what is holding back Windows Phone.
It is a combination of mediocre specs and mediocre quality.
Of course specs hurt in comparison; single core when dual core processors were becoming the norm for example, but I think it was more about quality.
Of the launch phones you had the Samsung Focus with ATT that had problems with memory cards (because it included a user accessable memory card, which was not in line with what Microsoft had intended). You had the HD7 on Tmobile. The HD7's build quality was noticably worse than it's predesessor the HD2 issues included loose fitting battery doors, loose volume, power, and camera keys that rattled. In addition to those issues, the internal micro sd cards used in the HD7 (not user accessable) have a higher than normal failure rate.
Carriers did not push them because of these issues. As much as many people think the salesman is just trying make money off of you when you purchase a new phone, they generally get paid on the service and not on the device. Sales people do not want you coming back with issues or returning your phone, so they will sell what they see the least issues with.