A completely open system is just asking for referrer-spam (which we already see in the logs) to grow into comment-spam. Sure, we could add captcha technology, that takes care of the automated spam. But what about quantity? The same message, over and over, is easy enough to eliminate. But you've got respondents who are more than happy to keep a flamewar alive, with completely new (yet unoriginal) comments. Do we implement a word limit per message? A comment limit per user? Is that fair, when we can, in the space we give ourselves, type as much as we like, yet require you to respond cogently in a limited space? (I get 2 hours to make my point, you get 5 seconds to make yours!) Sure, we could use some brevity ourselves, but that's hardly the point. What about quality? You've got the self-promoters out there who will use a comment merely as an excuse to pimp their own blog, their own seminar, their own product, and who care very little for the actual topic at hand. Do we filter all inbound messages? How will you know your message is truly in the queue, if it only shows up once we approve it? And the solution doesn't scale, anyway. We hardly have time to create content anymore, how would we have time to filter everything you say, possibly not even to us, but to each other? So do we let you moderate it yourselves? Many (most) people are incapable of self-restraint when given power over others; they won't just moderate down comments that are filled with vitriol, spam, or self-promotion; they'll moderate down comments that they simply disagree with, or don't understand. And for the moderation to be useful, it must have an effect, which means later readers won't see (by design) those comments, in order to moderate them back up. You can't exactly start with the comments pre-moderated-down, and expect someone to find the gems and moderate them up. Do we allow user-ratings rather than content-ratings? I hardly see how that would help, but some sites try it, the equivalent of shaming the person for making stupid comments, rather than just condemning those particularly offensive comments. There's some merit to the idea, but I don't see it actually working. There's no real shame online, no real reputation. Even if users aren't anonymous, even if we require real-ID verification, do you know these people? Do they care what you think of them? Then don't expect down-ratings to make a difference. Should we censor content? That's giving us our own powers back (we always would have the ability to censor anyway, but should we declare and use the power?) That's just asking for us to fall prey to the same moody behavior we'd accuse you of. I don't want to lie to you, tell you we invite debate and dissension, while putting in place a system that allows me to reject your comments out of hand. It's unfair to you when you post comments, and it's unfair to other readers who assume the page is a fair representation of the feedback received. And yet, if we don't censor, we've got all of your hate, anger, illogic, self-promotion, porn, and so forth on our page. Our page! Ah, the crux of the matter.
In the end, I don't think the problem is the technology of the comment system. We could build something. But should we? We have an inherent conflict of interest. We push content, and now we'd also host, moderate, and censor responses to it. You could accuse us of all sorts of vile manipulation. The reason you see nothing but positive feedback is that we hide all negative feedback. The reason you see nothing but vitriolic, illogical feedback is that we hide the intelligent feedback to make those who disagree with us look bad. Besides, we have to think about our (lackluster) google indexing! Why would we want to host on our page something we don't want to be responsible for? (Yes, you can hide content from the indexing engine, but that's not exactly being honest either, is it.)
What do I want? I want to publish content, and nothing more. It's not that I don't want you to talk to us (or each other) about it. It's not that I don't want a healthy debate. I just don't want to host it. Host it elsewhere. Make it not be my problem, my responsibility, my play-thing. And yet, if it's elsewhere, how will my readers know about it? Should I know about every forum discussion out there concerning each of my articles? Should I link to them? There's the conflict of interest again. It has to be automatic, out of my control. I've seen a very few in-browser systems to attach comments to a page, and they're not very well designed. They don't include any of the controls most forums have, many look like graffiti all over the subject page. They still require some central storage, which means that someone, somewhere, is responsible for [not] censoring your content, getting it backed up, making sure it stays online. You should own your content. So then it comes to this: get a blog! Go self-publish, and put your comments on your own page, and link to mine! Ah, the great trackback, spam of the blogosphere. Most of the time, it just annoys people. You're not going to visit all 30 sites listed on a page's trackback list, to see if anyone (or their own, personal readers) made any useful comments at that remote site. In fact, as far as you're concerned, it's just self-promotion. (And you're right -- the blogs talk about each other to generate hits, not because they're really trying to have a debate. You can't have a useful debate with that kind of ping-pong blogging. If they're serious, then they're also incompetent.) And trackbacks are content to be censored, anyway; I could just as easily remove an automatically-added trackback link as a comment.
Just so you know, google has a "link:" feature to help you find pages that link to a given page. If you want to comment on our articles, do so elsewhere, then link to the page you're talking about. Maybe, in some distant future, it'll mean something and your browser will prompt you "there are 8 ongoing discussions about this page, would you like to see them?" In the mean time ... sorry. So very sorry.