32

I looked here for an answer while writing a paper on evidence and scientific inference. I then saw the bold claims made by the website that the process goes as follows:

Anybody can ask a question
Anybody can answer
The best answers are voted up and rise to the top

What reason do we have to think that the best answers are voted up? I am fully confident that the website manages to make the answers that are voted up rise to the top, but what, if anything, underwrites that only the best answers are voted up. Sure, I can see that anybody who thinks some answer is better than another will upvote it, but why should I think that anybody is any good at telling good answers from bad answers. Also, what makes an answer good? Presumably what we get is instead some ranking of what others take to be good answers to the question. This is primarily evidence for what others believe makes an answer good or bad, but how can we infer anything from what others take to be good answers to what are good answers?

Maybe what makes an answer good is just that we agree that it is good. But when I'm doing philosophy I don't want to get at what we agree about, I want to get at what's true. Where do we get the connection from?

| improve this question | |
  • 3
    I guess I would say "best" is defined in many ways, and they are defining "best" as top-voted. But top-voted and correct are not identical. – virmaior Dec 4 '15 at 5:30
  • 5
    the most popular answers rise to the top. not anybody can ask a question. and not anybody can answer. (e.g. I cannot either ask questions or answer questions nor even comment at the Christianity SE which is controlled by a cabal that enforce their own Theological Correctness by use of censorship.) – robert bristow-johnson Dec 4 '15 at 8:28
  • 6
    It's evidently false that "only the best answers are voted up" by any definition of "best" I've ever thought to apply other than the wholly circular one, "that which gets most votes". So really this question seems to have a prerequisite, "how can we make the vague claim "the best answers are voted up" precise enough to be testable?". Adding the word "only" makes it easily testable and clearly false, so I think we can discard that naive reading of the claim and then we're into authorial intent ;-) Maybe they mean, "90% of the time, we think, on a good day". – Steve Jessop Dec 4 '15 at 9:13
  • 2
    @robertbristow-johnson In same cases (not necessarily yours) there may be confusion over the difference between anyone being able to ask a question and anyone being able to ask any quesiton - the latter of which is not guaranteed. I note in the Christianity.SE meta that there are some disputes about what is on or off-topic, and that happens on all the SEs that I participate in. While I have seen some users struggle to ask on-topic questions, and I can't speak to what happens at Christianity, I would say if people are actually blocked from asking outright, those would be exceptional cases. – Todd Wilcox Dec 4 '15 at 15:00
  • 2
    That's a "promotional blurb", the less evil sibling of "marketing blurb". It's not something that is meant to be logically analyzed. You are expected to take it in more on an emotional level. – x457812 Dec 5 '15 at 3:00
3

I've been watching my most-popular ever answer recently. It was slightly more popular than others, and the gap increased substantially over time.

Then the 2nd-most popular answer (mid-30s upvotes compared to mid-50s upvotes for my own) was selected as the answer. From that point on, that answer was getting upvotes at a substantially faster rate than my own.

I strongly believe that the answer nearest the top of the list gets the most votes, rather than vice versa.

Upvotes are the result of positioning. To them that hath shall be given.

| improve this answer | |
  • I'm giving my upvote to this! You answer the question in the way that I already stacked the deck when asking (yes, it's flawed), and you gave an explanation that has empirical backing from x-phi. Ordering effects are a thing. If we can take any lesson from the work in experimental philosophy I think it is that if we prime people into what the popular answer is then doing philosophy by polling is at its most hopeless. Don't get me wrong, I love stackexchange when I'm asking questions about latex formating. I still think that methodologically it has very limited use in philosophy. – KKell Apr 18 '16 at 19:40
18

The Stack Exchange methodology is based on the original Stack Overflow, which is for CS and programming questions. For those types of questions, the answers are objective in the fact that they either solve the OP's problem or they don't. People are free to upvote for the wrong ones, and it does happen occasionally, but overall, there are enough people who know what they are talking about that the most accurate and informative replies "prevail". More often, what happens is whoever asked the question is actually going to go try out the different solutions provided, and will be able to empirically verify which one is best. They will then choose the correct answer, which helps future users in gleaning out relevant information.

The Philosophy Stack Exchange and other humanities SE's are obviously more subjective and hence more susceptible to opinion/consensus biases. But ideally, we are dealing with philosophy (and other fields) as academic disciplines for which a certain amount of objectivity is possible. It is my experience that objective questions (Such as "Is Derrida considered an analytic philosopher?") tend to result in objective answers ("No, he isn't"). The onus in Stack Exchanges like the philosophy SE is really on the askers to ask the right questions, more so than on the users to upvote the right answers.

| improve this answer | |
  • 1
    On occasion the highest voted and/or accepted answers just leave me sitting there all "No, just noooo...". :-) However, the system seems to work in that the usefulness of answers tends to be much higher than other Q&A style sites I've used. So, it's a success despite not being totally "correct". – Brian Knoblauch Dec 4 '15 at 12:43
  • 1
    Just created an account to say that, if this question and answer were extended just slightly, they would probably be a very good fit on Meta.SE. – Panzercrisis Dec 4 '15 at 15:06
  • 3
    @robertbristow-johnson I have to admit a strange sense of pleasure looking at all of the SEs, from Philosophy to Christianity to Worldbuilding to Mathematics. Each community seems to have decided on their own set of aspirations for the site. I rather appreciate the knowledge that none of them appear to have it "right." – Cort Ammon Dec 4 '15 at 16:24
  • not to forget that typically each moderator on each SE have their own standards and this in the end just confuses people because two very similar questions get treated very differently – user100487 Dec 5 '15 at 3:31
  • It is "Stack Exchange" and "Stack Overflow", not "stack exchange" and "StackOverflow" (the last section, "Proper Use of the Stack Exchange Name"). – Peter Mortensen Dec 6 '15 at 9:25
4

SE* do not provide assurances of truth; they provide information.

First off, it is a collection of short articles -- just the written content of the answers will provide information, e.g. summaries and references to other works, individuals' experiences, etc. However, the key feature of SE beyond providing a platform for user generated content, is the voting system.

At the most bare level they provide information on the "goodness" of answers, where goodness is defined in the tautological sense: i.e. good answers are those answers that are voted up (similar to the tautological definition of fitness as propensity to reproduce sometimes used in describing evolution). There is a special vote that comes from the asker (accepted) and other votes that can come from anyone.

For questions that relate to clearly proscribed instrumental goals the interpretation of the goodness votes is pretty clear:

  • The asker asks question Q in order to achieve goal G
  • Answerers provide answers A1, A2, ...
  • The asker accepts (and usually upvotes) the answer, An that he/she used to achieve the goal.
  • Other people, with questions Q' and goals G', to come via search engines to question Q; if an answer, Am allows them to achieve their goals, they upvote it.
  • (As an aside, upvotes on Q indicate that many people searched in such a way as to find that question, and found it, and the answers, useful in achieving their goals)

There is a lot of truth in this -- not that the answers are true, but rather these records of what people have done, in terms of finding the question, achieving their goals, etc., are truthfully recorded and reproduced (assuming no software glitches).

In terms of practical reasoning this is usually sufficient -- you have self-reported information on problems other people have had, and which answers were sufficient to address them. This is probably the most common way of justifying ones beliefs in day-to-day life: relying on people's past experiences. In terms of formal/logical justification of the value of the answers it is clearly insufficient. If you assume that there is a close correspondence between the voting and the behaviours outlined above, then you can believe that SE provide at least (an analog of) empirical adequacy; whether you want to count this as "truth" or something like it is a word game.

Aside on the utility of sorting by votes: If you assume that the voting population is representative of the viewing population, then sorting by votes minimizes the average depth to which people have to search in order to find a suitable answer. This is another not truth-related, but informative aspect of the sites' designs.

For less concretely defined questions, the ability to make these higher level generalizations is correspondingly weaker. For example, if the answer relies on a sense of aesthetics (this can occur in programming!) then the resulting distribution of votes will reflect the aesthetic proclivities of the voting population.

Note that the SE model (I think not coincidentally) satisfies the critera set out by J. Suroweicki in The Wisdom of Crowds (1) diversity of opinion; (2) independence of members from one another; (3) decentralization; and (4) a good method for aggregating opinions.

Most obviously, this is an elaboration on Shiri's answer, and there is some overlap in this answer and Alexander S King's, in that I tried to more fully flesh out what "objective" means in the SE context, and Cort Ammon's, in terms of trying to flesh out the circularity.

(*) I'm using "SE" to refer to the set of Stack Exchange sites.

| improve this answer | |
  • Thanks for the thought out answer. I still don't think this works for my goal: When doing philosophy, my goal is always: Know the truth. I am only interested in answers that are best for achieving that goal. However, if other people with other goals approach the question and vote, that dillutes the usefulness of that vote. Moreover, I've done enough philosophy to know that peer disagreement is widespread and many philosophers must be wrong about many things. I can make use of the arguments provided in answers. I get absolutely no use out of the related upvotes. – KKell Apr 18 '16 at 20:00
2

I can't say their theory is right. Not only is the best answer not always voted up (I've seen plenty of counter examples), but the "best" answer is not always the "true" answer. Sometimes the best answer is technically false, but leads people towards inspiration.

However, the argument is intriguing in its circularity. The mere presence of the belief that "The best answers are voted up and rise to the top" encourages people to vote for the best answers, and an easy scripting tool puts those answers at the top. Thus, while you can't make the claim that their little pithy phrase is true, you can state that the mere act of stating it encourages best answers to be voted up and rise to the top.

| improve this answer | |
2

I think that the purpose of the Stack Exchange is "practically" fulfilled. I don't think their goal was to devise and extract the "truth" for any given question, it's primarily to help people come to a better (and obviously objective) understanding to their own question.

Everyone carries assumptions with their method of thinking and even if given a worded question with a worded answer, people will still understand things under their own premise. Your question is also semi-contradictory given that you're asking for a Yes/No answer to a question that is fundamentally full of a lot of other complicated assumptions and assumptions that everyone will carry differently.

I don't think "truth" is what the Stack Exchange is aiming for which is why they allow other answers to still exist even when one of them has been accepted as "correct" by the asker or unanimously voted as "correct" by the community. Whenever more than one person is involved in anything, you can never normally discern what the "truth" is anymore. There is only preference and the Stack Exchange can only try to solve this problem by making it community-driven to derive contributions from as many people as possible so as to find as many different viewpoints to a seemingly close-ended question. "Truth" is what each individual will take away for himself and the Stack Exchange will provide as much exposure to each person's version of the "correct" answer as possible.

| improve this answer | |
  • First: "Your question is also semi-contradictory given that you're asking for a Yes/No answer to a question that is fundamentally full of a lot of other complicated assumptions and assumptions that everyone will carry differently." - That's precisely the problem with using stackexchange for philosophy. Second: I'm not buying the truth relativism. We will disagree, and we may have rational disagreement but that doesn't make it "true for you but not for me." I see that the best answer need not be the true one (it might be wrong but helpful in some other way), but that's a different issue. – KKell Apr 18 '16 at 19:51
  • That's exactly my argument. The fact is that 'truth' is inherently very subjective and talks of true or false and right or wrong is not an appropriate stance to look at this kind of question. Your second comment is contradictory. If people disagree it means they believe in different things respectively. Like you say the best answer might not be the true one, but what is 'true'? That's all a matter of perspective. One person believes x is true, another, y. There is no such thing as absolute truth as long as there are disagreements so how can one's personal pursuit of truth not be relative? – Shiri Apr 19 '16 at 9:24
  • To add, the 'best' answer is never going to be the one that everyone agrees with fully. The purpose, like you said, is to be helpful in some way and top voted ones are statistically favoured to be helpful to a greater number of people just as a greater number of people found that particular answer more helpful than the rest. So talks about absolute truth and what is 'right' does not fit into the context of a collaborative knowledge base such as the Stack Exchange. – Shiri Apr 19 '16 at 9:27
1

Voting is not a method that produces truth; it is a decision taking procedure that minimizes resentment.

There is however a statistical reason that supports the procedure. In large samples fluctuations decrease, so it takes a relatively smaller number of experts to turn the vote on the right answer.

A very crude example: If you have 100 random yes/no answers, they will be divided most probably somewhere in the 40-60 interval, so 20 non-random opinions are need to produce the right answer. A 1000 random yes/no will fall most probably in the 470-530 interval, so 60 experts are needed.

| improve this answer | |
-2

Good luck.

I'm sorry to say this, but your apparently astute observations about the problems with defining "truth" have not escaped the attention of philosophers. A good rule of thumb is that when you are quite sure you know better than Plato, Aristotle, Newton, Kant, Heidegger, etc., then perhaps you should think again.

The absurd up/down "democratic" process of arriving at consensus is obviously not a process of arriving at "truth." But is there any reasonable definition of "truth" as long as one incontrovertible argument remains standing, as one always will?

If you have a good, incontrovertible way of defining "truth," please, announce it to the world. But if you have not even read any of those thinkers in innumerable historical contexts who have attempted to do precisely that, I don't think anyone will take you seriously.

Sorry, this is a cranky answer. But the fact that you "want" to get at "what's true" is rather immaterial. It's not exactly news that people want things.

| improve this answer | |
  • Thanks for the cranky answer to the cranky post. I don't think in what I said I tried to give some positive answer to what truth is. I've read some by all those folks, some even on what they say about truth. Heck, I'm even writing another paper arguing that someone else is wrong about what relative truth is. I was just particularly miffed at the flippant way that whoever built this website phrased this, and when I'm in "philosophy mode" I tend to take what I read seriously. It's just absurd to claim that better answers get upvoted. ps.: Truth: whatever turns justified belief into knowledge ;) – KKell Dec 4 '15 at 5:19
  • 1
    A few years ago I actually tried to read Atlas Shrugged. I despised Ayn Rand, but only based on second hand sources, and decided that out of intellectual honesty I should read it for myself. About halfway through the preface, she actually stated that her writing is better than Aristotle's and Plato's, at which point I concluded that all those people making fun of her were right, and stopped reading. – Alexander S King Dec 4 '15 at 6:53
  • @AlexanderSKing, it's amazing that Rand gets any juice, but i am starting to see that it's becoming politically coupled. several GOP candidates like Rand. – robert bristow-johnson Dec 4 '15 at 8:33
  • 1
    @AlexanderSKing. Well, she was a screenwriter, so she knew how to produce the bodice-rippers of capitalism. Unlike you, alas, many kept reading, including Alan Greenspan, Paul Ryan, and others in a position to prove that a little philosophy may be worse than none. But we digress.... – Nelson Alexander Dec 4 '15 at 13:30

Not the answer you're looking for? Browse other questions tagged or ask your own question.