32

I looked here for an answer while writing a paper on evidence and scientific inference. I then saw the bold claims made by the website that the process goes as follows:

Anybody can ask a question
Anybody can answer
The best answers are voted up and rise to the top

What reason do we have to think that the best answers are voted up? I am fully confident that the website manages to make the answers that are voted up rise to the top, but what, if anything, underwrites that only the best answers are voted up. Sure, I can see that anybody who thinks some answer is better than another will upvote it, but why should I think that anybody is any good at telling good answers from bad answers. Also, what makes an answer good? Presumably what we get is instead some ranking of what others take to be good answers to the question. This is primarily evidence for what others believe makes an answer good or bad, but how can we infer anything from what others take to be good answers to what are good answers?

Maybe what makes an answer good is just that we agree that it is good. But when I'm doing philosophy I don't want to get at what we agree about, I want to get at what's true. Where do we get the connection from?

KKell
  • 488
  • 3
  • 9
  • 3
    I guess I would say "best" is defined in many ways, and they are defining "best" as top-voted. But top-voted and correct are not identical. – virmaior Dec 04 '15 at 05:30
  • 5
    the **most popular** answers rise to the top. **not** anybody can ask a question. and **not** anybody can answer. (e.g. I cannot either ask questions or answer questions nor even comment at the [Christianity SE](http://christianity.stackexchange.com/) which is controlled by a cabal that enforce their own Theological Correctness by use of censorship.) – robert bristow-johnson Dec 04 '15 at 08:28
  • 6
    It's evidently false that "only the best answers are voted up" by any definition of "best" I've ever thought to apply other than the wholly circular one, "that which gets most votes". So really this question seems to have a prerequisite, "how can we make the vague claim "the best answers are voted up" precise enough to be testable?". Adding the word "only" makes it easily testable and clearly false, so I think we can discard that naive reading of the claim and then we're into authorial intent ;-) Maybe they mean, "90% of the time, we think, on a good day". – Steve Jessop Dec 04 '15 at 09:13
  • @AnonPhilGrad0815 can you clarify what makes a good answer to your question? Rereading it a few times, I'm wondering if this question is answerable within the SE format (regardless of the merits of questioning the capacity of this format to arrive at "best answers") – virmaior Dec 04 '15 at 12:31
  • Maybe this format is not for everyone. I for one see value from the community feedback, gaining multiple perspectives as questions are evaluated, improved, and if possible, answered. Group input is probably the best way to learn in *more* subjective domains like philosophy .. since "truth" can prove elusive, and semantic gaps need to be sorted out .. – sourcepov Dec 04 '15 at 14:08
  • 2
    @robertbristow-johnson In same cases (not necessarily yours) there may be confusion over the difference between anyone being able to ask a question and anyone being able to ask **any** quesiton - the latter of which is not guaranteed. I note in the Christianity.SE meta that there are some disputes about what is on or off-topic, and that happens on all the SEs that I participate in. While I have seen some users struggle to ask on-topic questions, and I can't speak to what happens at Christianity, I would say if people are actually blocked from asking outright, those would be exceptional cases. – Todd Wilcox Dec 04 '15 at 15:00
  • 1
    Another interesting question to ask about SE/SO is whether the processes used produce *useful* answers/information. As an analogy, one way to view the "truth" of our understanding of physics is by what we are able to do with it. Sending people to the Moon and returning them alive is a validation of our physics "knowledge", as is every successful GPS routing (which requires general relativity). By that metric, I can anecdotally validate SE since I have learned much and solved many problems using high-voted answers here (not always top-voted, granted, but never the "worst" answers, either). – Todd Wilcox Dec 04 '15 at 15:12
  • 1
    I don't have the rep needed to answer, but one plausible interpretation is that Condorcet's jury theorem guarantees that if the individual voters are at least, say, 1% more likely to choosing the best answer than any other answer (when there are multiple answers) then, as the number of voters increases, the chance that the best answer is top voted approaches 100%. – Charles Dec 04 '15 at 15:48
  • 1
    Any methodology that relies on perfect execution by a human is fundamentally flawed, if you want to look at it that way. But that wipes out entire branches of philosophy. – corsiKa Dec 04 '15 at 20:25
  • 2
    That's a "promotional blurb", the less evil sibling of "marketing blurb". It's not something that is meant to be logically analyzed. You are expected to take it in more on an emotional level. – x457812 Dec 05 '15 at 03:00
  • @Charles maybe, but you still need to define "best" and that's something SE/SO does via votes and those come from voters which means that in SEs that don't compile/execute it's going to be hard to make sure the people voting know what they're talking about. – virmaior Dec 05 '15 at 05:16
  • @virmaior: The jury theorem applies for any version of 'best' (it's doesn't know or care which one) -- the hard part is showing that the assumptions are fulfilled. – Charles Dec 05 '15 at 06:25
  • 1
    The theorem may apply to *many* theories of best, but it assumes the jury has the slightest clue which way is up in terms of some particular idea of "best." Problematically, for many SEs, there are competing simultaneous ideas of best which completely wreck the Condorcet theory. – virmaior Dec 05 '15 at 06:34
  • Well OF COURSE the methodological assumptions of Stack Exchange are flawed. But that doesn't stop the sites being useful. – Dawood ibn Kareem Dec 05 '15 at 22:59
  • [It is "Stack Exchange", not "StackExchange"](http://stackoverflow.com/legal/trademark-guidance) (the last section, "Proper Use of the Stack Exchange Name"). – Peter Mortensen Dec 06 '15 at 09:25
  • @robertbristow-johnson: The "cabal" in question is comprised of moderators elected by the community. I'm one of them. We don't push "theological correctness"; much to the contrary, we actually enforce a rule against attempting to push theological correctness. Anyone can ask questions or answer them, *until they lose the privilege of doing so* by repeated abusive, off-topic or low-quality posts, and in that aspect Christianity.SE is no different than any other SE site. Strict community standards are necessary to keep the site from being overrun by trolls. (You should have seen the first year!) – Mason Wheeler Dec 06 '15 at 12:54
  • @MasonWheeler, i have never written nor implied that the cabal was not elected. i have never written an abusive answer nor an off-topic answer. the "low-quality" is a label that the cabal has made and i refute they might not have been popular. nonetheless, the response from the cabal was simply and nakedly **censorship**, which has been demonstrated from the beginning. – robert bristow-johnson Dec 06 '15 at 17:09
  • **Please no discussion in comments**. Take it to chat, folks – Joseph Weissman Dec 31 '15 at 02:06
  • I'm glad to see that a cranky comment in the middle of final paper crunchtime generated such heated discussion. I should probably have phrased this more carefully, seeing that with 2k views this will probably be read by more people than anything serious I hope to publish one day. – KKell Apr 18 '16 at 19:38

7 Answers7

18

The Stack Exchange methodology is based on the original Stack Overflow, which is for CS and programming questions. For those types of questions, the answers are objective in the fact that they either solve the OP's problem or they don't. People are free to upvote for the wrong ones, and it does happen occasionally, but overall, there are enough people who know what they are talking about that the most accurate and informative replies "prevail". More often, what happens is whoever asked the question is actually going to go try out the different solutions provided, and will be able to empirically verify which one is best. They will then choose the correct answer, which helps future users in gleaning out relevant information.

The Philosophy Stack Exchange and other humanities SE's are obviously more subjective and hence more susceptible to opinion/consensus biases. But ideally, we are dealing with philosophy (and other fields) as academic disciplines for which a certain amount of objectivity is possible. It is my experience that objective questions (Such as "Is Derrida considered an analytic philosopher?") tend to result in objective answers ("No, he isn't"). The onus in Stack Exchanges like the philosophy SE is really on the askers to ask the right questions, more so than on the users to upvote the right answers.

Alexander S King
  • 26,984
  • 5
  • 64
  • 187
  • 1
    On occasion the highest voted and/or accepted answers just leave me sitting there all "No, just noooo...". :-) However, the system seems to work in that the usefulness of answers tends to be much higher than other Q&A style sites I've used. So, it's a success despite not being totally "correct". – Brian Knoblauch Dec 04 '15 at 12:43
  • 1
    Just created an account to say that, if this question and answer were extended just slightly, they would probably be a very good fit on Meta.SE. – Panzercrisis Dec 04 '15 at 15:06
  • 3
    @robertbristow-johnson I have to admit a strange sense of pleasure looking at all of the SEs, from Philosophy to Christianity to Worldbuilding to Mathematics. Each community seems to have decided on their own set of aspirations for the site. I rather appreciate the knowledge that none of them appear to have it "right." – Cort Ammon Dec 04 '15 at 16:24
  • not to forget that typically each moderator on each SE have their own standards and this in the end just confuses people because two very similar questions get treated very differently – user100487 Dec 05 '15 at 03:31
  • [It is "Stack Exchange" and "Stack Overflow", not "stack exchange" and "StackOverflow"](http://stackoverflow.com/legal/trademark-guidance) (the last section, "Proper Use of the Stack Exchange Name"). – Peter Mortensen Dec 06 '15 at 09:25
  • I've deleted most of the comments, because comments are not for extended discussion (and moreover they are ceased to be comments on the answer). I did find the discussion interesting myself but when things go off the rails in the comments, they all go down. – virmaior Dec 07 '15 at 00:02
  • @PeterMortensen thanks for pointing that out. Will edit. – Alexander S King Dec 07 '15 at 17:02
  • I actualy disagree. The answers in philosophy are every bit as objective as in math or computer science. You're free to be a relativist about truth, but I'm not, neither in math nor in philosophy. The difference is not one of objectivity, but one of difficulty. It is a heck of a lot harder to figure out the truth to a philosophical question, and often that path will need further clarification or disambiguation of concepts, as well as considering previously unexamined conceptual connections. This is much harder to evaluate than a simple upvote procedure makes it seem. – KKell Apr 18 '16 at 19:47
  • @KKell I understand and sympathize with your view point, and as I said, those answering Philosophy SE questions should strive to objectivity. But complete objectivity is impossible even in natural sciences, [see the underdetermination of scientific theories](http://philosophy.stackexchange.com/a/31845/13808), so it stands to reason that complete objectivity is impossible in philosophy as well. – Alexander S King Apr 18 '16 at 20:52
  • More reasons why this is a terrible platform: I actually dispute that. More modestly, I have an article in the works rebutting one of the arguments against objective favoring of theories by the evidence. But I can't argue about that in these comments and I can't contact you in any other way, so I have no way to continue the conversation. I repeat, this is a terrible platform for doing philosophy. – KKell Apr 19 '16 at 01:22
  • @KKell you can start a chat room and tag whoever you want to invite to chat on the topic. I myself am struggling with the relativism implied by Quine and others, see my posts on the topic. – Alexander S King Apr 19 '16 at 03:55
5

I've been watching my most-popular ever answer recently. It was slightly more popular than others, and the gap increased substantially over time.

Then the 2nd-most popular answer (mid-30s upvotes compared to mid-50s upvotes for my own) was selected as the answer. From that point on, that answer was getting upvotes at a substantially faster rate than my own.

I strongly believe that the answer nearest the top of the list gets the most votes, rather than vice versa.

Upvotes are the result of positioning. To them that hath shall be given.

Euan M
  • 192
  • 3
  • I'm giving my upvote to this! You answer the question in the way that I already stacked the deck when asking (yes, it's flawed), and you gave an explanation that has empirical backing from x-phi. Ordering effects are a thing. If we can take any lesson from the work in experimental philosophy I think it is that if we prime people into what the popular answer is then doing philosophy by polling is at its most hopeless. Don't get me wrong, I love stackexchange when I'm asking questions about latex formating. I still think that methodologically it has very limited use in philosophy. – KKell Apr 18 '16 at 19:40
4

SE* do not provide assurances of truth; they provide information.

First off, it is a collection of short articles -- just the written content of the answers will provide information, e.g. summaries and references to other works, individuals' experiences, etc. However, the key feature of SE beyond providing a platform for user generated content, is the voting system.

At the most bare level they provide information on the "goodness" of answers, where goodness is defined in the tautological sense: i.e. good answers are those answers that are voted up (similar to the tautological definition of fitness as propensity to reproduce sometimes used in describing evolution). There is a special vote that comes from the asker (accepted) and other votes that can come from anyone.

For questions that relate to clearly proscribed instrumental goals the interpretation of the goodness votes is pretty clear:

  • The asker asks question Q in order to achieve goal G
  • Answerers provide answers A1, A2, ...
  • The asker accepts (and usually upvotes) the answer, An that he/she used to achieve the goal.
  • Other people, with questions Q' and goals G', to come via search engines to question Q; if an answer, Am allows them to achieve their goals, they upvote it.
  • (As an aside, upvotes on Q indicate that many people searched in such a way as to find that question, and found it, and the answers, useful in achieving their goals)

There is a lot of truth in this -- not that the answers are true, but rather these records of what people have done, in terms of finding the question, achieving their goals, etc., are truthfully recorded and reproduced (assuming no software glitches).

In terms of practical reasoning this is usually sufficient -- you have self-reported information on problems other people have had, and which answers were sufficient to address them. This is probably the most common way of justifying ones beliefs in day-to-day life: relying on people's past experiences. In terms of formal/logical justification of the value of the answers it is clearly insufficient. If you assume that there is a close correspondence between the voting and the behaviours outlined above, then you can believe that SE provide at least (an analog of) empirical adequacy; whether you want to count this as "truth" or something like it is a word game.

Aside on the utility of sorting by votes: If you assume that the voting population is representative of the viewing population, then sorting by votes minimizes the average depth to which people have to search in order to find a suitable answer. This is another not truth-related, but informative aspect of the sites' designs.

For less concretely defined questions, the ability to make these higher level generalizations is correspondingly weaker. For example, if the answer relies on a sense of aesthetics (this can occur in programming!) then the resulting distribution of votes will reflect the aesthetic proclivities of the voting population.

Note that the SE model (I think not coincidentally) satisfies the critera set out by J. Suroweicki in The Wisdom of Crowds (1) diversity of opinion; (2) independence of members from one another; (3) decentralization; and (4) a good method for aggregating opinions.

Most obviously, this is an elaboration on Shiri's answer, and there is some overlap in this answer and Alexander S King's, in that I tried to more fully flesh out what "objective" means in the SE context, and Cort Ammon's, in terms of trying to flesh out the circularity.

(*) I'm using "SE" to refer to the set of Stack Exchange sites.

Peter Mortensen
  • 133
  • 5
Dave
  • 5,261
  • 1
  • 18
  • 51
  • Thanks for the thought out answer. I still don't think this works for my goal: When doing philosophy, my goal is always: Know the truth. I am only interested in answers that are best for achieving that goal. However, if other people with other goals approach the question and vote, that dillutes the usefulness of that vote. Moreover, I've done enough philosophy to know that peer disagreement is widespread and many philosophers must be wrong about many things. I can make use of the arguments provided in answers. I get absolutely no use out of the related upvotes. – KKell Apr 18 '16 at 20:00
2

I can't say their theory is right. Not only is the best answer not always voted up (I've seen plenty of counter examples), but the "best" answer is not always the "true" answer. Sometimes the best answer is technically false, but leads people towards inspiration.

However, the argument is intriguing in its circularity. The mere presence of the belief that "The best answers are voted up and rise to the top" encourages people to vote for the best answers, and an easy scripting tool puts those answers at the top. Thus, while you can't make the claim that their little pithy phrase is true, you can state that the mere act of stating it encourages best answers to be voted up and rise to the top.

Cort Ammon
  • 17,336
  • 23
  • 59
2

I think that the purpose of the Stack Exchange is "practically" fulfilled. I don't think their goal was to devise and extract the "truth" for any given question, it's primarily to help people come to a better (and obviously objective) understanding to their own question.

Everyone carries assumptions with their method of thinking and even if given a worded question with a worded answer, people will still understand things under their own premise. Your question is also semi-contradictory given that you're asking for a Yes/No answer to a question that is fundamentally full of a lot of other complicated assumptions and assumptions that everyone will carry differently.

I don't think "truth" is what the Stack Exchange is aiming for which is why they allow other answers to still exist even when one of them has been accepted as "correct" by the asker or unanimously voted as "correct" by the community. Whenever more than one person is involved in anything, you can never normally discern what the "truth" is anymore. There is only preference and the Stack Exchange can only try to solve this problem by making it community-driven to derive contributions from as many people as possible so as to find as many different viewpoints to a seemingly close-ended question. "Truth" is what each individual will take away for himself and the Stack Exchange will provide as much exposure to each person's version of the "correct" answer as possible.

Shiri
  • 129
  • 3
  • First: "Your question is also semi-contradictory given that you're asking for a Yes/No answer to a question that is fundamentally full of a lot of other complicated assumptions and assumptions that everyone will carry differently." - That's precisely the problem with using stackexchange for philosophy. Second: I'm not buying the truth relativism. We will disagree, and we may have rational disagreement but that doesn't make it "true for you but not for me." I see that the best answer need not be the true one (it might be wrong but helpful in some other way), but that's a different issue. – KKell Apr 18 '16 at 19:51
  • That's exactly my argument. The fact is that 'truth' is inherently very subjective and talks of true or false and right or wrong is not an appropriate stance to look at this kind of question. Your second comment is contradictory. If people disagree it means they believe in different things respectively. Like you say the best answer might not be the true one, but what is 'true'? That's all a matter of perspective. One person believes x is true, another, y. There is no such thing as absolute truth as long as there are disagreements so how can one's personal pursuit of truth not be relative? – Shiri Apr 19 '16 at 09:24
  • To add, the 'best' answer is never going to be the one that _everyone_ agrees with fully. The purpose, like you said, is to be helpful in some way and top voted ones are statistically favoured to be helpful to a greater number of people just as a greater number of people found that particular answer more helpful than the rest. So talks about absolute truth and what is 'right' does not fit into the context of a collaborative knowledge base such as the Stack Exchange. – Shiri Apr 19 '16 at 09:27
1

Voting is not a method that produces truth; it is a decision taking procedure that minimizes resentment.

There is however a statistical reason that supports the procedure. In large samples fluctuations decrease, so it takes a relatively smaller number of experts to turn the vote on the right answer.

A very crude example: If you have 100 random yes/no answers, they will be divided most probably somewhere in the 40-60 interval, so 20 non-random opinions are need to produce the right answer. A 1000 random yes/no will fall most probably in the 470-530 interval, so 60 experts are needed.

sand1
  • 3,676
  • 9
  • 19
-2

Good luck.

I'm sorry to say this, but your apparently astute observations about the problems with defining "truth" have not escaped the attention of philosophers. A good rule of thumb is that when you are quite sure you know better than Plato, Aristotle, Newton, Kant, Heidegger, etc., then perhaps you should think again.

The absurd up/down "democratic" process of arriving at consensus is obviously not a process of arriving at "truth." But is there any reasonable definition of "truth" as long as one incontrovertible argument remains standing, as one always will?

If you have a good, incontrovertible way of defining "truth," please, announce it to the world. But if you have not even read any of those thinkers in innumerable historical contexts who have attempted to do precisely that, I don't think anyone will take you seriously.

Sorry, this is a cranky answer. But the fact that you "want" to get at "what's true" is rather immaterial. It's not exactly news that people want things.

Nelson Alexander
  • 13,331
  • 3
  • 28
  • 52
  • Thanks for the cranky answer to the cranky post. I don't think in what I said I tried to give some positive answer to what truth is. I've read some by all those folks, some even on what they say about truth. Heck, I'm even writing another paper arguing that someone else is wrong about what relative truth is. I was just particularly miffed at the flippant way that whoever built this website phrased this, and when I'm in "philosophy mode" I tend to take what I read seriously. It's just absurd to claim that better answers get upvoted. ps.: Truth: whatever turns justified belief into knowledge ;) – KKell Dec 04 '15 at 05:19
  • 1
    A few years ago I actually tried to read Atlas Shrugged. I despised Ayn Rand, but only based on second hand sources, and decided that out of intellectual honesty I should read it for myself. About halfway through the preface, she actually stated that her writing is better than Aristotle's and Plato's, at which point I concluded that all those people making fun of her were right, and stopped reading. – Alexander S King Dec 04 '15 at 06:53
  • @AlexanderSKing, it's amazing that Rand gets any juice, but i am starting to see that it's becoming politically coupled. several GOP candidates like Rand. – robert bristow-johnson Dec 04 '15 at 08:33
  • 1
    @AlexanderSKing. Well, she was a screenwriter, so she knew how to produce the bodice-rippers of capitalism. Unlike you, alas, many kept reading, including Alan Greenspan, Paul Ryan, and others in a position to prove that a little philosophy may be worse than none. But we digress.... – Nelson Alexander Dec 04 '15 at 13:30