I posted this question on LessWrong.com where it was suggested i come here. I also posted there about an open source decision making web site I am working on called WikiLogic. The site has a 2 minute explanatory animation if you are interested. I wont repeat myself but the tl;dr is that it will follow the Wikipedia model of allowing everyone to collaborate on a giant connected database of arguments where previously established claims can be used as supporting evidence for new claims.

The raw deduction element of it works fine and would be great in a perfect world where such a thing as absolute truths existed, however in reality we normally have to deal with claims that are just the most probable. My program allows opposing claims to be connected and then evidence to be gathered for each. The evidence will create a probability of it being correct and which ever is highest, gets marked as best answer. Principles such as Occams Razor are applied automatically as long list of claims used as evidence will be less likely as each claim will have its own likelihood which will dilute its strength.

However, my only qualification in this area is my passion and I am hitting a wall with some basic questions. I am not sure if this is the correct place to get help with these. If not, please direct me somewhere else and I will remove the post.

The arbitrarily chosen example claim I am working with is whether “Alexander the Great existed”. This has the useful properties of 1: an expected outcome (that he existed - although, perhaps my problem is that this is not the case!) and 2: it relies heavily on probability as there is little solid evidence.

One popular claim is that coins were minted with his face on them. I want to use Bayes to find how likely a face appearing on a coin is for someone who existed. As I understand it, there should be 4 combinations:

Existed; Had a coin minted
Existed; Did not have a coin minted
No Existed; Had a coin minted
No Existed; Did not have a coin minted

The first issue is that there are infinite people who never existed and did not have a coin made. If I narrow it to historic figures who turned out not to exist and did not have a coin made it becomes possible but also becomes subjective as to whether someone actually thought they existed. For example, did people believe the Minotaur existed?

Perhaps I should choose another filter instead of historic figure, like humans that existed. But picking and choosing the category is again so subjective. Someone may also argue that woman inequality back then was so great that the data should only look at men, as a woman’s chance of being portrayed on a coin was skewed in a way that isn’t applicable to men.

I hope i have successfully communicated the problem i am grappling with and what i want to use it for. If not, please ask for clarifications. A friend in academia suggested that this touches on a problem with Bayes priors that has not been settled. If that is the case, is there any suggested resources for a novice with limited free time, to start to explore the issue?

  • 1
    $\begingroup$ Priors are like axioms in logic. In logic, an axiom is assumed true with no proof necessary and perhaps proof is actually impossible. So your site would be driven mainly by your priors rather than truth. It will be a site where things are more likely if they agree with your most deeply-held beliefs, which is fine but isn't exactly what you are seeking. That said, read the Wikipedia page on Dempster Shafer Theory -- especially the Criticisms section -- and you may find this a more useful approach. $\endgroup$ – Wayne Aug 14 '16 at 15:17
  • $\begingroup$ Do they have to be axioms? Why can we not delve into them and argue for why a prior should/shouldn't be the case? A "proof" as in a perfect truth might be impossible but i believe you could get to a point where the argument for a given prior convinces everyone. Would you disagree? Can you give an example? If i am wrong with the above then yes, the site will just reinforce whoever got to pick the priors/axioms in which case i would ditch the project! I accidentally sent this before finishing reading the wiki so i will come back to comment again after. $\endgroup$ – WikiLogic Aug 14 '16 at 15:56

First, let me say that I don't have anything personal against you when I say something like "your beliefs" or "you and those you designate" in my comments. It's not you per se, it's someone in general.

You should probably look into Dempster Shafer Theory, which is a way to fuse beliefs. (Carefully read the Criticisms section. When it works it works well, when it's misapplied, it fails spectacularly.) Or perhaps Fuzzy Logic, Possibility Theory, and other topics I imagine the Dempster Shafer Theory page on Wikipedia would link to.

You should also look at Artificial Intelligence (AI) research projects, like Cyc, that are already attempting to create a base of knowledge/experience upon which AI might build. It's a very hard problem and it's my impression that the consensus may even be that it's a dead end.

On a personal-reaction note, what you're proposing sounds to me a little like Google's proposed idea that they might be able to not only search but also return the "truth" of what it finds. That creeps me out: Google deciding what's true? Think it might get political or reinforce a currently-popular consensus maybe? I've already seen the negative effects of Google's Friend's effect on search results, and that's way too much power, and you can never underestimate Confirmation Bias.

Last, one of the reasons science is currently experiencing a replication crisis, in my opinion, is that we're all become modelators (if I may coin a phrase for idolators/worshippers of models). We over-simplify and then when a computer spits out an answer with the proper p-value, we say it's true. Not to discourage you, but to add a warning to your enthusiasm. You have an admirable drive and as you branch out into Logic, Artificial Intelligence, Bayesian Statistics, etc, you'll learn a lot and may find some linkages between them that lead to breakthroughs.

Just be skeptical of your ideas, look for bugs in your code, look for exceptions in your logic, beat your ideas and system up, and in the end you may have a nugget of gold. That's the Scientific Method.

  • $\begingroup$ Ah, i made the same mistake of pressing enter and sending the above! To continue > I want to leave it open for a little while longer to encourage more posts, however the Dempster link was spot on and has given me direction. I will need to spend some time reading and digesting the info in that area before i can say more. One common misconception with the idea behind WL is that its heavily based on machine learning. I actually want to crowd source the arguments as much as possible. $\endgroup$ – WikiLogic Aug 14 '16 at 19:32
  • $\begingroup$ I will take away the grunt work of dumb calculations such as working out what probability you are left with after adding a chain of them but the program will not second guess if they should be added or not. It will be up to the community to question and raise issues in the chains of reasoning. The p-value issue is one that i dont know much about but i hope WL will help by highlighting the importance of any given scientific claim. If we see that one experiment is foundational for many subsequent claims, we know we need to make sure that is correct and invest in further testing it. Thanks again! $\endgroup$ – WikiLogic Aug 14 '16 at 19:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.