How do I know if my data is meaningful? I am a 17 year old conducting a very scientific experiment into the best method for pulling Christmas crackers in order to get the highest chance of winning the prize. I have 1000 crackers and a team of volunteers, however I am unsure how to process the results, while I am very keen to pursue a career in maths - I have been taught very little so far :( and would really appreciate some simple advice on how to show if my data is meaningful or just a random occurrence.
Currently my data will be collected to give me a %-chance of winning for each method (Number of wins/Total tries). However if I find that twisting a cracker as you pull gave 60% of the test subjects a win. Would this be a statistical outcome or just a fluke? (Assuming the control to be 50%) and if 60% isnt, where is the cutoff? Is there one? I apologize for my naivety in the world of statistics!
I have not run the experiment yet so any suggestions (for example collect every event individually if this helps the analysis) would be much appreciated also.
If there is a simple answer to this question then a few links for me to discover my answers would be very useful indeed.
Thank you and Merry Christmas!
 A: I think running simulations can give good intuition on how common an outcome may be given random chance.
Say you have 10 volunteers who try 100 crackers each. The first five volunteers use strategy #1, while the second five use strategy #2. Both strategies have 50% chance of being successful on any given attempt. Then we can make this simple simulation using R:
# Set the parameters for the simulation
Nsim=10000; Ncrackers=100; Nvolunteers=10; Pwin1=.5; Pwin2=.5;
StrategyUsed=c(rep(1,Nvolunteers/2),rep(2,Nvolunteers/2))

dat=matrix(nrow=Ncrackers, ncol=Nvolunteers) # Create matrix to hold the Individual Results
Percent.Success=matrix(nrow=Nsim, ncol=Nvolunteers) # Create matrix to hold the Success Rates

for(sim in 1:Nsim){
  # Simulate the results for each volunteer 
  for(v in 1:Nvolunteers){
    if(StrategyUsed[v]==1){
      dat[,v]<-sample(x=c("win","lose"), size=Ncrackers, 
                      prob=c(Pwin1,1-Pwin1), replace=TRUE)
    }
    if(StrategyUsed[v]==2){
      dat[,v]<-sample(x=c("win","lose"), size=Ncrackers, 
                      prob=c(Pwin2,1-Pwin2), replace=TRUE)
    }
    Percent.Success[sim,v]=100*length(which(dat[,v]=="win"))/Ncrackers
  }
}

#Calculate the difference in average percent success between the two strategies
SuccessDifferences<-rowMeans(Percent.Success[,1:5])-rowMeans(Percent.Success[,6:10])


#Plot a Representative Result
boxplot(c(Percent.Success[1,])~StrategyUsed, range=0, col=c("Red","Blue"), 
        names=c("Strategy #1", "Strategy #2"), ylab="Percent Success")
stripchart(c(Percent.Success[1,])~StrategyUsed, method="jitter", 
           jitter=.2, vertical=T, pch=21,  bg="Grey", cex=1.5, add=T)

#Plot Distribution of all Results
par(mfrow=c(3,1))
hist(Percent.Success[,1:5], col="Red", xlim=c(0,100), 
     xlab="Percent Success", main="Strategy #1")
hist(Percent.Success[,6:10], col="Blue", xlim=c(0,100), 
     xlab="Percent Success", main="Strategy #2")
hist(SuccessDifferences, col="Grey", 
     xlab="Average Difference", main="Strategy #1 - Strategy #2")


#Calculate the percent of simulation results beyond threshold = 10%
100*length(which(abs(SuccessDifferences)>10))/Nsim

Here is an example of one of the results:

Here the top two charts are distribution of percent success from all the simulations. The bottom is the distribution of the differences between the two strategies on average:

The last line calculates the percent of time the differences between the two groups was greater than 10%. For these specific simulations it was only 0.16%.
    #Calculate the percent of simulation results beyond threshold = 10%
    > 100*length(which(abs(SuccessDifferences)>10))/Nsim
[1] 0.16

Is an outcome that occurs 0.16% of the time rare enough to be considered not random chance? That is up to you. Also if there were a differences between volunteers using the two strategies it is not correct to then assume the differences is due to the strategy. Other possibilities need to be ruled out, for example maybe volunteers 1:5 got a different batch of crackers than volunteers 6:10, etc.
Edit:
Also:

I have not run the experiment yet so any suggestions (for example
  collect every event individually if this helps the analysis) would be
  much appreciated also.

Yes, collect as much data as is practical to collect then plot it vs time and anything else you can think of. The reason is that if you find patterns in the individual events (eg many wins early, fewer later) then there may be other processes going on that may be interesting. For example, perhaps learning is occurring, or fatigue/boredom. These other aspects may differ between the groups and result in differences confounding the theorized effect due to strategy. 
The important thing is to be ready to rule out as many explanations for an observed effect as possible, then the ones not ruled out may explain them. Exploring the data that way would not mean you can conclude any apparent patterns are not "random chance", but it may cast doubt on interpreting the results in favor of the strategy effect. Just ruling out that two groups are exactly the same is not very informative without that.
A: How many methods are you comparing? If you are just comparing 2 methods which are both trialed for each cracker then it is simple. If you have 3 or more methods that could add some complications.
Assuming you have just 2 methods then you can easily test if method A is better than B by assuming that they are both equally good (so the chance of success is 50%) and show that your results would be very unlikely if the chance of success was 50%.
The critical value for this test is 
$0.5+0.5\times \frac{1.645}{\sqrt{N}}$
0.5 is the expected rate of winning if A and B are equally good, $0.5\times \frac{2.32}{\sqrt{N}}$ is the amount you'd expect the results to vary if you take N samples. 
So taking the example you gave, if you pull 1000 crackers the critical value is $0.5+0.5\times \frac{2.32}{\sqrt{1000}}=0.54$  which means that it is unlikely to have a success rate above 54%. The 2.32 figure in the formula corresponds to being 99% sure that the success rate would be under 54% so if you do get a success rate of 60% then you're over 99% sure that one method is better than the other. With 1000 crackers you'll probably get results showing you are 99% sure of, but in case you want to split up the crackers between several tests you can replace 2.32 with 1.64 in the calculations and you will be 95% sure of the results instead of 99% sure.
