0
$\begingroup$

I'm working on comparing 2 algorithms with an experimental protocol that produce 100 folds for each one.

As a result, I found that my algorithm got (49.29 $\pm$ 1.69) and the baseline got (50.40 $\pm$ 2.16). I applied ANOVA and other tests and I always got a p-value of 0.60.

Method: Deep learning.

Goal: comparing 2 algorithms (mine and another)

Field: computer-vision

Hypothesis ($\alpha=0.05$):

  • H0: the mean of the results are equal.
  • Ha: the mean of the results are unequal. (advantage go to the adversary)

Results:

  • Population : 2 ( Method 1 and method 2)

  • sample size = 100

  • $P=0.6$ and $\alpha = 0.05$

  • $P > 0.05 $ $=>$ no significant difference

accuracy distribution: bleu: my model, Green: baseline

Can a reviewer reject the conclusion? (Quality)

If you need more information, please ask them in a comment.

$\endgroup$

closed as unclear what you're asking by mkt, Peter Flom Apr 16 at 19:26

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ What do you want to show? If you want to show the two are equivalent then ANOVA is not appropriate (it is a test of difference). In such cases use equivalence testing. $\endgroup$ – ReneBt Apr 8 at 12:04
  • $\begingroup$ Please don't say that your problem is urgent. Remember that you are asking strangers on the internet to volunteer their time to help you for free. $\endgroup$ – gung Apr 8 at 20:28
  • $\begingroup$ OK, I understand $\endgroup$ – zeronoid Apr 8 at 20:44
  • $\begingroup$ Reviewers can do anything. $\endgroup$ – Peter Flom Apr 16 at 19:26
3
$\begingroup$

There is no possible way for us to know whether reviewers will accept or reject your results.

Even researchers with decades of experience don't always know what will be accepted or rejected by whom - they may have a better idea than people new to the field, but they are still surprised fairly frequently.

Also, you have told us nothing about your study, your methods, your hypotheses, your conclusions. Nor have you told us about the conference you are submitting to and its record of rejecting and accepting papers. Nor have you told us about whether your conclusions are interesting or surprising, nor whether your effect size estimates are large or small, precise or imprecise.

Even with all that information, we'd be guessing. But right now, we can't even make an educated guess.

$\endgroup$
  • $\begingroup$ Could you recheck my question? I change it $\endgroup$ – zeronoid Apr 8 at 19:47
  • $\begingroup$ What does "result be rejected" mean? $\endgroup$ – Peter Flom Apr 8 at 19:51
  • $\begingroup$ Can a reviewer state that the Ha should be accepted $\endgroup$ – zeronoid Apr 8 at 19:54
  • 2
    $\begingroup$ Reviewers can do anything. Your question is still unclear. I think you just have to submit and hope, like the rest of us. $\endgroup$ – Peter Flom Apr 8 at 19:56
  • $\begingroup$ I add more information. $\endgroup$ – zeronoid Apr 8 at 20:16
1
$\begingroup$

As @Peter_Flom pointed out, you don't provide a lot of context.

Assuming A) that you have set up a procedure that creates correct estimates of the error under CV, and B) assuming you correctly calculated the uncertainty of the CV estimate, your results would simply be that there is no significant difference between the algorithms (you don't need a test, it's pretty obvious from the widths of your CIs).

The core question is if you have done B) right. Maybe you could clarify how you derive the +/- estimates.

$\endgroup$
  • $\begingroup$ Could you recheck my question? I change it $\endgroup$ – zeronoid Apr 8 at 19:47

Not the answer you're looking for? Browse other questions tagged or ask your own question.