There are 3 problems with the explanations:
For testing, you mention that you calculate "the" average score. You are looking to compare 2 computer programs, so unless you know the actual expectancy for one of them (which you don't in this setting), you'll have to calculate two averages: one for the old program, one for the new and test if the difference is equal to 0.3 (Tip: a more interesting test would be to test if the difference in the means is >0, i.e. a right-tailed test.)
As @BruceET correctly points out in the comment on the OP, we accept (i.e. can't reject) the null Hypothesis $H_0$ if the test statistic is within the critical interval, i.e. lower than the critical value (if we have a symmetric distribution), not greater. If it's greater, we reject the null and accept the alternative.
Lastly, the term "threshold" and "cutoff-off" seem very vague (almost arbitrary) and gives the impression that you consider the critical interval and p-values to be different tests - which they are not. To clarify this, see longer answer below.
(Frequentist) Hypothesis testing
Generally speaking, the critical interval and the p-value answer the same question with regard to the test statistic. They are not 2 different 'tests' per se, but rather 2 different ways of calculating the exact same thing.
The idea of hypothesis testing is to setup two hypotheses: the null $H_0$ and the alternative $H_1$. Then express them in terms of a statistical parameter. [In your case: $H_0:$ new program performs on average 0.3 better than old one. Reformulated in terms of the average score $\mu$: $H_0: \mu_{new}-\mu_{old}=0.3$]. Then formulate any random variable that can be calculated from sample data and transform it in such a way that its distribution is known if $H_0$ were true. This new transformed random variable is called the test statistic. [In your case we can use the averages of 100 empirical scores: If we assume that the scores are $iid$ then we can assume $\bar{X}_{new}\approx \mu_{new}$ and $\bar{X}_{old}\approx \mu_{old}$ by the LLN. We can assume $\bar{X}_{new}\overset{a}{\sim} \mathcal{N}(\mu_{new},\frac{\sigma_{new}^2}{n})$ by the CLT. Analogously for $\bar{X}_{old}$. Our test statistic is $t=\bar{X}_{new}-\bar{X}_{old} \Leftrightarrow t \overset{a}{\sim} \mathcal{N}(0.3,\frac{s^2}{n})$, where $s$ is the sample standard deviation of the differences.]
Let's say we have a random variable $t$ (a test statistic) and we know its distribution assuming $H_0$ is true. We then set a significance level e.g. $\alpha=5\%$, and calculate (for a two-tailed test) the 2 quantiles of the distribution of $t$ that satisfy that at least $1-\alpha=95\%$ of probability mass is between them. These two quantiles enclose the critical interval. If $quantile_1=-quantile_2$, i.e. if the distribution is symmetric, then we say $|quantile_1|$ is the critical value and compare it to $|t|$. If $|t|<|quantile_1|$ then we can't reject $H_0$. So the critical interval describes the space that our test statistic has to be in in order for $H_0$ not be rejected.
The p-value is kind of the opposite: it focuses on the area outside of the critical interval. It calculates the probability of $t$ or more extreme values occurring if $H_0$ were true. If this probability is larger than our significance level $\alpha$ then we do not reject.
You can use either of the two: critical interval or p-value, they will lead to the same conclusion since one follows from the other and vice-versa. (A third option would be comparing the random variable to the confidence intervals - still same result.) Note that $\alpha$ is set in advance and used for both. Below a plot to visualize all these values. In the plot, the null is accepted (not rejected).
