In a presentation I saw recently, a two-sided t-test was repeated with jackknifed subsets of the original data in order to assess the result's "robustness".

In detail, they took a random half of the first group and a random half of the second group (with a couple hundred samples each), ran a t-test, recorded the result, and repeated this for 100 times. Then, they used the count of significant p-values to estimate the robustness of their original result (a t-test with the full dataset).

I am more of a "stats-consumer" with weak theoretical background and have never seen anything like this before. My impression about jackknifing / bootstrapping was always that you use it to "recreate" a null distribution in order to assess the extremeness of your result, which is quite different to the procedure described above. However, it seems to be a simple and interesting approach to see whether only a few samples determine the result. Thus, I wanted to read up on the theory behind it (how many samples one should use, what contingencies to consider, etc.) but couldn't really find anything. Now I am sceptical. Is this actually a valid strategy and if so, does it have a specific name or term associated with it? Could you point me to related literature?

In a presentation I saw recently, a two-sided t-test was repeated with jackknifed subsets of the original data in order to assess the result's "robustness".

In detail, they took a random half of the first group and a random half of the second group (with a couple of hundred samples each), ran a t-test, recorded the result, and repeated this for 100 times. Then, they used the count of significant p-values to estimate the robustness of their original result (a t-test with the full dataset).

I am more of a "stats-consumer" with a weak theoretical background and have never seen anything like this before. My impression about jackknifing / bootstrapping was always that you use it to "recreate" a null distribution in order to assess the extremeness of your result, which is quite different to the procedure described above. 

However, it seems to be a simple and interesting approach to see whether only a few samples determine the result. Thus, I wanted to read up on the theory behind it (how many samples one should use, what contingencies to consider, etc.) but couldn't really find anything. 

Now I am sceptical. Is this actually a valid strategy and if so, does it have a specific name or term associated with it? Could you point me to related literature?

In a presentation I saw recently, a two-sided t-test was repeated with jackknifed subsets of the original data in order to assess the result's "robustness".

In detail, they took a random half of the first group and a random half of the second group (with a couple hundred samples each), ran a t-test, recorded the result, and repeated this for 100 times. Then, they used the count of significant p-values to estimate the robustness of their original result (a t-test with the full dataset).

I am more of a "stats-consumer" with weak theoretical background and have never seen anything like this before. My impression about jackknifing / bootstrapping was always that you use it to "recreate" a null distribution in order to assess the extremeness of your result, which is quite different to the procedure described above. However, it seems to be a simple and interesting approach to see whether only a few samples determine the result. Thus, I wanted to read up on the theory behind it (how many samples one should use, what contingencies to consider, etc.) but couldn't really find anything. Now I am sceptical. Is this actually a valid strategy and if so, does it have a specific name or term associated with it?

In a presentation I saw recently, a two-sided t-test was repeated with jackknifed subsets of the original data in order to assess the result's "robustness".

In detail, they took a random half of the first group and a random half of the second group (with a couple hundred samples each), ran a t-test, recorded the result, and repeated this for 100 times. Then, they used the count of significant p-values to estimate the robustness of their original result (a t-test with the full dataset).

I am more of a "stats-consumer" with weak theoretical background and have never seen anything like this before. My impression about jackknifing / bootstrapping was always that you use it to "recreate" a null distribution in order to assess the extremeness of your result, which is quite different to the procedure described above. However, it seems to be a simple and interesting approach to see whether only a few samples determine the result. Thus, I wanted to read up on the theory behind it (how many samples one should use, what contingencies to consider, etc.) but couldn't really find anything. Now I am sceptical. Is this actually a valid strategy and if so, does it have a specific name or term associated with it? Could you point me to related literature?