Bootstrap resampling for constructing hypothesis test I need to use bootstrap resampling to test the significant difference between two datasets (data1 & data2). I have already used bootstrap resampling to estimate the confidence interval of the mean for a single datase. However, I am absolutely lost when trying to use bootstrap resampling for testing the null hypothesis of whether any two datasets are different. To be more specific, the reasons for my confusion come from the following: 
1- My two datasets are not numeric, they are a set of specific words (Rainy, Sunny, cloudy). I want to test the difference between data1 and data2 in predicting the weather. So my datasets would be somthing like:
      data1   data2   correct-prediction
 day1 rainy   rainy   rainy
 day2 cloudy  sunny   sunny
 day3 cloudy  rainy   rainy

How to convert these two datasets - data1 and data2 - to appropriate numeric data (vectors) for applying bootstrap resampling.
2- what is the mechanism of using bootstrap resampling for testing the null hypothesis of whether any two datasets are different. Should I resample each dataset (after converting it to appropriate numeric values) and compute the means for the bootstraps, then making the comparisons? 
 A: From the description above it seems you are mainly interested in comparing the performance of the two data-sets. If this is the case:


*

*Select a measure of multi-class classification performance that fits your purpose (e.g. F1-score >>here you find a bunch of other metrics).

*Calculate the performance for data1 and data2. This is the observed performance of each method.

*Calculate the difference in performance for the two methods.

*You are interested in estimating how (un-)likely the observed difference is under the null-hypothesis that both methods perform equally good/bad.

*To test this you generate a distribution of performances on randomly sampled data. Run both methods X times (let's say X=5ooo), each time drawing random data with replacement (e.g. day 3, day3, day1). The size of each sample should correspond to the size of your data-sets. Calculate and record the performance difference at each iteration for each bootstrap-sample.

*You now got a distribution of performance differences that can be expected for data1 and data2.

*Now compare your observed performance difference with this distribution (e.g. via histogram): How extreme is this difference on this distribution? The percentile rank of your observed performance difference on the distribution is is the p-value you are looking for.
A: As for your more general question, how to use bootstrapping for hypothesis testing, in my understanding (which might be wrong ...), bootstrapping in the usual sense is not well adapted for hypothesis testing.  It is better for estimating standard erors of estimates or even confidence intervals.  The reason is that bootstrapping consists in resampling from the data as a way of estimating the sampling distribution of some statistic or estimator.  That is, bootstrapping is (trying to) estimate the sampling distribution with the actually working data generating process (or with other words, with the true values of the underlying parameters) while for hypothesis testing you need the sampling distribution under the null hypothesis, that is, some hypothesized parameter values, not the actual ones. 
So for hypothesis testing somewhat similar in spirit to bootstrapping, maybe look into monte carlo  tests or permutation tests. 
