In order to measure the information density like proposed in section 3.2 of this paper I need a symmetric positive definite Kernel function. For this purpose I want to use the Gaussian Kernel like suggested in the paper.

When using the RBF Kernel from scikit-learn the user guide states that when setting gamma to sigma negative square the kernel becomes a gaussian kernel of variance sigma squared. Here is the quote:

enter image description here

Where do I get the value for Sigma?
Is it the standard deviation of something?
Is Sigma calculated from the two input vectors xi and xj or do I need two whole sets of data?
Is the calculated value for sigma then really only raised to the power of -2?

Here is my current Python code for the problem:

def gaussian_kernel(x_i, x_j):

    # if gamma = sigma negative square then the kernel is known as the
    # Gaussian kernel of variance sigma square
    sigma = 0 # how to calculate sigma and sigma negativ squared?
    gamma = sigma**-2 # <- is this even correct?
    kernel_result = rbf_kernel(x_i, x_j, gamma)

    return kernel_result 
  • $\begingroup$ The question would be: What are you trying to achieve? I guess you want to use SVM to predict something (?)... Usually there is nobody who tells you what value you should use (except for the case when you have some specific prior knowledge about the data). The remark is of a 'purely cosmetic' nature. Usually $\gamma$ is a so-called 'hyperparameter' meaning that for each training run of the model, it is a fixed, preset parameter. People usually figure out the 'best' value for $\gamma$ by simply attempting to run and evaluate the model with many different (guessed) values for $\gamma$... $\endgroup$ – Fabian Werner Apr 2 '20 at 14:33
  • $\begingroup$ @FabianWerner I updated my qestion to answer your question. I am not using a SVM but want to calculate how informative a sample is in a dataset $\endgroup$ – JohnDizzle Apr 2 '20 at 14:47
  • $\begingroup$ It seems as if they want to fit a Gaussian Process to something. In that case, $\sigma$ still is a hyperparameter but a good values can be figured out via likelihood maximization, see mlss2011.comp.nus.edu.sg/uploads/Site/lect1gp.pdf. $\endgroup$ – Fabian Werner Apr 3 '20 at 20:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.