1
$\begingroup$

My question is to do with

1) how to identify the best kernel function to use (for instance Epanechnikov, Gaussian, triangle etc) for earnings on formal and informal sector using Stata

2) how do I work out the bandwidth which would bring out the best estimation.

| cite | improve this question | |
$\endgroup$
  • $\begingroup$ Software-specific details are usually off-topic here. See stats.stackexchange.com/help/on-topic for guidance. $\endgroup$ – Nick Cox Jun 26 '17 at 15:08
  • 2
    $\begingroup$ The kernel is usually of somewhat lesser importance than the bandwidth. How are you defining "best" here? $\endgroup$ – Glen_b Jun 27 '17 at 3:50
  • $\begingroup$ @Glen_b♦...In the sense that, which kernel function is much more likely to show a true reflection of my earnings on formal and informal sector.....thanks. $\endgroup$ – david Jun 27 '17 at 10:10
  • 5
    $\begingroup$ Sorry, that's pretty much completely useless. What do you actually mean by "true reflection" here? I'm trying to figure out what we're trying to do best at, in a precise mathematical sense, rather than going through iterations where you come up with new ways to say something too vague to do anything with and we ask you what that means. This will pretty much require either very precise language or some actual algebraic expression. $\endgroup$ – Glen_b Jun 27 '17 at 11:03
  • 4
    $\begingroup$ @Glen_b is bang on here, I am afraid, and being an expert doesn't qualify anyone to say in abstraction what is best for you and your data. Practically, try out different bandwidths and see which one seems to be the best where you can make most sense of the patterns shown. It's banal but important that you want to avoid under- and over-smoothing. $\endgroup$ – Nick Cox Jun 27 '17 at 14:02
1
$\begingroup$

Check out the webpage: https://jakevdp.github.io/blog/2013/12/01/kernel-density-estimation/

1) You can test them separately. keep some of your data as validation data, do the KDE without the validation data, look at the likelihood of the validation data in the KDE model. The kernel which gives the highest likelihood is probably the best kernel.

2) You can do cross-validation to get the best parameter. There is a section "Bandwidth Cross-Validation in Scikit-Learn" in the link, which shows you how to do it in a couple of lines.

EDIT: Here is a demonstration of how you would do it (code mainly taken from the link). The code is in Python, which is easy to use for this kind of application:

nsamp=500

data=np.concatenate([np.array(np.random.normal(loc=0.1, scale=0.2, size=nsamp)), \
                     np.array(np.random.normal(loc=2.9, scale=0.8, size=nsamp))])
data=np.reshape(data,(2*nsamp,1))

#seperate into validation and training
val_data=data[0:int(nsamp*0.9)]
train_data=data[int(nsamp*0.9):]

#look at the data with a histogram
plt.hist(train_data,bins=100, normed=True)

#1. now do the KDE with Gaussian kernel with cross validation
grid = GridSearchCV(KernelDensity(kernel='gaussian'), {'bandwidth': np.linspace(0.01, 1.5, 20)}, cv=5) # 20-fold cross-validation
grid.fit(train_data)
print("grid.best_params_: " + str(grid.best_params_))

#get the best estimator
kde_gauss=grid.best_estimator_

##to play arounf and see the effects of changing the bandwidth
#kde_gauss = KernelDensity(kernel='gaussian', bandwidth=0.1)
#kde_gauss.fit(train_data)

#what is the likelihood of validation data
kde_gauss.score(val_data)

#look at the fitted pdf
plt.plot(np.linspace(-1,5,1000), np.exp(kde_gauss.score_samples(np.reshape(np.linspace(-1,5,1000),(1000,1)) )))


#2. now do the KDE with tophat kernel with cross validation
grid = GridSearchCV(KernelDensity(kernel='tophat'), {'bandwidth': np.linspace(0.01, 1.5, 20)}, cv=5) # 20-fold cross-validation
grid.fit(train_data)
print("grid.best_params_: " + str(grid.best_params_))

#get the best estimator
kde_tophat=grid.best_estimator_

#look at the fitted pdf
plt.plot(np.linspace(-1,5,1000), np.exp(kde_tophat.score_samples(np.reshape(np.linspace(-1,5,1000),(1000,1)) )))

#what is the likelihood of validation data
kde_tophat.score(val_data)
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for the prompt response ''user2137591''. Since am not really used to this,,,, what sign do i look out for to pick out the ''highest likelihood'' thanks $\endgroup$ – david Jun 26 '17 at 15:06
  • $\begingroup$ you need to evaluate the fitted KDE model at your validation data points. For example if you use KernelDensity from sklearn.neighbors, then you have a function, which does that for you: kde.score_samples() $\endgroup$ – user166243 Jun 26 '17 at 15:11
  • $\begingroup$ Thanks again...i have taken a look at it and it is becoming clearer as i go through them... $\endgroup$ – david Jun 27 '17 at 18:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.