I'm following this paper:
And I'm trying to understand specifically how the k-means approach works when learning feature filters in a convolution map. The section I'm reading is III. A. page 2. From what I understand, we start out with a large set of normalized/ whitened filters,... say 3600 filters taken from various sample of training data and the key is to find a reduced set of filters,... maybe 96 filters that have a minimized euclidean distance from the 3600 original filters.
I understand that we a solving for a dictionary D which contain all learned filters with that small euclidean distance in their respective clusters. But I don't understand why we multiply D * s(i) in the following equation (1) of the paper:
Sum||(D * s(i) − x(i))||^2
If s(i) is a basis vector that belongs to D, then what point does D serve. I'm actually really confused how the k-means process is supposed to work under the particular theme that this paper talks about. What do they mean by that "one hot encoder" and if we're summing over the difference of s(i) and x(i), why would s and x have the same index? Wouldn't we want to see which cluster x belongs to to minimize our squares -> s(j) - x(i) : sum over j, and assign x to the proper s.
Also they don't explain the process of minimization. I understand how minimization works in normal k-clustering, but not in this context. How do you switch from minimizing D and then switching to minimize s? Lastly, how do we initialize s(i), do we just take s(i) from a column of D?