# How does expectation maximization relate to weighted least squares?

For the past few days I have been trying to implement the EM-algorithm in order to segment stores into k-clusters. What I already did was derivation of the complete-log-likelihood and also performed the E-step. Where it goes wrong is the M-step.

I showed that I need to optimize the following (See derivation):

$$-\frac{1}{2} \sum^{N T} P_{i s}\left(\log (S_{it})-\alpha^{[s]}-\beta^{[s]} \log \left(P_t\right)\right)^2$$

When I take the derivative with respect to $$\alpha^{[s]}$$ and apply first order conditions, there shows up a $$\beta^{[s]}$$ in the expression and vice versa.

The main problem is my rusty linear algebra, because to solve this I should store $$\alpha^{[s]}$$ and $$\beta^{[s]}$$ in vector $$\theta^{[s]}$$ and optimize with respect to $$\theta^{[s]}$$, such that I get an expression that depends on the $$P_t$$ and $$Log(S_{it})$$

Additionally I need to show that this expression is basically the WLS solution

Q1: What is the derivation of the optimal $$\theta^{[s]}$$ Q2: How does this solution relation to WLS?

I added my derivations as an attachment to give extra context if someone is interested.

Context of the model:

Note: There are multiple stores and each store has time observations. Hence, I compute a density per store by taking all the time-observations of that store into account.

We consider a model for the sales of a particular product, measured at the store level at a weekly frequency. Denote the sales of the product in store $$i=1, \ldots, N$$ in week $$t=1, \ldots, T$$ by $$S_{i t}$$. The sales are explained by the price of the product at time t, that is, $$p_t$$. All stores have the same price in any given week. We specify the following heterogeneous model $$\log S_{i t}=\alpha_i+\beta_i \log p_t+\varepsilon_{i t}, \quad \varepsilon_{i t} \sim N(0,1) .$$

The store-specific parameters are assumed to follow a mixture distribution with $$K$$ segments. $${ }^1$$ We use $$C_i \in\{1, \ldots, K\}$$ to denote the (unobserved) cluster to which store $$i$$ belongs. Denote $$\operatorname{Pr}\left[C_i=c\right]=\pi_c, c=1, \ldots, K$$ with $$\pi_c \geq 0$$ and $$\sum_{c=1}^K \pi_c=1$$. For stores in segment $$c$$ the intercept and price elasticity equal $$\alpha^{[c]}$$ and $$\beta^{[c]}$$, respectively. Therefore $$\alpha_i=\alpha^{[c]} \text { and } \beta_i=\beta^{[c]} \text { if } C_i=c .$$

Finally, denote $$\theta=\left(\alpha^{[1]}, \beta^{[1]}, \ldots, \alpha^{[K]}, \beta^{[K]}\right)^{\prime}$$ and $$\pi=\left(\pi_1, \ldots, \pi_K^{\prime}\right)^{\prime}$$.

E step: Taking expectation w.r.t. $$C$$ on the loglikelihood results in a "weighted" log likelihood.
• The last two boxes of your photo forms a linear system of two equations and two unknowns, $\alpha$ and $\beta$. Solve the linear system will get you the answer. Another, notation simpler but mathmatically equivalent method, is to represent your target function in matrix form and let the gradient of $\theta^{[1]}$ equals zero. Commented Dec 2, 2023 at 15:44