Skip to main content
added 223 characters in body
Source Link
Thoth
  • 123
  • 4

I am reading about PCA and I came up with a some contradictions. PCA based on this post, PCA optimization problem is given by:

$$\begin{aligned} \max_{w} \quad & w^T\Sigma w\\ \textrm{s.t.} \quad & w^Tw=1\\ \end{aligned}\quad(\text{P1})$$ where $\Sigma$ is the covariance matrix of data. My questions are:

  1. In this post, it was stated by @whuber that $w^T\Sigma w$ is not Convex. How is that possible as $\nabla w^T\Sigma w=\Sigma \succeq 0$ (link)?
  2. Also, in the same post it is stated that the unit sphere $S^{n-1}\subset\mathbb{R}^n$ is decidedly not convex. How it is that possible as it is proven that $w^Tw=1$ is convex?Also, in the same post it is stated that the unit sphere $S^{n-1}\subset\mathbb{R}^n$ is decidedly not convex. How it is that possible as it is proven that $w^Tw=1$ is convex?
  3. Also, if $w^T\Sigma w$ is convex what is the intuition of maximizing it?

Could you please some one shade some light?

EDIT: I removed the second question. I just realized that we are dealing with a non-convex domain which cause as a non-convex problem, that is why we maximize (P1). Please, correct me If I am wrong.

I am reading about PCA and I came up with a some contradictions. PCA based on this post, PCA optimization problem is given by:

$$\begin{aligned} \max_{w} \quad & w^T\Sigma w\\ \textrm{s.t.} \quad & w^Tw=1\\ \end{aligned}\quad(\text{P1})$$ where $\Sigma$ is the covariance matrix of data. My questions are:

  1. In this post, it was stated by @whuber that $w^T\Sigma w$ is not Convex. How is that possible as $\nabla w^T\Sigma w=\Sigma \succeq 0$ (link)?
  2. Also, in the same post it is stated that the unit sphere $S^{n-1}\subset\mathbb{R}^n$ is decidedly not convex. How it is that possible as it is proven that $w^Tw=1$ is convex?
  3. Also, if $w^T\Sigma w$ is convex what is the intuition of maximizing it?

Could you please some one shade some light?

I am reading about PCA and I came up with a some contradictions. PCA based on this post, PCA optimization problem is given by:

$$\begin{aligned} \max_{w} \quad & w^T\Sigma w\\ \textrm{s.t.} \quad & w^Tw=1\\ \end{aligned}\quad(\text{P1})$$ where $\Sigma$ is the covariance matrix of data. My questions are:

  1. In this post, it was stated by @whuber that $w^T\Sigma w$ is not Convex. How is that possible as $\nabla w^T\Sigma w=\Sigma \succeq 0$ (link)?
  2. Also, in the same post it is stated that the unit sphere $S^{n-1}\subset\mathbb{R}^n$ is decidedly not convex. How it is that possible as it is proven that $w^Tw=1$ is convex?
  3. Also, if $w^T\Sigma w$ is convex what is the intuition of maximizing it?

Could you please some one shade some light?

EDIT: I removed the second question. I just realized that we are dealing with a non-convex domain which cause as a non-convex problem, that is why we maximize (P1). Please, correct me If I am wrong.

Source Link
Thoth
  • 123
  • 4

Some tricky details about PCA non-convexity

I am reading about PCA and I came up with a some contradictions. PCA based on this post, PCA optimization problem is given by:

$$\begin{aligned} \max_{w} \quad & w^T\Sigma w\\ \textrm{s.t.} \quad & w^Tw=1\\ \end{aligned}\quad(\text{P1})$$ where $\Sigma$ is the covariance matrix of data. My questions are:

  1. In this post, it was stated by @whuber that $w^T\Sigma w$ is not Convex. How is that possible as $\nabla w^T\Sigma w=\Sigma \succeq 0$ (link)?
  2. Also, in the same post it is stated that the unit sphere $S^{n-1}\subset\mathbb{R}^n$ is decidedly not convex. How it is that possible as it is proven that $w^Tw=1$ is convex?
  3. Also, if $w^T\Sigma w$ is convex what is the intuition of maximizing it?

Could you please some one shade some light?