Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
2 added 373 characters in body
source | link

As you're likely aware, a vector in a $D$-dimensional space can be described by a direction and a magnitude. The direction requires $D-1$ values to describe, and the magnitude requires one value to describe.

Suppose we pick some directional unit vector $\hat{v}.$ By "unit vector" I mean that it has magnitude one. A vector $w$$z$ that satisfies $w \cdot \hat{v} = \sum_j w_j \hat{v}_j = 0$$z \cdot \hat{v} = \sum_j z_j \hat{v}_j = 0$ is perpendicular to the directional vector. The set of all vectors that can be perpendicular to $\hat{v}$ definesforms a $D-1$ dimensional hyperplane that goes through the origin. This is easy to see, because the constraint that it's perpendicular to $\hat{v}$ is really just the equation of a $D-1$ dimensional plane in $D$ dimensions, i.e. $$\sum_{j=1}^D \hat{v}_j w_j = 0.$$$$\sum_{j=1}^D \hat{v}_j z_j = 0.$$ This is a linear combination of the components of $w$$z$ with a zero intercept. It exhibits plane-like behavior because, if you reduce the value of one component of $w$$z$ by a certain amount, you must increase other component values in a linear fashion, so that the sum still reaches zero.

Any input vector $x$ in a perceptron can be expressed as $x = r \hat{v}$$$x = r \hat{v}$$ where $r \in \mathbb{R}$ is a magnitude, and $\hat{v} \in \mathbb{R}^D$ is a directional vector that satisfies $|\hat{v}|=1.$ In fact, the perceptron only cares about the direction $\hat{v}$, not the magnitude $r$. To see this, look at the classification procedure for a perceptron: $$ f(x;w) = \left\{ \begin{split} 1 \hspace{1mm} \text{if} \hspace{1mm} x \cdot w >0 \\ 0 \hspace{1mm} \text{if} \hspace{1mm} x \cdot w \le 0 \end{split} \right. $$ where $w$ is the weight vector of the perceptron and I'm implicitly including the constant bias component to $x.$ It's easy to see that the conditions above for $x \cdot w$ equally apply to $\hat{v} \cdot w.$ Thus you can replace $x$ with $\hat{v}$ in the above condition and the perceptron function is unchanged.

As we established, a plane in $w$-space can be defined by the equation $\hat{v} \cdot w = 0.$ This, to answer your question, is how an input vector can be represented by a plane. In factTo be more precise, an input direction defines the plane, but the plane is invariant to the magnitude, of the vector. $r,$(The equation of a plane is identical if you multiply the whole thing by a constant.) So there's one missing piece of information on the plane, with regards to the vector. However, but as we established, so is the perceptron algorithm doesn't care about the magnitude of the vector, so for all intents and purposes, the plane describes the vector entirely, to all the detail we care about.  

If our value of $w$ falls on or below the plane defined by an input direction $\hat{v}$ (where by "below" I mean on the opposite side of the plane than $\hat{v}$) then the perceptron classifies this input as a "zero", otherwise it classifies it as a "one". To see why this is the case, we need only take advantage of a very elementary result of linear algebra, namely that $$ |w \cdot \hat{v}| = |w| |\hat{v}| \cos \theta = |w| \cos \theta, $$ where $\theta$ is the angle between $w$ and $\hat{v}$ (i.e. the angle between $w$ and $x$). The perceptron condition then becomes that the point is classified as a "one" if $\theta < \pi/2$ and a "zero" if $\pi/2 < \theta \le \pi.$$\pi/2 \le \theta \le \pi.$

(To answer your question 3, they're not different. The input vector is the training case.)

As you're likely aware, a vector in a $D$-dimensional space can be described by a direction and a magnitude. The direction requires $D-1$ values to describe, and the magnitude requires one value to describe.

Suppose we pick some directional unit vector $\hat{v}.$ By "unit vector" I mean that it has magnitude one. A vector $w$ that satisfies $w \cdot \hat{v} = \sum_j w_j \hat{v}_j = 0$ is perpendicular to the directional vector. The set of all vectors that can be perpendicular to $\hat{v}$ defines a $D-1$ dimensional hyperplane that goes through the origin. This is easy to see, because the constraint that it's perpendicular to $\hat{v}$ is really just the equation of a $D-1$ dimensional plane in $D$ dimensions, i.e. $$\sum_{j=1}^D \hat{v}_j w_j = 0.$$ This is a linear combination of the components of $w$ with a zero intercept. It exhibits plane-like behavior because, if you reduce the value of one component of $w$ by a certain amount, you must increase other component values in a linear fashion, so that the sum still reaches zero.

Any input vector $x$ in a perceptron can be expressed as $x = r \hat{v}$ where $r \in \mathbb{R}$ is a magnitude, and $\hat{v} \in \mathbb{R}^D$ is a directional vector that satisfies $|\hat{v}|=1.$ In fact, the perceptron only cares about the direction $\hat{v}$, not the magnitude $r$. To see this, look at the classification procedure for a perceptron: $$ f(x;w) = \left\{ \begin{split} 1 \hspace{1mm} \text{if} \hspace{1mm} x \cdot w >0 \\ 0 \hspace{1mm} \text{if} \hspace{1mm} x \cdot w \le 0 \end{split} \right. $$ where I'm implicitly including the constant bias component to $x.$ It's easy to see that the conditions above for $x \cdot w$ equally apply to $\hat{v} \cdot w.$ Thus you can replace $x$ with $\hat{v}$ in the above condition and the perceptron function is unchanged.

As we established, a plane in $w$-space can be defined by the equation $\hat{v} \cdot w = 0.$ This, to answer your question, is how an input vector can be represented by a plane. In fact, the plane is invariant to the magnitude, $r,$ of the vector, but as we established, so is the perceptron algorithm.  

If our value of $w$ falls on or below the plane defined by an input direction $\hat{v}$ (where by "below" I mean on the opposite side of the plane than $\hat{v}$) then the perceptron classifies this input as a "zero", otherwise it classifies it as a "one". To see why this is the case, we need only take advantage of a very elementary result of linear algebra, namely that $$ |w \cdot \hat{v}| = |w| |\hat{v}| \cos \theta = |w| \cos \theta, $$ where $\theta$ is the angle between $w$ and $\hat{v}$ (i.e. the angle between $w$ and $x$). The perceptron condition then becomes that the point is classified as a "one" if $\theta < \pi/2$ and a "zero" if $\pi/2 < \theta \le \pi.$

(To answer your question 3, they're not different. The input vector is the training case.)

As you're likely aware, a vector in a $D$-dimensional space can be described by a direction and a magnitude. The direction requires $D-1$ values to describe, and the magnitude requires one value to describe.

Suppose we pick some directional unit vector $\hat{v}.$ By "unit vector" I mean that it has magnitude one. A vector $z$ that satisfies $z \cdot \hat{v} = \sum_j z_j \hat{v}_j = 0$ is perpendicular to the directional vector. The set of all vectors that can be perpendicular to $\hat{v}$ forms a $D-1$ dimensional hyperplane that goes through the origin. This is easy to see, because the constraint that it's perpendicular to $\hat{v}$ is really just the equation of a $D-1$ dimensional plane in $D$ dimensions, i.e. $$\sum_{j=1}^D \hat{v}_j z_j = 0.$$ This is a linear combination of the components of $z$ with a zero intercept. It exhibits plane-like behavior because, if you reduce the value of one component of $z$ by a certain amount, you must increase other component values in a linear fashion, so that the sum still reaches zero.

Any input vector $x$ in a perceptron can be expressed as $$x = r \hat{v}$$ where $r \in \mathbb{R}$ is a magnitude, and $\hat{v} \in \mathbb{R}^D$ is a directional vector that satisfies $|\hat{v}|=1.$ In fact, the perceptron only cares about the direction $\hat{v}$, not the magnitude $r$. To see this, look at the classification procedure for a perceptron: $$ f(x;w) = \left\{ \begin{split} 1 \hspace{1mm} \text{if} \hspace{1mm} x \cdot w >0 \\ 0 \hspace{1mm} \text{if} \hspace{1mm} x \cdot w \le 0 \end{split} \right. $$ where $w$ is the weight vector of the perceptron and I'm implicitly including the constant bias component to $x.$ It's easy to see that the conditions above for $x \cdot w$ equally apply to $\hat{v} \cdot w.$ Thus you can replace $x$ with $\hat{v}$ in the above condition and the perceptron function is unchanged.

As we established, a plane in $w$-space can be defined by the equation $\hat{v} \cdot w = 0.$ This, to answer your question, is how an input vector can be represented by a plane. To be more precise, an input direction defines the plane, but the plane is invariant to the magnitude of the vector. (The equation of a plane is identical if you multiply the whole thing by a constant.) So there's one missing piece of information on the plane, with regards to the vector. However, as we established, the perceptron algorithm doesn't care about the magnitude of the vector, so for all intents and purposes, the plane describes the vector entirely, to all the detail we care about.

If our value of $w$ falls on or below the plane defined by an input direction $\hat{v}$ (where by "below" I mean on the opposite side of the plane than $\hat{v}$) then the perceptron classifies this input as a "zero", otherwise it classifies it as a "one". To see why this is the case, we need only take advantage of a very elementary result of linear algebra, namely that $$ |w \cdot \hat{v}| = |w| |\hat{v}| \cos \theta = |w| \cos \theta, $$ where $\theta$ is the angle between $w$ and $\hat{v}$ (i.e. the angle between $w$ and $x$). The perceptron condition then becomes that the point is classified as a "one" if $\theta < \pi/2$ and a "zero" if $\pi/2 \le \theta \le \pi.$

(To answer your question 3, they're not different. The input vector is the training case.)

1
source | link

As you're likely aware, a vector in a $D$-dimensional space can be described by a direction and a magnitude. The direction requires $D-1$ values to describe, and the magnitude requires one value to describe.

Suppose we pick some directional unit vector $\hat{v}.$ By "unit vector" I mean that it has magnitude one. A vector $w$ that satisfies $w \cdot \hat{v} = \sum_j w_j \hat{v}_j = 0$ is perpendicular to the directional vector. The set of all vectors that can be perpendicular to $\hat{v}$ defines a $D-1$ dimensional hyperplane that goes through the origin. This is easy to see, because the constraint that it's perpendicular to $\hat{v}$ is really just the equation of a $D-1$ dimensional plane in $D$ dimensions, i.e. $$\sum_{j=1}^D \hat{v}_j w_j = 0.$$ This is a linear combination of the components of $w$ with a zero intercept. It exhibits plane-like behavior because, if you reduce the value of one component of $w$ by a certain amount, you must increase other component values in a linear fashion, so that the sum still reaches zero.

Any input vector $x$ in a perceptron can be expressed as $x = r \hat{v}$ where $r \in \mathbb{R}$ is a magnitude, and $\hat{v} \in \mathbb{R}^D$ is a directional vector that satisfies $|\hat{v}|=1.$ In fact, the perceptron only cares about the direction $\hat{v}$, not the magnitude $r$. To see this, look at the classification procedure for a perceptron: $$ f(x;w) = \left\{ \begin{split} 1 \hspace{1mm} \text{if} \hspace{1mm} x \cdot w >0 \\ 0 \hspace{1mm} \text{if} \hspace{1mm} x \cdot w \le 0 \end{split} \right. $$ where I'm implicitly including the constant bias component to $x.$ It's easy to see that the conditions above for $x \cdot w$ equally apply to $\hat{v} \cdot w.$ Thus you can replace $x$ with $\hat{v}$ in the above condition and the perceptron function is unchanged.

As we established, a plane in $w$-space can be defined by the equation $\hat{v} \cdot w = 0.$ This, to answer your question, is how an input vector can be represented by a plane. In fact, the plane is invariant to the magnitude, $r,$ of the vector, but as we established, so is the perceptron algorithm.

If our value of $w$ falls on or below the plane defined by an input direction $\hat{v}$ (where by "below" I mean on the opposite side of the plane than $\hat{v}$) then the perceptron classifies this input as a "zero", otherwise it classifies it as a "one". To see why this is the case, we need only take advantage of a very elementary result of linear algebra, namely that $$ |w \cdot \hat{v}| = |w| |\hat{v}| \cos \theta = |w| \cos \theta, $$ where $\theta$ is the angle between $w$ and $\hat{v}$ (i.e. the angle between $w$ and $x$). The perceptron condition then becomes that the point is classified as a "one" if $\theta < \pi/2$ and a "zero" if $\pi/2 < \theta \le \pi.$

(To answer your question 3, they're not different. The input vector is the training case.)