Multiple regression - how to calculate the predicted value after feature normalization? I'm currently doing the Andrew Ng machine learning course on coursera, and in Week2 he discusses feature scaling.
I have seen the lecture and read many posts; I understand the reasoning behind feature scaling (basically to make gradient descent converge faster by representing all the features on roughly the same scale). 
My problem arises when I try to do it. I'm using Octave, and I have the the code for gradient descent with linear regression set up: it computes the 'theta' matrix for the hypothesis just fine for non-scaled values, giving accurate predictions. 
When I use scaled values of input matrix X and output vector Y, the values of theta and the cost function J(theta) calculated are different than from the un-scaled values. Is this normal? How do I 'undo' the scaling, so that when I test my hypothesis with real data, I get accurate results?
For reference, here is the scaling function I am using (in Octave):
function [scaledX, avgX, stdX] = feature_scale(X)
    is_first_column_ones=0; %a flag indicating if the first column is ones
    sum(X==1)
    if sum(X==1)(1) == size(X)(1)   %if the first column is ones
        is_first_column_ones=1;
        X=X(:,2:size(X)(2));    %strip away the first column;
    end

    stdX=std(X);
    avgX=mean(X);

    scaledX=(X-avgX)./stdX;

    if is_first_column_ones
        %add back the first column of ones; they require no scaling.
        scaledX=[ones(size(X)(1),1),scaledX];
    end
end

Do I scale my test input, scale my theta, or both?
I should also note that I'm scaling as such:
scaledX=feature_scale(X);
scaledY=feature_scale(Y);

where X and Y are my input and output respectively. Each column of X represents a different feature (the first column is always 1 for the bias feature theta0) and each row of X represents a different input example. Y is a 1-D column matrix where each row is an output example, corresponding to the input of X.
eg: X = [1, x, x^2]
 1.00000    18.78152   352.74566
 1.00000     0.61030     0.37246
 1.00000    21.41895   458.77124
 1.00000     3.83865    14.73521

Y =
99.8043
 1.8283
168.9060
-29.0058

^ this is for the function y=x^2 - 14x + 10
 A: You should perform feature normalization only on features - so only on your input vector $x$. Not on output $y$ or $\theta$. When you trained a model using feature normalization, then you should apply that normalization every time you make a prediction. Also it is expected that you have different $\theta$ and cost function $J(\theta)$ with and without normalization. There is no need to ever undo feature scaling.
A: I understand the concepts explained in the previous 2 answers i.e. after we do feature scaling and calculate the intercept (θ0) and the slope (θ1), we get a hypothesis function (h(x)) which uses the scaled down features (Assuming univariate/single variable linear regression)

h(x) = θ0 + θ1x' -- (1)

where

x' = (x-μ)/σ -- (2)

(μ = mean of the feature set x; σ = standard deviation of feature set x)
As Yurii said above, we don't scale the  target i.e. y when doing feature scaling. So to predict y for some xm, we simply scale the new input value and feed it into the new hypothesis function (1) using (2)

x'm = (xm-μ)/σ -- (3)

And use this in (1) to get the estimated y. And I think this should work perfectly fine in practice.
But I wanted to plot the regression line against the original i.e. unscaled features and target values. So I needed a way to scale it back. Coordinate geometry to the rescue! :)
Equation (1) gives us the hypothesis with the scaled feature x'. And we know (2) is the relation between the scaled feature x' and the original feature x. So we substitute (2) in (1) and we get (after simplification):

h(x) = (θ0 - θ1*μ/σ) + (θ1/σ)x -- (4)

So to plot a line with the original i.e. unscaled features, we just use the intercept as (θ0 - θ1*μ/σ) and the slope as (θ1/σ).
Here is my complete R code which does the same and plots the regression line:
if(exists("dev")) {dev.off()} # Close the plot

rm(list=c("f", "X", "Y", "m", "alpha", "theta0", "theta1", "i")) # clear variables

f<-read.csv("slr12.csv") # read in source data (data from here: http://goo.gl/fuOV8m)

mu<-mean(f$X) # mean

sig<-sd(f$X) # standard deviation

X<-(f$X-mu)/sig # feature scaled

Y<-f$Y # No scaling of target
m<-length(X)

alpha<-0.05

theta0<-0.5

theta1<-0.5

for(i in 1:350) {
    theta0<-theta0 - alpha/m*sum(theta0+theta1*X-Y)
    theta1<-theta1 - alpha/m*sum((theta0+theta1*X-Y)*X)
    print(c(theta0, theta1))
}
plot(f$X,f$Y) # Plot original data
theta0p<-(theta0-theta1*mu/sig) # "Unscale" the intercept
theta1p<-theta1/sig # "Unscale" the slope
abline(theta0p, theta1p, col="green") # Plot regression line

You can test that theta0p and theta1p are correct above by running lm(f$Y~f$X) to use the inbuilt linear regression function. The values are the same!
> print(c(theta0p, theta1p))
[1] 867.6042128   0.3731579
> lm(f$Y~f$X)

Call:
lm(formula = f$Y ~ f$X)

Coefficients:
(Intercept)          f$X  
   867.6042       0.3732  


