2
$\begingroup$

This question is essentially same as this one. The question is: How do you calculate conditional probability of a node in Bayesian network when it has a continuous node as a parent? However, I cannot understand its answer given by @steffen. My problems with the answer are as follows:

(I use the exact same variables as used in the previous question. cost and harvest are both continuous variables).

What is the justification in mapping the current value of harvest to a parameter describing the distribution of cost? For example,

$p(Cost=c|Harvest=h)=N(\alpha * h + \beta,\sigma_2^2)(c)$

for some $\alpha$ and $\beta$ and fixed $\sigma^2$.

I carried out an experiment to check how this works. I modeled harvest as $N(\mu=10,\sigma^2=1)$. I modeled cost as $N(\mu=2,\sigma^2=0.5)$. Now, I want $P(cost=10 | harvest=10)$ which should be close to $0$. Now if we take some value of $\alpha$ and $\beta$, say $1$ and $0$ (and $\sigma^2=1$), we get a high probability according to the above equation.

So it is clear that $\alpha, \beta$ and $\sigma^2$ play a major role. Now do I do some sort of fitting to determine these values? If yes, exactly how? Let's say I have 1000 values for cost and harvest, and I want to calculate $\alpha, \beta, \sigma^2$ such that it closely approximates the true $P(cost | harvest)$. (Let's assume that I did a very fine sampling of the space of cost and harvest to get true $P(cost | harvest)$ as a discrete conditional probability distribution).

I hope my understanding is right. If not, please correct it.

$\endgroup$
2
$\begingroup$

In short, there is no real justification for selecting these distributions and the linear relationship. It is an assumption made for simplification. In general, any model f(h) which minimizes the error $\sum_ {(h,c)}|f(h)-c|$ can be used, given $(h,c)$ as observed data points. It is "just" a regression problem, an area with its own multitude of algorithms. Even the error function used can be discussed, whatever is the best fit to the practical problem at hand.

Let's assume the assumptions are true and that both harvest and cost can be modelled as normal distribution with the mentioned relationship.

  • The parameters for the harvest distribution can be estimated using the standard formulas for mean and standard deviation for a sample ($\mu_{harvest}=\frac{1}{n}\sum h$ etc.)
  • Now: $\alpha$ and $\beta$ are just the parameters of a textbook linear regression. Given the linear regression line (model), $\mu_{cost|h}=\alpha*h+\beta$ and $\sigma^2_{cost}=\frac{\sum_{(h,c)}(c-(\alpha*h+\beta))^2}{n}$ (see here). But note that more sophisticated linear regression algorithms are available which solve the same problem and might be better fit.

Here is example code using R

# In this example
true_alpha <- 2
true_beta <- 0.5

true_mu_harvest <- 10
true_sigma_harvest <- 2

true_sigma_cost <- 0.1

# create some example data
set.seed(42)
h <- rnorm(1000,mean=true_mu_harvest,sd=true_sigma_harvest)
c <- as.numeric(lapply(h,function(x_h){rnorm(1,mean=true_alpha*x_h+true_beta,sd=true_sigma_cost)}))

dat <- data.frame("harvest"=h,"cost"=c)

# now this example data is all that we have when we need to learn the  parameters
print(paste("mu_harvest=",mean(h)))
print(paste("sigma_harvest=",sqrt(var(h))))

# learn a simple regression model
fit <- lm(cost~harvest,data=dat)
# calculate estimated alphas and betas, compare to true values above
print(paste("alpha=",alpha <- coefficients(fit)[2]))
print(paste("beta=",beta <- coefficients(fit)[1]))

# the estimated sigma of cost|h, compare to true_sigma_cost
print(paste("sigma_cost",sigma_cost<- sum(residuals(fit)^2)/(1000-2)))

# now, for every h, we can calculate the mu of the distribution of cost
mu_c <- as.numeric(lapply(h,function(x_h){alpha*x_h+beta}))
# or simpler in the case of the learning data
mu_c2 <- fitted(fit)
$\endgroup$
  • $\begingroup$ To get $\alpha$ and $\beta$ using linear regression, we should know values of $P(cost=c | harvest=h)$ (or $\mu_{cost | h}$ in above example), but it is unknown. Therefore, I still have a doubt about how to do regression. If we do some very fine sampling and get $P(c | h)$ as if c,h are discrete, then I can get $\alpha$ and $\beta$. But I am not sure if I would be doing it right? As a side note, I think you meant to write $\sigma^2_{cost}=\frac{\sum_{(h,c)}(c-\alpha*h-\beta)^2}{n}$. Notice two minus signs. $\endgroup$ – Parag S. Chandakkar Jan 29 '15 at 22:17
  • $\begingroup$ Also, can't we use something like this - conditional distribution of Y given X=x. $\endgroup$ – Parag S. Chandakkar Jan 29 '15 at 22:18
  • $\begingroup$ @ParagS.Chandakkar As soon as you have some tuples (h,c) as learning data, you can learn a regression model. You do not need P(c|h). $\endgroup$ – steffen Jan 30 '15 at 8:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.