# What is proposal distribution in Importance sampling

I want to learn importance sampling using a simple example. Consider the following example code which implement importance sampling using python for a simple Bayesian network.

I've read that we fix the value of evidences, so here we fix "y" (which is taking umbrella) as True and want to estimate the probability of sunny weather if we observe someone has took umbrella.

I don't get why it uses uniform distribution to generate sunny and rainy samples, I expected to use the prior probabilities to generate them and then adjust the weight based on the probability of "y" given the weather in each sample.

import random

# Define probabilities
p_sunny = 0.7
p_rainy = 0.3
p_y_given_sunny = 0.2
p_y_given_rainy = 0.8

# Generate 10 samples using the proposal distribution
samples = []
for _ in range(10):
# Sample X from the proposal distribution
x_sample = random.choice(['sunny', 'rainy'])

# Calculate weight based on evidence Y = yes
if x_sample == 'sunny':
weight = p_sunny * p_y_given_sunny
else:
weight = p_rainy * p_y_given_rainy

samples.append((x_sample, weight))

# Normalize weights
total_weight = sum(w for _, w in samples)
normalized_samples = [(x, w/total_weight) for x, w in samples]

print("Generated Samples:")
for x, weight in normalized_samples:
print(f"X = {x}, Weight = {weight:.4f}")

# Calculate posterior probabilities
posterior_sunny = sum(weight for x, weight in normalized_samples if x == 'sunny')
posterior_rainy = sum(weight for x, weight in normalized_samples if x == 'rainy')

print("\nPosterior Probabilities:")
print(f"P(X = sunny | Y = yes) = {posterior_sunny:.4f}")
print(f"P(X = rainy | Y = yes) = {posterior_rainy:.4f}")


I rewrote the program by sampling sunny and rainy weather based on the given prior probabilities (0.7 and 0.3) instead of uniform sampling.

Now the weight for each sample is the p_y_given_sunny or p_y_given_rainy depending on the value of weather in each sample. Since we sampled the weather based on the original distribution, we don't need to multiply these probabilities to p_sunny and p_rainy as we did before.

So, in general, we can fix the value of evidence ("y" here) and sample the rest of variables like before using probability values of Bayesian network. Just to adjust each sample we must set the weights to the probability of the evidence under the given sample according to Bayesian network (p_y_given_sunny if weather is sunny and p_y_given_rainy if weather is rainy). The other solution was also correct, just we should take into account the p_sunny and p_rainy in each sample because we sampled them uniformally which differs from original distribution.

import random

# Define probabilities
p_sunny = 0.7
p_rainy = 0.3
p_y_given_sunny = 0.2
p_y_given_rainy = 0.8

# Generate 10 samples using the proposal distribution
samples = []
for _ in range(1000):
# Sample X from the proposal distribution
x_sample = random.choices(['sunny', 'rainy'],weights=[p_sunny, p_rainy])
x_sample = x_sample[0]
# Calculate weight based on evidence Y = yes
if x_sample == 'sunny':
weight = p_y_given_sunny
else:
weight = p_y_given_rainy

samples.append((x_sample, weight))

# Normalize weights
total_weight = sum(w for _, w in samples)
normalized_samples = [(x, w/total_weight) for x, w in samples]

print("Generated Samples:")
for x, weight in normalized_samples:
print(f"X = {x}, Weight = {weight:.4f}")

# Calculate posterior probabilities
posterior_sunny = sum(weight for x, weight in normalized_samples if x == 'sunny')
posterior_rainy = sum(weight for x, weight in normalized_samples if x == 'rainy')

print("\nPosterior Probabilities:")
print(f"P(X = sunny | Y = yes) = {posterior_sunny:.4f}")
print(f"P(X = rainy | Y = yes) = {posterior_rainy:.4f}")

# Compute marginal probability of evidence Y = yes
p_y_yes = p_y_given_sunny * p_sunny + p_y_given_rainy * p_rainy

# Compute posterior probabilities using Bayes' theorem
posterior_sunny = (p_y_given_sunny * p_sunny) / p_y_yes
posterior_rainy = (p_y_given_rainy * p_rainy) / p_y_yes

print("Posterior Probabilities (Exact Calculations):")
print(f"P(X = sunny | Y = yes) = {posterior_sunny:.4f}")
print(f"P(X = rainy | Y = yes) = {posterior_rainy:.4f}")