# Generating synthetic data for robust regression?

I was running some test to see the benefits of using robust regression over ols. For that purposes, I created synthetic data in the following manner, A is the random normal matrix(100k*500), x0 random normal vector(generated using np.random.normal). Then I generate the response variable b as

b=Ax0
noise=np.ramdom.normal(0,1,A.shape[0]);
scaled_noise=0.25*norm(b)/norm(noise)*noise


To add the outliers I choose random 1% entries(i's) of scaled_noise and make them scaled_noise[i]=1000*scaled_noise[i] and call this vector outlier_noise and set bn=b+outlier_noise.

With the data above I run ols and robust regression(using weighted least square algo with huber loss ) with A,bn and compare the two by the relative error $\frac{||b_{pred}-b||}{||b||}$, where $b_{pred}=A\hat{x}$ and $\hat{x}$ is the optimal solution retured by the algorithms. I observe that my relative error in ols is $1.00014$ and and robust regression its $0.99996$, which seems pretty close. So I was wondering if there is a better way of generating synthetic data so that we can make the case for need for robust regression strong? I tried changing my noise to laplace in the above code(noise=np.random.laplace(0,1,A.shape[0])) but not much difference. Any hints and suggestions are greatly appreciated. Thanks a lot.

A more dramatic illustration can be made by adding a cluster of distinct outliers.

Adapting from the scikit-learn documentation, try this:

from sklearn import datasets
import numpy as np

# Generate data with outliers: 1000 samples total, of which 50 are outliers
n_outliers = 50
data_with_outliers_X, data_with_outliers_y, data_with_outliers_coef = datasets.make_regression(n_samples=1000, n_features=1, n_informative=1, noise=10, coef=True, random_state=0)
np.random.seed(0)
data_with_outliers_X[:n_outliers] = 3 + 0.5 * np.random.normal(size=(n_outliers, 1))
data_with_outliers_y[:n_outliers] = -3 + 10 * np.random.normal(size=n_outliers)


Then you can regress the response data_with_outliers_y on covariates data_with_outliers_X.