If I extend this argument, would removing other value also impact the relationship?
Almost certainly yes. The regression line slope depends on all observations, so removing any point will change the estimate, and therefore also the $p$ value.
(It is possible, but unlikely, that there are data points with no influence on the coefficient estimate. Removing such a point will not change the parameter estimate, but will change the $p$ value slightly, since this depends on the parameter estimate and the number of observations.)
Removing a data point may or may not move your $p$ value above or below the magic threshold of $p < .05$.
which statistical operation would help me conclude if the relationship that I observe here is truly significant or not?
Statistical significance depends on the observed data (and a correctly specified model). There is no "true" significance. Either the null hypothesis holds (e.g., there is no relationship), or it does not (e.g., there is a relationship) - and many statisticians will say that the null hypothesis never holds in the first place. All we can say given data (and a model) is whether the data are consistent with a null hypothesis or not.
Thus, if you have no reason to suspect a data error, you should take your $p$ value as it is. It would be good to be cautious about any conclusions you draw and note explicitly that significance depends on a single data point. Also, note that the difference between the slopes you estimate with and without this one data point will probably in itself not be significant (Gelman & Stern, 2006).
Would boostraping help me ?
Bootstrapping is just another way of looking at given data. It will give slightly different results. You may get $p<.05$ with the full dataset with a bootstrap - or not, and the same if you omit this observation. (And as above, a bootstrapped and a parametric parameter estimate may not be significantly different.) If all you are interested in is significance, then I would not worry overly about differences between a parametric and a bootstrap analysis.
Edge cases like these are one main reason to be careful not to over-interpret statistical significance. If you want more precision in your estimates, there is no way around collecting (a lot) more data.