1
$\begingroup$

And what is the opposite? "Global" methods? What the differences are? I have found reference to "local" methods in this answer about KNN: https://stats.stackexchange.com/a/104261/107213

$\endgroup$
4
  • $\begingroup$ For your information, this is not very popular distinction. Honestly, I don't recall seeing it anywhere. $\endgroup$
    – Tim
    Commented Dec 12, 2018 at 20:04
  • $\begingroup$ It might be more productive to think of $k$-NN as a method that has no learned parameters (it just finds nearby points), while something like a logistic regression learns weights (coefficients) for features. $\endgroup$
    – Sycorax
    Commented Dec 12, 2018 at 21:03
  • $\begingroup$ @Sycorax, you can learn optimal k in KNN, it's like an effective window size in kernel methods, where it turns out to be the most important characteristic, maybe more important than the type of kernel itself $\endgroup$
    – Aksakal
    Commented Dec 12, 2018 at 21:08
  • $\begingroup$ @Aksakal You can optimize $k$, most commonly with cross-validation, but it's not amenable to estimation from the training data in the same manner that the coefficients of a logistic regression: $k$ is a hyperparameter, not a parameter. $\endgroup$
    – Sycorax
    Commented Dec 12, 2018 at 21:11

1 Answer 1

2
$\begingroup$

I don't think there's a precise definition or an exhaustive list of local methods in data science. I bet that the term local (nonlocal is the opposite) came either from physics or mathematics into what you call a data science.

In physics local means that an object is influence only by its immediate surroundings. In the old days locality was restricted to a literal collision or some sort of hard connection between objects through some kind of a medium like ether. This was before the notion of forces such as gravity, and their modern understanding where there's no need for any medium at all, the fields go through vacuum too.

So, today the meaning of locality in physics could be linked to the speed of information transfer. For instance, Earth and Moon interact still locally despite being somewhat far away. Here we mean that their movements are described by gravity force/field, and the disturbances would be passed along by speed of light, and, consequently proximity matters. Ceteris paribus, distant objects impact us less than objects in close proximity. This is very convenient, because although every object interacts with every other object in the universe through gravity force, you need to account for only a handful of bodies to calculate the trajectory of the satellites.

Hence, by analogy the local method in data science would be those where you deal with immediate surroundings, whatever it means in the context. For instance, kernel methods would be one example. Here, although all observations go in, they are weighted by the distance to the given observation: farther the distance, the smaller is the weight. This way, we make the interactions local. Nearest neighbors methods are obviously quite local by construction.

There are nonlocal phenomena in physics, where the disturbance would be transferred faster than the speed of light. For instance, entanglement in quantum mechanics can be interpreted as such.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.