What are some examples in Machine Learning where overfitting is necessary or inevitable?

Google just released the landmark dataset:

https://research.googleblog.com/...

This is a case where instance-level recognition is important.

Overfitting can be loosely seen as instance recognition because you would like to recognize a specific instance of a landmark, in this case.

In other words, instance recognition and overfitting are very similar.

If, for example, you decide to use a convolutional neural network (CNN) for instance-level recognition then you would be okay if the CNN overfit the dataset in someway.

Overfitting is like memorization, so in landmark recognition you would like for your model to memorize the landmarks, not completely because you want the model to tolerate certain variations, like lens distortions.

The Eiffel Tower for example appears the same from the same angle and varies slightly as you change view angle.

So you can build a model that memorizes different view points of the Eiffel Tower. That means deliberately overfitting the model to different view points of the landmarks so that the model can be good at telling you whether the scene is that of the Grand Canyon, the Egyptian Pyramid or non, for example.

On the other hand, category recognition is where you would love the system to be agnostic to a very wide variety of appearances and thus overfitting in category recognition is bad.

In instance-level recognition like landmark recognition you only want tolerance to view point, lighting changes (day and night), weather (winter snowy to warm) as well as occlusion.

Thus overfitting can be important in such a case but should still be controlled depending on the task.

Hope this helps.

Answered by Chomba Bupe on Quora.com