What is Occam's razor (principle of parsimony)?

• A. Prefer the simplest solution that adequately explains the data/task when multiple solutions exist.
• B. Always choose the most complex model for accuracy.
• C. Prefer models with the most features.
• D. Discard any model that uses deep learning.

Answer: (A)

Occam's razor is a parsimony guideline: when several explanations or models can explain the data adequately, the simplest one is preferred. This matters because simpler explanations tend to generalize better to new data and are less likely to rely on unnecessary assumptions. In practice, you’re balancing fit with complexity—you want an explanation that captures the essential structure without overloading it with parameters or features. A simpler model that explains the observations well will usually perform better on unseen cases than a more complex one that fits the training data more closely but may fail to generalize.

The other ideas would push toward more complexity regardless of its necessity, which can lead to overfitting and poorer real-world performance. Occam's razor isn't about rejecting powerful methods like deep learning outright; it's about choosing the simplest adequate approach when possible.