Quantcast
Channel: Trending Questions - Cross Validated Meta
Viewing all articles
Browse latest Browse all 364

If someone asks "What is the intuition behind the idea that for linear regression, ...", does the answer have to be only about the linear regression?

$
0
0

Since this question asks for an intuition behind the idea:

What is the intuition behind the idea that for linear regression, the number of observations should exceed the number of parameters?

I thought I might also add something that is not about a linear regression but might lead to the same intuition. Therefore, I wrote about the problem of a logistic regression instead, and my answer got deleted. I see the point that this was too far-fetched, but my idea when I heard that observations should exceed the number of parameters was that Why is logistic regression particularly prone to overfitting in high dimensions? also showed this intuition to me.

I doubt that it was even right to see both questions as dealing with the same intuition, so that this question here is not about reopening the deleted answer (unless someone wants to). Yet, the answer was not deleted for being wrong, as far as I can see, but just since it was not about linear regression, at least if you check the only upvoted remark for this (which was again upvoted right before the answer was deleted).

If I show that in other models, you also have to avoid that the number of parameters (or dimensions) are too many, is this clearly off-topic for a question that asks about "the intuition of the idea that for linear regression, ..."? Mind that the question also becomes clearer in the body:

If a population model has k independent variables and 1 intercept, whyare k+1 observations required to perform OLS estimates?

What is the intuition behind this?

But even then, the other link about logistic regression might lead to this intuition, or take anything else that is not about linear regression and still helps understanding this.

Is the wording of the question right? English is not my mother tongue, and I understood it as a much broader question.


Viewing all articles
Browse latest Browse all 364

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>