When researchers do studies specifically designed to isolate different factors, it's different. Please stop ignoring this part of my argument!!
Assume the most extreme case of assortative mating possible. Every child inherits two genes that list the exact IQ of its parents.
Do those genes correlate with the IQ of the child? If they do, there's only a few ways for that to happen. The causal factor could be how they're parented, or the environment, or the genes themselves. If you correct for the first two across a statistically large sample, and still see an effect, then it must be the last one.
> not a measure of any single person
I know. I'm saying each data point inside the statistic is arrived at in a specific way. Cause and effect only go via certain paths. It's easy to make mistakes about cause and effect by forgetting about paths, but you can categorize them and only some paths are possible. Any statistically resilient correlation has a cause somewhere.
I'm not trying to stake out a position about whether IQ is in any sense genetically causal or fixed. I'm saying that it's much more complicated than the Wikipedia page on "Heritability of IQ" would suggest. That's the only reason I dipped into this thread. You can believe whatever you want to believe, but this is an actively (indeed, furiously) studied open question, and the answer is definitely not "twin studies from 20 years ago set a heritability number that resolves the question".