Alright, I'll put on my Reviewer Number 3 hat and say that I learned some neat things from your work, including that the National Swine Improvement Federation. I'll try and do a halfway decent job here.

> I would love if an expert could expand on the relationship between mixed effects and Empirical Bayes.

A real expert? Here you go:

http://statweb.stanford.edu/~ckirby/brad/LSI/monograph_CUP.p...

Read it, all of it, but particularly chapter 1, section 2.5, and chapters 8, 10, and 11. Why does testing, effect size estimation, and high-dimensional analysis have anything to do with anything? Because...

1) independence is largely a myth 2) you are likely to have multiple ratings per reviewer on your site, whether your generating distribution is nearly-continuous (0-10, mean-centered) or discrete (0/1, A/B/C). If you discard this, you are throwing away an enormous amount of information, and failing utterly to understand why a person would estimate not just the variance but the covariance even for a univariate response.

The second point is the one that matters.

Also, "empirical Bayes" is in modern parlance equivalent to "Bayes". What's the alternative? "Conjectural Bayes"? (Maybe I should quit while I'm ahead, pure frequentists may be lurking somewhere)

> I welcome your feedback though on how I could make this point clearer.

For starters, edit. Your post is too damned long.

Think about where you are getting diminishing returns and why. Is there ever a realistic situation where your ratings site would not keep track of who submitted the rating? (It's certainly not going to be an unbiased sample, if so; the ballot box will get stuffed) So if you have to keep track of who's voting, you automatically have information to decompose the covariance matrix, and everything else logically follows.

A univariate response with a multivariate predictor (say, rating ~ movie*rater) can have multiple sources of variance, and estimating these from small samples is hard. When you use a James-Stein estimator, you trade variance for bias. You're shrinking towards movie-specific variance estimates, but you almost certainly have enough information to shrink towards movie-centric and rater-centric estimates of fixed and random effects, tempered by the number of ratings per movie and the number of ratings per rater. (Obviously you should not have more than one rating per movie per rater, else your sample cannot be unbiased).

I think you will return to this and write a much crisper, more concise, and more useful summary once this sinks in. I could be wrong. But you'll have learned something deeply useful even if I am. I do not think you can lose by it.