Yes, clear limitation of an objective rating system is they can't factor in the degree of difficulty in individual contests, match ups etc. Unless they do have a way of doing that, but I doubt it.
We can do that by watching, you are spot on. The trouble is, by watching once, through one set of eyes, we miss so much that they notice. So their deficit here that you are correctly indentifying might have an error of +-10% built into it over a whole season. Our error factor from just watcing once I am willing to bet is closer to the 30-50% range. We don't even record what we see, we just process the part we see and memorise only part of that well enough to recall faithfully post match.
