100%He's always struck me as an introvert, which often times gets taken as arrogance by people with a lack of maturity.
As you say, he's eccentric, and in a country like this (particularly playing a sport like footy) you're going to be branded as a weirdo through no fault of your own.
Some just can't wrap their heads around those who skew from the typical 'blokey' culture of footy, which seems to be Fyfe's way from what I've seen of him.
The Bob Murphy interview with him is a classic example. He appears to be a really deep thinker - at least by sportsmen standards - and that can come off as pretentious to those who haven't been around people like that.