By default? No. That's just really bad and inappropriate wording.There's little to nothing in that that supports the idea that we are female by default. You seem to be relying on the notion that because it's the SRY gene that drives male development, the absence of that gene is the "do nothing" option i.e. female until otherwise driven to male.
Emma Hilton addresses this in the X thread I linked to.
Regardless, sex is determined at conception, and it can be observed within a few weeks, before differentiation, and that's the key point.
What the article states is that we all start off as females, for the first 6-7 weeks of our gestation period. It's only once testosterone starts being produced, around 9 weeks, that the male gonads start to develop.