I think it's more how they're built, not the size.
I need something patterned on a dress shirt that can be tucked in at the waist and tapers to the wrist. I guess you could say there's just an overabundance of fabric in BDUs to make for a looser fit than I want.
The BDU pants on the other hand fit relatively well with a decent belt, the loose fit there is actually helful when trying to run/jump/etc.
My BDUs DO fit like that.
I wear my BDU top tucked in at games, and whenever I'm wearing any gear.
The shoulder seam should be right at the lower curve of your shoulder. Any lower, and that shirt is too big.
I'm telling you, let me know your measurements, and I'll help you out.
Here's how my sh*t fits (I'm very skinny, too!)
That top is too big (Large Long). I now wear Medium X Long. It's narrower in the body, sleeves, and waist and longer.
Trust me bro, I wear either a BDU, ACU, or ABDU to work
every day, and have done so for the past year and a half.