Well, when it comes down to it, things like the stock, furniture, muzzle accessory, sights, etc. can all be pretty easily changed. Same goes for barrel length, etc.
What it comes down to is the receiver (and by association, probably the barrel and its lining, because you're not likely to see a barrel change). Aside from what's already been said, the Yugo is significantly heavier than the Russian, as the receiver is quite a bit thicker. It's noticeable by a side-by-side comparision, but other than that, I'm not sure how to quantify it.
Also, I've yet to see a Yugo that didn't also have an import roll on the barrel that said "Yugo 7.62x39" right above where the stock and gas block end. I think most have been imported by CAI - all the ones I've seen have said so.