defining sets from abelian groups

I want to say something along the lines of "You can define abelian groups in the internal logic of Set, but you can't define sets in the internal logic of Ab." Is this intuition supportable? Ideally there would be a precise technical sense in which it is provably the case. Vaughan Pratt