Yeah that sounds like typical problem. Bearing quality can have some effect as does worn bearings. If the bearing falls out of the axle carrier easily, instead of fitting tightly, it is likely time for new ones. I usually don't worry about it as much on my basher as I do on the racing cars.
would going to an aluminum axle carrier help, I have the same issue with a used roller I am building on, I have some washers ordered to shim between the wheel and wheel nut, so I will see what I need, I remember Brian had written in a recent post about running a Revo with aluminum carriersand he uses those because he had gotten a good deal on them, I have thought of upgrading to the RPM carriers