The expectation that object weight increases with size guides the control of manipulatory actions [1-6] and also influences weight perception. Thus, the size-weight illusion, whereby people perceive the smaller of two equally weighted objects to be heavier, is thought to arise because weight is judged relative to expected weight that, for a given family of objects, increases with size [2, 7]. Here, we show that the fundamental expectation that weight increases with size can be altered by experience and neither is hard-wired nor becomes crystallized during development. We demonstrate that multiday practice in lifting a set of blocks whose color and texture are the same and whose weights vary inversely with volume gradually attenuates and ultimately inverts the size-weight illusion tested with similar blocks. We also show that in contrast to this gradual change in the size-weight illusion, the sensorimotor system rapidly learns to predict the inverted object weights, as revealed by lift forces. Thus, our results indicate that distinct adaptive size-weight maps, or priors, underlie weight predictions made in lifting objects and in judging their weights. We suggest that size-weight priors that influence weight perception change slowly because they are based on entire families of objects. Size-weight priors supporting action are more flexible, and adapt more rapidly, because they are tuned to specific objects and their current state.