Endergonic and Exergonic Reactions?

I read that with endergonic reactions (reactions in which energy is taken in) molecules are synthesized (anabolism), and with exergonic reactions (reactions in which energy is released) molecules are broken down (catabolism). This idea seems somewhat counter-intuitive to me. Shouldn't the opposite be true? Wouldn't energy be needed in order to break something down rather than being released?