It depends.
Are you talking about a laptop with built-in 2-channel speakers?
If so, then as Jaded says, it's a mechanical contact in the 3.5mm Jack socket that opens when you shove in the headphones. Un-plugging the 'phones should allow the contacts to spring closed, and the speakers should workee again. If not, then the contacts need a bit of jiggling / cleaning. Try some plugging / un-plugging cycles.
If it's a desktop machine with a fancier multi-channel sound hardware, it may be more complicated. The headphone jack on the front of the PC may be a seperate output from the sound hardware in addition to the array of jacks on the back, where the speakers plug in. For example, on my system here, it's got 7.1 channel speakers plugged in the back. If I plug headphones in the front, the Realtek HD Audio driver detects this and asks 'Hey, was that headphones you just plugged in?' I say 'yes', and in mutes the rear outputs and down-mixes the 7.1 to 2-channel for the phones.
So in this case, it's not a simple dumb mechanical thing, software is involved.