Definitely possible.
You'd need a USB audio dongle because:
a) the RPi's audio output isn't very good quality (not that this really matters for headphones)
b) the RPi has no audio input
You'll also need a button to wire up to some of the GPIO pins to act as the record button, an LED to let you know the recording state would be nice to have too.
Battery power is not a problem, just use a USB power pack.
preferably starting some seconds before the record button is pressed and to continue until the button is pressed again.
(My emphasis). I take it that's the reason why you can't just use an existing dictaphone or other digital recorder. Again it's perfectly possible for the Pi to be buffering the last
n seconds of audio just in case the record button is pressed.
Post processing into mp3 is easy.
So, it's possible but I doubt it's commercially available. It would take someone with the right skills (and bits) less than a day to knock up a prototype I suppose.