I have an Olympus EPL-1, and aspire to an EP-5. Being a rangefinder kinda guy, I like the compact style. OK, so a four thirds sensor isn't going to have the same features as full frame, but DSLRs are such heavy beasts, I couldn't imagine carrying one round every day.
I like having the extensive features, and programmable functions, but there's an awful lot of fluff I don't use.
One thing I would say is that I much prefer to use an electronic viewfinder, which spoils the line of the camera, and makes it more bulky, but certainly improves handling.
The GX7 is similar to the EP5, but not so retro stylish. If you like modern-looking cameras, it's got a lot to recommend it. And the integrated viewfinder makes it a really nice option.
The OMDs are nice, if you like the mini-SLR styling. And, given that I use an EVF anyway, there's no reason why I shouldn't have an integrated hump on top. I'm very attracted to the EM10, which is very programmable, and with ergonomic controls instead of having to rely on a crappy little wheel on the back. Although the EM10 is technically below the EM5 in the range, it has the better sensor and engine.
The X1Pro is a fine camera, and the extra size of the APS-C sensor makes a difference. It's renowned for being a tad slow in focus for its class, but the user control is quite extensive. Bigger, of course, than the micro four thirds options, it still isn't too heavy or awkward to handle.
There's a criticism that the Fuji hasn't got a wide enough range of lenses, but that's debateable. The lenses that are available are really excellent, if pricy.
Micro four thirds gives you an ever-increasing range of lenses, as both Olympus and Panasonic extend their offer, while third parties are aiming for a bigger share of the market.
One other thing worth considering: Olympus have in-body stabilisation, which means you can use legacy lenses (m43 is small enough that just about any lenses can be adapted), and have them stabilised, whereas Panasonic lenses have their own stabilisation.
But it's a very personal thing. Get to handle as many of the cameras as you possibly can before making a decision.