I have an x100s and if I could only own one camera, this would be the one I keep. Here's why: focal lengths don't actually matter that much. People think that they can't shoot a portrait with a wide lens or they can't shoot street with an eighty five. It's rubbish - you work with what you have and it frees you to stop worrying about gear.
I fucking LOVE my fuji because it's a cracking lens in front of a sensor that gives me great quality files, all wrapped up in a body that now I've gaffer taped it up, looks like nothing at all. Nobody looks at it and nobody's intimidated by it. Also, it's small - so small that it's always in my bag, wherever I go. DSLRs, even crop frame ones are fucking huge and you won't carry it with you unless you're actually out to make a photograph. When you're carrying a DSLR, you'll also want to carry the other random shit that comes with one. Extra lenses, maybe a flashgun, a tripod, whatever. Then you're weighed down with CRAP.
Get a Fuji (or something like it) and stop letting your camera get in the way of making photographs.