I guess the organisers of the Tour of Switzerland and TdF can choose who they want to invite or ban, it's their events, and they probably see such a high profile sportsman being caught using illegal drugs as being a bad image for the sport in general, and their events in particular. Any sort of drug issues are going to be seen as much higher profile these days, even when they are not directly linked to performance enhancement.
And yes, he is an idiot, but that's hardly unique amongst high profile personalities.