from your description they probably are bumble bees. It is about the right time of year for them too.
If so they will make nest and there will never be more than a few hundred bees in it; bumble bee colonies are small. Come late summer it will be done and dusted; if you don't want a repeat next year then block off the entrance (in the winter, or once this year's colony is finished). They are not aggressive by nature, so unless you obstruct their flight path or start doing something right next to the nest then they will just leave you alone.
Most bumble bee species (in theory there are about ten different types) in the UK are endangered, so not the best thing if you destroy them. You may not get a precise identification of species (even from an expert) because the morphology of individual bees (i.e. size and markings) can vary with how the conditions in the colony are food-wise. A colony that is doing well will tend to produce more uniform bees IME, whereas one that is struggling (or was struggling earlier in the year) will tend to produce more smaller bees that are not so representative of the species.
Old bees (and 'old' means a few weeks of being active flyers) look different to young bees too; young bees are hairier, where older ones appear to 'go bald' in places where their hair is rubbed off, and the bee usually looks darker-coloured where this has happened. The hair doesn't regrow or anything.
If they are a lot smaller, but not wasps or honey bees, they could be mason bees. Past that there are several hundred other obscure species of other bee, but most of these are 'solitary bees', eg you may see lots in a wall or something, but they each make their own tiny nest rather than share a communal colony per se.
cheers