Yebbut, tea in India is usually served in a way that's quite different from typical British tea. Small cup, very sweet, with cardamom - and other spices, but that's predominant. So it makes sense, in a British context, to call that 'chai'.
As for masala, which I presume is the other word you're referring to, it might translate as 'spice' but in use it's more like 'mixture of spices or herbs'. It doesn't have to be spicy in the chilli, peppery sense. Garam, of course, means hot in both senses. If you consider 'garam chai' as an English phrase with Hindi origins, it kind of makes sense. If you think it's Hindi, it's a bit silly. Pretty much every language borrows words from others and distorts them from their original meanings.