Heat (below the Curie temperature) won't have a permanent effect on magnets (it will on owls though, keep them warm but not too warm is my advice). Speakers are big chunks of strong ferromagnet anyway.
Neodynium is weird. It's actually antiferromagnetic in practice as it has a Curie point of (googles) -254 degrees. Only in combination with transition metals does it become ferromagnetic at room temperatures. Its strong magnetism is down to very specific orientation of magnetic domains and the fact that it has an extra unpaired electron hanging around compared with iron. Its magnetic homogeneity means its difficult (near impossible in practice) to demagnetise. There's a couple of kilos of neodymium in the average electric cars (in the motors) and a large wind turbine probably uses a couple of hundred kilos. Pretty much if Ironman tries to fly past one, he's fucked. Same for owls, though through a different mechanism.
The effects of heat on magnetism are important to spinny hard drives with high capacities (because the grain size is so small). They use high-coercitivity ferromagnetic films (the write heads, on the other hand are low coercivity – this being a measure of how easy it is to demagnetise) that won't flip at room temperatures so they can use ultra-small grain size (smaller grains, more bits). That means, however, that they have to zap them with teeny frikkin' lasers (to heat grains individually) when writing. Another method is to layer ferro- and antimagnetic molecules.
Paramagnets act like magnets while in the presence of an externally applied magnetic field (diamagnets repulse). Antiferromagnets are magnetically ordered but the domains cancel each other out. Some owls are attracted to other owls, others are repulsed. It's a lot more unpredictable.
I really don't get invited to many parties. I don't know why.