If the 4000mAh battery is a usb power pack then it is 5V rather than 3.6V. In which case 2500mAh at 5V is very close to 4 x 2500mAh at 1.2V.
The 4 AA pack is probably above 5V when fully charged but quickly drops to about 4.8V and then gradually drops to 4.4V as it discharges. The usb power pack probably maintains a more even 5V.
You'd think that, but it's the mah value of the 3.6v (nominal) lithium cell inside, not the 5v output. It makes for bigger numbers, and we all know more megapixels is good...
The usb pack does have a dc-dc converter in there to increase the output to 5v (or in some cases to drop it from 7.2v...). Which adds some inefficiency, which is why you'll never actually get 3 hours of light for your 3w lamp from a 9wh battery pack, tho you should get >2.5 hours.
How accurate that 5v is depends a lot on the quality of the pack. As well as any intelligence (in the loosest sense of the word) is built in to it. You may find it i anywhere from 4.9-5.1v. You may also find that it will only output 2.5w (500ma), at 5v, unless your device has the right chip to talk to the pack and ask for extra power. Depending on the pack...
In the interests of bigger, more impressive looking numbers, marketers of USB battery packs cite the amp-hour capacity at the battery, not the USB output. On a really bad day, batteries in series get counted twice.
This is unhelpful when you're doing real-world capacity calculations.
Yep. Hence my 3.6v (nominal) rather than 5v.
Most lithium battery packs will quote the wh as well as the mah, of you look closely enough on the label. Largely because planes don't allow above a 100wh of lithium battery in your hand luggage, so printing the wh saves some maths for the security goon.
J