How much of a difference does it actually make? Does it account for the trouncing we got in the Ashes?
How much of a difference? In a closely fought contest this sort of tampering can significantly alter a match outcome.
The advantage shifts between bowlers and batters through several phases of the ball's lifetime. With swing bowlers that can also employ reverse swing the batting team has a window of opportunity for more easily accumulating runs when the ball is in a condition that is less conducive to swing. This is between the new ball, that has a clean, pronounced seam that can be used to steer the ball* (shining one side assists and prolongs this phenomenon while the seam ages and becomes less effective both as an aerodynamic device and pronounced high point that provides deviation off the pitch) and the old ball where the significant difference in the two hemispheres induces a different aerodynamic effect that acts in the opposite direction to the earlier swing.^
If a bowling side can artificially induce reverse swing earlier then they limit the amount of time that the batters have where the ball is easier to hit and score runs from. The reverse swinging ball is also more difficult to play than the conventionally swinging ball and if the bowler is particularly skilled, more likely to take wickets or even approach being unplayable.
Minor alterations to the ball can switch it on when it comes to reverse swing so intentionally roughing up the ball is an effective way of shifting the balance of the game in the bowlers' favour.
During the Ashes England weren't beaten by Australia's ability to swing the old ball. If there was any tampering it would have had a minimal effect on the outcome. In other contests though it could make a profound difference.
* the raised seam, if orientated correctly, induces lower pressure turbulence on one side of the ball, this turbulent air detaches readily from the ball and creates an even lower pressure area. The laminar flow on the other side is higher pressure air that more readily 'wraps' around the ball and detaches in the same low pressure area on the far side of the ball and thus the ball is pushed into the lower pressure area.
^the hard, polished surface with numerous irregularities from impacts, plus a widening quarter seam disrupts airflow more than the soft, seude like 'natural' side of the old ball that is thought to carry an insulating micro- boundary layer of turbulence that allows that side to preserve laminar flow for longer.