Curiously I asked this question recently to a small group containing yacfers and both Tim Pike and I had In Which We Serve at the top or near the top of our relative lists.
Here's a clue, old B&W war movies (especially those made during WWII), see them in the cinema if you can as they pack a hell of an emotional punch that TV reduces. Last year I bought a stupidly big TV and all sorts of old films come alive on it on the bigger screen in ways I didn't expect, especially old WWII films (and most especially the better British WWII films. Both In Which We Serve and The Cruel Sea look completely different with the texture and movement that a bigger screen gives to the water).
I would argue that Gaudalcanal Diary is a more powerful film than Saving Private Ryan because it has more emotional realism. When it was made, the issue was still possibly in doubt, therefore while it has a propaganda angle and the various clumsy stereotypes, it's still implying that the war can be lost (because they want you to buy war bonds).
Too far after WWII and many films lose that perspective.
I just rewatched 'A Wing and a Prayer' this evening which was a WWII flag-waver about the Battle of Midway "Where is our Navy, Why doesn't it Fight". It was made during the war so it lacks certain information but cannot lie about the immediate outcome of war on people.
Cross of Iron for me is probably one of the best war films ever made. Interestingly my father said it was the only war film that he felt ever gave a decent impression of what it was like to be wounded in combat.
I think it's a masterpiece. Land and Freedom is also arguably another masterpiece which is high on my top X list of war films.
I freely admit to being a complete obsessive on this subject. On my DVD shelves, war films are broken down by historical period and then further subdivided by theatre of war (mainly for WWII films). I suspect I have more to offer on this subject.