Go to Control Panel - > Internet Options. Then there's a button to delete Temporary Internet Files. That should get rid of most of it.
Or if you want to delete them manually, go to Folder Options, then select the options for "show hidden files" and "show protected operating system files" or similar. Then you should be able to see that folder.