Author Topic: Linux files system ext2 vs ext3  (Read 1954 times)

woollypigs

  • Mr Peli
    • woollypigs
Linux files system ext2 vs ext3
« on: 05 May, 2008, 11:56:49 am »
I was sorting out my hard drives yesterday and noticed when copying large amount of files from one drive to another the speed was dropping down past snail speed. Both drives are ext3, one is 80Gb the other is 100Gb.

So I was thinking that I better run a "scan disk" (fsck) on the drive, after a bit of poking around I found the e2fsck command. But first I had to unmount the drive and it came up and asked me if i wanted to empty the deleted folder first, clicked yes and saw that there was around 48000 files there (yes I have moved and deleted a lot) but after 1 minute of deleting it was saying that it was deleting 62540'ish file out of 48000'ish files ?!?!?!

I did the e2dsck on the drive and it found a lot of errors, and after a re-mount the copying of times are still down at stupid slow speed, 500ish KB/sec normally around the 1-2Mbit/sec/

All this have made me wonder what is the different between ext2 and ext3 I know the ext3 is a journal drive but what does that mean, anything that is better to do on ext3 that is not good to do on ext2 ? I have tried to look it up on the www but it is over my head can some one please explain.

Current mood: AARRRGGGGHHHHH !!! #bollockstobrexit

pdm

  • Sheffield hills? Nah... Just potholes.
Re: Linux files system ext2 vs ext3
« Reply #1 on: 05 May, 2008, 01:17:42 pm »
A Journaling file system is simply a file system that uses a metadata file to record changes until they are finally committed.

For example.
Assume the file system is in a stable and completely committed state. If you switch it off then and switch it on, then it will still be 100% OK.
Assume you now initiate a file transaction. This entail a series of steps: data is written to a part of the disk, the bits of the disk written to are "remembered" in memory and then written to the filesystem index areas, timestamps are updated, etc. This transaction involves writing to several different areas of the physical disk in sequence. If the process is interrupted half way through, the file system integrity has suffered and is "corrupt". The file check program has the task of rooting out these corruptions and reverting the file system to complete integrity. In the process, however, some unwritten stuff will, of course, be lost and some stuff that has been written will be left hanging in "orphan" files. With a journaled file system, the information pertaining to this series of steps is first written to a metafile or journal file on the disk. During the next few moments, the instructions in this journal are then executed as before but a track of what has been done is recorded as "done" in the journal. If the process is interrupted at any point, all that is needed is for the system to carry on where it left off and so end up with a cosher file system with no data loss. The only time that data can be (theoretically) lost is if the insult occurs during journal writing. However, writing the journal is a much faster and simpler process that doing the full file system write and thus data loss is much less likely. Also, if the journal has not been written, then the file system is by definition still intact and cannot be corrupt. This makes recovery from power outages blindingly fast and much more efficient since it is merely a process of recovering and completing the journal instructions and then purging the journal.

ext2 is a non-journalled file system.
ext3 is ext2 with an added journal file system.
When a system is started up, all file systems are checked. with ext3, the journal is looked at and recovered. However, this is only done a set number of times or before a certain interval. Thereafter, a full belt and braces scan of the file system is performed to catch any possible errors that may have escaped the file system.
Most "modern" file systems are journaling systems.
The very latest, ZFS from Sun, is another step beyond that with automatic though level integrity checking, dynamic create and destroy and hardware replacement facilities. It also has built in striping, mirroring and raid facilities, built in snapshot and rollback facilities, data duplication to any level required and much more. It is very impressive but requires about 1G memory and a decent processor just to run the file system!

sas

  • Penguin power
    • My Flickr Photos
Re: Linux files system ext2 vs ext3
« Reply #2 on: 05 May, 2008, 08:18:14 pm »
It's worth emphasizing that journaling is only designed to prevent file system corruption, not corrupt files. E.g. If the computer crashes in the middle of writing a 1GB data file, you'll loose that file (or maybe get half of it), but the rest of the filesystem should be fine.
I am nothing and should be everything

bobajobrob

Re: Linux files system ext2 vs ext3
« Reply #3 on: 06 May, 2008, 04:30:54 pm »
One thing to bear in mind with ext3 is that resize2fs doesn't support it, or didn't last time I looked.

vorsprung

  • Opposites Attract
    • Audaxing
Re: Linux files system ext2 vs ext3
« Reply #4 on: 06 May, 2008, 04:42:24 pm »
You might like to use a different method for copying the files

Maybe they are going slow for some reason other than the file system

Re: Linux files system ext2 vs ext3
« Reply #5 on: 06 May, 2008, 04:47:30 pm »
How are each of the drives connected?

Internal: IDE / SATA / SCSI?
External: Firewire / USB ?

A long time ago I saw some shocking performance copying between two USB 2.0 external drives under Linux. I reconnected one through the firewire connector and the copying speed was back to normal.

Copying method was piped tar. i.e.:

Code: [Select]
(cd /mnt/drive1; tar -cvf - dir_list ) | (cd /mnt/drive2; tar -xf - )

P.S. And I'll own up to doing the amazingly stupid thing of including the 'z' option on both tar commands the first time I did it. Derrrrrrr.
"Yes please" said Squirrel "biscuits are our favourite things."

woollypigs

  • Mr Peli
    • woollypigs
Re: Linux files system ext2 vs ext3
« Reply #6 on: 06 May, 2008, 04:55:54 pm »
All the drives are the old IDE connection, managed to copy everything off. And have re-formated the drive to ext3, so lets see how it take it  when I copy 43Gb files to it tonight.
Current mood: AARRRGGGGHHHHH !!! #bollockstobrexit