SIGforum.com    Main Page  Hop To Forum Categories  The Lounge    Question about HDD Formatting Formats...
Go
New
Find
Notify
Tools
Reply
  
Question about HDD Formatting Formats... Login/Join 
Member
Picture of erj_pilot
posted
Could use some guidance from the Borg Collective on formatting a RAID 0 array and which formatting option will compress data the most, if there is even any difference.

Backstory...I have an ASUSTOR NAS containing two (2) 20TB HDD arranged in RAID 1 (mirrored) for redundancy. In addition to the NAS, I have two (2) RAID controller boxes each containing five (5) 1TB HDD's; both boxes are configured in RAID 0, giving me spanned data storage of 5TB each.

I currently have well less than 5TB of data on the primary NAS drive**, so periodically, I will backup that drive to one of the 5TB RAID boxes, alternating from box to box every time I perform an additional backup. Whenever I perform this backup process, I format the spanned drives and then launch the backup. This is all done through ASUSTOR'S utilities.

So here comes the question...whenever I perform an update on the RAID box, I have several options to choose from for formatting:
    EXT4
    FAT16/FAT32
    exFAT
    EXT3
    NTFS
    HFS+

Up to this point, I've always used NTFS, but was just wondering if any of the other formats would compress data any better and conserve disk space across the RAID. Would you recommend anything other than NTFS??

Thanks!!

** Used space: 4,224,766,836,736 bytes / 3.84TB



"If you’re a leader, you lead the way. Not just on the easy ones; you take the tough ones too…” – MAJ Richard D. Winters (1918-2011), E Company, 2nd Battalion, 506th Parachute Infantry Regiment, 101st Airborne

"Woe to those who call evil good, and good evil... Therefore, as tongues of fire lick up straw and as dry grass sinks down in the flames, so their roots will decay and their flowers blow away like dust; for they have rejected the law of the Lord Almighty and spurned the word of the Holy One of Israel." - Isaiah 5:20,24
 
Posts: 11066 | Location: NW Houston | Registered: April 04, 2012Reply With QuoteReport This Post
Shall Not Be Infringed
Picture of nhracecraft
posted Hide Post
Just Windows devices to access your NAS, or do you have Mac OS devices that will need access as well?


____________________________________________________________

If Some is Good, and More is Better.....then Too Much, is Just Enough !!
Trump 2024....Make America Great Again!
"May Almighty God bless the United States of America" - parabellum 7/26/20
Live Free or Die!
 
Posts: 9693 | Location: New Hampshire | Registered: October 29, 2011Reply With QuoteReport This Post
Member
Picture of creslin
posted Hide Post
EXT3 & EXT4 are both used primarily for UNIX OS's.
NTFS is for Windows
FAT16/32 is crazy old and nobody uses it anymore except for really small partitions/drives.
HFS+ is also pretty old - it's a legacy file system used by MacOS back in the day.

Honestly what clients you use is a fairly irrelevant question.
If the drives were directly connected to the OS - then sure - that matters.
But since it's all going over the network it'll be using (likely) SMB/CIFS or (possibly) NFS.





This is where my signature goes.
 
Posts: 1584 | Location: Kernersville, NC | Registered: June 04, 2015Reply With QuoteReport This Post
Member
Picture of erj_pilot
posted Hide Post
Thanks, y'all.

The files are stored on the NAS, which has its own inherent Operating System. The RAID drives are connected to the NAS via USB 3.0 and the NAS interface I'm running is on my Windows 10 system, so that's why I was choosing NTFS. From the sounds of it, seems like that's the appropriate format to use for my purposes.

Appreciate the input and assistance! [like button]



"If you’re a leader, you lead the way. Not just on the easy ones; you take the tough ones too…” – MAJ Richard D. Winters (1918-2011), E Company, 2nd Battalion, 506th Parachute Infantry Regiment, 101st Airborne

"Woe to those who call evil good, and good evil... Therefore, as tongues of fire lick up straw and as dry grass sinks down in the flames, so their roots will decay and their flowers blow away like dust; for they have rejected the law of the Lord Almighty and spurned the word of the Holy One of Israel." - Isaiah 5:20,24
 
Posts: 11066 | Location: NW Houston | Registered: April 04, 2012Reply With QuoteReport This Post
Member
Picture of sigcrazy7
posted Hide Post
Does your NAS not support ZFS or BTRFS? BTRFS is light years ahead of anything on your list. Its ability to self-heal along with the efficiency of snapshot replication and data de-duplication makes the benefits enormous. Using btrfs with a Raid 5,6, or 10 makes the benefits of btrfs through the roof.

Of the filesystems on your list, I'd go with EXT4.

ETA: I just read your last post, and I see that you're using external drives, so that means btrfs is out. I still recommend EXT4. Why use a windows filesystem when the drives are connected to a Unix OS?



Demand not that events should happen as you wish; but wish them to happen as they do happen, and you will go on well. -Epictetus
 
Posts: 8292 | Location: Utah | Registered: December 18, 2008Reply With QuoteReport This Post
Member
Picture of erj_pilot
posted Hide Post
BTRFS appears to be available on select ASUSTOR NAS models...just not on my particular model. From what I'm reading, ASUSTOR's ADM versions 4.0 and higher have been upgraded to Linux kernel 5.4, so yes...smoother operation on the NAS itself. Discussed toward the bottom of the page here:

https://www.asustor.com/adm/adm4_0

I'm not tech savvy enough to know anything about the SYBA RAID enclosures. They're connected to the ASUSTOR via USB 3.0. I assume they don't run an on-board OS and the enclosure is controlled by firmware programming on a chip on the enclosure's board and the DIP switches on the back panel. Linky to SYBA enclosure:

SYBA RAID Enclosure

I'm going to go out on a limb, in my limited knowledge, and say it doesn't matter what formatting option(s) is used to format the drives housed in the SYBA enclosures; 5TB is 5TB...don't think any one format will compress data better than the other.

Thanks again for all the input and knowledge!!



"If you’re a leader, you lead the way. Not just on the easy ones; you take the tough ones too…” – MAJ Richard D. Winters (1918-2011), E Company, 2nd Battalion, 506th Parachute Infantry Regiment, 101st Airborne

"Woe to those who call evil good, and good evil... Therefore, as tongues of fire lick up straw and as dry grass sinks down in the flames, so their roots will decay and their flowers blow away like dust; for they have rejected the law of the Lord Almighty and spurned the word of the Holy One of Israel." - Isaiah 5:20,24
 
Posts: 11066 | Location: NW Houston | Registered: April 04, 2012Reply With QuoteReport This Post
Optimistic Cynic
Picture of architect
posted Hide Post
As noted in previous posts, the file system on the disks themselves do not inherently provide data compression, this would be done by the OS or the RAID controller before the data gets written to disk, and would shown as a separately configured item in the RAID setup screen. There are two file system formats that are at issue here, what is actually on the disks themselves, and how the array is presented to the system OS. It is not at all unlikely that the former is proprietary to the RAID vendor, or a proprietary modification to open source file system software, perhaps including a compression pipeline (which you would know because they would advertise this as a value-added feature), and/or some sort of parity/data integrity provision (other than that implied by the RAID level selected). The options presented in the OP imply that these are the file system types available to be presented to the OS, and if you are running a newer version of Windows, NTFS makes the most sense for this.

I will also add, like with many aspects of life, that software free lunches are hard to come by. The problem with compressing data that goes on a storage media is that it greatly magnifies the consequences of data corruption. Configuring an array as a RAID 0 stripe is somewhat risky in and of itself, writing a compressed data and metadata stream to that array means that a single disk error, one flipped bit, can render the entire data store unreadable. It is possible to get most of the benefits of compression by using a file-by-file strategy (think ZIP files), the consequence of corruption being that you only lose one file.
 
Posts: 6975 | Location: NoVA | Registered: July 22, 2009Reply With QuoteReport This Post
Thank you
Very little
Picture of HRK
posted Hide Post
Interesting topic.

I have an older NT server with a DPT SCSI RAID controller, 3 drives, is there an enclosure that I could move these to without having to reformat the drives.

Don't/can't do that or I'd lose the data, the NT partition isn't operating, but the RAID is still there, just can't boot to windows.

Yes, NT SCSI, RAID, kept the drives and server thinking I'd like to see if I can bring it back to life, recover the data for historical reasons.

Maybe the way to go is to pull the old NT drive put it in an enclosure, try to build a new drive and pull the setups and files off the old drive if it will come up as a secondary...
 
Posts: 24720 | Location: Gunshine State | Registered: November 07, 2008Reply With QuoteReport This Post
Member
Picture of erj_pilot
posted Hide Post
architect...




"If you’re a leader, you lead the way. Not just on the easy ones; you take the tough ones too…” – MAJ Richard D. Winters (1918-2011), E Company, 2nd Battalion, 506th Parachute Infantry Regiment, 101st Airborne

"Woe to those who call evil good, and good evil... Therefore, as tongues of fire lick up straw and as dry grass sinks down in the flames, so their roots will decay and their flowers blow away like dust; for they have rejected the law of the Lord Almighty and spurned the word of the Holy One of Israel." - Isaiah 5:20,24
 
Posts: 11066 | Location: NW Houston | Registered: April 04, 2012Reply With QuoteReport This Post
  Powered by Social Strata  
 

SIGforum.com    Main Page  Hop To Forum Categories  The Lounge    Question about HDD Formatting Formats...

© SIGforum 2024