[Noisebridge-discuss] [RUG] Data integrity, SMART and SSD's

[Ceren Ercen]

Traditionally, it's been thought that SMART is a good predictor of drive 
failure and should be integrated into automated hardware monitoring in 
live environments where data integrity is obviously a concern.

Then this Google study came out, and as far as most reviews of the paper 
go, it seems that "the drive’s self-monitoring data (S.M.A.R.T.) does 
not reliably predict failure and that the drive temperature and usage 
levels are not proportional to failure."

http://www.google.com/url?q=http://docs.google.com/viewer%3Fa%3Dv%26q%3Dcache:q7nqEvSwhZIJ:labs.google.com/papers/disk_failures.pdf%2BSMART%2Bgoogle%2Bhard%2Bdrive%2Bfailure%26hl%3Den%26gl%3Dus%26pid%3Dbl%26srcid%3DADGEEShvzuvEYXdh1yhO0lkXfTbeRY56hp7UC0GJysAZRtxmUu_ftFJlKMx08F_72uDEXlI4tiFJme5OLduNdpGRYeMnrOzabAUUpmf5Z_idY2urJldpF9ONfykBSbHIow18LW4k7EdO%26sig%3DAHIEtbSbzeLsH6IpSBA54BnpOkzpICCWiQ&ei=enskS44BlOKzA_eKvOAO&sa=X&oi=gview&resnum=1&ct=other&ved=0CAoQxQEwAA&usg=AFQjCNG5EwWMT7Iu3g0P-4uRgGMSaGrq4A

http://www.myce.com/news/Hard-Disk-SMART-data-is-ineffective-at-predicting-failure-13049/

And the implication seems to be that there are probably significantly 
better places to spend your effort and money if you're trying to ensure 
data integrity than intensive SMART monitoring. RAID, journalling, zfs, 
etc.

- Ceren's $0.02

John Magolske wrote:
> I'm looking for strategies around maintaining data integrity. The
> primary machine in question where data is gathered, manipulated &
> stored is a laptop with an SSD hard-drive (actually a CF card + CF to
> IDE adapter). This is rsync'd to backups on external USB hard-drives.
> 
> At the last RUG (Rsync Users Group) meeting there was mention of
> the usefulness of SMART (Self-Monitoring, Analysis and Reporting
> Technology) systems, which are built into many (all?) ATA, IDE and
> SCSI-3 hard drives.
> 
>>From SMARTCTL(8):
> 
>     The purpose of SMART is to monitor the reliability of the hard
>     drive and predict drive failures, and to carry out different
>     types of drive self-tests.
> 
> Issuing `smartctl -a` to my Transcend & SanDisk CF cards gets:
> 
>     SMART support is: Ambiguous - ATA IDENTIFY DEVICE words 85-87
>     don't show if SMART is enabled.
> 
>     SMART support is: Unavailable - Packet Interface Devices
>     [this device: Write-once (optical disk)] don't support ATA SMART
> 
> I also understand that SMART doesn't work with USB hard drives. So it
> appears that nowhere would the benefits of SMART be present in this
> particular situation. Which leaves me me wondering...
> 
> How helpful is SMART in maintaining data integrity?
> 
> Is there a way to gain the benefits of SMART in the above scenario?
> I'm thinking maybe by doing the rsync backups to a hard-drive built
> into another computer... Or maybe there's an external hard-drive with
> a little computer built in that allows SMART to function?
> 
> I'm a bit concerned about keeping the primary repository of a
> collection of data on a CF card. If some cells fail on read (is that
> possible? or do they only fail on write?) and that is rsync'd to an
> external hard-drive target, what are the chances those errors will
> be propagated without being detected? In general, what would be some
> recommended tools & strategies to ensure ongoing data integrity?
> 
> TIA for any suggestions,
> 
> John
> 
>