"HPA/DCO data unavailable, cannot determine hidden sector status" and DIF/DIX (aka PI) protection Information field or type 2 formatting.

[thebearnecessities]

Hi there, Is there anyway of getting nwipe to not give this error, because its not a security/data risk I care about, but it might freak out the person who gets the report lol.

[thebearnecessities]

perhaps there a method or PRNG that doesn't check for HPA/DCO?

[PartialVolume]

A command line option to disable HPA/DCO could be added, this would have to be reflected in the report that HPA/DCO check is disabled, but would give a green tick and no warnings.

How many drives do you see this on and are you using the very latest version of nwipe?

I'd like to hear what others feel about HPA/DCO checks and what the message might be for a drive that won't respond to HPA/DCO commands like some recent enterprise discs and discs attached via some usb adapters.

[thebearnecessities]

We got this warning from three out of a batch of 6 we were wiping.

But to be fair, we hadn't seen it for the previous 20 devices we ran nwipe on.

We are running the latest version of nwipe (that from memory was released last November-ish?).

Quick background, we are a charity in Scotland, UK that is taking laptops from local businesses and wiping the hard disks and then refurbishing the laptops for good causes.

So when I send the reports to the businesses to reassure them that their data has been written over successfully, the exclamation marks and large red warning text is gonna spook them, lol

My basic understanding is that this HPA/DCO hidden sector is not used by the OS to store any user data, so it is not any type of security issue. But that was just from a quick Google as I had never heard of HPA/DCO sectors before.

Loving your software, it's helped our charity a lot, because we don't have any budget for blancco or similar.

Is there a way we can make a donation for your efforts?

[PartialVolume]

@thebearnecessities Any donations are always welcome as it does help towards the purchase of some of the hardware. Either via github at https://github.com/sponsors/PartialVolume or via https://www.buymeacoffee.com/Shredos

I'll see if I can improve HPA/DCO status reporting when it comes to ??? Identifying the drive as not supporting HPA/DCO so it responds with a more appropriate HS? N/A and without the exclamation is definitely preferable if nwipe knows the drive doesn't support HPA/DCO. SAS especially could do with some improvement. SATA, I think seems to be generally ok but there maybe some drives to don't support it.

[bandit8623]

just ran 80 drives. all of them gave this error. these are 4kn drives with type 2 protection enabled. on a second batch of these many just plain failed to finish wiping alltogether. reran wipe using a different program and wiped fine so drives are not bad.

getting rid of this error/ warning would be ideal

.36 version running on mint

SEAGATE ST8000NM0065

vendor: SEAGATE
product: ST8000NM0065
revision: K004
compliance: SPC-4
user capacity: 8,001,563,222,016 bytes [8.00 TB]
logical block size: 4096 bytes
formatted with type 2 protection
8 bytes of protection information per logical block
lu is fully provisioned
rotation rate: 7200 rpm
form factor: 3.5 inches

[bandit8623]

i/o errors. i even rebuilt a new system and swapped raid card . made no diff.
zeros .no verification. no last pass
yes same zero wipe
killdisk industrial
all non raid using lastest firmware on lsi 2308 it mode hbas.

[PartialVolume]

Interesting, how far into the wipe would you say it got before failing with an I/O errors?

[PartialVolume]

I'm not familiar with killdisk industrial but a quick review of the manual suggests that write error detection can be disabled in the processing attributes, assuming I've understood it correctly. I'm just checking, but I'm assuming you had write error detection enabled?

ShredOS/nwipe always aborts on the first write error it detects by design, however an option to override this maybe be in the pipeline. That's why I'm interested in how far it got in percentage before generating an I/O error.

[bandit8623]

10% then i/o errors on 3 to 4 drives . left run over night and all but 3 had failed to i/o errors by morning. i/o error code -1.

tried 3 diff times diff system and cards, all errors seemed to happen around the same time each run.

[PartialVolume]

10% is a long way into the drive and if three drives wiped ok it would suggest that nwipes ability to write and sync the drive is working ok. When the drive syncs the os is told to flush the disc cache to the disc. The return status is then detected as provided by fdatasync to determine whether the data was written to the disc without any write error. Nwipe would then terminate that particular wipe as the disc would then be destined for physical destruction.

So, how to prove where the problem is?

• Confirm killdisc industrial processing attributes are set to detect write errors.
• use the verify zeros method in nwipe to check if the disc is zero wiped by kill disc.
• what's the reallocated sector count like on the drives that fail? Have a lot of sectors been reallocated or is it in good shape, i.e zero reallocated sectors.

[bandit8623]

by default we have ignore read and write, because we need the wipe to finish no matter what. actually that would be a good option to add to nwipe.
actualy default for killdisk is to ignore read and write errors.. we leave default there

all the drives are 100% health. no realocated sectors. no read or write errors.

wondering if its more to do with the type 2 protection. example - i know trunas core has issues with it but trunas scale can handle. just throwing out ideas

health.txt

[PartialVolume]

all the drives are 100% health. no realocated sectors. no read or write errors.

Not all are at 100% health, there's 3 drives amongst those 84 that are reporting 0% health, imminent failure.

wondering if its more to do with the type 2 protection. example - i know trunas core has issues with it but trunas scale can handle. just throwing out ideas

That's a possibility, personally I've not tested with drives that are type 2 protection formatted. As I've read, the controller formats them as a slightly larger size than 512 or 4096 and uses that as some error correction? But presents the correct block size back to the OS. Apparently they can be reformatted as standard blocks but I guess it depends on the controller.

Anybody that has a better knowledge of these drives please feel free to jump in.

[PartialVolume]

If you are running ShredOS it will be in the root folder of the virtual drive, ALT F1 or under /sent/. How did you burn ShredOS to USB, dd, Rufus or Ventoy? If you are running nwipe from a distro you need to specify the log file on the command line. ShredOS auto creates the log file.

[bandit8623]

using mint.
can you specify how i set it via command line?

[PartialVolume]

sudo ./nwipe --logfile=nwipe.log

[PartialVolume]

sudo ./nwipe --help to see all the available options. I think you mentioned you were using v0.36?

[bandit8623]

yep. using .36.

and thanks!

[bandit8623]

killdisk at about 50% done no stops yet. ill run them again using nwipe after. may be a few days before i get info for you.
edit-
ill just stop the killdisk as i know it will go through and rerun the nwipe to get logs faster.

[kevin-bbinc]

Similar to u/thebearnecessities, I'm also clearing drives for donations.

Forgive my ignorance, but is it possible that this affects SAS drives specifically? I recently cleared 8 drives, 1 SATA and 7 SAS, and the SATA drive cleared as expected without any warnings in the report. I googled if HPA/DCO is an ATA-specific technology, and it seems to be, but I don't know enough about SAS drives to say it's conclusive.

If it matters, I'm using an older SuperMicro chassis with a backplane connected to an HBA to clear them.

[bandit8623]

update here, drives doing same thing with same batch. ill hopefully have a log to share in the next few days. will that log show up where the certs normal go?

[PartialVolume]

Just to clarify, so you ran wipes on 16 drives that were not type 2 and they all wiped successfully without any errors. I think we need to format some SAS drives as type 2 and run our own tests and see if we can reproduce this.

[PartialVolume]

For reference, can you post the lspci output of the server that generates the I/O errors on type 2 formatted drives. Thanks.

[bandit8623]

they were completely diff models but yes. on the lspci output ill see what i can do. no longer have those drives in that system.

[PartialVolume]

The more I read about type 2 and Linux the more I just want to put a check in for a type 2 formatted drive that says reformat the drive without type 2 protection then wipe.

So type 2 formatted drives, also known as DIF/DIX (Data Integrity Field/Data Integrity Extension) and T10-DIF/DIX are all to do with 8 bytes of data are are associated with every block. So on disk a 512 block on disc occupies 520 bytes.

Having read a few documents and articles on this subject, there have been a few issues and it would seem there still are some issues about using this DIF/DIX protection on any Linux based system except one designed specifically for the purpose.

I've read (which may or may not be true) that ..

• No Linux file-system supports DIF/DIX which may be because Linux memory management doesn't support it if you are using cached I/O (default for Linux) BUT if your program uses DIRECT I/O then it should work as long as your driver supports DIF/DIX. This might be fixed in later versions of the kernel.
• mspt3sas apparently does support DIF/DIX but you have to enable it when the mspt3sas driver loads by appending mpt3sas.prot_mask=0x7f to the kernel command line. Which opens up a whole new set of questions when dealing with mixed discs like ShredOS/Nwipe does.
• Your HBA needs to support DIF/DIX and have the correct firmware.

How does all this affect ShredOS/Nwipe?

• Nwipe operates in cached I/O mode, DIRECT I/O is a feature I would like to add. Whether that is still an issue on a modern kernel I'm not sure. Apparently memory management can move cached I/O about in memory which can result in the wrong CRC being calculated against the wrong data (intermittently). This might explain how a minority of zero fill wipes succeeded while others failed but seemingly at random points in a wipe. If nwipe performed DIRECT I/O and managed it's own disk buffer, then once the buffer was dynamically allocated we wouldn't be moving it about or realloc'ed it.
• Nwipe or ShredOS would have to recognise a type 2 formatted drive then check the kernel command line for mpt3sas.prot_mask=0x7f and if not present, require the user to add the text and reboot. We would need to determine whether it was necessary to remove this for non type 2 disks?. If that is correct then it becomes a problem with a server that is wiping a mix of type 2 and standard SAS discs simultaneously. It would be nice if mpt3sas adopted the correct mode based on the disc, but then again, maybe it does and mpt3sas.prot_mask=0x7f is not necessary in a later version.
• How does this complicate things when nwipe is being run on a Linux distro that is not ShredOS, when your boot disk is non type 2 formatted.
• The T10-DIF/DIX apparently supports the application (or does it mean the mpt3sas driver generating the CRC for each block.

So we maybe have the following options:

• Add some code to nwipe that says the disk is type 2 formatted, reformat with type 0 protection and then wipe.
• Spend a lot of time trying to get ShredOS/nwipe to work reliably with type 2 formatted disks and possibly the other modes. If the kernel and driver handled all this then maybe there wouldn't be a problem.

Personally, I think there are more important features that need adding to ShredOS/nwipe first before working on this which seems fairly niche.

As always, your thoughts are appreciated and if there is anybody with the suitable programming skills that wanted to make this their pet project you are always welcome to submit a PR. However, bear in mind you will need a suitable DIF/DIX capable server and at least 20 DIF/DIX discs formatted with type 2 protection to be simultaneously wiped, in order to be sure there aren't any driver or memory management issues that might cause false I/O errors.

[bandit8623]

Great information.
I would say type 2 protectred drives are not the majority so what you say about working on other features makes total sense. maybe just a warning check on the drives? like wipe at own risk may not work?

i would agree features like direct i/o would be a higher priority than making type 2 work 100% of the time. I come accross alot of diff drives and types (sata sas hdd ssd) so if you need me to test something let me know.

as for the mpt3sas my systems i was using are using a mptsas2 cards latest firm. LSI 9206-16e 6Gb/s 16-Port SATA SAS PCI-e 3.0 x8 External HBA SAS2308- firm 20.00.07.00
since the program killdisk likely uses direct i/o (uses like 5gig ram when wiping 80 drives) that finishes fine so im thinking the cached i/o may be the ticket. with direct i/o much less cpu usage as well.

let me know if i can help further.

[PartialVolume]

@kevin-bbinc

Forgive my ignorance, but is it possible that this affects SAS drives specifically? I recently cleared 8 drives, 1 SATA and 7 SAS, and the SATA drive cleared as expected without any warnings in the report. I googled if HPA/DCO is an ATA-specific technology, and it seems to be, but I don't know enough about SAS drives to say it's conclusive.

The SAS drives we test with don't support DCO/HPA, hence the [HS? ???] Status response, however it possible some drives do, although at the moment we don't know which models those are or even if they exist. SAS drives do use the ata command set as used by SATA drives however they don't necessarily support all commands including those to retrieve the HPA/DCO status, so theoretically
they could support HPA/DCO. However, HPA/DCO is or has been dropped by Seagate on some enterprise drives or so I believe so it's quite normal to not be able to determine the HPA/DCO status as the drive doesn't respond to the commands or if it does responds with invalid data.

[kevin-bbinc]

Gotcha. I appreciate the explanation!