Disk is unusable for minutes at a time #10
Comments
|
Doing some more research into this, and it seems that when I changed |
|
I have never seen this problem. Given that the problem resolves itself when you use a single thread, I am wondering whether the problem is some form of race condition when you are using multiple threads. |
|
That was my thought too, but I recently restarted my computer, and the problem had oddly fixed itself with no changes to the code. I guess it's some odd Windows stuff. |
|
Here is something to know: if you process is a console one and you accidentally make a selection in the console window, this can result in the whole process freezing under some circumstances until you unselect the text. This has bitten me a couple of times. |
|
No, it definitely isn't that, especially since none of my callback code reports to actually be running. I had set up an atomic integer that increments/decrements and prints its value when entering or exiting the callback, and it just stopped getting called during those hangs. It happened even while I didn't have a debugger attached. If this issue pops up again, I'll make sure to screen record it. |
|
I have this issue too. Here's how I reproduce it:
#include <stdlib.h>
#include <stdio.h>
#define WIN32_LEAN_AND_MEAN
#include <Windows.h>
int wmain(int argc, wchar_t** argv)
{
if (argc < 3) return -1;
int bufsize = _wtoi(argv[1]);
UINT8* buf = (UINT8*) malloc(bufsize);
HANDLE h = CreateFileW(argv[2], GENERIC_READ, 0, NULL,
OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
UINT64 progress = 0;
while (true)
{
DWORD bytesRead;
if (!ReadFile(h, buf, bufsize, &bytesRead, NULL)) break;
if (bytesRead == 0) break;
progress += bytesRead;
printf("%lld\n", progress);
}
CloseHandle(h);
return 0;
}(in my case args are "32768 file")
|
Whew, and I thought I was the crazy one. |
|
I think the problem lies in the kernel (driver or Windows). I can reproduce it without locks and file mappings in the user-mode callbacks. It doesn't occur while reading a file randomly in threads. It occurs while reading sequentially. Also I set a lock in |
|
I tried to trace some kernel code and noticed |
|
@extratype thanks for the debugging on this. I do not have time to look at this right now, but I am hoping to look at it soon. If you find any additional information regarding this problem please let me know. |
|
I have reviewed Also I found the drive does not hang with caching disabled ( |
|
Call stack of 7-zip while rawdisk hangs: I found an interesting thread: https://community.osr.com/discussion/198906/image-file-disk-driver-blocking-on-filter-manager |
|
For testing I replaced Also I tried to debug the hang, here's what I got: The OSR thread seems right. We may not need caching with memory mapping. Windows already caches the data read from Found another thread: https://community.osr.com/discussion/272478 |
|
I thought I solved this issue, but it was not. I used non-cached I/Os in my project. It was working at first, but when I turned off real-time scanning in the antivirus solution, it started freezing again. |
|
There's a similar project called wnbd and they had the same issue: cloudbase/wnbd#30. Having Here is a patch: I don't claim any copyrights for this. Please use it whatever you want. |
|
@extratype thanks for your extensive research on this. The reason for passing no Has your patch completely resolve the issue for you? |
I had read the comment. I think so too. Because WinSpd is a disk driver? I don't know.
Yes for rawdisk. It no longer freezes with the #10 (comment). But WIndows may freeze when the write cache is full if you use synchronous buffered I/O (regular |
- Remove Utils::Callback (and therefore std::function) overhead by directly using function pointers - Removed redundant copy with a static buffer (also allows the code to be multi threaded when winfsp/winspd#10 gets fixed) - Builds the service executable with a static CRT to prevent dll calls on memcpy
- Remove Utils::Callback (and therefore std::function) overhead by directly using function pointers - Removed redundant copy with a static buffer (also allows the code to be multi threaded when winfsp/winspd#10 gets fixed) - Builds the service executable with a static CRT to prevent dll calls on memcpy
|
@extratype, could you get the ball rolling and write a PR with your patch to close this issue? |
|
@WorkingRobot Sorry, I don't agree with the contributor agreement so I don't raise a PR. The patch is simple, you can submit it if you want. I don't need credit, just mention this issue in your PR :) |
|
I am also facing this issue in a simple RAM-Disk I made using this project. 1 Thread: Whole benchmark completes as expected. 2 Threads: Occasionally freezes during "Preparing.. Creating Test File", completes with greater performance than the previous test (as one would expected) with the sole exception being the second test row which is a lot slower, during which the disk seemingly becomes unresponsive.
4 or more Threads: Issues continues to grow, new but same Event Viewer entries Using #14 patch: Using a custom winspd-x64.dll build from #14 the will cause the issues to vanish, I do however understand from the explanation in #14 that this patch may not solve the underlying issue at hand. Here's my C++ RAM-Disk I cobbled together using bits from rawdisk.c
#include <iostream>
#include <winspd/winspd.h>
static BOOLEAN Read(SPD_STORAGE_UNIT* StorageUnit,
PVOID Buffer, UINT64 BlockAddress, UINT32 BlockCount, BOOLEAN FlushFlag,
SPD_STORAGE_UNIT_STATUS* Status)
{
const auto block_length = StorageUnit->StorageUnitParams.BlockLength;
const size_t size = static_cast<size_t>(BlockCount) * block_length;
const void* start = static_cast<char*>(StorageUnit->UserContext) + block_length * BlockAddress;
memcpy(Buffer, start, size);
return TRUE;
}
static BOOLEAN Write(SPD_STORAGE_UNIT* StorageUnit,
PVOID Buffer, UINT64 BlockAddress, UINT32 BlockCount, BOOLEAN FlushFlag,
SPD_STORAGE_UNIT_STATUS* Status)
{
const auto block_length = StorageUnit->StorageUnitParams.BlockLength;
const size_t size = static_cast<size_t>(BlockCount) * block_length;
void* start = static_cast<char*>(StorageUnit->UserContext) + block_length * BlockAddress;
memcpy(start, Buffer, size);
return TRUE;
}
static SPD_STORAGE_UNIT_INTERFACE RawDiskInterface =
{
Read,
Write
};
int main()
{
constexpr unsigned long threads = 4;
constexpr size_t gb = 8;
// Set storage unit parameters
const SPD_STORAGE_UNIT_PARAMS params = {
.Guid = { 1234, 5678, 9012, {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h'} },
.BlockCount = static_cast<UINT64>(1024) * 1024 * 1024 / 512 * gb,
.BlockLength = 512,
.WriteProtected = false,
.CacheSupported = false,
.UnmapSupported = false,
.MaxTransferLength = 64 * 1024,
};
// Try to create storage unit
SPD_STORAGE_UNIT* storage_unit = nullptr;
if (const auto error = SpdStorageUnitCreate(nullptr, ¶ms, &RawDiskInterface, &storage_unit); error != ERROR_SUCCESS)
{
std::cerr << "Storage unit creation failed! Error code " << error << "\n";
return static_cast<int>(error);
}
std::cerr << "Storage unit created successfully\n";
// Allocate buffer
void* buffer = malloc(params.BlockCount * params.BlockLength);
if (buffer == nullptr)
{
std::cerr << "Malloc failed";
return 1;
}
// Initialize buffer with MBR + partition
SPD_PARTITION partition = {
.Type = 7,
.BlockAddress = 4096 >= params.BlockLength ? 4096 / params.BlockLength : 1,
};
partition.BlockCount = params.BlockCount - partition.BlockAddress;
SpdDefinePartitionTable(&partition, 1, static_cast<UINT8*>(buffer));
storage_unit->UserContext = buffer;
// Try to start dispatcher
if (const auto error = SpdStorageUnitStartDispatcher(storage_unit, threads); error != ERROR_SUCCESS)
{
std::cerr << "Storage unit dispatcher start failed! Error code " << error << "\n";
return static_cast<int>(error);
}
std::cerr << "Storage unit dispatcher started successfully using " << storage_unit->DispatcherThreadCount << " threads\n";
// Wait until dispatcher stops
SpdStorageUnitWaitDispatcher(storage_unit);
// Cleanup allocated buffer
free(buffer);
return 0;
}System info
Edition Windows 11 Pro Processor 12th Gen Intel(R) Core(TM) i7-12700K 3.60 GHz |
In my current use of WinSpd, there are somewhat rare occasions, usually while the disk is being read from, when it completely freezes up and refuses to respond for a while. When that happens, the disk's active time spikes to 100%, despite not having any active transfers (as reported by Task Manager.) During this "hang time", none of my callbacks are running either. In perfmon, it reports that the "Current Disk Queue Length" is 1, with all other values being 0. Because both the "% Disk Time" and "% Idle Time" is 0%, it leads me to believe that there is some sort of an internal problem. My mounted drive is write protected, meaning that there should be no other interferences.
The text was updated successfully, but these errors were encountered: