Confirmed, should be fixed in this slightly newer build (will be same name though): Snapshot build · subspace/subspace@21194a8 · GitHub
This time I won’t be able to test since I’ve moved all of my PC’s to the test build 
I hope someone else who is running Feb 19 release can test, otherwise I believe it’s not a big deal. Even me can fix it by running the Mar 8 release for couple of minutes.
I believe moving to feb-19 and back would still trigger the same bug, but I did test myself and it did upgrade from feb-19 to the test build successfully, so I’m quite confident it will work as expected.
Thanks, I just want to enjoy this moment. 200 rewards, no miss.
2 Likes
It should still prove successfully either way, just less efficiently. We already have a lot of flags, I’m not sure we want even more of that. What makes things more complex is that best method is disk-specific, this is why by forcing one for all you can make things worse.
There is a Prometheus metrics for this. If you want to see this visually, there are community tools or you can try Space Acres, which shows this visually too.
When we know the application has worked the way it is designed to be, I think It’s still good for farmers to see the prove approach and prove time at every eligible reward chance, so they can see by themselves what hardware/connection is good enough. Giving more info for farmers to optimize their own hardware is always good and helpful.
Honestly I don’t know in any condition, that the whole sector prove can be faster than concurrent chunks prove? But if we can have a flag like: --prove-method with 3 values (auto, concurrent-chunk, whole-sector). It’s still good. If this is set manually by farmers for any reason, the benchmark can skip at start.
thanks.
I see your point, but this is kind what benchmarks are supposed to do. You can run them on any disk for any amount of time and see what results it yields. I’ll may add proving time to logs still.
I have such example in VM with NVMe SSD connected via USB adapter. While concurrent reads take many seconds for proving due to massive concurrency, sequential reads at ~800MB/s are much faster than that and allow farmer to prove successfully every time. The fact that you don’t have this issue doesn’t mean it doesn’t exist an the goal is to make it “just work”, which I guess we mostly achieved for proving.
I spent some time with Windows Performance Analyzer looking at my latency spike but I can’t see anything obvious, no rogue processes spamming drive with requests, no driver issues etc. I tried simulating Subspace’s read access and there are occasional dips in performance but nothing lasting several seconds.
Probably some hardware compatibility, file drivers or maybe drive issue. I’ll plot another MX500 and see if it’s doing better. Perhaps drive is simply faulty. Or it’s rearranging data and will stop with these freezes in a few weeks.
No misses with this test build, I’m so happy I did not lose one reward all night.
It does look to be replotting now though so I’m still keeping an eye on it but so far so good!
No problems were found.
Fix shipped in following releases: