New farm results in `Single disk farm I/O error: Invalid argument (os error 22)`

Support
1

Issue Report

Environment

  • Operating System: Multiple Linux
  • Space Acres/Advanced CLI/Docker: Advanced CLI

Problem

The first time a new farm is instantiated the user sees an error similar to the below. Appears on XFS and ext4. Relevant Discord thread.

2024-10-25T00:49:05.706930Z ERROR {farm_index=13}: subspace_farmer::commands::cluster::farmer: Farm creation failed error=Single disk farm I/O error: Invalid argument (os error 22)
Error: Single disk farm I/O error: Invalid argument (os error 22)

Caused by:
    Invalid argument (os error 22)
2

Tried one of the NVMe devices I have, then tried loop device, not reproducible on my end :confused:

@xorinox could I get access to a VM with problematic drive so I can debug this further? Ideally with 8+ cores and 16G+ of RAM so I can recompile things there a few times.

My SSH public key can be found on GitHub: https://github.com/nazar-pc.keys

3

I will look into this, possibly you can get access to the physical machine, I assume no root privileges needed?

Can you provide exact details on how your NVMe devices were formatted and mounted?

4

If SSD needs to be re-formatted for the issue to reproduce then I probably need access to root, but if not (not clear to be from the previous testing whether simply removing files is sufficient) then I don’t need any privileged access.

I usually create VMs with libvirt, so it is easy to dedicate one PCIe device to it and throw VM away later.

5

The issue is only reproducible, when the NVME drives are formatted with 4k blocks. When I format them back to 512 byte blocks, the issue cannot be produced.

1 Like
6

This will fix it: Use only 4096 bytes sectors for Direct I/O by nazar-pc · Pull Request #3189 · autonomys/subspace · GitHub

Thanks for the bug report, access to the machine for testing and everything else.

1 Like