Farm errored and stopped

Support Incentivized Testnet
1 
2024-04-18T14:28:03.241433Z  INFO {farm_index=8}: subspace_farmer::single_disk_farm::plotting: Plotting sector (49.59% complete) sector_index=1765
2024-04-18T14:30:49.792546Z  INFO {farm_index=10}: subspace_farmer::single_disk_farm::plotting: Plotting sector (12.90% complete) sector_index=459
2024-04-18T14:33:30.219046Z  INFO {farm_index=9}: subspace_farmer::single_disk_farm::plotting: Plotting sector (49.14% complete) sector_index=1749
2024-04-18T14:33:30.357976Z  INFO subspace_farmer::commands::farm::dsn: DSN listening on /ip4/192.168.0.225/tcp/30533/p2p/12D3KooWFGpshykbRya4FwRVY7mEyN5XV36DE1YrhJFqESMjUcgN
2024-04-18T14:33:30.359034Z  INFO subspace_farmer::commands::farm::dsn: DSN listening on /ip6/2408:820c:270b:7800:cd5:c8c:32ec:3d59/tcp/30533/p2p/12D3KooWFGpshykbRya4FwRVY7mEyN5XV36DE1YrhJFqESMjUcgN
2024-04-18T14:33:30.360067Z  INFO subspace_farmer::commands::farm::dsn: DSN listening on /ip6/2408:820c:270b:7800:1d1d:d97e:5d90:7326/tcp/30533/p2p/12D3KooWFGpshykbRya4FwRVY7mEyN5XV36DE1YrhJFqESMjUcgN
2024-04-18T14:34:30.238634Z  WARN {farm_index=9}: subspace_farmer::single_disk_farm::plotting: Failed to send sector index for initial plotting error=send failed because receiver is gone
2024-04-18T14:34:30.279850Z ERROR subspace_farmer::commands::farm: Farm exited with error farm_index=9 error=Failed to retrieve farmer info: Request timeout
2024-04-18T14:35:00.282076Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=9 error=Failed to retrieve farmer info: Request timeout
2024-04-18T14:35:30.287705Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=9 error=Failed to retrieve farmer info: Request timeout
2024-04-18T14:35:58.871013Z  INFO {farm_index=6}: subspace_farmer::single_disk_farm::plotting: Plotting sector (48.69% complete) sector_index=1733
2024-04-18T14:36:00.300278Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=9 error=Failed to retrieve farmer info: Request timeout
2024-04-18T14:36:18.108757Z ERROR jsonrpsee_core::client::async_client: [backend]: Networking or low-level protocol error: WebSocket connection error: i/o error: An existing connection was forcibly closed by the remote host. (os error 10054)
2024-04-18T14:36:18.121118Z ERROR subspace_farmer::commands::farm: Farm exited with error farm_index=6 error=Slot info notification stream ended
2024-04-18T14:36:30.311415Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=9 error=Failed to retrieve farmer info: Request timeout
2024-04-18T14:36:48.123664Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=6 error=Slot info notification stream ended
2024-04-18T14:37:00.312097Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=9 error=Failed to retrieve farmer info: Request timeout
2024-04-18T14:37:18.130830Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=6 error=Slot info notification stream ended
2024-04-18T14:37:30.322422Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=9 error=Failed to retrieve farmer info: Request timeout
2024-04-18T14:37:48.135751Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=6 error=Slot info notification stream ended
2024-04-18T14:38:00.328049Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=9 error=Failed to retrieve farmer info: Request timeout
2024-04-18T14:38:18.141297Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=6 error=Slot info notification stream ended
2024-04-18T14:38:21.002175Z  INFO {farm_index=7}: subspace_farmer::single_disk_farm::plotting: Plotting sector (48.89% complete) sector_index=1740
2024-04-18T14:38:30.332912Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=9 error=Failed to retrieve farmer info: Request timeout
2024-04-18T14:38:40.266170Z ERROR jsonrpsee_core::client::async_client: [backend]: Networking or low-level protocol error: WebSocket connection error: i/o error: An existing connection was forcibly closed by the remote host. (os error 10054)
2024-04-18T14:38:40.307675Z ERROR subspace_farmer::commands::farm: Farm exited with error farm_index=7 error=Slot info notification stream ended
2024-04-18T14:38:48.152854Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=6 error=Slot info notification stream ended
2024-04-18T14:38:49.004291Z ERROR jsonrpsee_core::client::async_client: [backend]: Networking or low-level protocol error: WebSocket connection error: i/o error: An existing connection was forcibly closed by the remote host. (os error 10054)
2024-04-18T14:38:49.004641Z ERROR jsonrpsee_core::client::async_client: [backend]: Networking or low-level protocol error: WebSocket connection error: i/o error: An existing connection was forcibly closed by the remote host. (os error 10054)
2024-04-18T14:38:49.007810Z ERROR subspace_farmer::commands::farm: Farm exited with error farm_index=1 error=Slot info notification stream ended
2024-04-18T14:38:49.011700Z ERROR subspace_farmer::commands::farm: Farm exited with error farm_index=3 error=Slot info notification stream ended
2024-04-18T14:38:49.035343Z ERROR jsonrpsee_core::client::async_client: [backend]: Networking or low-level protocol error: WebSocket connection error: i/o error: An existing connection was forcibly closed by the remote host. (os error 10054)
2024-04-18T14:38:49.035610Z ERROR jsonrpsee_core::client::async_client: [backend]: Networking or low-level protocol error: WebSocket connection error: i/o error: An existing connection was forcibly closed by the remote host. (os error 10054)
2024-04-18T14:38:49.107388Z ERROR subspace_farmer::commands::farm: Farm exited with error farm_index=10 error=Slot info notification stream ended
2024-04-18T14:38:49.466676Z ERROR jsonrpsee_core::client::async_client: [backend]: Networking or low-level protocol error: WebSocket connection error: i/o error: An existing connection was forcibly closed by the remote host. (os error 10054)
2024-04-18T14:38:49.552743Z ERROR jsonrpsee_core::client::async_client: [backend]: Networking or low-level protocol error: WebSocket connection error: i/o error: An existing connection was forcibly closed by the remote host. (os error 10054)
2024-04-18T14:38:50.687916Z ERROR jsonrpsee_core::client::async_client: [backend]: Networking or low-level protocol error: WebSocket connection error: i/o error: An existing connection was forcibly closed by the remote host. (os error 10054)
2024-04-18T14:38:53.326248Z ERROR subspace_farmer::commands::farm: Farm exited with error farm_index=8 error=Slot info notification stream ended
2024-04-18T14:38:53.330465Z ERROR subspace_farmer::commands::farm: Farm exited with error farm_index=5 error=Slot info notification stream ended
2024-04-18T14:38:53.334663Z ERROR subspace_farmer::commands::farm: Farm exited with error farm_index=2 error=Slot info notification stream ended
2024-04-18T14:38:53.336831Z ERROR subspace_farmer::commands::farm: Farm exited with error farm_index=4 error=Slot info notification stream ended
2024-04-18T14:39:00.335811Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=9 error=Failed to retrieve farmer info: Request timeout
2024-04-18T14:39:10.311785Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=7 error=Slot info notification stream ended
2024-04-18T14:39:18.154866Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=6 error=Slot info notification stream ended
2024-04-18T14:39:19.020896Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=1 error=Slot info notification stream ended
2024-04-18T14:39:19.020891Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=3 error=Slot info notification stream ended
2024-04-18T14:39:19.115403Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=10 error=Slot info notification stream ended
2024-04-18T14:39:23.337257Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=8 error=Slot info notification stream ended
2024-04-18T14:39:23.337267Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=5 error=Slot info notification stream ended
2024-04-18T14:39:23.337283Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=2 error=Slot info notification stream ended
2024-04-18T14:39:23.350229Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=4 error=Slot info notification stream ended
2024-04-18T14:39:30.344609Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=9 error=Failed to retrieve farmer info: Request timeout
2024-04-18T14:39:40.321784Z ERROR subspace_farmer::commands::farm: Farm errored and stopped farm_index=7 error=Slot info notification stream ended

It’s keeps printing “Farm errored and stopped” error message for about 4 hours until I noticed it and restarted the farmer.

2

Looks like connection between farmer and node was lost

3

I’ve seen the case about connection lost, the farmer didn’t behave like this, it just quit after the “WebSocket connection error” without keeping printing the “Farm errored and stopped” message.

The issue happened again within 5 minutes after I restarted the farmer. Then I restarted the PC and launched the farmer, it’s working well till now.

Thanks for reply.