No rewards for 5 days, Benchmark Audit looks OK

Support
1

Issue Report

No rewards for 3 days, while benchmark Audit looks OK

Environment

  • Operating System:
    Ubuntu 18.04 x64
  • Pulsar/Advanced CLI/Docker:
    Advanced CLI

Problem

Just no reward at all for 5 days.
2 Farmer node with 300GB NVMe SSD respectively.
What are the reasons.

[Paste any errors or relevant logs here]
./subspace-farmer-ubuntu-x86_64-skylake-gemini-3h-2024-mar-22 benchmark audit /home/fact/subspace/farm/
Benchmarking audit/plot/rayon/regular
Benchmarking audit/plot/rayon/regular: Warming up for 3.0000 s
Benchmarking audit/plot/rayon/regular: Collecting 10 samples in estimated 5.2755 s (660 iterations)
Benchmarking audit/plot/rayon/regular: Analyzing
audit/plot/rayon/regular
time: [7.7439 ms 7.8280 ms 7.9222 ms]
thrpt: [ 35659 GiB/s 36088 GiB/s 36480 GiB/s]

2

Can you provide your benchmark prove result also? I trust it’s good but just in case.

A few more questions:

  • Is your node synced?
  • How many sectors have you plotted on your farmers?
  • If possible, share the last 20 log lines of your farmer that you’ve run pretty long (a day) without any reward
3

node is synced

Node Log:
2024-03-26T13:56:23.755518Z INFO Consensus: substrate: :sparkles: Imported #803537 (0xdb3c…8540)
2024-03-26T13:56:28.320324Z INFO Consensus: substrate: :sparkles: Imported #803538 (0x2851…1957)
2024-03-26T13:56:28.730649Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803538 (0x2851…1957), finalized #715731 (0xaeb1…3477), :arrow_down: 12.6kiB/s :arrow_up: 16.7kiB/s
2024-03-26T13:56:33.730924Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803538 (0x2851…1957), finalized #715731 (0xaeb1…3477), :arrow_down: 15.1kiB/s :arrow_up: 16.5kiB/s
2024-03-26T13:56:36.865332Z INFO Consensus: substrate: :sparkles: Imported #803539 (0x54dd…2f2d)
2024-03-26T13:56:38.731109Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803539 (0x54dd…2f2d), finalized #715731 (0xaeb1…3477), :arrow_down: 8.4kiB/s :arrow_up: 11.4kiB/s
2024-03-26T13:56:40.668698Z INFO Consensus: substrate: :sparkles: Imported #803540 (0x319f…b16a)
2024-03-26T13:56:43.731367Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803540 (0x319f…b16a), finalized #715731 (0xaeb1…3477), :arrow_down: 8.9kiB/s :arrow_up: 9.5kiB/s
2024-03-26T13:56:44.615639Z INFO Consensus: substrate: :sparkles: Imported #803541 (0x1ec3…3f3b)
2024-03-26T13:56:48.731616Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803541 (0x1ec3…3f3b), finalized #715731 (0xaeb1…3477), :arrow_down: 11.0kiB/s :arrow_up: 16.2kiB/s
2024-03-26T13:56:53.731855Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803541 (0x1ec3…3f3b), finalized #715731 (0xaeb1…3477), :arrow_down: 9.9kiB/s :arrow_up: 15.9kiB/s
2024-03-26T13:56:55.799903Z INFO Consensus: substrate: :sparkles: Imported #803542 (0x4aa7…9afc)
2024-03-26T13:56:58.732104Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803542 (0x4aa7…9afc), finalized #715731 (0xaeb1…3477), :arrow_down: 12.9kiB/s :arrow_up: 17.5kiB/s
2024-03-26T13:56:59.223438Z INFO Consensus: substrate: :sparkles: Imported #803543 (0xa1b0…156a)
2024-03-26T13:57:03.732356Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803543 (0xa1b0…156a), finalized #715731 (0xaeb1…3477), :arrow_down: 9.5kiB/s :arrow_up: 15.5kiB/s
2024-03-26T13:57:07.884197Z INFO Consensus: substrate: :sparkles: Imported #803544 (0x5844…9605)
2024-03-26T13:57:08.732762Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803544 (0x5844…9605), finalized #715731 (0xaeb1…3477), :arrow_down: 13.9kiB/s :arrow_up: 17.0kiB/s
2024-03-26T13:57:12.991536Z INFO Consensus: substrate: :sparkles: Imported #803545 (0x677e…89d1)
2024-03-26T13:57:13.733135Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803545 (0x677e…89d1), finalized #715731 (0xaeb1…3477), :arrow_down: 11.1kiB/s :arrow_up: 13.2kiB/s
2024-03-26T13:57:16.175362Z INFO Consensus: substrate: :sparkles: Imported #803546 (0x3a89…a868)
2024-03-26T13:57:18.733558Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803546 (0x3a89…a868), finalized #715731 (0xaeb1…3477), :arrow_down: 11.2kiB/s :arrow_up: 13.5kiB/s
2024-03-26T13:57:18.756144Z INFO Consensus: substrate: :sparkles: Imported #803547 (0xb8f7…be05)
2024-03-26T13:57:19.729160Z INFO Consensus: substrate: :sparkles: Imported #803546 (0x7fee…a51d)
2024-03-26T13:57:23.733853Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803547 (0xb8f7…be05), finalized #715731 (0xaeb1…3477), :arrow_down: 14.2kiB/s :arrow_up: 19.1kiB/s
2024-03-26T13:57:28.734482Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803547 (0xb8f7…be05), finalized #715731 (0xaeb1…3477), :arrow_down: 9.0kiB/s :arrow_up: 12.5kiB/s
2024-03-26T13:57:33.734728Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803547 (0xb8f7…be05), finalized #715731 (0xaeb1…3477), :arrow_down: 10.0kiB/s :arrow_up: 15.0kiB/s
2024-03-26T13:57:38.735166Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803547 (0xb8f7…be05), finalized #715731 (0xaeb1…3477), :arrow_down: 13.8kiB/s :arrow_up: 21.5kiB/s
2024-03-26T13:57:43.735414Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803547 (0xb8f7…be05), finalized #715731 (0xaeb1…3477), :arrow_down: 10.8kiB/s :arrow_up: 15.8kiB/s
2024-03-26T13:57:48.736042Z INFO Consensus: substrate: :zzz: Idle (9 peers), best: #803547 (0xb8f7…be05), finalized #715731 (0xaeb1…3477), :arrow_down: 10.6kiB/s :arrow_up: 16.8kiB/s
2024-03-26T13:57:51.217203Z INFO Consensus: substrate: :sparkles: Imported #803548 (0xb3a2… 9a36)
2024-03-26T13:57:53.736507Z INFO Consensus: substrate: :zzz: Idle (8 peers), best: #803548 (0xb3a2…9a36), finalized #715731 (0xaeb1…3477), :arrow_down: 11.2kiB/s :arrow_up: 13.1kiB/s
2024-03-26T13:57:55.559721Z INFO Consensus: substrate: :sparkles: Imported #803548 (0x12fc… a961)

One of the farmer Log:
2024-03-26T08:38:07.840474Z INFO subspace_farmer::farmer_cache: Piece cache sync 8.02% complete
2024-03-26T08:39:35.057012Z INFO subspace_farmer::farmer_cache: Piece cache sync 12.02% complete
2024-03-26T08:41:02.496002Z INFO subspace_farmer::farmer_cache: Piece cache sync 16.03% complete
2024-03-26T08:42:21.363383Z INFO subspace_farmer::farmer_cache: Piece cache sync 20.04% complete
2024-03-26T08:43:59.199702Z INFO subspace_farmer::farmer_cache: Piece cache sync 24.05% complete
2024-03-26T08:45:29.968672Z INFO subspace_farmer::farmer_cache: Piece cache sync 28.06% complete
2024-03-26T08:47:32.543667Z INFO subspace_farmer::farmer_cache: Piece cache sync 32.06% complete
2024-03-26T08:49:19.596265Z INFO subspace_farmer::farmer_cache: Piece cache sync 36.07% complete
2024-03-26T08:51:45.345130Z INFO subspace_farmer::farmer_cache: Piece cache sync 40.08% complete
2024-03-26T09:09:54.842203Z INFO subspace_farmer::farmer_cache: Piece cache sync 44.09% complete
2024-03-26T09:12:29.301461Z INFO subspace_farmer::farmer_cache: Piece cache sync 48.10% complete
2024-03-26T09:13:43.770073Z INFO subspace_farmer::farmer_cache: Piece cache sync 52.10% complete
2024-03-26T09:15:06.035799Z INFO subspace_farmer::farmer_cache: Piece cache sync 56.11% complete
2024-03-26T09:16:16.978969Z INFO subspace_farmer::farmer_cache: Piece cache sync 60.12% complete
2024-03-26T09:17:25.252991Z INFO subspace_farmer::farmer_cache: Piece cache sync 64.13% complete
2024-03-26T09:18:40.493967Z INFO subspace_farmer::farmer_cache: Piece cache sync 68.14% complete
2024-03-26T09:19:59.230236Z INFO subspace_farmer::farmer_cache: Piece cache sync 72.14% complete
2024-03-26T09:21:15.789962Z INFO subspace_farmer::farmer_cache: Piece cache sync 76.15% complete
2024-03-26T09:22:35.789395Z INFO subspace_farmer::farmer_cache: Piece cache sync 80.16% complete
2024-03-26T09:24:12.838896Z INFO subspace_farmer::farmer_cache: Piece cache sync 84.17% complete
2024-03-26T09:25:20.593577Z INFO subspace_farmer::farmer_cache: Piece cache sync 88.18% complete
2024-03-26T09:26:39.198429Z INFO subspace_farmer::farmer_cache: Piece cache sync 92.18% complete
2024-03-26T09:27:44.253917Z INFO subspace_farmer::farmer_cache: Piece cache sync 96.19% complete
2024-03-26T09:28:57.985354Z INFO subspace_farmer::farmer_cache: Finished piece cache synchronization
2024-03-26T09:53:19.781180Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (0.45% complete) sector_index=1
2024-03-26T10:16:44.875539Z INFO subspace_farmer::commands::farm::dsn: DSN listening on /ip4/172.17.0.1/udp/30533/quic-v1/p2p/12D3KooWFyKR1YDb3s5CMqGrQaZVHvth1xhoxySrpP5p2chw3JQ7
2024-03-26T10:16:44.876394Z INFO subspace_farmer::commands::farm::dsn: DSN listening on /ip4/172.17.0.1/tcp/30533/p2p/12D3KooWFyKR1YDb3s5CMqGrQaZVHvth1xhoxySrpP5p2chw3JQ7
2024-03-26T10:17:48.436076Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (0.90% complete) sector_index=2
2024-03-26T10:42:01.309512Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (1.36% complete) sector_index=3
2024-03-26T11:06:33.280886Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (1.81% complete) sector_index=4
2024-03-26T11:30:39.926444Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (2.26% complete) sector_index=5
2024-03-26T11:54:43.309865Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (2.71% complete) sector_index=6
2024-03-26T12:01:42.455477Z WARN hickory_proto::xfer::dns_exchange: failed to associate send_message response to the sender
2024-03-26T12:01:42.455620Z WARN hickory_proto::xfer::dns_exchange: failed to associate send_message response to the sender
2024-03-26T12:18:54.778901Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (3.17% complete) sector_index=7
2024-03-26T12:43:05.273730Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (3.62% complete) sector_index=8
2024-03-26T13:07:24.552848Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (4.07% complete) sector_index=9
2024-03-26T13:31:28.455838Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (4.52% complete) sector_index=10
2024-03-26T13:55:28.307896Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (4.98% complete) sector_index=11

4

This line tells us that you’ve only plotted 12 sectors (sector 0 to sector 11) couple of minutes ago. Even you pledge 300GB, you’re actually farming only 11GiB. So it’s understandable that your ETW (estimated to win) is many days. So be patient, when more sectors are plotted, you’ll have more chance to win reward.

Subspace has a unique feature, it has both block and vote reward. Right now, on average, each block has 1 block reward and 9 vote reward. So, I believe you will receive your first reward soon. This is the log line on farmer that will tell you win a reward.

2024-03-26T13:56:18.036813Z INFO {farm_index=4}: subspace_farmer::reward_signing: Successfully signed reward hash 0x6fa3e76757ff976a5a1888605f558494ed0b1ed99105a1b338a37bdc3838905f

As soon as you receive your first reward, your wallet will be visible on the block explorer.

2 Likes
5

1 sector = 1GB, is this correct?

6

If the farmer has to be exit for some reason and run again. Will farmer has to farm from sector 0 again?

7

1 sector = 1GiB, slightly bigger than 1GB.
If you re-run farmer, of course, it will continue from the last sector, not from sector 0. You can try it by yourself right now :smiley:

8

Benchmark Prove attached, Could you please teach how to understand the numbers in it?

./subspace-farmer-ubuntu-x86_64-skylake-gemini-3h-2024-mar-22 benchmark prove /home/fact/subspace/farm
Benchmarking prove/plot/rayon/regular/concurrent-chunks
Benchmarking prove/plot/rayon/regular/concurrent-chunks: Warming up for 3.0000 s

Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 6.8s.
Benchmarking prove/plot/rayon/regular/concurrent-chunks: Collecting 10 samples in estimated 6.7540 s (10 iterations)
Benchmarking prove/plot/rayon/regular/concurrent-chunks: Analyzing
prove/plot/rayon/regular/concurrent-chunks
time: [806.36 ms 812.31 ms 818.28 ms]
Benchmarking prove/plot/rayon/regular/whole-sector
Benchmarking prove/plot/rayon/regular/whole-sector: Warming up for 3.0000 s

Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 11.1s.
Benchmarking prove/plot/rayon/regular/whole-sector: Collecting 10 samples in estimated 11.112 s (10 iterations)
Benchmarking prove/plot/rayon/regular/whole-sector: Analyzing
prove/plot/rayon/regular/whole-sector
time: [1.4583 s 1.4791 s 1.4981 s]
Found 2 outliers among 10 measurements (20.00%)
2 (20.00%) low mild