Farming cluster plot time

Support
1

Issue Report

To plot a new farm, farming-cluster takes more time than regular farmer.

Environment

  • Operating System: Ubuntu 18.04
  • Pulsar/Advanced CLI/Docker: CLI

Problem

My server takes 7 minutes for each sector to plot with regular farmer.
The same server takes 31 minutes with farming-cluster (The server runs controller, cache, plotter and farmer, without other node).

Why the cluster farmer performance is worse than regular farmer?

[Paste any errors or relevant logs here]

6014 and 6013 is the name of my x86_64 Server with CPU 12 core 4.4GHz.
6014 single node cluster 1GB plot: 31 mins
2024-05-28T11:39:41.035386Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (0.00% complete) sector_index=0
2024-05-28T12:10:44.884067Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Initial plotting complete

6014 single node cluster 2GB plot: 48 mins (+17)
2024-05-28T13:11:56.462001Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (0.00% complete) sector_index=0
2024-05-28T13:11:56.464066Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (50.00% complete) sector_index=1
2024-05-28T13:59:53.956365Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Initial plotting complete

6014 single node cluster 3GB plot: 57 mins (+9)
2024-05-28T14:11:29.280458Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (0.00% complete) sector_index=0
2024-05-28T14:11:29.282809Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (33.33% complete) sector_index=1
2024-05-28T14:11:29.284934Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (66.67% complete) sector_index=2
2024-05-28T15:08:45.901572Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Initial plotting complete

6014 single node cluster 4GB plot: 68 mins (+11)
2024-05-28T22:57:49.265806Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (0.00% complete) sector_index=0
2024-05-28T22:57:49.267754Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (25.00% complete) sector_index=1
2024-05-28T22:57:49.269688Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (50.00% complete) sector_index=2
2024-05-28T22:57:49.271688Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plotting sector (75.00% complete) sector_index=3
2024-05-29T00:05:47.153511Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Initial plotting complete

6014 6013 cluster 4GB plot: 101 mins
2024-05-29T00:24:30.859609Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plottiing sector (0.00% complete) sector_index=0
2024-05-29T00:24:30.861712Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plottiing sector (25.00% complete) sector_index=1
2024-05-29T00:24:30.863937Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plottiing sector (50.00% complete) sector_index=2
2024-05-29T00:24:30.866185Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Plottiing sector (75.00% complete) sector_index=3
2024-05-29T02:05:13.996870Z INFO {farm_index=0}: subspace_farmer::single_disk_farm::plotting: Initial plotting complete

2

Plotting in a cluster can be limited by the cache and networking. Specifically when plotter needs pieces it will request pieces from controller and controller will request them from cache. Make sure cache is fully synced first or else controller will have to reach out to DSN first. Also note that every message goes to and from NATS server in this architecture. So if NATS server is on a separate machine from controller and cache, it will be much slower even if everything seems to run on the same machine.

3

I set cache size to 8GiB. What are the rules to define the proper cache size?

./subspace-farmer cluster --nats-server nats://192.168.0.10:4222 cache path=/home/fact/subspace/cluster/cache,size=8GiB

4

Cache size must be at least 200GB at the moment.

5

What’s the function of the cache during farming/plotting?

6

Cache allows your plotter to retrieve pieces from local storage rather than downloading from the Internet. If you store the whole history locally in cache it allows plotter to avoid reaching out to Internet completely.

7

The subspace-node will download the whole history (About 60GB so far) from internet for the first time.
Can plotter retrieve pieces from node (LAN speed should be much quick than internet) instead of cache?
Please correct me if my understanding is wrong.

8

No, node doesn’t store those pieces. It retrieves them, decodes blocks and throws pieces away afterwards. It will also prune older blocks as is sycns further.

10

Hi Nazar, could you please have a look at 2 questions about cache.

  1. In farming-cluster, is controller doing the piece sync, right? because its log shows “2024-06-03T02:22:55.661963Z INFO subspace_farmer::farmer_cache: Piece cache sync 99.34% complete”

  2. After piece cache sync completed, where is the local file position?

11

Controller is orchestrating piece cache sync and other operations. Piece cache is stored at path you specified during starting of the cache component.