# High memory usage for June 11th release

**URL:** <https://forum.autonomys.xyz/t/high-memory-usage-for-june-11th-release/4222>\
**Category:** Support\
**Created:** [June 15, 2024, 4:42pm UTC](https://forum.autonomys.xyz/t/high-memory-usage-for-june-11th-release/4222 "2024-06-15T16:42:13Z")\
**Posts on this page:** 7\
**Page:** 2

<div class="post-metadata">

**Author:** ![nazar-pc](https://yyz1.discourse-cdn.com/flex011/user_avatar/forum.autonomys.xyz/nazar-pc/32/6_2.png) [@nazar-pc](https://forum.autonomys.xyz/u/nazar-pc)\
**Post date:** [June 27, 2024, 2:23pm UTC](https://forum.autonomys.xyz/t/high-memory-usage-for-june-11th-release/4222/24 "2024-06-27T14:23:53Z")

</div>

> [@P3-1452](#):
>
> It fell to 12G Immediately after closing the second instance. I should say that the second instance was plotting to some same ssd’s as the first instance, though I don’t know if it’s connected to the memory issue.

Very odd! This might be related, but at the same time I wouldn’t expect memory to be attributed to the process 🤔

> [@P3-1452](#):
>
> Regarding many farms on one ssd instead of one farm - I still do that because the initial piece cache sync takes way longer for a 4TB farm than 500GB farm. Of course I could always start earlier…

It only does so because it downloads all the pieces upfront. Without that you’ll have to re-download them from the network all the time, which is even slower. I don’t think you are saving time by doing this. In fact with the amount of space pledged it’ll be the fastest to all all disks at once with one farm per disk from the very beginning.

> [@gensyn\_good](#):
>
> After running for a period of time, the farmer process will exit!

Thanks for the report, but the information isn’t really actionable. Please read the discussion above to see what information I need to be able to figure out what is going on. Simply saying you’re running farmer on Windows and it exits isn’t enough unless it happens to absolutely everyone all the time, which is quite clearly not the case.

---

<div class="post-metadata">

**Author:** ![nazar-pc](https://yyz1.discourse-cdn.com/flex011/user_avatar/forum.autonomys.xyz/nazar-pc/32/6_2.png) [@nazar-pc](https://forum.autonomys.xyz/u/nazar-pc)\
**Post date:** [June 27, 2024, 2:25pm UTC](https://forum.autonomys.xyz/t/high-memory-usage-for-june-11th-release/4222/25 "2024-06-27T14:25:43Z")

</div>

> [@P3-1452](#):
>
> Sometime after it started plotting, I found the first instance with 38G memory.

It’d be nice to have logs from that run

---

<div class="post-metadata">

**Author:** ![P3-1452](https://avatars.discourse-cdn.com/v4/letter/p/8baadc/32.png) [@P3-1452](https://forum.autonomys.xyz/u/P3-1452)\
**Post date:** [June 27, 2024, 7:42pm UTC](https://forum.autonomys.xyz/t/high-memory-usage-for-june-11th-release/4222/26 "2024-06-27T19:42:26Z")

</div>

I ran it again on Intel  
one instance with all plots and after sometime added a second instance just plotting. At 18:24:  
the first was replotting and with 11G  
the second was plotting and with 32G. [https://we.tl/t-Nwdd2CyLkD](https://we.tl/t-Nwdd2CyLkD)  
Both with concurrency 2.  
33G was the highest I’ve seen and after some time it came down.

 ![IMG_5995](https://canada1.discourse-cdn.com/flex011/uploads/subspace/original/2X/6/6b1ef3e3d16733a6177e7f763c5930367f83cdea.jpeg)

---

<div class="post-metadata">

**Author:** ![nazar-pc](https://yyz1.discourse-cdn.com/flex011/user_avatar/forum.autonomys.xyz/nazar-pc/32/6_2.png) [@nazar-pc](https://forum.autonomys.xyz/u/nazar-pc)\
**Post date:** [June 27, 2024, 9:01pm UTC](https://forum.autonomys.xyz/t/high-memory-usage-for-june-11th-release/4222/27 "2024-06-27T21:01:10Z")

</div>

Hm… so a single 4T farm resulted in 33G of memory usage, very interesting. If you restat it now that piece cache sync is done and everything has settled, does it use a lot of memory again? Does memory usage grow gradually or fairly quickly, I’m curious for example what is memory usage like right after first sector was plotted?

For things that I was not yet able to reproduce I’m looking for anecdotal evidence of various patterns.

You can also try farmer from [Snapshot build · subspace/subspace@f6ed626 · GitHub](https://github.com/subspace/subspace/actions/runs/9697908890) that has some optimizations, though nothing that would obviously reduce memory usage in second case with just plotting dramatically.

I’m also very curious if `--plot-cache false` make any difference to memory usage (it is `true` on the second farmer where you’re plotting).

---

<div class="post-metadata">

**Author:** ![nazar-pc](https://yyz1.discourse-cdn.com/flex011/user_avatar/forum.autonomys.xyz/nazar-pc/32/6_2.png) [@nazar-pc](https://forum.autonomys.xyz/u/nazar-pc)\
**Post date:** [June 28, 2024, 4:07pm UTC](https://forum.autonomys.xyz/t/high-memory-usage-for-june-11th-release/4222/28 "2024-06-28T16:07:23Z")

</div>

Assuming memory usage reason if of the same nature as the one fixed for some users in `jun-11`, this farmer build should help (don’t use node from this build, just farmer): [Snapshot build · subspace/subspace@2469498 · GitHub](https://github.com/subspace/subspace/actions/runs/9715660890)

---

<div class="post-metadata">

**Author:** ![P3-1452](https://avatars.discourse-cdn.com/v4/letter/p/8baadc/32.png) [@P3-1452](https://forum.autonomys.xyz/u/P3-1452)\
**Post date:** [July 2, 2024, 7:19pm UTC](https://forum.autonomys.xyz/t/high-memory-usage-for-june-11th-release/4222/29 "2024-07-02T19:19:43Z")

</div>

Ok I can report that after using this version for a few days, its memory usage low and stable, also when running two instances on each machine (amd (concurrency 3) Intel (concurrency 2) ).

---

<div class="post-metadata">

**Author:** ![nazar-pc](https://yyz1.discourse-cdn.com/flex011/user_avatar/forum.autonomys.xyz/nazar-pc/32/6_2.png) [@nazar-pc](https://forum.autonomys.xyz/u/nazar-pc)\
**Post date:** [July 3, 2024, 12:16am UTC](https://forum.autonomys.xyz/t/high-memory-usage-for-june-11th-release/4222/30 "2024-07-03T00:16:16Z")

</div>

Nice, thanks for confirming! We’ll ship it in the next release then. Appreciate all the experiments.

I wish I knew why it helps on Windows though 😅

[Previous page](https://forum.autonomys.xyz/t/high-memory-usage-for-june-11th-release/4222.md?page=1)
