StakeWars2. Any problem

Good afternoon, dear developers.
Now the StakeWars2 stage has started and I’m having some problems.
Ryzen 9 7950x/64gb DDR5/NVMe/Ubuntu 20.04-22.04/Internet line from 100 megabits to 800 megabits configurations are used everywhere

Question #1.
AutoID domain (domain 1).
Every now and then the logs start to look weird.

When logs turn into “normal” logs, there is a lack of synchronization, even after adding a boot node. Domain has no peers, node is not synchronized, height is specified as 0 in telemetry.

Ports 30333, 30433 and 40333 are open. Also rebooted and changed internet providers - did not help. What do you recommend to do?
Node startup string:

ExecStart=/usr/local/bin/subspace-node run --chain gemini-3h --name pentium1-60 --base-path /root/subspace/subspace-node/ --blocks-pruning archive-canonical --state-pruning archive-canonical – --domain-id 1 --listen-on /ip4/0. 0.0.0.0/tcp/40333 --in-peers 150 --out-peers 150 --bootstrap-nodes /dns/bootstrap-0.autoid.gemini-3h.subspace.network/tcp/30334/p2p/12D3KooWFoiz2iTkmnnSqiL2oQRhGzaqgtUjYNz2jyWKQqgPXgx9

Question #2
Nova domain (domain 0).
Yesterday created operator #89 (Astral - Subspace Explorer - Gemini 3h - Operator #undefined) . I wrote operator-id in the node service file and restarted

( ExecStart=/usr/local/bin/subspace-node run --chain gemini-3h --name 386sx-40 --base-path /root/subspace/subspace-node/ --blocks-pruning archive-canonical --state-pruning archive-canonical – --domain-id 0 --operator-id 89 --listen-on /ip4/0. 0.0.0.0.0/tcp/40333 --in-peers 200 --out-peers 200 ). Nominated coins from the wallet from which the operator was created ( st6dd7VxFUoPaC2kJb9MTttEUsJkzvMYPLBHpMBPLhX38MuF1v ). Nominated coins from the second wallet (st6gzmTVLW5SQdF81Vcw8m7me8338Ru6JZ5okwH8VW1KFUGef ). Today friends nominated from their wallets.
Bottom line: 80 TSSC are displayed, no information about nominators, balance has not changed since yesterday. What to do?

Can you provide this as text please?

The Astral back-end is having some issues. Please remain patient while it catches up. You can check certain things on Subscan or PolkadotJS if you cannot wait,

WASM backtrace:

error while executing at wasm backtrace:

  1. 0: 0x205fe - subspace_runtime.wasm! sp_to: :storage: :get: :hb5f9479d1155afcb
  2. 1: 0xd1a50 - subspace_runtime.wasm! frame_ _support:: storage::unhashed: :get: :h53f368b0f3d8815c
  3. 2: 0xa975f - subspace_runtime.wasm!pallet_domains: :pallet: :Pallet: :submit_bundle: :{{closure}}::h971ba069d4b76e9c

3: 0x93ecb - subspace_runtime.wasm!<pallet_domains: :pallet::Call as frame_support::traits: :dispatch: :UnfilteredDispatchable>::dispatch_bypass_filter::{{closure}}::h6aca46ccdfab76af

4: 0x8bd17 - subspace_runtime.wasm!<subspace_runtime: :RuntimeCall as frame_support: :traits: :dispatch: :UnfilteredDispatchable>::dispatch_bypass_filter::hb64ee335b46d1ee1

5: 0x8a079 - subspace_runtime.wasm!<subspace_runtime: :RuntimeCall as sp_runtime::traits::Dispatchable>: :dispatch::h02a30185e46dcf82

6: 0x1bb442 - subspace_runtime.wasm!frame_executive:: Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::apply_extrinsic::h03a2ef97cc6358d3

7: 0x207c6c - subspace_runtime.wasm!Core_execute_block

2024-07-11T13:02:01.6023057 WARN Consensus: sync: Error importing block 0x84e1c52c3020dd512f93e18941b452d7fc8370cf2ba9d7ae4c8d382c9b2b3e59: consensus error: Inner block import error: Import failed: Error at calling runtime api: Execution failed: Execution aborted due to trap: host code panicked while being called by the runtime: Externalities not allowed to fail within runtime: “Trie lookup error: Database missing expected key: 0x548f57fc06ab47e8342d61159015e857133985f409b5762b865b06154566f31b”

WASM backtrace:

error while executing at wasm backtrace:

PS. The mistake was yesterday. I pulled the text from the screenshot

1 Like

@ning @ved is this something we’ve seen before?

“The Astral back-end is having some issues. Please remain patient while it catches up. You can check certain things on Subscan or PolkadotJS if you cannot wait”

Astral, wallet found:

Subscan:

Addendum.
It is disconcerting that it is in astral that other operators have a different balance from the starting balance and they show the number of nominators. I don’t have it, although it’s been almost a long time.

error: Database missing expected key: 0x548f57fc06ab47e8342d61159015e857133985f409b5762b865b06154566f31b”

This error happened on the consensus chain, when it tried to execute a submit_bundle extrinsic in the block, it looks like the node’s storage was corrupted. Does the node start from scratch (i.e. sync from the genesis block)? @user7

After a while, the logs look like this

It is weird but your node seems to resume normal.

At first glance, yes, but
The node has 40 consensus peers and 0 domain peers. Synchronization does not proceed further. Doesn’t seem to be working properly(
This situation has been going on for more than 24 hours.

See Domain synchronization - #3 by ning

Yeah, I looked it up. However, there is unfortunately no solution in that thread.

I am the 71st operator. In the past few days, there have been instances where the 71st operator has not registered

Many operators have been slashed by a bug in the protocol. If you are seeing something like Operator [x] is not registered on the Runtime. this is somewhat expected.

My startup command is

./subspace-node run
–chain gemini-3h
–name 7950x3doptane
–blocks-pruning archive-canonical
–state-pruning archive-canonical
–base-path /home/root1/sub/node

–domain-id 1
–operator-id 71
–bootstrap-nodes /dns/bootstrap-0.autoid.gemini-3h.subspace.network/tcp/30334/p2p/12D3KooWFoiz2iTkmnnSqiL2oQRhGzaqgtUjYNz2jyWKQqgPXgx9

How do I solve or handle this problem that I have encountered

thank you very much indeed