Fix calibnet wallet test flakiness

[sudo-shashank]

Summary of changes

Changes introduced in this pull request:

• Add a wait_for_msg helper that blocks on Filecoin.StateWaitMsg until a message lands on-chain and asserts ExitCode == 0, replacing the hand-rolled poll_until_state_search_msg so there's a single "wait for inclusion" helper.
• Call it after each send in the calibnet wallet tests so balance assertions run only once the transaction is included.

Reference issue to close (if applicable)

Closes

Other information and links

Change checklist

• I have performed a self-review of my own code,
• I have made corresponding changes to the documentation. All new code adheres to the team's documentation standards,
• I have added tests that prove my fix is effective or that my feature works (if possible),
• I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

Outside contributions

• I have read and agree to the CONTRIBUTING document.
• I have read and agree to the AI Policy document. I understand that failure to comply with the guidelines will lead to rejection of the pull request.

Summary by CodeRabbit

Release Notes

• Tests

  • Enhanced test reliability for message pool operations with improved polling and confirmation mechanisms
  • Refactored wallet transfer and delegation tests to deterministically wait for on-chain message confirmation before verifying balance changes

• Chores

  • Reorganized test helper utilities for improved maintainability and code reuse across integration tests

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 17dfa6d0-1f99-4919-8e52-f775c8cd6e7d

📥 Commits

Reviewing files that changed from the base of the PR and between f44eda1 and 9c17a39.

📒 Files selected for processing (3)

• tests/calibnet_mpool_tools.rs
• tests/calibnet_wallet.rs
• tests/common/calibnet_wallet_helpers.rs

🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

• filecoin-project/lotus (manual)

---

Walkthrough

The PR replaces poll_until_state_search_msg (polling Filecoin.StateSearchMsg) with a new wait_for_msg function that calls Filecoin.StateWaitMsg and validates Receipt.ExitCode. It also refactors Backend into a struct with public run_raw, run, and send methods handling keystore serialization and retry logic, then updates all calibnet wallet and mpool integration tests to use these new helpers.

Changes

Calibnet integration test helper and test refactor

Layer / File(s)	Summary
Shared helper refactor: Backend methods, wait_for_msg, HTTP client tests/common/calibnet_wallet_helpers.rs	Adds module doc; centralizes POLL_TIMEOUT, POLL_WAIT_TIME, and retry constants with run_command; introduces LOCAL_KEYSTORE_LOCK to serialize local keystore CLI access; moves wallet CLI logic into Backend::run_raw, Backend::run, and Backend::send with min-gas-price retry; makes poll public; switches HTTP client timeout to POLL_TIMEOUT; replaces poll_until_state_search_msg with wait_for_msg backed by Filecoin.StateWaitMsg and Receipt.ExitCode validation.
Wallet test helpers and wait_for_msg call sites tests/calibnet_wallet.rs	Introduces funded_delegated_addr cached initializer, Backend::balance, poll_until_changed/poll_until_funded, and import_wallet; updates export_import_roundtrip, wallet_delete, market_add_balance_message_on_chain, send_to_filecoin_address, send_to_eth_equivalent, delegated_send, and delegated_remote_send to call wait_for_msg on message CIDs before asserting balance changes.
Mpool test helpers and wait_for_msg call site tests/calibnet_mpool_tools.rs	Adds forest_cli, mpool_nonce, and poll_until_pending_nonce helpers; refactors mpool_replace_auto_unblocks_pending into a parameterized rstest over Backend::Local/Backend::Remote using backend.send; replaces the final assertion from poll_until_state_search_msg with wait_for_msg.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• ChainSafe/forest#6983: Directly precedes this PR's refactor of tests/calibnet_wallet.rs and tests/common/calibnet_wallet_helpers.rs, establishing the Backend enum and wallet helper structure that this PR extends.
• ChainSafe/forest#7084: Introduced the mpool nonce-fix and mpool replace calibnet integration tests in tests/calibnet_mpool_tools.rs that this PR refactors and extends with shared CLI/polling helpers.

Suggested labels

RPC

Suggested reviewers

• hanabi1224
• LesnyRumcajs

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Fix calibnet wallet test flakiness' accurately describes the main objective of the PR, which is to address flakiness in calibnet wallet tests by introducing the wait_for_msg helper and integrating it into wallet tests.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch shashank/fix-flaky-wallet-test

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch shashank/fix-flaky-wallet-test

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

tests/calibnet_mpool_tools.rs (1)

63-65: ⚡ Quick win

Preserve wait_for_msg error details in test failures.

Using is_ok() discards the root-cause error chain from wait_for_msg; emitting {err:#} will make failed CI runs much easier to diagnose.

Suggested patch

-    assert!(
-        wait_for_msg(&cid).await.is_ok(),
-        "mpool replace --auto should replace message {cid} from {addr} at nonce {nonce}."
-    );
+    wait_for_msg(&cid).await.unwrap_or_else(|err| {
+        panic!(
+            "mpool replace --auto should replace message {cid} from {addr} at nonce {nonce}: {err:#}"
+        )
+    });

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/calibnet_mpool_tools.rs` around lines 63 - 65, The assertion on
wait_for_msg result uses is_ok() which discards the actual error information
from the Result, making it difficult to diagnose test failures in CI. Instead of
checking is_ok() on the await result of wait_for_msg, capture the Result and use
pattern matching or conditional logic to extract and include the actual error
details (using format strings like {err:#}) in the assertion failure message.
This way, when the test fails, the root cause error information will be visible
to help with debugging.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/common/calibnet_wallet_helpers.rs`:
- Around line 311-314: The wait_for_msg function currently relies on the HTTP
client's implicit 120-second timeout without explicit error context. Wrap the
rpc_call invocation with tokio::time::timeout(POLL_TIMEOUT, ...) and add
.context() with a descriptive message to provide clearer diagnostics when the
function times out or fails. This approach should follow similar patterns
already established in the codebase, such as those found in stateful_tests.rs,
to ensure consistency and improve test observability.

---

Nitpick comments:
In `@tests/calibnet_mpool_tools.rs`:
- Around line 63-65: The assertion on wait_for_msg result uses is_ok() which
discards the actual error information from the Result, making it difficult to
diagnose test failures in CI. Instead of checking is_ok() on the await result of
wait_for_msg, capture the Result and use pattern matching or conditional logic
to extract and include the actual error details (using format strings like
{err:#}) in the assertion failure message. This way, when the test fails, the
root cause error information will be visible to help with debugging.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: e263a229-62ee-4189-b215-c5d5c0db9504

📥 Commits

Reviewing files that changed from the base of the PR and between 4edfd01 and 72089f7.

📒 Files selected for processing (3)

• tests/calibnet_mpool_tools.rs
• tests/calibnet_wallet.rs
• tests/common/calibnet_wallet_helpers.rs

🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

• filecoin-project/lotus (manual)

[akaladarshi]

Can you place all utils function in a common file so they can be used across tests instead of keeping them separate.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.37%. Comparing base (d1cb4f8) to head (bd4cedd).
✅ All tests successful. No failed tests found.

Additional details and impacted files

see 7 files with indirect coverage changes

---

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d1cb4f8...bd4cedd. Read the comment docs.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.