Refactor bdk-cli structure by tvpeter · Pull Request #278 · bitcoindevkit/bdk-cli

tvpeter · 2026-04-21T12:22:46Z

Description

This PR restructures the bdk-cli codebase to move away from a monolithic handling model towards a modular architecture. The primary goal is to improve code readability, simplify the addition of new features, and decouple core logic into smaller modules.

Fixes #273

Notes to the reviewers

While trying to achieve the above-stated goal, I took maximum care to avoid introducing new bugs. As a result, the existing execution logic was preserved as much as possible.

This PR introduces two new subdirectories under src:

handlers: This contains all the subcommands' execution logic grouped logically according to the top-level commands. Modules include the config, descriptor, key, offline, online and repl modules. They each contain structures for the subcommand, execution logic, and implementation of the top-level enum.
utils: This subdirectory contains helper functions used across the app. They are also grouped logically according to related functionality. So far, we have the descriptor-related module, the output that formats responses, and the types module that defines response structures.

Other notable changes

main module is retained with added functionality of routing requests to the right module for execution.
All client-related functionalities have been moved from various modules into the client module, and it defines the enum that holds all the available clients, as well as implements the functionalities such as broadcasting transactions and syncing operations.
The AppCommand trait that defines how each subcommand should be structured and the execute method they should implement.
The AppContext struct that holds the universal parameters that subcommands may need. Each subcommand's execute method accepts this struct and uses the params for its execution.
The output module in the util subdirectory attempts to present a uniform presentation layer by having an output trait, the FormatOutput trait, which is implemented by all types presenting data response to the user.
Collapsed the wallet subdirectory into the persister module that defines all structures and functionalities related to persistence.
Introduced a type state pattern for the AppContext: To minimise runtime errors, the Init, OfflineOperations and OnlineOperations state structs were introduced, and AppContext is generic over these states. This moved the validation from runtime to compile time.
Introduced FormatOutput::write_out and made it generic over the Write trait to allow for flexibility in testing
Extracted all initialisation logic (database, config loading and persistence initialisation) into runtime module. Also, added the RuntimeWallet enum to return options of a persisted wallet and a standard non-persisted wallet. The WalletRuntime struct is a wrapper that determines the type of wallet needed and initialises it and handles loading of a wallet from config.

Request-Response Cycle

The following outlines how requests are handled and responses are generated after introducing changes to the structure:

When a user enters a set of commands, those commands are received and parsed by the Clap library into Rust structures, such as CliOpts, WalletOpts, and subcommands. This process takes place in the commands module.
The parsed request is then passed to the main module. The main module uses the runtime module to interpret the request and initialize any necessary resources (such as wallets, databases, and clients) needed to fulfill the request. It then routes the request to the appropriate handler for execution.
Once a particular handler receives the request along with all the required resources, it processes the specific command using the AppContext and prepares a response.
After processing is complete, the handler calls the output module to format the result, serialize it into JSON, and return it to the terminal.

Changelog notice

In the key derive subcommand, when printing help message, the clap arg for env=“PATH” often pull all system and displays all system paths. This has been changed to derivation_path I.e. env=“DERIVATION_PATH” to prevent clap from picking the system paths and rendering it
Moved the wallet subdirectory that served as persister into the persister module
Moved DatabaseType enum from commands module into persister module
Removed the —pretty flag

Checklists

All Submissions:

I've signed all my commits
I followed the contribution guidelines
I ran cargo fmt and cargo clippy before committing

codecov · 2026-04-21T12:26:17Z

Codecov Report

❌ Patch coverage is 4.10811% with 1774 lines in your changes missing coverage. Please review.
✅ Project coverage is 24.79%. Comparing base (943b4b6) to head (63f769a).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
src/handlers/offline.rs	0.00%	398 Missing ⚠️
src/handlers/online.rs	0.00%	263 Missing ⚠️
src/utils/common.rs	8.02%	172 Missing ⚠️
src/client.rs	0.00%	165 Missing ⚠️
src/utils/descriptors.rs	0.00%	140 Missing ⚠️
src/handlers/config.rs	0.00%	93 Missing ⚠️
src/handlers/repl.rs	0.00%	89 Missing ⚠️
src/main.rs	13.40%	84 Missing ⚠️
src/handlers/dns/mod.rs	0.00%	82 Missing ⚠️
src/handlers/descriptor.rs	0.00%	60 Missing ⚠️
... and 7 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #278      +/-   ##
==========================================
- Coverage   26.97%   24.79%   -2.18%     
==========================================
  Files          10       22      +12     
  Lines        3882     3678     -204     
==========================================
- Hits         1047      912     -135     
+ Misses       2835     2766      -69

Flag	Coverage Δ
rust	`24.79% <4.10%> (-2.18%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

notmandatory · 2026-05-02T21:57:03Z

concept ACK, this is a big change might be a good idea to do a live walk through some time on discord to help with the review process.

tvpeter · 2026-05-02T22:00:30Z

concept ACK, this is a big change might be a good idea to do a live walk through some time on discord to help with the review process.

Alright, anytime we are chanced.

Also, I would want most of the PRs to get in before this one because it will be challenging for others to rebase and fix conflicts.

- move all blockchain clients code into the backend subdirectory - move commands.rs content into commands subdirectory

- split util fns into `descriptors` and `common`

- move persister into wallet subdirectory - update imports in payjoin module

- split handlers into top level commands config, key and descriptors

- move handler fns into offline, online, wallets modules

- split handler fns into repl - add entry point to the handlers subdir - add wallet subdir entry point

- add trait for formatting outputs - add types for presenting data to the output - refactor offline mod to use types

- add types for descriptor, key and wallets mods

- add simple table helper fn for creating tables - refactor types to use simple table helper - add type defs for key, wallets and descriptors

- rebase master for bip322 feature - update types to use simple table helper

notmandatory

ACK 188e68d

I like the new structure and added generic/context type safety for adding new commands. It's a big change but certainly also a big improvement.

As discussed on the walk-through today I know you still have some other PRs that need to be merged before this one and then incorporated, but once those are in it looks ready to go.

I focused my review on the overall design and did a simple manual smoke test: generate descriptors, config wallet, get new address, send btc to wallet, sync wallet, send btc from wallet.

- Update bdk_wallet to v2.4.0 and other deps - Add wallet events to RPC, electrum, esplora and kyoto clients

- add compile taproot descriptor with randomized unspendable internal key

- Adds payjoin persistence

sdmg15

Overall looks good.
So far I have two questions:

Why does payjoin requires a folder, couldn't things inside it falls either under utils or handlers? Because for me when having it as a separate folder it feels like most of the features would also have their own dedicated folders which isn't the case.
Most of the .rs aren't exported in the module tree which makes rust-analyzer unable to find those symbols. Is this intentional?

sdmg15 · 2026-07-07T15:14:15Z

 ))]
 mod payjoin;
-#[cfg(any(feature = "sqlite", feature = "redb"))]
 mod persister;


Why is this not feature-gated anymore?

Because I moved the new_wallet fn into the module, which the db gating does not apply to.

sdmg15 · 2026-07-07T15:17:47Z

+
+            cmd.execute(&mut ctx)?.write_out(std::io::stdout())?;
+        }
+        CliSubCommand::Completions { shell: _ } => unimplemented!(),


I think there's missing implementation of this?

I have added it in the new updated push

sdmg15 · 2026-07-07T15:55:36Z

+            Ok(RuntimeWallet::Standard(Box::new(new_wallet(
+                self.network,
+                &self.wallet_opts,
+            )?)))


Is this additional check needed? Doesn't the one on lines 94-97 covers this?

Thank you for catching this

sdmg15 · 2026-07-07T16:09:06Z

+};
+
+impl OfflineWalletSubCommand {
+    pub fn execute(&self, ctx: &mut AppContext<OfflineOperations<'_>>) -> Result<(), Error> {


Just a nit: can we have an alias in the form of type OfflineCtx<'a> = AppContext<OfflineOperations<'a>>; something similar to Online.

Okay. I will revisit this after the merge.

tvpeter · 2026-07-07T16:59:28Z

Overall looks good. So far I have two questions:

Why does payjoin requires a folder, couldn't things inside it falls either under utils or handlers? Because for me when having it as a separate folder it feels like most of the features would also have their own dedicated folders which isn't the case.

Most of the .rs aren't exported in the module tree which makes rust-analyzer unable to find those symbols. Is this intentional?

The payjoin subdirectory contains all payjoin-related modules -the db, the ohttp relays, and the feature impl itself. I have thought long about it and could not come up with a better way to restructure it, so I felt it is better it retains the structure. If you have a better way that you think it should be restructured, you can suggest it here.

sdmg15 · 2026-07-08T14:11:50Z

Overall looks good. So far I have two questions:

Why does payjoin requires a folder, couldn't things inside it falls either under utils or handlers? Because for me when having it as a separate folder it feels like most of the features would also have their own dedicated folders which isn't the case.

Most of the .rs aren't exported in the module tree which makes rust-analyzer unable to find those symbols. Is this intentional?

The payjoin subdirectory contains all payjoin-related modules -the db, the ohttp relays, and the feature impl itself. I have thought long about it and could not come up with a better way to restructure it, so I felt it is better it retains the structure. If you have a better way that you think it should be restructured, you can suggest it here.

I don't want to hold this back but I'm wondering what can stay at top level of the src and what can go inside the handlers?

My thoughts were that components that belong to the same functionalities will logically fall into handlers as a lib.

For example for the DNS feature you just added I was expecting a structure similar to:

src/handlers/
├── config.rs
├── descriptor.rs
├── dns
│   ├── dns_payment_instructions.rs
│   └── mod.rs

Likewise for the payjoin:

src/handlers/
├── config.rs
├── descriptor.rs
├── payjoin
│   ├── db.rs
│   ├── mod.rs
│   └── ohttp.rs

I think this helps grouping related files of a feature together.

tvpeter · 2026-07-08T14:22:49Z

I don't want to hold this back but I'm wondering what can stay at top level of the src and what can go inside the handlers?

My thoughts were that components that belong to the same functionalities will logically fall into handlers as a lib.

For example for the DNS feature you just added I was expecting a structure similar to:
src/handlers/
├── config.rs
├── descriptor.rs
├── dns
│   ├── dns_payment_instructions.rs
│   └── mod.rs
Likewise for the payjoin:
src/handlers/
├── config.rs
├── descriptor.rs
├── payjoin
│   ├── db.rs
│   ├── mod.rs
│   └── ohttp.rs
I think this helps grouping related files of a feature together.

Good suggestion. I will implement. Thank you

tvpeter · 2026-07-08T15:52:15Z

Good suggestion. I will implement. Thank you

Done.

- migrate dns payment instruction feature

tvpeter · 2026-07-08T18:57:55Z

@sdmg15 if you are satisfied, you can ack so I will merge. Thank you

sdmg15

LGTM
ACK a08225e

sdmg15 · 2026-07-08T14:08:03Z

+}
+
+#[derive(Parser, Debug, Clone, PartialEq)]
+pub struct NewAddressCommand {}


Non blocking: I think there's unneeded braces for all theses unit type structs.

Removed the braces for all unit types

- remove braces for unit type structs

6000dd5 test: Add tests for dns payment instn feature (Vihiga Tyonum) b4e1081 fix(Justfile): fix test-threads to max 2 (Vihiga Tyonum) f809a7e test: add more tests for createtx, bip322 (Vihiga Tyonum) a993315 test: Add sp createtx test (Vihiga Tyonum) d710838 test: add test for multiple recipients and sp code (Vihiga Tyonum) a040a55 test: add tests for repl, transactions & send_all (Vihiga Tyonum) f17a7ef fix(repl): Remove null printed after repl output (Vihiga Tyonum) d425f5b test: split tests according to typestate (Vihiga Tyonum) 3293764 fix(proxy_opts): add saving and reading proxy_opts (Vihiga Tyonum) 3465513 ref(verbose): Drop `verbose` flag from tests (Vihiga Tyonum) 702d259 test(online): Add test transaction full cycle (Vihiga Tyonum) c013da0 test: Add integration tests for offline wallet ops (Vihiga Tyonum) 0d3c1a9 test: Add wallets, descriptor, compile & config (Vihiga Tyonum) d8e2887 test: Add helper fns & integration tests for key (Vihiga Tyonum) Pull request description:  ### Description  This PR builds on #278 and introduces integration tests for bdk-cli. It replaces manual `std::process::Command` boilerplate with the `assert_cmd` library, allowing us to perform black-box testing against the compiled binary. Features covered so far include: - key: generate, derive, and restore - wallets: list wallets - wallet config: save config, read config. - descriptor: generate descriptor - compile: policy compiler - offline wallet operations: new_address, unused_address, balance, unspent, transactions, policies, public_descriptor, create_tx, combine_psbt ### Notes to the reviewers  ## Changelog notice   - Introduces the BdkCli helper struct to inject context state into base commands. - Dropped `verbose` flag from wallets as it was applicable to only `Pbst` ### Checklists #### All Submissions: * [x] I've signed all my commits * [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk-cli/blob/master/CONTRIBUTING.md) * [x] I ran `cargo fmt` and `cargo clippy` before committing Top commit has no ACKs. Tree-SHA512: ea4adf734310f4ec494564f75b904835520b4625547859bf25c529b96f2f7fef823dc220dbe4eaf4856e2dd6fe06daa9f97bddc7a565f5e19185fdeea963202e

github-project-automation Bot added this to BDK-CLI Apr 21, 2026

tvpeter marked this pull request as ready for review April 21, 2026 12:59

tvpeter self-assigned this Apr 24, 2026

tvpeter added the enhancement New feature or request label Apr 24, 2026

tvpeter force-pushed the refactor/restructure-project branch from 8fc06d8 to 2ce28b2 Compare April 29, 2026 14:10

tvpeter added this to the CLI V4.0.0 milestone May 6, 2026

notmandatory moved this to In Progress in BDK-CLI May 21, 2026

va-an mentioned this pull request May 21, 2026

feat: add BIP353 DNS payment instructions #236

Merged

6 tasks

tvpeter force-pushed the refactor/restructure-project branch from 04dd213 to de1fc98 Compare May 21, 2026 20:04

tvpeter force-pushed the refactor/restructure-project branch 3 times, most recently from 53a6566 to 0c89aea Compare June 4, 2026 11:35

tvpeter added 15 commits June 5, 2026 13:43

ref(commands): mv commands & clients into subdirs

5c286d6

- move all blockchain clients code into the backend subdirectory - move commands.rs content into commands subdirectory

ref(config): move config into config subdir

739dec3

ref(utils): refactor utils into utils subdir

2482044

- split util fns into `descriptors` and `common`

ref(persister): mv persister into wallet subdir

eb3a6aa

- move persister into wallet subdirectory - update imports in payjoin module

ref(handlers): split handlers into config, key

03307c9

- split handlers into top level commands config, key and descriptors

ref(handlers):mv handler fns into offline & others

bf0551b

- move handler fns into offline, online, wallets modules

ref(handlers): split handler fns into repl

0bf3e66

- split handler fns into repl - add entry point to the handlers subdir - add wallet subdir entry point

ref(main): update main entry point

3e2eec8

refactor(handlers): Add types for outputting data

a973273

- add trait for formatting outputs - add types for presenting data to the output - refactor offline mod to use types

ref(handlers): add types for desc, key & wallets

055e1fd

- add types for descriptor, key and wallets mods

ref(types): add simple table helper

990f30a

- add simple table helper fn for creating tables - refactor types to use simple table helper - add type defs for key, wallets and descriptors

ref(handlers): rebase bip322 feature

90017cd

- rebase master for bip322 feature - update types to use simple table helper

ref(utils): use types in desc output in utils

ada3485

revert mod names for command and clients

eaa4e11

update namespace and update types

7a0e68b

tvpeter mentioned this pull request Jun 30, 2026

feat(utxo-locking)!: Add commands to lock, unlock and list locked UTXOs #294

Open

6 tasks

notmandatory approved these changes Jun 30, 2026

View reviewed changes

notmandatory mentioned this pull request Jul 1, 2026

Payjoin persistence #242

Merged

4 tasks

tvpeter added 5 commits July 4, 2026 11:13

ref(create_tx): enforce single recp for sendall

cb9634f

feat(WalletEvents): Add wallet events to clients

6a6390f

- Update bdk_wallet to v2.4.0 and other deps - Add wallet events to RPC, electrum, esplora and kyoto clients

ref(compile):compile tr desc with rand unspend key

249948e

- add compile taproot descriptor with randomized unspendable internal key

ref(payjoin): add payjoin persistence

c2ca2dd

- Adds payjoin persistence

Merge 'master' into refactor/restructure-project

fcf881c

tvpeter force-pushed the refactor/restructure-project branch from de36eee to b7abe68 Compare July 5, 2026 02:47

feat(sp): fix command_requires_db fn match arm

b3fe860

tvpeter force-pushed the refactor/restructure-project branch from b7abe68 to b3fe860 Compare July 7, 2026 16:08

sdmg15 reviewed Jul 7, 2026

View reviewed changes

tvpeter force-pushed the refactor/restructure-project branch from 24710d7 to e86b45b Compare July 8, 2026 16:16

tvpeter added 4 commits July 8, 2026 18:47

Merge branch 'master' into ref/restructure-project

77a891b

feat(dns-payment): port dns payment instruction

5951c17

- migrate dns payment instruction feature

feat(handlers): mv feats into subdirs in handlers

5da7660

drop verbose flag

8dc0503

tvpeter force-pushed the refactor/restructure-project branch from e3b3355 to a08225e Compare July 8, 2026 17:49

sdmg15 approved these changes Jul 8, 2026

View reviewed changes

revert parallel request wallet parameter

63f769a

- remove braces for unit type structs

tvpeter force-pushed the refactor/restructure-project branch from a08225e to 63f769a Compare July 8, 2026 20:56

tvpeter merged commit 898d08c into bitcoindevkit:master Jul 8, 2026
7 of 9 checks passed

github-project-automation Bot moved this from Ready to Review to Done in BDK-CLI Jul 8, 2026

tvpeter mentioned this pull request Jul 9, 2026

Add a section in the README that explains how to add new features to the new design. #298

Open

Uh oh!

Conversation

tvpeter commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Notes to the reviewers

Changelog notice

Checklists

All Submissions:

Uh oh!

codecov Bot commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

notmandatory commented May 2, 2026

Uh oh!

tvpeter commented May 2, 2026

Uh oh!

notmandatory left a comment

Choose a reason for hiding this comment

Uh oh!

sdmg15 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tvpeter commented Jul 7, 2026

Uh oh!

sdmg15 commented Jul 8, 2026

Uh oh!

tvpeter commented Jul 8, 2026

Uh oh!

tvpeter commented Jul 8, 2026

Uh oh!

tvpeter commented Jul 8, 2026

Uh oh!

sdmg15 left a comment

Choose a reason for hiding this comment

Uh oh!

sdmg15 Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tvpeter commented Apr 21, 2026 •

edited

Loading

codecov Bot commented Apr 21, 2026 •

edited

Loading

sdmg15 Jul 8, 2026 •

edited

Loading