Important: Any time you upgrade bcoin, you must execute npm rebuild
before restarting.
- A new
u
bucket is added totxdb
that stores only spendable coins indexed byaccount
andvalue
. - A new multi-algorithm coin selection. (Lowest Larger, Branch and Bound, and Single Random Draw)
To upgrade wallet, we need to copy coins from c
bucket to u
bucket in txdb
.
To do this you can run:
node ./migrate/walletdb7to8.js /path/to/bcoin/wallet
- Added
getnodeaddresses
which returns entries from the hostlist. - Updated
estimatesmartfee
to return attributefeerate
(previouslyfee
).
- Support for multiple compact filters added as per BIP158 specification. To enable the support of multiple filters the data previously in
~/.bcoin/index/filter
has been moved to~/.bcoin/index/filter/BASIC
. - The filter indexer now also stores the filter hashes corresponding to the blockfilters.
To upgrade to the multiple compact filter support we need to move filter data from ~/.bcoin/index/filter
to ~/.bcoin/index/filter/BASIC
and generate filter hashes for the existing filters.
To do this you can run:
node ./migrate/indexerdb0to1.js /path/to/bcoin/index
-
Support for bech32m has been added. bcoin can now validate and send BTC to addresses for witness programs with versions > 0. The address indexer has also been updated to retrieve the new addresses. A bug was fixed where the indexer may return data for a witness version 0 address even if a version 1 address was queried, if the two addresses had the same data (hash). After upgrading bcoin, a full rescan may be necessary to re-index these collisions correctly. To accomplish this:
- Stop bcoin (
bcoin-cli rpc stop
orctrl-C
) - Delete the existing addr indexer data (
rm -rf ~/.bcoin/index/addr
) - Restart bcoin
- The address indexer will automatically begin re-indexing the chain. It may take up to 24 hours.
- Stop bcoin (
-
The logging module
blgr
has been updated. Log files will now be rolled over at around 20 MB and timestamped. Only the last 10 log files will be kept on disk and older log files will be purged. These values can be configured by passing--log-max-file-size
(in MB) and--log-max-files
.
- Fixed wallet RPC method
importprunedfunds
.
- Fixed a bug with the wallet 'conflict' event.
- Added an endpoint to the wallet HTTP API at
GET /
for an aliveness check. It returns the currentversion
andnetwork
type.
The way that block data is stored has changed for greater performance, efficiency, reliability and portability.
- Block and undo block data has been moved from LevelDB into flat files.
- The transaction and address indexes have been moved into separate LevelDB databases.
- The transaction index has been de-duplicated, and will reduce disk usage by
half for those running with
txindex
enabled. - The
txindex
andaddrindex
can now be enabled after the initial block download. - The
addrindex
has been sorted to support querying for large sets of results, and will no longer cause CPU and memory exhaustion issues. - The
addrindex
will correctly distinguish betweenp2pkh
andp2wpkh
addresses. - BIP158 compact block filters can be computed and indexed (
filterindex
).
To upgrade to the new disk layout it's necessary to move block data
from LevelDB (e.g. ~/.bcoin/chain
) to a new file-based block
storage (e.g. ~/.bcoin/blocks
), and remove txindex
and addrindex
data from the chain database, for those that have that feature enabled.
To do this you can run:
node ./migrate/chaindb4to6.js /path/to/bcoin/chain
The migration will take 1-3 hours, depending on hardware. The block data
will now be stored at /path/to/bcoin/blocks
. After the data has been moved
the chain database will be compacted to free disk space.
Alternatively, you can also sync the chain again, however the above migration will be faster as additional network bandwidth won't be used for re-downloading the blocks.
For those with txindex
and addrindex
enabled, the indexes will be
regenerated by rescanning the chain on next startup. This process can
take multiple hours (e.g. 8 hours) depending on hardware and the
index. Please take the potential downtime in re-indexing into account
before upgrading.
- Client module
/lib/client
was created from the external dependencybclient
. NodeClient
methodestimateFee
now returns JSON instead of a number.- Added
fee
command tobcoin-cli
. - Added
getBlockHeader
toNodeClient
and correspondingheader
command to CLI. - Added
getFilter
toNodeClient
and correspondingfilter
command to CLI.
PUT /wallet/:id
Creating a watch-only wallet now requires anaccountKey
argument. This is to prevent bcoin from generating keys and addresses the user can not spend from.POST /wallet/:id/create
- Now has a
sign
argument for optional signing of transactions. - Now has a
template
option, that will skip templating inputs whensign = false
, but you can enable it if necessary. It does not have an effect whensign = true
. - Exposes
blocks
, which will be used for fee estimation ifrate
is not set. - Exposes
sort
(Defaulttrue
), that can be used to disable BIP69 sorting.
- Now has a
- Bug fix addresses for the
getnewaddress
command with various networks. - Deprecate the
ismine
andiswatchonly
fields from thevalidateaddress
command and addisscript
,iswitness
,ischange
,witness_version
andwitness_program
to partially match the v0.18.0 Bitcoin Core release (26a2000b0177fd2668b7d82e5aa52829cf2bfdf6) - Add wallet RPC
getaddressinfo
to returnismine
andiswatchonly
with the correct values instead of their previous values which were hard-coded tofalse
. Also returnsaddress
,scriptPubKey
,isscript
,iswitness
,witness_version
andwitness_program
. (a28ffa272a3c4d90d0273d9aa223a23becc08e0e)
Several CPU and memory exhaustion issues have been resolved with some additional arguments for querying multiple sets of results for addresses that have many transactions.
GET /tx/address/:address
has several new arguments:after
,reverse
andlimit
. Theafter
argument is a txid, for querying additional results after a previous result. Thereverse
argument will change the order that results are returned, the default order is oldest to latest. Thelimit
argument can be used to give results back in smaller sets if necessary.POST /tx/address
has been deprecated. Instead, query for each address individually withGET /tx/address/:address
with the expectation that there could be many results that would additionally need to be queried in a subsequent query using theafter
argument to request the next set.POST /coin/address
andGET /coin/address/:address
are deprecated as coins can be generated using results from/tx/address/:address
and querying by only a range of the latest transactions to stay synchronized. Coins could otherwise be removed from results at any point, and thus the entire set of results would need to be queried every time to discover which coins have been spent and are currently available.GET /
has new fields.indexes.{addr,tx,filter}
for the status of indexers.
- Both
getblock
andgetblockheader
returnbits
in base 16 as a string instead of base 10 and a number.
- Regtest params have been updated to correspond with other bitcoin implementations for cross-testing. Key prefixes have been updated and SegWit is always active (39684df6051b599da9ed1383cb7e3f8605806a74). The regtest seeds have also been cleared so that the node will not attempt to connect to itself.
- Testnet seeds have been updated (8b6eba165b090edfc1ff5dbc9c422758d60b6342)
- Wallet and node HTTP and RPC is now more clearly identified. The wallet
HTTP server will log with the context
wallet-http
and the RPC aswallet-rpc
. The node HTTP will log with the contextnode-http
and the RPC asnode-rpc
. There was also a reduction in the duplicate logs in the case of an error for the node RPC, and the addition of method logging for the wallet RPC. - There is now also additional logging during wallet rescans.
- Pool options now has a
--discover
option exposed, and the--only
node option will disable discovery of new nodes. - The option for
coin-cache
has been removed. This setting was causing issues during sync with out-of-memory errors and was making performance worse instead of better. - The database location for indexes can be configured via the
--index-prefix
option. Default locations areprefix
+/index
(e.g.~/.bcoin/testnet/index/tx
and~/.bcoin/testnet/index/addr
).
- Script has been updated for policy flags. For example
VERIFY_CONST_SCRIPTCODE
to disableOP_CODESEPARATOR
in non-SegWit scripts and make a positivefindAndDelete
result invalid. - SegWit scripts that terminate with a stack size not equal to one
now have an error of
CLEANSTACK
instead ofEVAL_FALSE
. VERIFY_UPGRADABLE_NOPS
now does not apply toOP_CHECKLOCKTIMEVERIFY
andOP_CHECKSEQUENCEVERIFY
.
- Switched to use a security-focused rewrite of
mocha
calledbmocha
, for further details see: https://github.com/bcoin-org/bmocha - Data has been updated to be in sync with other implementations around policy-only script validation.
- Tests now cleanly close with all timers being cleared.
- Configuration file
wallet.conf
won't be read during test runs, which was causing issues with some testing environments.
- The transaction index methods are now implemented at
node.txindex
:getMeta(hash)
getTX(hash)
hasTX(hash)
getSpentView(tx)
- The address index method
getHashesByAddress
is now implemented atnode.addrindex
:getHashesByAddress(addr)
It now acceptsAddress
instances rather thanAddress|String
and the results are now sorted in order of appearance in the blockchain.getHashesByAddress(addr, options)
A new options argument has been added with the fields:after
- A transaction hash for results to begin after.limit
- The total number of results to return at maximum.reverse
- Will give results in order of latest to oldest.
- The following methods require
node.addrindex.getHashesByAddress
in conjunction withnode.txindex.getTX
andnode.txindex.getMeta
respectively, and now includes a new options argument as described above forgetHashesByAddress
:node.getMetaByAddress(addr, options)
node.getTXByAddress(addr, options)
- The following method has been deprecated:
getCoinsByAddress(addr)
- A new module for storing block data in files.
- Use of
buffer-map
for storing hashes (see #533). - Use of
bsert
for assertions. SIGINT
handling will close the full node, spvnode and wallet.- Using the bcoin library in a REPL now has auto-completion by pressing tab.
- Various documentation updates.
- Mempool fix to add non-standard (non-SegWit) to reject cache.
- Dependencies are committed to the repository and bcoin does not depend on npm.
- Updates to dependencies including
bcrypto
to version > 5. - Various small fixes to run bcoin in a browser.
The latest bcoin release contains almost a total rewrite of the wallet. A migration is required.
Bcoin has also been modularized quite a bit. Users installing globally from npm should be sure to run:
$ npm update -g bcoin
For users who cloned the respository themselves, it might be wise to simply
remove the node_modules
directory and do a fresh npm install
.
$ cd bcoin/
$ git pull origin master
$ rm -rf node_modules/
$ npm install
$ npm test
Once upgrading is complete, a migration script must be executed. The migration
script resides in ./migrate/latest
as an executable file.
A bcoin prefix directory must be passed as the first argument.
Example:
$ ./migrate/latest ~/.bcoin
What this does:
- Renames
chain.ldb
tochain
. - Renames
spvchain.ldb
tospvchain
. - Renames
walletdb.ldb
towallet
. - Runs a migration (
chaindb3to4.js
) onchain
. - Runs a migration (
chaindb3to4.js
) onspvchain
. - Runs a migration (
walletdb6to7.js
) onwallet
.
The chain migration should be minor, and only taxing if you have
--index-address
enabled.
If you have a testnet, regtest, or simnet directory, the migration must also be run on these, e.g.
$ ./migrate/latest ~/.bcoin/testnet
Also note that the --prefix
argument now always creates a fully-fledged
prefix directory. For example --prefix ~/.bcoin_whatever --network testnet
will create a directory structure of ~/.bcoin_whatever/testnet/
instead of
~/.bcoin_whatever
. Please update your directory structure accordingly.
Wallet and Node are now separated and use different persistent configuration files.
Some configuration options have moved to wallet.conf
, which is stored in the prefix
directory next to the existing bcoin.conf
.
All configuration options in wallet.conf
will have the wallet-
prefix added implicitly. The prefix is still needed when using CLI/ENV configuration
methods. The wallet now has it's own HTTP server, so options such as
wallet-http-host
must also be specified along with http-host
if you require this setting (required for Docker networking).
Wallet-specific settings such as api-key
and wallet-auth
have moved to wallet.conf
. If using CLI/ENV options, these are prefixed with wallet-
.
Various configuration method examples are shown below.
Note some config options (eg. network
) are automatically passed to wallet plugin
if specified through CLI/ENV, but should be specified in wallet.conf
for bwallet-cli
.
Example using wallet.conf:
network: testnet
wallet-auth: true
api-key: hunter2
http-host: 0.0.0.0
Example using CLI options:
./bin/node --network=testnet --http-host=0.0.0.0 --wallet-http-host=0.0.0.0 --wallet-api-key=hunter2 --wallet-wallet-auth=true
Example using ENV:
BCOIN_NETWORK=simnet BCOIN_HTTP_HOST=0.0.0.0 BCOIN_WALLET_HTTP_HOST=0.0.0.0 BCOIN_WALLET_API_KEY=hunter2 BCOIN_WALLET_WALLET_AUTH=true ./bin/node
The bcoin cli
interface has been deprecated and is now replaced with a
separate tool: bcoin-cli
. bcoin wallet
has also been deprecated and is
replaced with bwallet-cli
. Both bcoin-cli
and bwallet-cli
can be
installed with the bclient
package in npm.
The main wallet API changes have to do with changes in the structure of the HTTP server itself. The bcoin wallet HTTP server will now always listen on it's own port. Standard ports are:
main: 8334
testnet: 18334
regtest: 48334
simnet: 18558
The default account object is no longer present on the wallet. To retrieve it,
request /wallet/:id/account/default
instead. More minor changes to the JSON
output include the removal of id
and wid
properties on a number of objects.
Please update your code accordingly.
For socket.io events, all wallet
prefixes have been removed. For example,
wallet join
should be replaced with simply join
. All wallet-related events
(tx
, confirmed
, etc) now receive the wallet ID as the first argument. The
tx
event will receive [id, details]
instead of [details]
.
The _admin
endpoints have been moved up a level. Instead of requesting
/wallet/_admin/backup
(for example), request /backup
instead. If
--wallet-auth
is enabled, a new flag --admin-token
is accepted. Admin token
is a 64 character hex string and allows access to all admin routes. For
example, GET /backup?token=xxx...
. It also allows access to any wallet on the
HTTP or websocket layer (websockets who use the admin token can join a wallet
of *
, which notifies them of events on all wallets).
- wallet/http - A
conflict
event will be correctly emitted for all double spends. - wallet/http - Account balances are now indexed and will appear on account objects via the API.
- bcoin - Bcoin has been rewritten using class syntax. Objects will not be inheritable with legacy prototypal inheritance.
- pkg - Bcoin is once again buildable with browserify.
- pkg - Numerous modules have been moved out to separate dependencies. For
example, the
lib/crypto
module has been moved to thebcrypto
package (and also heavily optimized even further). - bcoin - The main
bcoin
module has been rewritten in terms of what it exposes and how. Conventions have changed (Address
instead ofaddress
). Due to the use of class syntax, something likebcoin.Address(str)
is no longer possible. Please take a look atlib/bcoin.js
to get a better idea of what has changed. - net - Support for BIP151 and BIP150 has been dropped.
- pkg - Ignored
bcoin*
files in npmignore have been removed. This fixes the npm install.
-
config - Options using megabyte units are now calculated properly again (6182df044228f9215938e7d314435f3f2640acca, a630d23a97b68f189a85105856fedc4e9e515754, 7728a0047053d4c368e60426e5fc7cc812d54caf).
-
address - Bech32 addresses are now supported (6acef06cbc87a3051ba238a2fb640562e718135e). This changes the semantics of the
Address
object: to support bech32,Address.fromBase58
calls should be replaced withAddress.fromString
. Likewise,addr.toBase58
calls should be replaced withaddr.toString
-
rpc -
getblockbyheight
is now exposed via JSON-RPC. It takes the same parameters as thegetblock
call, requiring a height instead of block hash (12d3ee6f9582afa9a3ba8984c63dcbc27b8db57e). -
bin -
bcoin --version
andbcoin --help
now exit with appropriate messages (f2f94a800e37c5dbdda6920fa6b85fbd485c212a). -
net - The p2p pool now exposes an
--only
option (a1d0948f2e528c5d77d6502659fafd064b1e693b). -
mempool - The mempool
indexAddress
option should now work correctly (fba9b46d253c19bbf8e662d9d75ab03dc9e20a78). -
rpc - JSON-RPC calls now properly cast booleans a la bitcoin core (dd49ee076196d2353783e3044185165dbac4aeb9).
-
rpc - Various RPC calls have been improved and updated (c78707ed6a71ce46c41c4e43ecb505e78a84b380, c1e684dc12d0a86573a905d54d4f81fce921987a, 5bde338a53117b1bd0fd92df0abc23d95180ab32).
-
rpc - Retroactive pruning is now available via the
pruneblockchain
call (f0bc6d5925419ba4a8289fa7828efc48ecc152d4). -
http - Getting block by height via the Rest API now works again (df4c8cc68c965bd818a5004354d2652751d4a702).
-
net - Peers who serve invalid orphans are now punished properly (0ceca23cb5a3d724c79c6bf587ede5d473df8486).
-
utils - An implementation of GCS filters is now supported (b994c278f25408886c3095d0c24123baaf07f78f).
-
http - The
witness
option is now properly exposed on the Rest API for segwit wallets (f04ad612b216becd35765c6e231f7820c7eee358). -
deps - Node.js >=v7.6.0 is now a required dependency (a0bd9680fed07c5eb37c227d160b0868f8adaf31).
-
build - The browser build has switched from browserify to webpack (19f236f74072d473123d20282d2119f6d9130458).
-
bcoin - The codebase has been rewritten to use all ES6 and ES7 features supported by node.js (aa05bb5df79d9a3af53060a4c0c066226f6e9e4c, 78d62c73b82e1953999d1cf80c90ed2035d4996e, e00472891df5934d8fc3aa63662f852816aa86b0, c53f4cf89e46d9de8ab7f65430310567558fe03f, 8c7279518f5341a2482a79ac98f0574468541edc).
-
workers - The worker pool has been refactored to solve the dissonance between the browser and node.js (27c60ce76e57af1695d78f912227d93194812c88).
-
net - DNS requests should now timeout sooner (647b6909c6d527eb82f6d789c88a23b2f8a60126).
-
http - Satoshi values are now required for all JSON serialization, as opposed to BTC strings which were used in the past (2f51fd1c5066f194a5a52383f4dd45497b882706).
-
bin - The
--no-wallet
flag is now exposed for nodes who want to run without a wallet. -
chain - BIP91 support is now exposed via the
--bip91
option. Thesegwit
andsegsignal
rules will be automatically added when callinggetblocktemplate
. To enable bip91 on an existing database--force-flags
must be passed. This will force bip91 to be enforced, but will not invalidate any previous blocks (bcoin cli reset [height]
may need to be used). -
chain - BIP148 support is now exposed via the
--bip148
option. This likewise must be enabled with--force-flags
on existing chains. This has the same potential "reset" implications as the--bip91
option.
This release contains a few non-backward-compatible changes.
Bcoin now requires node.js >=7.6.0 due to the use of ES6/7 features, and for the sake of more stability.
Bcoin's rest API now assumes satoshi values for in all JSON serialization. This is a breaking change for code that is not aware of it. All code which hits the rest API must be updated to use only satoshi values as opposed to BTC strings.
In other words, this:
{
"unconfirmed": "1.12",
"confirmed": "1.12"
}
Becomes this:
{
"unconfirmed": 112000000,
"confirmed": 112000000
}
- networks - Fixed simnet wpkh prefix.
- http -
wallet join
without wallet auth has been fixed for responses. This was causing a hanging issue with the client.
- networks - Simnet params have been fixed.
- cli - Chain reset call has been fixed.
- wallet/http - Create wallet route modified
(
POST /wallet/:id?
changed toPUT /wallet/:id
). - wallet/http - Create account route modified
(
POST /wallet/:id/account/:account?
changed toPUT /wallet/:id/account/:account
). - wallet/http -
auth
socket.io event name for wallet auth changed towallet auth
. - config -
payout-address
option was changed tocoinbase-address
. - node - Plugin system is now exposed. See #156.
- config - The internal API for the config object has been rewritten and is now more reusable, particularly by node plugins.
- http/rpc - Both the HTTPBase and RPCBase objects now allow "mounting" by other rpc and http servers.
- wallet - The wallet code has been completely removed from the node, and now resides entirely within one module. The wallet is exposed only as a plugin or a separate server.
- rpc -
prioritisetransaction
is now exposed properly (deltaFee
s are now tracked on mempool entries). - rpc - Proper
id
and error codes are now implemented. - rpc - Several
getblocktemplate
improvements have been implemented for more accuracy. e.g.curtime
will now be updated each call. - mining - The internal miner API has been rewritten, and now mimics stratum in a sense.
- chain - Faster verification with checkpoints.
- net - Fixed a potential block stalling issue.
- net - Hardcoded seeds for main added. Makes for better shipping with browsers.
- wsproxy/proxysocket - DNS resolution is no longer exposed.
- logger - Log files now trim to 20mb on boot.
- hostlist - A persistent
hosts
file is now written by default.
- mempool - Trimming now removes dependency chains by cumulative fee rate.
- mempool - Cumulative descendant fees are now updated properly when removing a transaction.
- net - Preliminary upnp support for adding port mappings.
- chain/mempool/miner - Various atomicity fixes and extra sanity checking.
- pool/peer - Peer height is now tracked and exposed on the RPC as
bestheight
.
- mempool - Fixed critical fee estimator bug causing throwing in the mempool.
- http - Always display spent coins in tx routes (e.g.
/tx/[txid]
). - mempool - An on-disk mempool is now exposed via
--persistent-mempool
(also makes fee data persistent). - chain -
chain.add
now takes aflags
parameter to avoid POW and non-contextual checks if necessary. - net - HostList is now potentially persistent with a
hostLocation
option. - net - Smarter stall behavior and timeouts.
- http - Better bitcoind compatability for JSON-RPC.
- miner - Better fee rate comparisons.
- deps - Upgrade deps, fix build on arm and windows.
- miner - Optimized TX sorting.
- rpc - Improved getblocktemplate to provide more accurate results to bitcoind.
- miner - Improved fee rate sorting.
- rpc - Fix incompatibilities in rpc api (getblocktemplate & submitblock).
- pool - Increase max header chain failures to 500 (prevents the initial sync from reverting to getblocks so frequently).
- wsproxy: Fixed proof of work handling in websocket proxy (43c491b).
- chain: Optimized MTP and network target calculations (1e07d1b).
- wallet: Implemented "smart" coin selection (304f0e7e).
- protocol: Increased default network fees for main (09c2357).
- http: Fix for handling
DELETE
http method (393dd5d). - miner: Improved handling of default reserved size and sigops (f2964e0 and 7104e4c).
- Initial tagged release.