2013 wallet.dat fails to load on version 29

[takuma sato]

Some have been claiming that it's very easy to migrate old wallets into new versions. but Im already seeing this is not as easy as advertised. Did the test by running Bitcoin 0.8.6 on a Tails live session. The file is hosted in bitcoin.org and the SHA256SUMS.asc file has a valid signature by Gavin Andresen. So I created a wallet.dat file, created some addresses, saved the wallet.dat file and moved it into my fully synced Knots 29 node. I tried opening it with just the GUI and it gets stuck loading forever. I then tried to open it with the bitcoin-qt's console with loadwallet wallet.dat (I placed it inside "wallets" folder where I think it's the modern default spot to place wallets) and it returned this error:

Code:

loadwallet wallet.dat

Wallet file verification failed. "/home/takuma/software/.bitcoin/wallets/wallet.dat" corrupt. Try using the wallet tool bitcoin-wallet to salvage or restoring a backup. (code -4)

This action generated these 3 files:

/.bitcoin/wallets/database/log.0000000001 (a binary file of 1MB)
/.bitcoin/wallets/db.log
/.bitcoin/.walletlock (empty file)

db.log contains this:

file unknown has LSN [some number], past end of log at [some number]
Commonly caused by moving a database from one database environment
to another without clearing the database LSNs, or by removing all of
the log files from a database environment
Page 0: metadata page corrupted
Page 0: could not check metadata page
wallet.dat: DB_VERIFY_BAD: Database verification failed

I really doubt the file got corrupted, I created it yesterday. I think this is just bad backwards compatibility. So it doesn't sound as easy as just drop your old wallet file into the latest version and that's it. Now im not sure what are the necessary steps to get it working.

If you want to recreate this experiment, you can download the binaries here and try generating a wallet.dat file then try to load it into a modern version:

https://bitcoin.org/bin/insecure/bitcoin-core-0.8.6/

If you cannot run it because it lacks qt4 libraries, you'll have to download them:

https://archive.debian.org/debian/pool/main/q/qt4-x11/

Specifically, get these:
https://archive.debian.org/debian/pool/main/q/qt4-x11/libqt4-core_4.8.2+dfsg-11_amd64.deb
https://archive.debian.org/debian/pool/main/q/qt4-x11/libqt4-network_4.8.2+dfsg-11_amd64.deb
https://archive.debian.org/debian/pool/main/q/qt4-x11/libqt4-gui_4.8.2+dfsg-11_amd64.deb
I think it worked with these 3, but if there any missing files then the linux shell should point to them, just download them.

Place them properly there:

Code:

bin/64/qt4bundle/lib/
bin/64/qt4bundle/plugins/

Then run bitcoin-qt calling those libraries:

Code:

LD_LIBRARY_PATH=$PWD/qt4bundle/lib:$LD_LIBRARY_PATH QT_PLUGIN_PATH=$PWD/qt4bundle/plugins ./bitcoin-qt 

Then you can test and try to actually load an old walet.dat into a new version.

I have not tried with the actual wallet.dat files but it's not looking great when it comes to expecting a smooth backwards compatibility experience.

[achow101]

saved the wallet.dat file

Please clarify what you actually did, as there is no "save" action.

This is extremely unlikely to be a backwards compatibility issue. Looking at the log snippet you provided, it is highly likely that the error is a result of only copying the wallet.dat file. Contrary to popular belief, the wallet.dat file is not the only file you have to copy when copying a wallet from one node to another. You also have to copy that database folder.

This is especially true for older versions of Bitcoin Core as shutdown did not always compact everything into the wallet.dat file. Furthermore, copying the wallet.dat without unloading the wallet (or stopping Bitcoin Core) will result in data loss, and therefore detected corruption, because the log files in the database directory contains data before it gets committed into the wallet.dat file.

This behavior is also one of the reasons that we wanted to move off of BDB. SQLite can be configured to specifically not do that and to always write into the wallet.dat file.

[takuma sato]

saved the wallet.dat file

Please clarify what you actually did, as there is no "save" action.

By saving I meant, I exited 0.8.6 bitcoin-qt, copied the wallet.dat and then pasted it into the hard drive to open it with the knots 29 bitcoin-qt.

This is extremely unlikely to be a backwards compatibility issue. Looking at the log snippet you provided, it is highly likely that the error is a result of only copying the wallet.dat file. Contrary to popular belief, the wallet.dat file is not the only file you have to copy when copying a wallet from one node to another. You also have to copy that database folder.

This is especially true for older versions of Bitcoin Core as shutdown did not always compact everything into the wallet.dat file. Furthermore, copying the wallet.dat without unloading the wallet (or stopping Bitcoin Core) will result in data loss, and therefore detected corruption, because the log files in the database directory contains data before it gets committed into the wallet.dat file.

This behavior is also one of the reasons that we wanted to move off of BDB. SQLite can be configured to specifically not do that and to always write into the wallet.dat file.

I've never heard you needed the chainstate files. I may or not have a copy of the chainstate files of the old Bitcoin Core install where the actual wallet with private keys came from. It may still be there in some old Windows install of Bitcoin Core. Im assuming there is a way to salvage the private keys from the wallet.dat without chainstate files otherwise I would be screwed.

Im hoping this is a particular thing that happened by doing this on a Tails live session, perhaps something went wrong there. I will try with the real wallet files on the airgap computer just in case, if I get the same error then im going to need some good ideas in case I cannot find the chainstate files from whatever install the wallet was last used in. Would actually be insane you need these files to backup your keys but I will try to find them anyway.

[achow101]

I've never heard you needed the chainstate files.

I did not say chainstate files.

The files in database/ are not for the chainstate, they are the BDB log files and used for Write-ahead logging.

Im assuming there is a way to salvage the private keys from the wallet.dat without chainstate files otherwise I would be screwed.

There is no such guarantee. The log files will contain data that has not been written to the database file yet. Depending on when the wallet.dat was copied, the data may not yet exist in the database file and therefore losing the log files is catastrophic data loss.

Would actually be insane you need these files to backup your keys but I will try to find them anyway.

Would be insane to continue to use a database system that requires that.

[LoyceV]

Contrary to popular belief, the wallet.dat file is not the only file you have to copy when copying a wallet from one node to another. You also have to copy that database folder.

This may explain why I've seen countless topics over the years from people who can't open their old wallet.dat because it's corrupted.

[takuma sato]

I've never heard you needed the chainstate files.

I did not say chainstate files.

The files in database/ are not for the chainstate, they are the BDB log files and used for Write-ahead logging.

Im assuming there is a way to salvage the private keys from the wallet.dat without chainstate files otherwise I would be screwed.

There is no such guarantee. The log files will contain data that has not been written to the database file yet. Depending on when the wallet.dat was copied, the data may not yet exist in the database file and therefore losing the log files is catastrophic data loss.

Would actually be insane you need these files to backup your keys but I will try to find them anyway.

Would be insane to continue to use a database system that requires that.

Yeah I meant the database folder. Well the thing is, I managed to go into the old drive that had the full node, ages ago I was running a Windows node, and I kept updating that one. The last version I have saved is binaries for 0.16 so I assume that is what the bitcoin folder was running. There's a bunch of wallets from different years backed up. The last wallet file is from 2019. What I noticed is that there doesn't seem to be a database folder on the main folder, there is a database folder in the testnet folder but not on the main datadir folder. I don't remember deleting any folders so that's weird. I'll have to test those files, hopefully the most recent backups work.

[achow101]

The last wallet file is from 2019. What I noticed is that there doesn't seem to be a database folder on the main folder,

It depends on the version that was last in use. Newer versions would ensure that all the data from the log files ended up in the wallet.dat file, then delete the database folder, on shutdown. This would also happen periodically during operation. However, older versions and unclean shutdowns would still have the database folder.

[takuma sato]

The last wallet file is from 2019. What I noticed is that there doesn't seem to be a database folder on the main folder,

It depends on the version that was last in use. Newer versions would ensure that all the data from the log files ended up in the wallet.dat file, then delete the database folder, on shutdown. This would also happen periodically during operation. However, older versions and unclean shutdowns would still have the database folder.

Im looking at the debug.log and the last times I opened Bitcoin Core the version it says it was v0.19.0.1

Lines that have to do with the wallet:

init message: Verifying wallet(s)...
Using BerkeleyDB version Berkeley DB 4.8.30: (April 9 2010)
Using wallet C:\Users\takuma\AppData\Roaming\Bitcoin
BerkeleyEnvironment::Open: LogDir=C:\Users\takuma\AppData\Roaming\Bitcoin\database ErrorFile=C:\Users\takuma\AppData\Roaming\Bitcoin\db.log

some other lines here about blocks ad chainstate like "Opened LevelDB successfully" etc then

No coin database inconsistencies in the last 6 blocks

init message: Loading walet....
BerkeleyEnvironment::Open: LogDir=C:\Users\takuma\AppData\Roaming\Bitcoin\database ErrorFile=C:\Users\takuma\AppData\Roaming\Bitcoin\db.log
[default wallet] Wallet File Version = 60000
[default wallet] Keys: 0 plaintext, 1000+ encrypted, 1000+ w/ metadata, 1000+ total (im typing this by hand looking at the screen so im not putting exact numbers) Unknown wallet records: 1
[default wallet] Wallet completed loagin
init message : Rescanning...

Here it starts scanning and I have some AddToWallet new

So I was opening the node and the date says 2020 so I was still using the wallet there and reeving transactions... but there is no database folder, not even a file inside the database folder.

The testnet3 folder has a database folder with log. It also has the "ErrorFile=C:\Users\takuma\AppData\Roaming\Bitcoin\testnet3\db.log thing"

But for testnet3 the last version on the debug used was 0.15.1

So I don't get it, as I never did any manual updates of the wallet format. So either the wallet.dat will work without the database thing because Bitcoin Core v0.19.0.1 is modern enough to not need it, or somehow the database folder disappeared.

The last lines do not seem like there was an error during closing:

addcon: thread exit
Shutdown: In progress
torcontrol thread exit
net thread exit
msghand thread exit
opencon thread exit
scheluder thread interrupt
dumped mempool
[default wallet] releasing wallet
Shutdown: done

[achow101]

or somehow the database folder disappeared.

As I said, on a clean shutdown, the database folder is deleted by newer versions. 0.19 is new enough.

[apogio]

Sorry for jumping in, but as a much newer bitcoiner (I started 3-4 years ago), what is the database directory?

According to @achow101 it's deleted since version 0.19 and onwards, but what was its purpose? What did it include?

And more importantly, for newer versions, isn't it true that we only need to backup the wallet.dat files?

[LoyceV]

what is the database directory?

This one:

Code:

~/.bitcoin/database

According to @achow101 it's deleted since version 0.19 and onwards

He wrote it gets deleted on (clean) shutdown. I have that directory.

[apogio]

This one:

Code:

~/.bitcoin/database

He wrote it gets deleted on (clean) shutdown. I have that directory.

I'm running Core 29.1 but I don't have it. I guess it's an old directory that's not used anymore? The reason I'm asking is that I wanna make sure that keeping simply the wallet.dat for backup is enough.

[takuma sato]

or somehow the database folder disappeared.

As I said, on a clean shutdown, the database folder is deleted by newer versions. 0.19 is new enough.

Yes, I have observed this behavior, and now it makes sense. It explains why there is no database folder. The wallets work when I loaded them in Knots 29, with labels and all. The HD logo has a crossing line on it indicating it's not an HD. So the experiment may not have worked because Tails did not properly shut down the wallet and I also did not copy the database folder. But the actual wallet did work thankfully, now I need to plan on a watch-only setup. I will also test the migration tool with some backup.

An observation: When I loaded the wallet.dat file on Knots 29, the sha256 checksum changed. What changes are made to the original file?

This one:

Code:

~/.bitcoin/database

He wrote it gets deleted on (clean) shutdown. I have that directory.

I'm running Core 29.1 but I don't have it. I guess it's an old directory that's not used anymore? The reason I'm asking is that I wanna make sure that keeping simply the wallet.dat for backup is enough.

When you load a wallet, you should see a database folder, but when you close it, it disappears. If you don't see it, then im assuming this behaviour happens with dbd (berkeley) wallets and not sqlite wallets (more modern ones) which never need that database file.

And perhaps really old ones (2013) but that never went beyond the old versions (because I did updates on Bitcoin Core, I assume some updates were applied along the way on the wallet file as I updated) always need that database folder and it's never deleted? Not sure about that. On the experiment I did with the 0.8.x version, there was a database folder, even after closing it. I did not see any errors, but I think I also didn't the see the "Bitcoin is clossing please wait..." message, but I don't remember if that message was there back then. But doing this on Tails may not be reliable anyway, I just didn't want to install these binaries so used Tails for it.

[achow101]

I'm running Core 29.1 but I don't have it. I guess it's an old directory that's not used anymore?

It is still used if you use a legacy wallet. The directory is always located in the same directory as the wallet.dat file which may be different on your computer.

The reason I'm asking is that I wanna make sure that keeping simply the wallet.dat for backup is enough.

This whole discussion is about how it isn't.

Frankly, backing up a wallet by going to the filesystem is not recommended (or really supported) method of backup because of these kinds of issues. The only recommended way to backup a wallet is to use the backupwallet RPC or File > Backup Wallet in the GUI.

[apogio]

When you load a wallet, you should see a database folder, but when you close it, it disappears. If you don't see it, then im assuming this behaviour happens with dbd (berkeley) wallets and not sqlite wallets (more modern ones) which never need that database file.

Yeah, I figured, thanks for clarifying.

This whole discussion is about how it isn't.

Frankly, backing up a wallet by going to the filesystem is not recommended (or really supported) method of backup because of these kinds of issues. The only recommended way to backup a wallet is to use the backupwallet RPC or File > Backup Wallet in the GUI.

That's exactly how I'm doing it (using the GUI version). The output of this command (i.e what's actually saved) is a single wallet.dat file. When I read your post above, talking about backing up the database directory, I felt like asking, just for clarity.