Our wallet review process
We examine wallets starting at the code level and continue all the way up to the finished app that lives on your device. Provided below is an outline of each of these steps along with security tips for you and general test results.
Custody
Self-custodial: The user holds the keys
As part of our Methodology, we ask: Is the product self-custodial?
The answer is "yes". The user has control of their own keys.
Read more
Released
11th November 2021
Application build
We could not verify that the provided code matches the binary!
See the last Issue we created.
Passed 9 of 10 tests
We answered the following questions in this order:
We stopped asking questions after we encountered a failed answer.
The answer is "yes".
If the answer was "no", we would mark it as "Few users" and the following would apply:
The answer is "no". We marked it as "Few users".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Few users" and the following would apply:
We focus on products that have the biggest impact if things go wrong and this one probably doesn’t have many users according to data publicly available.
The answer is "yes".
If the answer was "no", we would mark it as "Fake" and the following would apply:
The answer is "no". We marked it as "Fake".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Fake" and the following would apply:
The bigger wallets often get imitated by scammers that abuse the reputation of the product by imitating its name, logo or both.
Imitating a competitor is a huge red flag and we urge you to not put any money into this product!
The answer is "yes".
If the answer was "no", we would mark it as "Not a wallet" and the following would apply:
The answer is "no". We marked it as "Not a wallet".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Not a wallet" and the following would apply:
If it’s called “wallet” but is actually only a portfolio tracker, we don’t look any deeper, assuming it is not meant to control funds. What has no funds, can’t lose your coins. It might still leak your financial history!
If you can buy Bitcoins with this app but only into another wallet, it’s not a wallet itself.
The answer is "yes".
If the answer was "no", we would mark it as "A wallet but not for Bitcoin" and the following would apply:
The answer is "no". We marked it as "A wallet but not for Bitcoin".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "A wallet but not for Bitcoin" and the following would apply:
At this point we only look into wallets that at least also support BTC.
The answer is "yes".
If the answer was "no", we would mark it as "Can't send or receive bitcoins" and the following would apply:
The answer is "no". We marked it as "Can't send or receive bitcoins".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Can't send or receive bitcoins" and the following would apply:
If it is for holding BTC but you can’t actually send or receive them with this product then it doesn’t function like a wallet for BTC but you might still be using it to hold your bitcoins with the intention to convert back to fiat when you “cash out”.
All products in this category are custodial and thus funds are at the mercy of the provider.
The product cannot be independently verified. If the provider puts your funds at risk on purpose or by accident, you will probably not know about the issue before people start losing money. If the provider is more criminally inclined he might have collected all the backups of all the wallets, ready to be emptied at the press of a button. The product might have a formidable track record but out of distress or change in management turns out to be evil from some point on, with nobody outside ever knowing before it is too late.The answer is "yes".
If the answer was "no", we would mark it as "Custodial: The provider holds the keys" and the following would apply:
The answer is "no". We marked it as "Custodial: The provider holds the keys".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Custodial: The provider holds the keys" and the following would apply:
A custodial service is a service where the funds are held by a third party like the provider. The custodial service can at any point steal all the funds of all the users at their discretion. Our investigations stop there.
Some services might claim their setup is super secure, that they don’t actually have access to the funds, or that the access is shared between multiple parties. For our evaluation of it being a wallet, these details are irrelevant. They might be a trustworthy Bitcoin bank and they might be a better fit for certain users than being your own bank but our investigation still stops there as we are only interested in wallets.
Products that claim to be non-custodial but feature custodial accounts without very clearly marking those as custodial are also considered “custodial” as a whole to avoid misguiding users that follow our assessment.
This verdict means that the provider might or might not publish source code and maybe it is even possible to reproduce the build from the source code but as it is custodial, the provider already has control over the funds, so it is not a wallet where you would be in exclusive control of your funds.
We have to acknowledge that a huge majority of Bitcoiners are currently using custodial Bitcoin banks. If you do, please:
- Do your own research if the provider is trust-worthy!
- Check if you know at least enough about them so you can sue them when you have to!
- Check if the provider is under a jurisdiction that will allow them to release your funds when you need them?
- Check if the provider is taking security measures proportional to the amount of funds secured? If they have a million users and don’t use cold storage, that hot wallet is a million times more valuable for hackers to attack. A million times more effort will be taken by hackers to infiltrate their security systems.
The answer is "yes".
If the answer was "no", we would mark it as "No source for current release found" and the following would apply:
The answer is "no". We marked it as "No source for current release found".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "No source for current release found" and the following would apply:
A wallet that claims to not give the provider the means to steal the users’ funds might actually be lying. In the spirit of “Don’t trust - verify!” you don’t want to take the provider at his word, but trust that people hunting for fame and bug bounties could actually find flaws and back-doors in the wallet so the provider doesn’t dare to put these in.
Back-doors and flaws are frequently found in closed source products but some remain hidden for years. And even in open source security software there might be catastrophic flaws undiscovered for years.
An evil wallet provider would certainly prefer not to publish the code, as hiding it makes audits orders of magnitude harder.
For your security, you thus want the code to be available for review.
If the wallet provider doesn’t share up to date code, our analysis stops there as the wallet could steal your funds at any time, and there is no protection except the provider’s word.
“Up to date” strictly means that any instance of the product being updated without the source code being updated counts as closed source. This puts the burden on the provider to always first release the source code before releasing the product’s update. This paragraph is a clarification to our rules following a little poll.
We are not concerned about the license as long as it allows us to perform our analysis. For a security audit, it is not necessary that the provider allows others to use their code for a competing wallet. You should still prefer actual open source licenses as a competing wallet won’t use the code without giving it careful scrutiny.
The product cannot be independently verified. If the provider puts your funds at risk on purpose or by accident, you will probably not know about the issue before people start losing money. If the provider is more criminally inclined he might have collected all the backups of all the wallets, ready to be emptied at the press of a button. The product might have a formidable track record but out of distress or change in management turns out to be evil from some point on, with nobody outside ever knowing before it is too late.The answer is "yes".
If the answer was "no", we would mark it as "Failed to build from source provided!" and the following would apply:
The answer is "no". We marked it as "Failed to build from source provided!".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Failed to build from source provided!" and the following would apply:
Published code doesn’t help much if the app fails to compile.
We try to compile the published source code using the published build instructions into a binary. If that fails, we might try to work around issues but if we consistently fail to build the app, we give it this verdict and open an issue in the issue tracker of the provider to hopefully verify their app later.
The product cannot be independently verified. If the provider puts your funds at risk on purpose or by accident, you will probably not know about the issue before people start losing money. If the provider is more criminally inclined he might have collected all the backups of all the wallets, ready to be emptied at the press of a button. The product might have a formidable track record but out of distress or change in management turns out to be evil from some point on, with nobody outside ever knowing before it is too late.The answer is "yes".
If the answer was "no", we would mark it as "Not reproducible from source provided" and the following would apply:
The answer is "no". We marked it as "Not reproducible from source provided".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Not reproducible from source provided" and the following would apply:
Published code doesn’t help much if it is not what the published binary was built from. That is why we try to reproduce the binary. We
- obtain the binary from the provider
- compile the published source code using the published build instructions into a binary
- compare the two binaries
- we might spend some time working around issues that are easy to work around
If this fails, we might search if other revisions match or if we can deduct the source of the mismatch but generally consider it on the provider to provide the correct source code and build instructions to reproduce the build, so we usually open a ticket in their code repository.
In any case, the result is a discrepancy between the binary we can create and the binary we can find for download and any discrepancy might leak your backup to the server on purpose or by accident.
As we cannot verify that the source provided is the source the binary was compiled from, this category is only slightly better than closed source but for now we have hope projects come around and fix verifiability issues.
The product cannot be independently verified. If the provider puts your funds at risk on purpose or by accident, you will probably not know about the issue before people start losing money. If the provider is more criminally inclined he might have collected all the backups of all the wallets, ready to be emptied at the press of a button. The product might have a formidable track record but out of distress or change in management turns out to be evil from some point on, with nobody outside ever knowing before it is too late.Application build test result
Update 2024-11-19
To automate building Nunchuk Bitcoin Wallet, we had to not only update the wallet specific files io.nunchuk.android.sh
and io.nunchuk.android.dockerfile
but also drastically improved our general Android App Bundle (AAB) test script testAAB.sh
which now for the first time could find an AAB to be reproducible.
Summary of Differences
Contents of diff_armeabi_v7a.txt:
Binary files /tmp/test_io.nunchuk.android_1.9.53/fromPlay-unzipped/armeabi_v7a/AndroidManifest.xml and /tmp/test_io.nunchuk.android_1.9.53/fromBuild-unzipped/armeabi_v7a/AndroidManifest.xml differ
Only in /tmp/test_io.nunchuk.android_1.9.53/fromPlay-unzipped/armeabi_v7a: META-INF
Only in /tmp/test_io.nunchuk.android_1.9.53/fromPlay-unzipped/armeabi_v7a: stamp-cert-sha256
Contents of diff_base.txt:
Binary files /tmp/test_io.nunchuk.android_1.9.53/fromPlay-unzipped/base/AndroidManifest.xml and /tmp/test_io.nunchuk.android_1.9.53/fromBuild-unzipped/base/AndroidManifest.xml differ
Binary files /tmp/test_io.nunchuk.android_1.9.53/fromPlay-unzipped/base/res/xml/splits0.xml and /tmp/test_io.nunchuk.android_1.9.53/fromBuild-unzipped/base/res/xml/splits0.xml differ
Binary files /tmp/test_io.nunchuk.android_1.9.53/fromPlay-unzipped/base/resources.arsc and /tmp/test_io.nunchuk.android_1.9.53/fromBuild-unzipped/base/resources.arsc differ
Only in /tmp/test_io.nunchuk.android_1.9.53/fromPlay-unzipped/base: stamp-cert-sha256
Contents of diff_en.txt:
Binary files /tmp/test_io.nunchuk.android_1.9.53/fromPlay-unzipped/en/AndroidManifest.xml and /tmp/test_io.nunchuk.android_1.9.53/fromBuild-unzipped/en/AndroidManifest.xml differ
Only in /tmp/test_io.nunchuk.android_1.9.53/fromPlay-unzipped/en: META-INF
Binary files /tmp/test_io.nunchuk.android_1.9.53/fromPlay-unzipped/en/resources.arsc and /tmp/test_io.nunchuk.android_1.9.53/fromBuild-unzipped/en/resources.arsc differ
Only in /tmp/test_io.nunchuk.android_1.9.53/fromPlay-unzipped/en: stamp-cert-sha256
Contents of diff_xhdpi.txt:
Binary files /tmp/test_io.nunchuk.android_1.9.53/fromPlay-unzipped/xhdpi/AndroidManifest.xml and /tmp/test_io.nunchuk.android_1.9.53/fromBuild-unzipped/xhdpi/AndroidManifest.xml differ
Only in /tmp/test_io.nunchuk.android_1.9.53/fromPlay-unzipped/xhdpi: META-INF
Binary files /tmp/test_io.nunchuk.android_1.9.53/fromPlay-unzipped/xhdpi/resources.arsc and /tmp/test_io.nunchuk.android_1.9.53/fromBuild-unzipped/xhdpi/resources.arsc differ
Only in /tmp/test_io.nunchuk.android_1.9.53/fromPlay-unzipped/xhdpi: stamp-cert-sha256
We tried to follow the guidelines in this issue, which is currently a work-in-progress.
We take the size of stamp-cert-sha256
$ wc -c fromPlay-unzipped/{base,armeabi_v7a,en,xhdpi}/stamp-cert-sha256
32 fromPlay-unzipped/base/stamp-cert-sha256
32 fromPlay-unzipped/armeabi_v7a/stamp-cert-sha256
32 fromPlay-unzipped/en/stamp-cert-sha256
32 fromPlay-unzipped/xhdpi/stamp-cert-sha256
128 total
Diffoscope results
base
armeabi_v7a
en
xhdpi
The pattern of files match the exceptions noted in the issue mentioned above.
We note that there are too many diffs to make this version reproducibile. Therefore, version 1.9.53 is nonverifiable
- We are documenting these diffs in our wiki: Excerpts in Reproducibility
Previous Review 2024-11-07 Reproducible verification for version 1.9.53
We followed the instructions from the provider regarding their reproducibility verification steps.
Using their apkdiff.py
Nunchuk has their own custom Python script to ascertain whether the built vs the device apks match or not. These were the results:
dannybuntu@MS-7978:~/nunchuk-android/reproducible-builds$ ./apkdiff.py ../apks/built-apks/splits/base-armeabi_v7a.apk ../apks/device-apks/split_config.armeabi_v7a.apk
APKs are the same!
dannybuntu@MS-7978:~/nunchuk-android/reproducible-builds$ ./apkdiff.py ../apks/built-apks/splits/base-xhdpi.apk ../apks/device-apks/split_config.xhdpi.apk
APKs are the same!
dannybuntu@MS-7978:~/nunchuk-android/reproducible-builds$ ./apkdiff.py ../apks/built-apks/splits/base-en.apk ../apks/device-apks/split_config.en.apk
APKs are the same!
dannybuntu@MS-7978:~/nunchuk-android/reproducible-builds$ ./apkdiff.py ../apks/built-apks/splits/base-master.apk ../apks/device-apks/base.apk
APK file classes.dex does not match
APKs are different!
We noticed that apkdiff.py excluded the following files:
def compareApkAndBundle(first, second):
FILES_TO_IGNORE = [
"resources.arsc",
"stamp-cert-sha256",
"assets/dexopt/baseline.prof",
"assets/dexopt/baseline.profm",
"AndroidManifest.xml",
]
def compareApks(first, second):
FILES_TO_IGNORE = [
"META-INF/MANIFEST.MF",
"META-INF/CERT.RSA",
"META-INF/CERT.SF",
"META-INF/BNDLTOOL.SF",
"META-INF/BNDLTOOL.RSA",
"stamp-cert-sha256",
"resources.arsc",
"res/xml/splits0.xml",
"AndroidManifest.xml",
"assets/dexopt/baseline.prof",
"assets/dexopt/baseline.profm",
]
We took the initiative to file an issue regarding these files.
Proceeding with unzipping the apks, normalizing their names and running diff -r
armeabi_v7a
dannybuntu@MS-7978:~/nunchuk-android/apks$ diff -r built-apks/armeabi_v7a/ device-apks/armeabi_v7a/
Binary files built-apks/armeabi_v7a/AndroidManifest.xml and device-apks/armeabi_v7a/AndroidManifest.xml differ
Only in device-apks/armeabi_v7a/: META-INF
Only in device-apks/armeabi_v7a/: stamp-cert-sha256
base
$ diff -r built-apks/base/ device-apks/base/
Binary files built-apks/base/AndroidManifest.xml and device-apks/base/AndroidManifest.xml differ
Binary files built-apks/base/assets/dexopt/baseline.prof and device-apks/base/assets/dexopt/baseline.prof differ
Binary files built-apks/base/classes2.dex and device-apks/base/classes2.dex differ
Binary files built-apks/base/classes3.dex and device-apks/base/classes3.dex differ
Binary files built-apks/base/classes4.dex and device-apks/base/classes4.dex differ
Binary files built-apks/base/classes5.dex and device-apks/base/classes5.dex differ
Binary files built-apks/base/classes6.dex and device-apks/base/classes6.dex differ
Binary files built-apks/base/classes.dex and device-apks/base/classes.dex differ
Binary files built-apks/base/res/xml/splits0.xml and device-apks/base/res/xml/splits0.xml differ
Binary files built-apks/base/resources.arsc and device-apks/base/resources.arsc differ
Only in device-apks/base/: stamp-cert-sha256
en
$ diff -r built-apks/en/ device-apks/en/
Binary files built-apks/en/AndroidManifest.xml and device-apks/en/AndroidManifest.xml differ
Only in device-apks/en/: META-INF
Binary files built-apks/en/resources.arsc and device-apks/en/resources.arsc differ
Only in device-apks/en/: stamp-cert-sha256
xhdpi
$ diff -r built-apks/xhdpi/ device-apks/xhdpi/
Binary files built-apks/xhdpi/AndroidManifest.xml and device-apks/xhdpi/AndroidManifest.xml differ
Only in device-apks/xhdpi/: META-INF
Binary files built-apks/xhdpi/resources.arsc and device-apks/xhdpi/resources.arsc differ
Only in device-apks/xhdpi/: stamp-cert-sha256
Until we have finalized the list of files that can be considered as excluded from diffs, the presence of multiple diffs, particularly in base-master.apk, such as:
- classes2.dex
- classes3.dex
- classes4.dex
- classes5.dex
- classes6.dex
would render version 1.9.53 as non-verifiable
Hashes of the APKs
dannybuntu@MS-7978:~/nunchuk-android/apks/built-apks/splits$ sha256sum *.apk
aaec6e500babbd1931db8485b99205468e426f6157df131f607aa69b6e821708 base-armeabi_v7a.apk
92e6c7ab6bd7335f9012cace0a71d2339b559cfa0a1ae56a798fb226bd676e83 base-en.apk
ff7a34f14d304b27991b00cb0148dbbb508108d64255e64047a2047865d5ac9e base-master.apk
72029da7b4d23eea8ddc7477ea6c618961e3c54971b1c257c549c62014396657 base-xhdpi.apk
dannybuntu@MS-7978:~/nunchuk-android/apks/device-apks$ sha256sum *.apk
59777895e5cc335505d9917dcbd71bd22162affae88d926c5ed92fee5216de08 base.apk
b743c962485d546ec74d8c7a21d7658d111fd5c61da465cefef188d82527cc6f split_config.armeabi_v7a.apk
1f8a07887f49898030894acfd47447b2fd56b7395b6c926ec2756427487d5252 split_config.en.apk
26fb6d2b6775d7a1044ccabfbc0c071ce22ba1095d4e3d5a664e1691dae07209 split_config.xhdpi.apk
We have updated the issue with this note.
Previous Review 2023-07-05: We have added a test script to check the reproducibility of Nunchuk 1.9.32. Unfortunately the result has diff on a lot of files, Which is also reported in an issue here. So the app is still not verifiable.
The app’s version for Android was released as open source under the GPLv3 here.
It’s build instructions are:
- Follow the build instructions for Nunchuk Android Native SDK.
- Publish the SDK to the local maven. Note the SDK version number.
- Open dependencies.gradle and update nativeSdkVersion to the SDK version you just published.
- Build and run the app on your device.
So, what are the “build instructions for Nunchuk Android Native SDK”? First of all, Xcode is required. That’s a bit of an issue for us, as all contributors who do builds do not have a Mac and Xcode only works on Mac. It’s not immediately clear if it really is required, so lets look further …
git submodule add --force -b main https://gitlab.com/nunchuck/libnunchuk.git
but the user nunchuck doesn’t exist on gitlab.com.
As Nunchuk Android’s code is on GitHub, we can guess the right repo might be nunchuk-io/libnunchuk.
Which is weird is that the repo’s git submodule also still points to GitLab, so the error is not only in the documentation.
Let’s try:
$ git clone https://github.com/nunchuk-io/nunchuk-android-nativesdk.git
$ cd nunchuk-android-nativesdk/
$ cat .gitmodules
[submodule "src/main/native/libnunchuk"]
path = src/main/native/libnunchuk
url = https://gitlab.com/nunchuck/libnunchuk.git
branch = main
$ git submodule update --init --recursive
Submodule 'src/main/native/libnunchuk' (https://gitlab.com/nunchuck/libnunchuk.git) registered for path 'src/main/native/libnunchuk'
Cloning into '/home/leo/tmp/nunchuk-android-nativesdk/src/main/native/libnunchuk'...
Username for 'https://gitlab.com':
This repo isn’t a public repo but as it might be private, git asks for a login. As discussed above, we’ll use libnunchuk from GitHub:
$ git submodule set-url src/main/native/libnunchuk https://github.com/nunchuk-io/libnunchuk.git
$ git submodule update --init --recursive
Cloning into '/home/leo/tmp/nunchuk-android-nativesdk/src/main/native/libnunchuk'...
Submodule path 'src/main/native/libnunchuk': checked out 'c168cf715cbe768b5cd5004609f2db6d0ebfe254'
...
error: object e35e28f52d20df27561b2780f6b9c86669a9de21: zeroPaddedFilemode: contains zero-padded file modes
fatal: fsck error in packed object
fatal: index-pack failed
fatal: clone of 'https://github.com/sqlcipher/sqlcipher' into submodule path '/home/leo/tmp/nunchuk-android-nativesdk/src/main/native/libnunchuk/contrib/sqlcipher' failed
Failed to clone 'contrib/sqlcipher'. Retry scheduled
So … we can’t clone that dependency. Seriously?
$ git clone https://github.com/sqlcipher/sqlcipher
Cloning into 'sqlcipher'...
remote: Enumerating objects: 15498, done.
remote: Counting objects: 100% (1910/1910), done.
remote: Compressing objects: 100% (816/816), done.
error: object e35e28f52d20df27561b2780f6b9c86669a9de21: zeroPaddedFilemode: contains zero-padded file modes
fatal: fsck error in packed object
fatal: index-pack failed
So for now we give up at this point and file this product as not verifiable.
Update 2023-01-06: As laid out in this issue, we managed to compile the app but with substantial differences:
Files ./GooglePlay/classes2.dex and ./LocalBuild/classes2.dex differ
Files ./GooglePlay/classes3.dex and ./LocalBuild/classes3.dex differ
Files ./GooglePlay/classes4.dex and ./LocalBuild/classes4.dex differ
Files ./GooglePlay/classes5.dex and ./LocalBuild/classes5.dex differ
Files ./GooglePlay/classes.dex and ./LocalBuild/classes.dex differ
Files ./GooglePlay/lib/arm64-v8a/libnunchuk-android.so and ./LocalBuild/lib/arm64-v8a/libnunchuk-android.so differ
Until these issues are resolved, the app is not verifiable.
Tests performed by Leo Wandersleb, Emanuel, mohammad, Daniel Andrei R. Garcia
Previous application build tests
5th July 2023 | 1.9.32 | |
6th January 2023 | 1.9.23 | |
15th December 2021 | 1.9.21 |
Do your own research
In addition to reading our analysis, it is important to do your own checks. Before transferring any bitcoin to your wallet, look up reviews for the wallet you want to use. They should be easy to find. If they aren't, that itself is a reason to be extra careful.