Our wallet review process
We examine wallets starting at the code level and continue all the way up to the finished app that lives on your device. Provided below is an outline of each of these steps along with security tips for you and general test results.
Custody
Self-custodial: The user holds the keys
As part of our Methodology, we ask: Is the product self-custodial?
The answer is "yes". The user has control of their own keys.
Read more
Released
We could not determine when this product was originally released.
Passed all 10 tests
We answered the following questions in this order:
The answer is "yes".
If the answer was "no", we would mark it as "Few users" and the following would apply:
The answer is "no". We marked it as "Few users".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Few users" and the following would apply:
We focus on products that have the biggest impact if things go wrong and this one probably doesn’t have many users according to data publicly available.
The answer is "yes".
If the answer was "no", we would mark it as "Fake" and the following would apply:
The answer is "no". We marked it as "Fake".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Fake" and the following would apply:
The bigger wallets often get imitated by scammers that abuse the reputation of the product by imitating its name, logo or both.
Imitating a competitor is a huge red flag and we urge you to not put any money into this product!
The answer is "yes".
If the answer was "no", we would mark it as "Not a wallet" and the following would apply:
The answer is "no". We marked it as "Not a wallet".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Not a wallet" and the following would apply:
If it’s called “wallet” but is actually only a portfolio tracker, we don’t look any deeper, assuming it is not meant to control funds. What has no funds, can’t lose your coins. It might still leak your financial history!
If you can buy Bitcoins with this app but only into another wallet, it’s not a wallet itself.
The answer is "yes".
If the answer was "no", we would mark it as "A wallet but not for Bitcoin" and the following would apply:
The answer is "no". We marked it as "A wallet but not for Bitcoin".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "A wallet but not for Bitcoin" and the following would apply:
At this point we only look into wallets that at least also support BTC.
The answer is "yes".
If the answer was "no", we would mark it as "Can't send or receive bitcoins" and the following would apply:
The answer is "no". We marked it as "Can't send or receive bitcoins".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Can't send or receive bitcoins" and the following would apply:
If it is for holding BTC but you can’t actually send or receive them with this product then it doesn’t function like a wallet for BTC but you might still be using it to hold your bitcoins with the intention to convert back to fiat when you “cash out”.
All products in this category are custodial and thus funds are at the mercy of the provider.
The product cannot be independently verified. If the provider puts your funds at risk on purpose or by accident, you will probably not know about the issue before people start losing money. If the provider is more criminally inclined he might have collected all the backups of all the wallets, ready to be emptied at the press of a button. The product might have a formidable track record but out of distress or change in management turns out to be evil from some point on, with nobody outside ever knowing before it is too late.The answer is "yes".
If the answer was "no", we would mark it as "Custodial: The provider holds the keys" and the following would apply:
The answer is "no". We marked it as "Custodial: The provider holds the keys".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Custodial: The provider holds the keys" and the following would apply:
A custodial service is a service where the funds are held by a third party like the provider. The custodial service can at any point steal all the funds of all the users at their discretion. Our investigations stop there.
Some services might claim their setup is super secure, that they don’t actually have access to the funds, or that the access is shared between multiple parties. For our evaluation of it being a wallet, these details are irrelevant. They might be a trustworthy Bitcoin bank and they might be a better fit for certain users than being your own bank but our investigation still stops there as we are only interested in wallets.
Products that claim to be non-custodial but feature custodial accounts without very clearly marking those as custodial are also considered “custodial” as a whole to avoid misguiding users that follow our assessment.
This verdict means that the provider might or might not publish source code and maybe it is even possible to reproduce the build from the source code but as it is custodial, the provider already has control over the funds, so it is not a wallet where you would be in exclusive control of your funds.
We have to acknowledge that a huge majority of Bitcoiners are currently using custodial Bitcoin banks. If you do, please:
- Do your own research if the provider is trust-worthy!
- Check if you know at least enough about them so you can sue them when you have to!
- Check if the provider is under a jurisdiction that will allow them to release your funds when you need them?
- Check if the provider is taking security measures proportional to the amount of funds secured? If they have a million users and don’t use cold storage, that hot wallet is a million times more valuable for hackers to attack. A million times more effort will be taken by hackers to infiltrate their security systems.
The answer is "yes".
If the answer was "no", we would mark it as "No source for current release found" and the following would apply:
The answer is "no". We marked it as "No source for current release found".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "No source for current release found" and the following would apply:
A wallet that claims to not give the provider the means to steal the users’ funds might actually be lying. In the spirit of “Don’t trust - verify!” you don’t want to take the provider at his word, but trust that people hunting for fame and bug bounties could actually find flaws and back-doors in the wallet so the provider doesn’t dare to put these in.
Back-doors and flaws are frequently found in closed source products but some remain hidden for years. And even in open source security software there might be catastrophic flaws undiscovered for years.
An evil wallet provider would certainly prefer not to publish the code, as hiding it makes audits orders of magnitude harder.
For your security, you thus want the code to be available for review.
If the wallet provider doesn’t share up to date code, our analysis stops there as the wallet could steal your funds at any time, and there is no protection except the provider’s word.
“Up to date” strictly means that any instance of the product being updated without the source code being updated counts as closed source. This puts the burden on the provider to always first release the source code before releasing the product’s update. This paragraph is a clarification to our rules following a little poll.
We are not concerned about the license as long as it allows us to perform our analysis. For a security audit, it is not necessary that the provider allows others to use their code for a competing wallet. You should still prefer actual open source licenses as a competing wallet won’t use the code without giving it careful scrutiny.
The product cannot be independently verified. If the provider puts your funds at risk on purpose or by accident, you will probably not know about the issue before people start losing money. If the provider is more criminally inclined he might have collected all the backups of all the wallets, ready to be emptied at the press of a button. The product might have a formidable track record but out of distress or change in management turns out to be evil from some point on, with nobody outside ever knowing before it is too late.The answer is "yes".
If the answer was "no", we would mark it as "Failed to build from source provided!" and the following would apply:
The answer is "no". We marked it as "Failed to build from source provided!".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Failed to build from source provided!" and the following would apply:
Published code doesn’t help much if the app fails to compile.
We try to compile the published source code using the published build instructions into a binary. If that fails, we might try to work around issues but if we consistently fail to build the app, we give it this verdict and open an issue in the issue tracker of the provider to hopefully verify their app later.
The product cannot be independently verified. If the provider puts your funds at risk on purpose or by accident, you will probably not know about the issue before people start losing money. If the provider is more criminally inclined he might have collected all the backups of all the wallets, ready to be emptied at the press of a button. The product might have a formidable track record but out of distress or change in management turns out to be evil from some point on, with nobody outside ever knowing before it is too late.The answer is "yes".
If the answer was "no", we would mark it as "Not reproducible from source provided" and the following would apply:
The answer is "no". We marked it as "Not reproducible from source provided".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Not reproducible from source provided" and the following would apply:
Published code doesn’t help much if it is not what the published binary was built from. That is why we try to reproduce the binary. We
- obtain the binary from the provider
- compile the published source code using the published build instructions into a binary
- compare the two binaries
- we might spend some time working around issues that are easy to work around
If this fails, we might search if other revisions match or if we can deduct the source of the mismatch but generally consider it on the provider to provide the correct source code and build instructions to reproduce the build, so we usually open a ticket in their code repository.
In any case, the result is a discrepancy between the binary we can create and the binary we can find for download and any discrepancy might leak your backup to the server on purpose or by accident.
As we cannot verify that the source provided is the source the binary was compiled from, this category is only slightly better than closed source but for now we have hope projects come around and fix verifiability issues.
The product cannot be independently verified. If the provider puts your funds at risk on purpose or by accident, you will probably not know about the issue before people start losing money. If the provider is more criminally inclined he might have collected all the backups of all the wallets, ready to be emptied at the press of a button. The product might have a formidable track record but out of distress or change in management turns out to be evil from some point on, with nobody outside ever knowing before it is too late.Application build test result
Update 2023-05-04: Emanuel from WalletScrutiny and Stephan Oeste were able
to reproduce the latest release under certain conditions and specific
environment configs, which are slightly different from our script that still
fails to reproduce the product. We will further investigate what is causing
different build results in
this thread
but have to assume version 0.99.98g to match the published code. This product
is reproducible under certain circumstances.
Update 2023-04-30: The latest version of Samourai Wallet, 0.99.98g, is still
with the same result as 0.99.98f.
===== Begin Results =====
appId: com.samourai.wallet
signer: 6ab9471c21d2cddd628172975cff8ba23584da41c6962df074eb56e4ef08d990
apkVersionName: 0.99.98g
apkVersionCode: 194
verdict: nonverifiable
appHash: 2e67af86400d69ae3ecb8b05e57e960d481800c15cd68bb204537a093fee99c8
commit: 534b9ecde412d6ed3e5e33788371091f66f2cc3c
Diff:
Files /tmp/fromPlay_com.samourai.wallet_194/assets/dexopt/baseline.prof and /tmp/fromBuild_com.samourai.wallet_194/assets/dexopt/baseline.prof differ
Files /tmp/fromPlay_com.samourai.wallet_194/classes3.dex and /tmp/fromBuild_com.samourai.wallet_194/classes3.dex differ
Only in /tmp/fromPlay_com.samourai.wallet_194/META-INF: CERT.RSA
Only in /tmp/fromPlay_com.samourai.wallet_194/META-INF: CERT.SF
Only in /tmp/fromPlay_com.samourai.wallet_194/META-INF: MANIFEST.MF
Revision, tag (and its signature):
===== End Results =====
Update 2022-11-02: As of now, Samourai Wallet is the
product
with the most users so we gave it yet another spin for their version 0.99.98f
with hash 0a5711195d96f13f41a71107f1b1035505b33afd3a299828e43e9d1b5101e9c0
even though the related issue is closed without resolution. Compilation was easy:
$ git clone https://code.samourai.io/wallet/samourai-wallet-android.git
$ cd samourai-wallet-android/
$ git checkout 0.99.98f
$ podman run -it --rm -v$PWD:/mnt --workdir=/mnt walletscrutiny/android
root@374550c30c4e:/mnt# apt update; apt install openjdk-11-jdk; ./gradlew assembleRelease
As always, the binaries from Play Store and compilation do not match:
$ unzip -d fromDownload path/to/Samourai\ 0.99.98f\ \(com.samourai.wallet\).apk
$ unzip -d fromBuild app/build/outputs/apk/production/release/app-production-release-unsigned.apk
$ diff -r from*
Binary files fromBuild/assets/dexopt/baseline.prof and fromDownload/assets/dexopt/baseline.prof differ
Binary files fromBuild/classes2.dex and fromDownload/classes2.dex differ
Only in fromDownload/META-INF: CERT.RSA
Only in fromDownload/META-INF: CERT.SF
Only in fromDownload/META-INF: MANIFEST.MF
While the bottom three lines are expected due to the signature, the other two are not ok. This product is not verifiable.
Update 2021-10-07: Erik Nylund reached out to let us know of his failed attempt to reproduce this app. He wrote he also took a look at the Samourai Wallet v0.99.97a. It seems “the number of files is way smaller now but still quite a diff in classes2.dex”. He also sent a link to a log of his attempt.
Update 2021-08-02: Samourai is currently certainly not reproducible as it’s even not possible to build it - due to an issue reported two months ago. We tried to build it again, as the Samourai devs don’t get tired to lie about the project’s reproducibility and we tried it again.
Update 2021-03-02: Samourai claims to be on F-Droid, implying … what exactly? FDroid.org has very strict rules about code being open source but FDroid itself is also open source and allows to add secondary repositories that might apply different rules and standards and that’s exactly what’s happening here. FDroid.org does not list Samourai but the Copperhead FDroid repository apparently does. As long as the binary on Google Play is not the same as the one on Copperhead, the presence on Copperhead has no relevance to the security of the 100k users that downloaded the app from Google Play. Smoke and mirrors from Samourai as always.
Update 2020-08-02: Samourai claims
which is a direct claim of falsehood of our findings. No other neutral party supported this claim so far and neither did the provider explain how such a verification should work or where our findings are wrong. This is so far the clearest lie and thus red flag about this wallet.
Update 2019-12-27: The provider closed the issue we had opened on their repository.
Update 2019-12-16: Samourai tweeted in response to us:
@SamouraiWallet Replying to @BashCo_ deterministic builds have not been a priority or goal at this stage of dev using the resources we have. The goals we have focused on (privacy, dojo, whirlpool, etc) we have continued to deliver on. There is limited value in this investment without expert audits for each release
The original review:
Samourai is still “early access” which means that there are no Google ratings or comments.
Their website claims the wallet is non-custodial:
Be your own Swiss Bank Fully non custodial software ensures you are always in control of your private keys. No email address, no ID checks, and no hassle. Just install and go.
Given claims like:
We are privacy activists who have dedicated our lives to creating the software that Silicon Valley will never build, the regulators will never allow, and the VC’s will never invest in. We build the software that Bitcoin deserves.
we are not surprised to not find who is behind this wallet.
But the build instructions on their GitHub are fairly simple:
Import as Android Studio project. Should build “as is”.
so lets see what we get when we do this:
/tmp/$ git clone git@github.com:Samourai-Wallet/samourai-wallet-android.git
/tmp/$ cd samourai-wallet-android
/tmp/samourai-wallet-android$ git tag
0.81
0.99.27-gb
0.99.87
0.99.88
/tmp/samourai-wallet-android$ git checkout 0.99.88
We open the folder in Android Studio, set the Build Variants as follows:
and build the APK.
The following is the full output of diffoscope. Red lines are what the playstore
version misses compared to the self compiled version and green lines are
additions. Right in the beginning we see the expected lines:
META-INF/MANIFEST.MF is different, META-INF/CERT.RSA and META-INF/CERT.SF
are exclusive to the playstore version as should be.
The rest of the diff is what makes the build not verifiable.
We left all the diff
here
(The diff was part of the review itself but that caused issues on some browsers.)
for the more curious to investigate but it’s obviously
too much to consider acceptable like we might conclude if it was only the .png
files that were different.
Tests performed by Leo Wandersleb, Emanuel, Mohammad Rafigh
Disclaimer
Our Analysis is not a full code review! We plan to make code reviews available in the future but even then it will never be a stamp of approval but rather a list of incidents and questionable coding practice. Nasa sends probes to space that crash due to software bugs despite a huge budget and stringent scrutiny.
Do your own research
In addition to reading our analysis, it is important to do your own checks. Before transferring any bitcoin to your wallet, look up reviews for the wallet you want to use. They should be easy to find. If they aren't, that itself is a reason to be extra careful.