Our wallet review process
We examine wallets starting at the code level and continue all the way up to the finished app that lives on your device. Provided below is an outline of each of these steps along with security tips for you and general test results.
Custody
Self-custodial: The user holds the keys
As part of our Methodology, we ask: Is the product self-custodial?
The answer is "yes". The user has control of their own keys.
Read more
Source code
Released
2nd August 2018
Application build
Without public source of the reviewed release available, this product cannot be verified!
See test resultPassed 5 of 9 tests
We answered the following questions in this order:
We stopped asking questions after we encountered a failed answer.
The answer is "yes".
If the answer was "no", we would mark it as "Fake" and the following would apply:
The answer is "no". We marked it as "Fake".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Fake" and the following would apply:
The bigger wallets often get imitated by scammers that abuse the reputation of the product by imitating its name, logo or both.
Imitating a competitor is a huge red flag and we urge you to not put any money into this product!
The answer is "yes".
If the answer was "no", we would mark it as "Not a wallet" and the following would apply:
The answer is "no". We marked it as "Not a wallet".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Not a wallet" and the following would apply:
If it’s called “wallet” but is actually only a portfolio tracker, we don’t look any deeper, assuming it is not meant to control funds. What has no funds, can’t lose your coins. It might still leak your financial history!
If you can buy Bitcoins with this app but only into another wallet, it’s not a wallet itself.
The answer is "yes".
If the answer was "no", we would mark it as "A wallet but not for Bitcoin" and the following would apply:
The answer is "no". We marked it as "A wallet but not for Bitcoin".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "A wallet but not for Bitcoin" and the following would apply:
At this point we only look into wallets that at least also support BTC.
The answer is "yes".
If the answer was "no", we would mark it as "Can't send or receive bitcoins" and the following would apply:
The answer is "no". We marked it as "Can't send or receive bitcoins".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Can't send or receive bitcoins" and the following would apply:
If it is for holding BTC but you can’t actually send or receive them with this product then it doesn’t function like a wallet for BTC but you might still be using it to hold your bitcoins with the intention to convert back to fiat when you “cash out”.
All products in this category are custodial and thus funds are at the mercy of the provider.
The product cannot be independently verified. If the provider puts your funds at risk on purpose or by accident, you will probably not know about the issue before people start losing money. If the provider is more criminally inclined he might have collected all the backups of all the wallets, ready to be emptied at the press of a button. The product might have a formidable track record but out of distress or change in management turns out to be evil from some point on, with nobody outside ever knowing before it is too late.The answer is "yes".
If the answer was "no", we would mark it as "Custodial: The provider holds the keys" and the following would apply:
The answer is "no". We marked it as "Custodial: The provider holds the keys".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Custodial: The provider holds the keys" and the following would apply:
A custodial service is a service where the funds are held by a third party like the provider. The custodial service can at any point steal all the funds of all the users at their discretion. Our investigations stop there.
Some services might claim their setup is super secure, that they don’t actually have access to the funds, or that the access is shared between multiple parties. For our evaluation of it being a wallet, these details are irrelevant. They might be a trustworthy Bitcoin bank and they might be a better fit for certain users than being your own bank but our investigation still stops there as we are only interested in wallets.
Products that claim to be non-custodial but feature custodial accounts without very clearly marking those as custodial are also considered “custodial” as a whole to avoid misguiding users that follow our assessment.
This verdict means that the provider might or might not publish source code and maybe it is even possible to reproduce the build from the source code but as it is custodial, the provider already has control over the funds, so it is not a wallet where you would be in exclusive control of your funds.
We have to acknowledge that a huge majority of Bitcoiners are currently using custodial Bitcoin banks. If you do, please:
- Do your own research if the provider is trust-worthy!
- Check if you know at least enough about them so you can sue them when you have to!
- Check if the provider is under a jurisdiction that will allow them to release your funds when you need them?
- Check if the provider is taking security measures proportional to the amount of funds secured? If they have a million users and don’t use cold storage, that hot wallet is a million times more valuable for hackers to attack. A million times more effort will be taken by hackers to infiltrate their security systems.
The answer is "yes".
If the answer was "no", we would mark it as "No source for current release found" and the following would apply:
The answer is "no". We marked it as "No source for current release found".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "No source for current release found" and the following would apply:
A wallet that claims to not give the provider the means to steal the users’ funds might actually be lying. In the spirit of “Don’t trust - verify!” you don’t want to take the provider at his word, but trust that people hunting for fame and bug bounties could actually find flaws and back-doors in the wallet so the provider doesn’t dare to put these in.
Back-doors and flaws are frequently found in closed source products but some remain hidden for years. And even in open source security software there might be catastrophic flaws undiscovered for years.
An evil wallet provider would certainly prefer not to publish the code, as hiding it makes audits orders of magnitude harder.
For your security, you thus want the code to be available for review.
If the wallet provider doesn’t share up to date code, our analysis stops there as the wallet could steal your funds at any time, and there is no protection except the provider’s word.
“Up to date” strictly means that any instance of the product being updated without the source code being updated counts as closed source. This puts the burden on the provider to always first release the source code before releasing the product’s update. This paragraph is a clarification to our rules following a little poll.
We are not concerned about the license as long as it allows us to perform our analysis. For a security audit, it is not necessary that the provider allows others to use their code for a competing wallet. You should still prefer actual open source licenses as a competing wallet won’t use the code without giving it careful scrutiny.
The product cannot be independently verified. If the provider puts your funds at risk on purpose or by accident, you will probably not know about the issue before people start losing money. If the provider is more criminally inclined he might have collected all the backups of all the wallets, ready to be emptied at the press of a button. The product might have a formidable track record but out of distress or change in management turns out to be evil from some point on, with nobody outside ever knowing before it is too late.The answer is "yes".
If the answer was "no", we would mark it as "Failed to build from source provided!" and the following would apply:
The answer is "no". We marked it as "Failed to build from source provided!".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Failed to build from source provided!" and the following would apply:
Published code doesn’t help much if the app fails to compile.
We try to compile the published source code using the published build instructions into a binary. If that fails, we might try to work around issues but if we consistently fail to build the app, we give it this verdict and open an issue in the issue tracker of the provider to hopefully verify their app later.
The product cannot be independently verified. If the provider puts your funds at risk on purpose or by accident, you will probably not know about the issue before people start losing money. If the provider is more criminally inclined he might have collected all the backups of all the wallets, ready to be emptied at the press of a button. The product might have a formidable track record but out of distress or change in management turns out to be evil from some point on, with nobody outside ever knowing before it is too late.The answer is "yes".
If the answer was "no", we would mark it as "Not reproducible from source provided" and the following would apply:
The answer is "no". We marked it as "Not reproducible from source provided".
We did not ask this question because we failed at a previous question.
If the answer was "no", we would mark it as "Not reproducible from source provided" and the following would apply:
Published code doesn’t help much if it is not what the published binary was built from. That is why we try to reproduce the binary. We
- obtain the binary from the provider
- compile the published source code using the published build instructions into a binary
- compare the two binaries
- we might spend some time working around issues that are easy to work around
If this fails, we might search if other revisions match or if we can deduct the source of the mismatch but generally consider it on the provider to provide the correct source code and build instructions to reproduce the build, so we usually open a ticket in their code repository.
In any case, the result is a discrepancy between the binary we can create and the binary we can find for download and any discrepancy might leak your backup to the server on purpose or by accident.
As we cannot verify that the source provided is the source the binary was compiled from, this category is only slightly better than closed source but for now we have hope projects come around and fix verifiability issues.
The product cannot be independently verified. If the provider puts your funds at risk on purpose or by accident, you will probably not know about the issue before people start losing money. If the provider is more criminally inclined he might have collected all the backups of all the wallets, ready to be emptied at the press of a button. The product might have a formidable track record but out of distress or change in management turns out to be evil from some point on, with nobody outside ever knowing before it is too late.
Application build test result
The following is the review of their Play Store app which equally applies to the App Store version:
Update 2020-12-03: This app is not obfuscated. Anybody can decompile at
least version 2.27.10 and inspect the code or modify it to compile it back
into an app. Clearly licensing stops honest people from publishing the result to
any users but we don’t see any argument left to not publish the code under a
non-permissive license.
Update 2019-12-20: This app and its review here were also discussed in a great episode of the Unhashed Podcast.
This app follows an interesting concept with a multi-signature setup.
The app contains a single-signature wallet like most wallets discussed here but it also allows to use 2-of-3, 3-of-5 and maybe other configurations.
So the client software is a wallet on its own but can we audit it? Neither on Google Play nor on their website nor on GitHub can we find a link to the source code.
Also on Stephan Livera’s podcast we found episode 182 with Casa’s Jameson Lopp and Nick Neuman where Lopp explains about Open Source:
Yeah. I mean, this is something that we’ve talked about, you know, ever since we started the company the various trade offs between what you can accomplish with free open source software versus, you know, a for profit company that may not open source all of the software that it’s writing. And when you’re looking at the multisig product that we’ve built, that is actually a very well diversified product because you end up using open source software, your firmware and hardware from a variety of different companies, which helps you both increase your level of security from a variety of different threats and decrease the likelihood that, you know, all of those different actors out there have been compromised and willing to coordinate, to work against you. It gets a little bit trickier, you know, when we’re talking about a closed source, single sig hot wallet, I mean, this is a riskier threat model.
fair points but … there is a single signature part to the wallet which they recommend to use like a checking account and as they recommend their multi-signature “gold” product for wallets starting at $1000, we assume this wallet is still meant for amounts up to $1000. And regardless of recommendations, users tend to forget they have $700 in their wallet and next time they check, Bitcion went up x20 …
Lopp goes on to say:
Now, when we’re talking about open source versus closed source in the context of mobile apps, then it gets even trickier because it is difficult if not impossible. And in fact, we have not yet really found a way to verify the build of a mobile app that is on the Apple store or the Google Play Store.
which is a direct hint to what we are doing at WalletScrutiny. Is he claiming we are promoting something that cannot exist? Or that he has not yet figured out, which given he is a professional in the space since before Bitcoin would be hard to believe? He explains:
The way that these mobile App Stores work is that they require the applications to be cryptographically signed by the developers in order to get pushed out to the store, but the actual build process for the app, the actual you know, attestation of what the code that’s that’s being run on the app is not really a part of the experience that Apple and Google provide.
What does he mean by that? Yes, the apps are signed. So what? For Reproducible Builds, a missing signature on part of the re-builder is perfectly fine to proof a match of the underlying code. Strip the signature from the Google Play version and compare.
He further claims:
The only real option, if you wanted to be sure of what the actual code was running as would end up having to build the mobile app yourself and load it onto your phone.
This is, theoretically possible, at least with Android, but it requires a fair amount of technical experience. It’s really not, the people who have the ability to do that. We are not really targeting them for this experience.
Reproducible Builds matter precisely so you can take advantage from public code audits without having to compile or audit the code for yourself. It is a complicated and technical topic and it is our mission to educate users and providers on the possibilities and limitations so users will demand reproducibility for peace of mind while professionals call out those who don’t follow best security practice thanks to relevant projects being subject to public scrutiny.
You know, this is meant to be for nontechnical people who are very early in their Bitcoin life cycle. So it really seems like from a free open source software security side of things, that it’s more of a feel good idea of having open source mobile apps, if you can’t actually verify.
Pardon? “open source software” is a “feel good idea”? This part of the interview is really getting hard to read without jumping up from the chair!
I mean, we are all familiar with the mantra of don’t trust verify, and if people could, you know, verify that the code we open sourced was the code that was out there on these stores. Then I think it would make a stronger argument for us to be open sourcing it.
Jameson Lopp we challenge you to show how WalletScrutiny is pointless. We close shop if you can show that our work is futile.
WalletScrutiny.com is not perfect but for now provides an important building block towards making wallets more secure by ensuring Build Reproducibility.
The missing parts of
- Actual code audits
- Release deceleration
depend on users and providers caring enough to appreciate the security implications of rushed releases and incentives bug bounties to audit code.
Build Reproducibility is important and possible and you claiming the exact opposite is an affront to those who hold up these pillars of transparency.
The other downside is that, like I said, these apps require a cryptographic signatures from the developers to be on the store. There’s also various functionality that we’re using such as some of the pieces of the seed backup that also require that. So even if you built the app on your own, you would not have a fully functioning Casa wallet, you know the user experience that we really intended. So there’s trade offs Yup.
With Build Reproducibility, the signature of the Google Play app can be applied to the self-built app. Sure, modifications are not possible but our fight is for transparency for the sake of security, not for the freedom to do modifications.
In defense of Lopp, he is talking about “Open Source”, not “public source”, so some statements apply to Open Source that would just not make sense for public source but let’s take
The Devil’s Advocate’s Perspective
Interestingly the FAQ section on their website states:
Can Casa ever access my Bitcoin?
Casa can never access your funds - even if we wanted to. With multisig, multiple keys are required to spend your Bitcoin, and Casa only ever has access to 1 key (for use in emergencies only). This also means that it’s impossible for Casa to ever stop you from accessing your Bitcoin. And that even if Casa were under attack, your funds would still be 100% safe from thieves or hackers. Casa was architected with a ‘Can’t Be Evil’ principle.
and
How does the setup process work?
For Gold memberships, you can bring your own hardware device(s), download the Keymaster app, and get started immediately. For Platinum and Diamond, Casa’s team walks you through a simple, curated setup process:
- Receive your hardware devices in the mail
- Includes everything you need in one discreet package.
- Book your Key Ceremony and graduate to multisig
- Your Client Advisor walks you through a personalized onboarding and OpSec review.
- Ongoing security and service
- We’ll keep you up to speed on all the security developments you need to know about.
- Message us any time you have a question, need a free device replacement, or just want to chat.
So if our assumption is correct that the client software is responsible for one of the keys in each setup, the above information leads us to grim conclusions.
Casa makes sure to always have control over the majority of your keys:
- They obviously have control over the key they promote as being under their control.
- They avoid scrutiny of the software wallet that holds a second key. In this configuration they invite you to “bring your own hardware device”. It’s only one of three after all.
- Should you want to use more than one hardware wallets, they ask you to accept shipment through them, making you vulnerable to supply chain attacks.
Combined with this quote from the above interview:
You know, this is meant to be for nontechnical people who are very early in their Bitcoin life cycle.
and this quote from their website:
How much Bitcoin can I store in Casa?
Gold is built to secure $1,000 - $50,000 worth of Bitcoin. We recommend Platinum if you are protecting $50,000 - $500,000 in BTC. Diamond has no upper limit, and is optimized for $100,000 and above.
it gives the impression Casa targets rich, non-technical users and lures them into using their products that are not auditable and thus leave room for backdoors capable of emptying all clients’ wallets at once.
Our Perspective
We don’t assume that Casa is out to steal your keys but we stand by the mantra “Don’t Trust. Verify!” and consider this wallet with or without multi-signature setup to be not verifiable.
Tests performed by Leo Wandersleb
Do your own research
In addition to reading our analysis, it is important to do your own checks. Before transferring any bitcoin to your wallet, look up reviews for the wallet you want to use. They should be easy to find. If they aren't, that itself is a reason to be extra careful.
<img src="https://walletscrutiny.com/images/social/iphone/com.casa.vault.png">
