Successful recovery of 70 ETH (EIP2333) in validator on the beacon chain (seed lost)

Originally published on reddit: https://www.reddit.com/r/ledgerwallet/comments/13kk6iz/successful_recovery_of_70_eth_eip2333_in/

TL;DR - Don't lose your recovery seed!

A client came to us for help trying to recover access to 64 ETH staked on the beacon chain on their ETH validators, plus rewards, so about 70 ETH total. The validators seed was lost.

See their posts on the ethstaker forum: https://www.reddit.com/r/ethstaker/comments/13bq8fh/lost_seed_possible_to_recover_with_ledger_nano/ and https://www.reddit.com/r/ethstaker/comments/13kl5nv/update_lost_seed_possible_to_recover_with_ledger/

Normally when you lose the validator seed, you lose all hope of withdrawing the funds. But the client was lucky that they had initialized thers validators and their ledger Nano S with the same seed phrase.

The issue was that the very tech-savvy client unfortunately lost their seed phrase due to unforeseen circumstances. So the only remaining copy of their validators seed was in their Nano S, and of course there is absolutely no way to extract the seed from the ledger.

Special private keys and signatures are needed for withdrawing ETH from validators, based on EIP2333 and using different cryptographic formulas, not those used for "normal" ETH transactions.

Not only is there currently no ledger app (yet) capable of generating those EIP2333 signatures with ledger devices, but also the Nano S does not even have enough RAM to generate those signatures. Normally, validators can generate those signatures, based on their seed phrase.

So the idea (suggested by Ledger Team) was to generate the EIP2333 private keys on the Nano S using the derivation paths used by the validators, extract them and use them with off-line tools to generate the needed signatures to rescue the ETH from the validators.

In order to do that, a custom recovery ledger app had to be developed and installed (i.e. side-loaded) on the client's ledger. We hoped that the firmware on their ledger (2.0.0) already had support for the new cryptographic functions (i.e. BLS12-381 elliptic curve) needed to generate those keys, since updating the ledger firmware is very risky if you don't have the seed (if the ledger resets, everything is lost!).

We developed the custom recovery app using the ledger development tools on a Debian 11 Linux system running in virtualbox on a Windows 11 host.

We first tested the custom recovery app after side-loading it on a test Nano S+, but the firmware on our S+ did not support some of the functions we needed, so we decided to test using a Nano S (firmware 2.1.0), and everything was working as expected. We were able to generate the EIP2333 private keys. It takes about 16 sec for the Nano S to generate each key. The derivation process is very CPU intensive for the BLS12-381 elliptic curve, and the CPU in the Nano S is quite slow.

We validated the EIP2333 keys generated by our ledger app on our test device by comparing them to those generated with the Ian Coleman EIP2333 tool, and at first it looked like the keys didn't match. We found out that it was due to a bug in Ian Coleman EIP2333 tool (adding a new-line after the last mnemonic word breaks the bip39 seed!!). So finally we could confirm that the keys were correctly generated by our app. Our client also confirmed that the keys generated by the Ian Coleman EIP2333 tool match those generated by other EIP2333 tools.

We then sent the virtualbox image to our client, and they were able to run it out-of-the box in virtualbox on their Windows 11 system. The next step was to check that the custom recovery app was able to generate to right keys on another test Nano S, this time with firmware 2.0.0 (the exact same firmware version as the precious ledger Nano S containing the validators seed), as this would tell us if a (potentially risky) firmware update on the precious ledger would be needed on not.

The recovery app was side-loaded on the client test Nano S, and it was able to generate correct EIP2333 keys.

The next step was to run the app on the precious Nano S that contained the validator seed.

We got a bit worried when the side-loading process generated an exception, not allowing us to install the recovery app on the device.

We figured out that there was probably not enough free space on the device because of other installed apps, so we uninstalled all apps (using the device dashboard "Uninstall all apps" function). Then our recovery app could be successfully side-loaded on the device. Relief!!!

The recovery app was then run on the client's device, and we were able to get all the EIP2333 keys needed to rescue the validator's ETHs. The keys were confirmed to be correct, based on the public keys that were known.

So it required significant work and development of a custom ledger app, but at the end this recovery was a success!

Comments: you can read and post comments on reddit, even if the post was "removed by reddit filters".

In the same Recovery series: Other crypto recovery reports by loupiote2

Contact loupiote2 via chat or message from reddit profile: https://www.reddit.com/user/loupiote2/