A peek inside the MISO war room – $350m incident response story

The Bug

  • The DutchAuction smart contract inherits the BoringBatchable utility contract that allows callers to batch different calls together.
  • There is a commitEth function in the auction contract that uses msg.value to know the amount of ETH commited by the user. If the user commits more ETH than the contract’s capacity, the contract refunds the extra ETH. This function is fine on its own but becomes a vulnerability when combined with the batch function from BoringBatachable.
  • The batch function works by making multiple DELEGATECALLs to itself (the Auction contract). An interesting aspect about DELEGATECALL is that they use the same global context as the parent call so every DELEGATECALL has the same msg.value that the parent call had.
  • This means that you can pass 1 ETH to the batch function and call the commitEth function a hundred times with 1 ETH as msg.value, every time.
  • Since the commitEth function uses msg.value to know the amount committed, it will get tricked into thinking that the user committed 100 ETH (1*100) instead of the actual amount (1 ETH). If the contract had only 1 ETH of capacity left, it will refund the remaining 99 ETH to the user.
  • In summary, the user can put in 1 ETH but get out 99 ETH as refund. The numbers can be scaled up or down.
Evil Bug

The learnings

  1. msg.value is a global variable and if you use it in a loop (batch), you may get unexpected results. If you have a function similar to batch, make sure to check the increase in balance manually rather than using the msg.value directly.
  2. Trust no one, not even msg.value or the auditors. MISO had gone through 3 audits and a few independent reviews. Verify everything on your own.
  3. Even if individual components of a contract work fine on their own, there’s no guarnatee that a cocktail of those components will work as expected. You must be careful when mixing such components.

The story

It was Monday night and I was casually scrolling through Reddit, laughing at a meme titled “One of those Mondays“. I was about to call it a day when Joe messaged me “You around?”. In hindsight, it was Joe’s version of the infamous “u up?”. He followed that with a link to Paradigm’s zoom call. My first thought was “Dang, joe selling us out”. I joined the call and saw Sam (and Georgios) waiting in there. It was at this moment that I knew, someone fucked up.

Georgios started describing the situation by mentioning that the Dutch Auction contract supports batching. This was enough information for me to connect the dots. I had reported the exact same vulnerability in a different project last month. In the interest of saving time (which is critical in these situations), I cut Georgios short, quickly confirmed my theory, and we ended the call to work on the next steps. The initial call lasted merely a minute.

Since a large amount of funds were at risk (~$350 million), we decided to keep the inner circle small and started the war room / incident response bridge call. The group was split into two teams, the first team’s job was to analyze our options and do a white hack if need be. The other team was doing the boring yet necessary work of comms, waking up people who controlled relevant keys, and stuff like that.

I had not audited or worked on MISO contracts but I did review them. I am a bit ashamed that I missed this in my review but I wouldn’t be a human if I didn’t make mistakes. Anyway, the tension was building up in the war room as time passed. To avoid stress, I decided not to convert the amount at risk from Million USD to Crore INR. My brain can’t process the amount without doing the conversion so I was able to work like there was nothing at risk. Brains are naive, can be tricked easily.

After the initial brainstorming, we concluded that we had 3 options:

  1. Do nothing and hope nobody notices.
  2. Rescue the funds by exploiting the vulnerability ourselves.
  3. Mitigate the vulnerability by closing/finalizing the auction.

The consensus was that the third approach makes the most sense. We started working on a smart contract that took a flash loan, bought out the auction, finalized the auction and, paid back the loan using the funds raised in the auction. This required some admin permission. People who had these permissions were still unaware of the situation which meant that the comms team finally had a purpose!

Over the next hour or so, we created and tested this flash loan contract on a mainnet fork. Everything was looking great, we even deployed it on mainnet and were ready to execute it as soon as we had the required admin permissions. On the other hand, the comms team was still waking up the folks who had the required permissions. Believe it or not, it’s hard to reach out to folks at 3.30 AM local time. I don’t know how the comms team pulled it off but we soon had everyone we needed on a call, ready to transfer the permissions.

While we are doing the final checks, we realized that the auction had naturally sold out. Someone bought in at the last moment. We no longer needed the flash loan to buy out the auction. All we had to do now is to finalize the auction so that the vulnerability is mitigated and the funds are taken out from the auction contract to a secure wallet. We already had the folks with relevant permissions on a call so we managed to finalize the auction immediately. Funds were now safe, the storm had passed.

Except, there was one more little problem. There was another auction going on with a similar vulnerability. It was a batch auction so we couldn’t finalize it before time by buying out the auction. The saving grace there was that although people were able to do fake commitments, they were not able to take out funds like in dutch auction because batch auction does not offer refunds. This meant, there were no funds at risk. The worse that could happen is a botched auction in which case the auction creator could refund all participants and redo the auction.

Nevertheless, we brainstormed mitigations for batch auctions. We noticed that the auction called an external contract to validate all commitments. We could change this contract and use something that only allowed valid commitments. Ideas were thrown around on how to do the validations (technical jargon ahead) –

  1. Limit gasLeft to allow batched commitments. This could’ve worked but was relatively complex to create and could cause bad UX if the wallets did not do a tight estimation of gas.
  2. Allow only one committment per tx.origin per block. This would’ve been ideal but it required state modification. The function was marked as view and since Solidity 0.5, STATICCALL is used for calling view functions. If we tried modifying state, it would have caused a revert even in legit committments. This is why solc 0.4 is still the best version (not really). If we had used solc 0.4, this method would have worked because solc 0.4 used regular calls even for view and pure functions.
  3. Ensure that the balance of the contract matches the total commitments. This method was clean and it worked. We created a POC and wrote some test cases within the next hour.

By the time we had verified the approach for the batch auction, the auction had almost come to an end. Since there were no funds at risk, We decided to let it run its natural course rather than trying to intervene at the last moment. The auction ended without any interventions or incidents. Finally, we could rest. This is when I decided to convert the million USD to crore INR to finally understand the magnitude of what had just happened. Overall, It was a thrilling experience, to say the least.

DISCLAIMER: This story is dramatized for entertainment and is filled with bad jokes. Forced Humour is how the author copes with stressful events, deal with it.

Leave a Comment

Your email address will not be published. Required fields are marked *