Keep it and tell him to loose your number.
Time is valuable and he wasted a bunch of your time.
I am not sure if you said how he paid for the deposit? Guessing any sleazeball even asking for $100 back is going to do a charge back on you if he is able to.
If that happens, be prepared to deal with it however you see fit.
I would post all his info publicly if he opts for that route.
sorry you have to deal with a guy like this. Once a deposit is paid, the deal is done.
Buyer walks after that, and they loose money.