This is a strange conversation. What the heck is a deposit, if not a good-faith down payment to cover the seller for lost opportunity while they get their shit together? No way I'd give this joker any money back; I already missed out on a few months o could have had full cash in hand and no pin taking up space. Market is cooling now too, this may cost you more than the deposit.
In the future, if you take a deposit, give them a forefit on date (i.e. - you pay within a month in full or deposit is forfeit and I move on).