The "con" in "con man" is literally "confidence." 90% of the scam is getting your trust. At that point, human nature kicks in, and we tend to not want to believe that our "friend" could be doing this.

Edit: For future reference, the anatomy of a con game
In Confessions of a Confidence Man, Edward H. Smith lists the "six definite steps or stages of growth" of a confidence game. He notes that some steps may be omitted.

Foundation Work
Preparations are made in advance of the game, including the hiring of any assistants required.
Approach
The victim is contacted.
Build-up
The victim is given an opportunity to profit from a scheme. The victim's greed is encouraged, such that their rational judgment of the situation might be impaired.
Pay-off or Convincer
The victim receives a small payout as a demonstration of the scheme's effectiveness. This may be a real amount of money, or faked in some way. In a gambling con, the victim is allowed to win several small bets. In a stock market con, the victim is given fake dividends.
The Hurrah
A sudden crisis or change of events forces the victim to act immediately. This is the point at which the con succeeds or fails.
The In-and-In
A conspirator (in on the con, but assumes the role of an interested bystander) puts an amount of money into the same scheme as the victim, to add an appearance of legitimacy to the scheme. This can reassure the victim, and give the con man greater control when the deal has been completed.
-- from the wiki page(https://en.wikipedia.org/wiki/Confidence_trick)
---------------------------
The midnight call was "The Hurrah," in your case.