Quoting Mama MacGyver*:" But do you really think a guy who is cheating on his girl would say something like that? Unless it ... [snip!] ... it was in the heat of an argument, I really don't think ANY guy would say something like that if he were REALLY cheating."
Absolutely they would, and it is a really common way people handle guilt. It's trying to approach something serious with comic relief.
DH and I are like how you explained you and your DH... we make comments like that all the time. But OP stated he is being weird with his phone and other red flags.
OP, I would just talk to him about it. There's probably a good chance he was just joking, but just tell him his comment threw you off guard.