My gut feel is this: SCO used the lawyers to make a quick buck, by waving an IP portfolio and a license agreement for the Unix System at people and expecting them to fold quickly. Their first two targets paid up - MS because it didn't want to face another big lawsuit with so many on the go at once (and one against UNIX would just be sooo much bad publicity), and Sun because it's getting enough bad press at the moment over its financials and didn't want to take any risks over Solaris.
IBM weren't so quick to fold. SCO went legal to try and push for payment. The lawyers came out, and then the bills started arriving, and SCO was out of its depth, as IBM decided to play a waiting game.
The rest is history.
(originally posted as a comment in another place)