That is true. There are two parts: Marital Confidences Privilege, and the Spousal Testimonial Privilege.
Marital Confidences Privilege is only for things married people may have said to one another, not things they have done. It applies to "confidential communications" and is only for things said to one another with no third party present. Both parties hold the privilege, and it survives divorce.
Then there is Spousal Testimonial Privilege, which is different. From Wiki: "The spousal testimonial privilege (or spousal immunity) can be used to prevent any party in a criminal case from calling the defendant's spouse to testify against the defendant about any topic. In federal court as a matter of common law, this privilege attaches to the witness spouse, not to the defendant; that is, the defendant's spouse can refuse to testify against the defendant, but the defendant may not prevent his or her spouse from testifying against the defendant.
This privilege does not survive the marriage; that is, after divorce, there is no right to refuse to testify against a defendant ex-spouse. This privilege may be restricted to testimony about events that occurred during the marriage, although in some jurisdictions it may apply to testimony about events occurring prior to the marriage.
These rules are suspended when the defendant is accused of crimes against his/her spouse or a child of the spouse.
There are some other interesting things you can read here: [ame]http://en.wikipedia.org/wiki/Spousal_privilege[/ame]