If you were a public defender or a private attorney with a government contract, that would not be a sufficient reason. If you were a private attorney retained directly by the defendant, you could move to withdraw in this situation, although you would probably word it more vaguely/delicately--e.g., "unresolvable conflicts with the client that cannot be disclosed due to privilege" lol. BUT if the trial is just a couple of months away, you might not be able to withdraw anyway.
Also, if you were a private defense attorney who regularly withdrew because your clients were unpleasant, you would probably run out of clients pretty quickly and starve.