This is one of the really annoying things about civil liberties. If you're going to take them seriously then you have to support them for even those people you disagree with.
It isn't necessarily true that governments can borrow more cheaply than companies. Look at the interest on marginal borrowing rather than average it over total borrowing.