Veganism has nothing to do with compassion. I'm not sure I have a satisfactory account of compassion, but I know that compassion has nothing to do with the fact that I don't torture and kill other people. Likewise, refraining from participating in the torture and killing of non-humans is not a compassionate act. It's a prerequisite for regarding another morally, the first step in approaching just relationships with them.
Compassion is often regarded as supererogatory, above and beyond the call of duty. Perhaps it's something we should all be striving for, but it's expected that we'll all fall short to varying degrees. This is problematic if we think of veganism is deriving from compassion or being an expression of compassion. We must move to an understanding of veganism that is not just something that an elite few achieve because they are so virtuous, but an imperative for us all. It also trivializes that injustices toward animals by suggesting that their slaughter is simply a the result of our not being nice enough.
Animal advocacy groups with names such as "Compassion over Killing" and "Mercy for Animals" (mercy being another dubiously applied concept) add to great confusion on this matter. The most important thing we owe non-humans is justice, which requires not treating them as things or killing them for our purposes. Let's talk about compassion once we have more consensus about what justice requires.