"I am not bound to win, but I am bound to be true. I must stand with anybody that stands right, and stand with while he is right, and part with him when he goes wrong." 

Abraham Lincoln 

I think this a great statement, but it depends on the situation and what they did wrong. Honestly, I'd rather help a person in the wrong to go on the path towards the right, so they quit being in the wrong. If you abandon someone solely because he goes wrong, you aren't being a good friend. for example, if someone is abusing prescription pills or using drugs I'll try to help out if I can, but I won't join them. If I see someone attempting to steal food or clothes, I'll pay for them if I could afford it, but I will not aid in a robbery. I would much rather be known as a helping hand than a person who left someone for dead.