"What's up" is extremely casual/slangy. Only use it with those you are close with. Now people use just "Sup" more I think.
"How's it going?" is the same thing, but less casual. I would still use it only with friends. If I was talking to a close teacher or someone like that, I would probably say "How is everything going?"
The more letters and words you take out, the more casual the sentence becomes.

"What's up" can either mean "how's it going" or "what's wrong", depending on the situation.

In that context, I think 'what's up' is seen as a vague greeting like 'hey' or 'hello' and 'what are you up to?'
If I saw my friend on the street, maybe I would say 'hey! what's up?'

'How's it going' is a casual way of saying 'how are you' but they are both informal greetings.

'What's up' can also be used in different situations/contexts:
'Hey, what's up? Are you okay?' (sympathetically)
Or if someone wants to ask you a question or for a favour, you could say: 'Yeah, what's up dude?'

Sorry! That might have been muddy! 'What's up' is a very vague term (: