You have asked about "to be with you" and "to be around you". They mean similar things. It's easier to explain them together.
"I want to be with you" by itself often means "I want to be your romantic partner". It is different to "I want to go to the circus with you" which means "I want both of us to go to the circus at the same time." You can also say "I want to be with you at the circus" but that is less natural. It is possible to be "with" somebody but not be physically near them, like in a long distance relationship.
"I want to be around you" means "I want to be physically near you." You can say "I want to be around you to make sure things go smoothly" or I want to be around you in case you need to ask a question."
@anzu_12 In that construction, they're not as different. Both sound like you enjoy spending time in the same place as him, without implying that you are his friend or boyfriend. He could just be a dick who does funny things.