Ich bin gestern Nachmittag mit seinem Auto gafahren.

or should it be a "habe" because of "mit seinem Auto"?

Can you call "mit seinem Auto" a direct object or merely an object of the preposition "mit"?

See a translation

gefahren*.

If there is a preposition involved it is not an object dictated by the verb, it is a prepositional phrase. In this case it is fine how it is.

If you wanted to use "haben", you could say:
Ich habe gestern Nachmittag sein Auto gefahren.

Here there is no preposition, it is actually an accusative object, so we use the verb "haben".

So to answer your question:
"mit" will always have a prepositional phrase, meaning it will be dativ afterward no matter what.
If you change it to "haben", then you can take away the "mit" in this case, because there is a transitive and intransitive (with or without object) version of "fahren".

There is also a bit of a change of meaning, as "mit seinem Auto" does not mean you necessarily drove it, but just used it as your means of transportation. Saying "ich fuhr das Auto" (Ich habe das Auto gefahren) will always mean you were the one who drove it.