Yes, they can bang into each other, or the rifle can get stuck behind the butt of the pistol, if they are on the same side.
Either of your ideas would work to minimize the problem.
I usually like to switch shoulders during the day for my slung rifle, though ... so that might not be a workable solution. Crossdraw in front might do better, or a front-slung AR with the pistol on strong-side on the belt would be fine, too. A pocket-carried .38 or little pistol is best for me, since it gives me a pistol's utility without the problems that can crop up from carry on the belt.
But the little gun is not as powerful as a service-size .357, or if I were going to carry a service-size 9mm, it doesn't have the capacity, and in no case will it be as easy to shoot as the larger pistols.
When I am down near the border hunting, I carry a full-size pistol along with my hunting rifle (or shotgun, if I'm after birds), end of story. If I'm just hiking, I still carry a full-size pistol, and I've sometimes taken the rifle along, too. When I am up north chasing elk, I might just decide that the little .38 in my pocket is enough.