I would be careful (I am sure you are) in shooting a bowling ball with any gun, but particularly with a small caliber or low-velocity bullet as there is a real good chance of a ricochet from a glancing-shot.
Could it be possible one bullet may have ricocheted towards the person's house who came to talk to you? In other words, is his property past the bowling ball and in the basic direction in which you were shooting?
I live in the wilderness, there are some nearby houses, but for all intents and purposes, I live in an extremely RURAL area.
When a person starts shooting his .22 rifle (or pistol) I often wonder what it is they are shooting at, as his house is about a mile away by driving, however it is only about 1/3 mile away from point a to point b. If he is shooting into a river that is between us or into a swamp that is also between us, there is a possibility that one of his bullets could ricochet in the direction of my house.
When he shoots (perhaps five to twenty rounds, sporatically - say once every month to several months), sometimes I will go out and just listen to hear if I can hear any of his projectiles either buzzing over my house or making their way through woods I own adjacent to my house. In the 25+ years that I have lived here, I have never heard a ricochet, so there is really nothing I can do. Yes, I can wonder where his bullets may be going, but that is the extent of it!
Now, another neighbor is totally ANTI-GUN! She is against any kind of shooting, no matter how legal it is. If I take a shot at vermin on my property (I have many skunks here and have since I have moved here), she will not hesitate to call me and lay right into me. I calmly tell her that I am shooting skunks that ruin my lawn and also live under a building I have that is on concrete sonatube footings (wooden floor structure off the ground at one end and basically on the ground at the other end). She had called the game warden about three years ago when I used my 12 gauge to kill a huge skunk. When the game warden came, I showed him the lawn all dug up, I showed him the dead skunk and I showed him the skunk scat (feces) that is visible under the corner of that out-building.
Once the game warden saw all that --- he knew immediately that this woman is just a complainer and told me what I am doing is perfectly fine. If I wanted to, I could shoot all day here, every day. Other than the noise, there is not much anyone could do. I try to respect those near me, so I only shoot something when necessary. To have someone complain just for the sake of complaining is really childish in my book!