I've shot several deer with a SKS, and a sizeable number with the .30/30.
Comparing these two is actually more like comparing apples to oranges.
Both are developing similar velocities and similar diameter bullet/weights, but from there the differences become profound.
As other posters noted, the .30/30 typically uses heavier, but blounter bullets (due to tubular magazines).
The problem is that the 7.62x39 hunting ammo usually doesn't turn out the "claimed" velocities. The Winchester and Remington 123-125gr factory loads I've chrono'd usually ran ~2,275fps from a 20" bbl SKS (Chinese).
FWIW; the last 150gr Winchester 150gr ammo I chrono'd ran 2,275fps from a 20"bbl (Marlin 336). Some have approached 2,400fps. Most 170's have been 2,100 to 2,250fps.
The reason your buddy lost the deer with the shot with the H.P.'s, is that all the H.P. ammo I've seen is the import mil-spec. stuff that is really a mild steel-copper coated FMJ jacket reversed and don't expand the way conventional SoftPt. or Hollowpt thin guilding metal jackets do. In essence, you buddy was still shooting steel jacketed ammo. It is modified to be legal to use in the U.S. where the various states require "expanding type" ammunition. Ditto some of the import Soft Pt. ammo I've seen/shot. I would also bet on a marginal lung shot. A "heart" shot that goes 3-4" low, will usually result in a small blood trail, and lost animal. Thats why I usually try for high lung shots in the shoulder. A high hit results in a spine shot, a low one in a heart shot.
I don't know about the 158gr 7.62x39 ammo you mentioned. I think that you may be confusing it with 7.62x54R. It is frequently loaded with 154-174gr bullets. NOT INTERCHANGEABLE with 7.62x39! It is for the Mosin-Nagant rifles. Power is similar to the 7.62x51 NATO, aka the .308.
In my experience, like another poster stated, the paramenters of good performing .30/30 ammo has been long worked out. The 150gr ammo is good for animals to 250lbs, the 170gr for 200lb and up. The 170gr Cor-lokts and Nosler Part. have a very good reputation amoung the locals in Maine and adjacent Canada on Black Bear and Moose. While he was stationed in the Air Force in Alaska, my brother encountered a lot of locals (interior native americans) who held the .30/30 w/170gr Federal/Nosler Partitions in near reverence. They used it for subsistence hunting and took many moose, caribou, and occasional bears. Their advice? Get close, shoot-em in the ribs, get ready to haul 'em home! Good advice wherever big game is shot!
In 20+yrs as a game warden, most deer I encounterd lost, were lost due to poor shot placement. At night, at 25mph from bouncing vehicle, with iron sights, at 200yds is not conducive to good shot placement! How many times did I hear 10+ shots, and no "dead deer" to be found!
Main reason so many deer are "lost" with the .30/30! Also, many hunters can't judge between 125yds and 225yds. Also, most .30/30's are zero'd at 100yds, instead of 150yds as they sould be (2.5" high at 100yds; 4" low at 200yds for 220yd "point-blank" range for deer- a 150yd zero).
I've not lost but one deer with a .30/30 and good bullets. (lost one with rear-lung/gut shot w/170gr Sierra FN, another w/150gr Nosler Solid Base- similar shot placement, both deer were walking pursuing does at the moment) None for the .223! I also take pains to place shots well.
Deer I've lost (more than I like to admit too), have been when I took shot placement for granted with either the .257Roberts,.270,.30/06,.338, and 12ga.
If you can hit where you aim, the 7.62x39 does about as well as the .30/30, assuming you use good bullets. If not, it fails about as often, too. Same goes for .270's and 7mm Mag'.s as well.
The biggest variable is the "Nut" holding the stock and manipulating the trigger!