Like a financial councilor of mine Dave Ramsey has always said that no one ever became a millionaire by collecting airline miles or any points.
Frankly, Dave Ramsey is an idiot. While his main message of not buying unnecessary items, new cars when you don't need them, etc. is a great point, he should be teaching people how to use credit cards and other forms of debt to their advantage. Not steering them away from it. There is no reason to walk into Best Buy and lay out cash for a $1,500 computer, yet that's exactly what he recommends doing. You not only pay exactly the same price, but you don't get any protection if the store tries to screw you over, you get nothing back money wise, etc. I think the problem is that a lot of people buy more when it's on a credit card and they don't immediately see the cost. This is what you have to avoid. You need to keep track of what you buy, not buy extra just because you don't see the immediate cost of it, not have cards with fees, never pay interest on a purchase, etc. The lesson should be to spend responsibly and not do this. Not to forfeit any rewards, protection, delayed cash flow advantages, on the same exact items you buy everyday. In addition to this, it's very hard to buy stuff online with "cash". I literally save thousands of dollars a year buying online versus at a B&M store and this is on items that I'd have to buy regardless.
Now that my opinion on Dave Ramsey rant is over, I have used PayPal quite a bit and I've never had a problem. If you buy gun or ammo purchases and they catch you, they will lock your account. Because of this some people don't like them. They also tend to side with the buyer in a dispute case which some people also don't like. However, if safety is your concern they are fairly safe and do a fairly good job at protecting the buyer. Most issues with information getting stolen, bank accounts having money withdrawn from them without authorization, etc. are user induced such as downloading an unofficial app like an above poster did. You need to make sure that you avoid this, avoid click on fake emails that look like they are from PayPal, but really are just trying to steal your password, etc.
For everyday online purchases though, I'd get a credit card (one without a fee and the best rewards you can find). You can generally save a ton of money by buying items online instead of at B&M stores and if you ever have a problem most credit cards will take care of it with just a phone call.