My wife, a HUGE Buffy fan, turned me on to Firefly - yes, I'm a product of the '70s sci-fi craze - and I really liked the show. But I can see why it failed; today's audiences are looking for psuedo reality, not escapism. I swear the mindset now is "don't make me think - just entertain me". Joss created a show that needs to be WATCHED, not just for the plot, but the subtle twists and turns that the charaters interaction with each other and the outside galaxy provide.
Remember, Joss is a big Star Wars fan - there's more than one referance to "SW" in the TV show and the movie.
And as for continuing the show, it would be easy. As was said earlier, "It's Hollywood!" A couple of examples: during her confinement, the blue hands found that River has telepathic powers, and can send and recieve thoughts instantly. Imagine what that would mean to communicating over the vast distances of space...
The Browncoats are still around, only hiding out - and some of them want to start the war again...
And there are always the ever-present villians to throw int the pot.
There would have to be some changes, but I could see it coming back; after all, who in 1970 would have believed "Star Trek" would ever return?