TFL on your hard drive

[45R]

Well most of us came from TFL and have used it for its great collective of information. There is lots of humor and wealth of information on Rich's site.

Recently some people have expressed interest in finding a way to copy TFL onto a disk or their HDDs to use as a reference guide, for research or just to browse into the past.

Well tonight I am trying a test run of some software I found online to see if I can download all of the treads on TFL. With luck maybe it will work. I'm trying this tonight. I'll post a report tomorrow to let you know how it all works out. There is alot to DL.


45R

 

[Justin]

I seem to recall that at one point the entire content of TFL was calculated. IIRC, it came to something like 45 gigs of data.

 

[Combat-wombat]

What, someone's gonna spend $150 on a new hard drive so we can see the threads without accessing the net? c'mon, just find it on the website!

 

[45R]

Justin-

45Gigs is alot of information to download. That will take me a few days. Thought it would be cool to see if it could be done. Thanks for the numbers.


Combat-Wombat-
If you can spend 700 bucks on a gun 150 is a few peanuts for alot of information. Would I do it no. A Gig at the most is what I would be willing to leave on my HDD to access information IF KEY WORD is "IF" I was not near a telephone or cable connection.


I've tried this utility (winht track) on a few websites. I works pretty good. I can take information with me on my laptop if the content is too long and to places were online access is not feasable.

 

[WonderNine]

I would like a CD of TFL, but I didn't think it would run 45GB's!

Might fit on one of those new quad layered DVD's?

It would be nice for people to have copies in case something ever happened to the server.

Maybe TFL will re-open someday.

I like this sight alot, but I still miss TFL!

 

[tyme]

It's under a gig compressed; it's probably about 60%/40% text/attachments, so ditching the attachments would probably get it within the ~700MB cd limit.

There are two caveats.

First, there's no good way to distribute it except in raw mysql or csv (or tsv) format. So it's not immediately useful to most people. However, this is an activist economy, and I'm sure people would write scripts to convert the mysql or csv files into pretty formatted pages, perhaps even prettier than vBulletin does it.

Second, Rich has to say it's okay and allow it. This, I suspect, is the taller hurdle.

I don't think trying to mirror it from TFL would be very productive. The generated pages have very little in the way of meta-information. So you won't be able to search very well or do much else with the data other than view it in a web browser, unless you do lots of post-processing. You're better off mirroring just the threads you like.

 

[Combat-wombat]

put it on a DVD, if you must have TFL offline. DVD-R/RW drives are becoming much more common.

 

[Mike Irwin]

Let me also remind everyone that TFL AND the contents are copyrighted.

To do this legally, you have to have the permission of the copyright owner.

 

[tyme]

I haven't ever thoroughly read the federal copyright statutes, but since TFL is obviously not a commercial entity, I would think copies for personal non-commercial use would be allowed without permission.

People posting were clearly doing so knowing their work was going into the public domain, or perhaps the ownership was merely transferred to TFL, but at any rate they did not retain control over how tfl might redistribute their work in the future.

 

[Mike Irwin]

I'm not 100% certain, as I haven't read the latest statutes & case law, either, but given that the copying would be of the entire site, including logos, and the site would be maintained in its original structure, I think that's going way to far for a "fair use" claim to be made.

Also, the fact that it's non-commercial doesn't matter.

IIRC, a copyright conveys protection based on the assumption that while no fee is being charged for the product (in this case access to TFL and it's information), the product does have intrinsic value as intellectual property.

My take is that given that the site clearly states that it is copyrighted, permission must be obtained before it can be copied/distributed, even freely.

 

[Rich Lucibella]

45R approached me for permission already. I told him it would be OK, assuming it was to a fairly small circle....I'd not expected the offer to be placed here. Still, if 45R wants to try and others are that interested, I'll not stand in the way.

45R....go ahead and attempt the file creation....I have real doubts that it can work. If it does, I'd ask that each user click "yes" to an Agreement that it not be used for commercial purposes. I'd also ask that we be given a list of all to whom the disks are distributed.

The email list needs to be carefully guarded. It consists of 16,000 individuals with a specific interest and I still get regular requests to buy/rent it. Non of the profile email addies should be contained on the disks.

As for TFL, I meant it when I said the site would remain up in perpetuity. That means it'll be up until such time as it shows zero activity and fades from memory. If I ever have reason to shut it down, I can assure you, I'll find someone else to host it.
Rich

 

[OF]

Hey there Rich!

You need to drop in more often. People invoke your name around here expecting you to show up like the Spanish Inquisition...but all there is are the crickets chirping.

How are things?

- Gabe

 

[Mike Irwin]

Well then! Problem nicely solved by the copyright holder. That's the way it should work.

I wouldn't doubt that you get requests for the mailing list. A list that focused could be valuable property.

 

[bogie]

Idea: If there's enough demand, and it can be made to fit on a single CD, sell copies for a modest amount, with the proceeds beyond the CD cost going to an activist organization?

 

[pytron]

I have already downloaded about 50% of the site and have a really extensive system for stripping the excess HTML and downloading the attachments. I have estimated that it will take between 1-3 CDs to fit all the threads plus the attachments. It is unclear at this point because sometimes threads are very long and others very short. Plus you never know how many attachments are in a thread until you get in to it.

I have turned the downloaded files into Windows Help files, which are really easy to use and pretty much anyone who has Windows could use them. Plus they have a full-text search with highlighting. Also, if the help files end up spanning multiple CDs, there are ways to link them together.

I have been putting off this project for a while, because it didn't seem like there is enough interest in it. But, if people are interested, and Rich was okay with it, I could finish the project and then send out CDs.

It would probably take me about a month to finish. I don't like hogging the bandwidth for TFL, so I've deliberately been working very slowly.

-Pytron

 

[cordex]

there's no good way to distribute it except in raw mysql or csv (or tsv) format.

Works for me.
But while the site is up (and "in perpetuity" is a pretty long time) there isn't much gained, excpet for potentially faster searching capability depending on server activity. For me, I mean.

 

[pytron]

I agree that there is a very limited use for TFL on CD. The best reasons I can think of are:

Portability - you can take it with you when you don't have internet access

Reliability - When the search function goes down, or TFL gets blocked by your ISP, or when the whole site goes down, you can still read it.

Speed - Reading and searching are much faster, especially if you have a slower internet connection.

That's why I decided to download the site for my personal use. Plus, it's kind of a piece of firearms history.

-Pytron

 

[Topgun]

Do you need permission

to copy it in longhand?

 

[Rich Lucibella]

Hey, Gabe. Keeping busy on this end...to say the least.
Bogie: Not likely...TFL was never commercial and we won't make it so in it's dormancy.
Topgun: Just you!

Everyone: Just to clarify, I've not offered a blanket approval to copy and distribute information from TFL. I've stated I'm not opposed to it, subject to control over who performs the work, who receives the copies and other items. I think you know, I'll not micro-manage the project, but I don't want to give anyone the impression that we are relinquishing any of TFL's rights under copyright law.

Rich