Yet Another Cycling Forum
General Category => The Knowledge => Ctrl-Alt-Del => Topic started by: Wowbagger on 27 March, 2008, 11:49:09 pm
-
My son downloaded all my posts for me using this simple php script:-
<?php
for ($i=0;$i<702;$i++) {
system("wget --html-extension --load-cookies ~/cookies.txt http://www.anothercyclingforum.com/index.php?action=profile\;u=1230\;sa=showPosts\;start=".($i*15));
}
?>
For this to work you need php installed and to log into the ACF forum in your web browser then locate your cookies.txt file or export cookies if using IE. Replace the "~/cookies.txt" with your location for the file. The 702 in the "for" statement was my number of pages of posts. You need to find your own number of pages using your profile and viewing your posts.
The pages will be downloaded into the current directory.
I hope this all makes sense. It didn't mean a thing to me.
-
I've done something similar. I'll work on some magic to convert the resulting HTML files into something more readable in the future.
-
Beyond me, I fear.
-
My son downloaded all my posts for me using this simple php script:-
<?php
for ($i=0;$i<702;$i++) {
system("wget --html-extension --load-cookies ~/cookies.txt http://www.anothercyclingforum.com/index.php?action=profile\;u=1230\;sa=showPosts\;start=".($i*15));
}
?>
For this to work you need php installed and to log into the ACF forum in your web browser then locate your cookies.txt file or export cookies if using IE. Replace the "~/cookies.txt" with your location for the file. The 702 in the "for" statement was my number of pages of posts. You need to find your own number of pages using your profile and viewing your posts.
The pages will be downloaded into the current directory.
I hope this all makes sense. It didn't mean a thing to me.
Yep that is about what I was thinking of.
You also need to replace the u=1230 with whatever your own ACF user ID is.
I suggest it might also be wise to have a short delay between each
download in the loop (e.g. sleep(5)). That would avoid hammering
the server, especially if a lot of people try the same trick.
-
Beyond me, I fear.
What OS do you use?
-
Winders XP.
-
Wouldn't it be better to pull entire threads?
-
Wouldn't it be better to pull entire threads?
Not sure. I was first of all interested in saving all my own work.
-
My version:
#!/usr/local/bin/python
import os
f = open("/tmp/acf-saves-list","w")
for i in range(120):
f.write("http://www.anothercyclingforum.com/index.php?action=profile;u=1465;sa=showPosts;start=%d\n"%(i*15))
f.close()
os.system("wget --wait=10 --random-wait --html-extension -p --convert-links --load-cookies ~/cookies.txt -i /tmp/acf-saves-list")
This has the (small) advantage of downloading all the prerequisites for the
pages so it looks correct even when ACF goes tits-up.
You may also want to save all your PMs.
-
I used wget, and I think I have all my posts. I'll trawl tomorrow and save "interesting" threads too.
I'm more interested in getting these .html web pages on my desktop converted into something useable though - at the moment they're identical to the on-line version.
-
Please can we have an 'Idiot's Guide' to this?
I don't know how to run a php script :( To be honest I wouldn't know one if it hit me in the face :D
-
That's all clever magic. I'm not sure I can get enough steam up on this machine to breach the ICBA and incompetence thresholds....
-
Please can we have an 'Idiot's Guide' to this?
I don't know how to run a php script :( To be honest I wouldn't know one if it hit me in the face :D
Seconded. Can it be done with FORTRAN ???
(Actually we don't even have a FORTRAN compiler here chiz)
-
Please can we have an 'Idiot's Guide' to this?
I don't know how to run a php script :( To be honest I wouldn't know one if it hit me in the face :D
Seconded. Can it be done with FORTRAN ???
(Actually we don't even have a FORTRAN compiler here chiz)
Shouldn't be a problem, a good programmer can write FORTRAN in any language :)
-
Sometimes I feel distinctly dumb compared to people who can do this sort of thing!
-
Please can we have an 'Idiot's Guide' to this?
I don't know how to run a php script :( To be honest I wouldn't know one if it hit me in the face :D
Seconded. Can it be done with FORTRAN ???
(Actually we don't even have a FORTRAN compiler here chiz)
Shouldn't be a problem, a good programmer can write FORTRAN in any language :)
Crses! Pwned agn :(
-
It's worth bearing in mind, that the more that people do this sort of thing, the faster it is likely to bring acf down, since eventually it'll reach some sort of bandwidth limit, and I doubt sam will rush to deal with it.
If there are certain things which you must have, then all well and good, but I wouldn't just download every post you ever made, "just in case", since out of context of the threads they are in, they are likely to look a bit strange anyway.
-
It's worth bearing in mind, that the more that people do this sort of thing, the faster it is likely to bring acf down, since eventually it'll reach some sort of bandwidth limit, and I doubt sam will rush to deal with it.
What I was going to add. Well said!
-
I just checked my browser history. On one particular day I looked at 1200 unique URLs at ACF.
Assuming each page has, on average, 10 posts that means I used up approximately the same bandwidth that day as I would have downloading 12000 posts, more than double the number I actually did.
The total file size for my 5500 posts was "only" 15MB.
-
WHY OH WHY OH WHY can't we import all the sodding posts into YACF? ??? Nothing else makes sense. This is all turning into a giant pain in the arras. >:(
-
I just checked my browser history. On one particular day I looked at 1200 unique URLs at ACF.
"working from home" again, eh?
::)
-
Beats "tele-working", i.e. lots of tele and not much work.
-
Is someone copying a load just now? Because old ACF is not working atm in The Igloo.
-
WHY OH WHY OH WHY can't we import all the sodding posts into YACF? ??? Nothing else makes sense. This is all turning into a giant pain in the arras. >:(
That is entirely possible, but unless somebody has the right credentials to be able to dump the entire database into one massive sql file, it's unlikely to happen.
If nobody has that access, I doubt the-guy-who-ran-ACF would be particularly forthcoming with the required details.
And my hacking days are long gone :P
-
Fortunately, I no interest at all in my posts. It's you lot I in back to re-read. So this isn't much help to me.
-
Aye
What I worry about is the loss of all that audax lore - and the PBP stuff. That was history-in-realtime.
-
Be nice to get the audax bits and the kit reviews. I'm not worried about saving any of my random mutterings :-X
-
I'd just like to point out, that even though Peter's son's script is very nifty and class A in terms of "thinking outside the box" (or whatever it's called these days) it will only ever work if the database is up and running. As soon as it dies (gets turned off, Sam drops it if you like), it won't work.
For those who have made careful, considered posts - I'd suggest doing a copy/paste action now.
Would that be correct RZ?
-
I've looked through my posts on the other acf, and I have decided that nothing I posted was that earth shattering or deep and meaningful. I did go through and delete some of my old threads, but I really can't be bothered.
A few days ago I said that I wouldn't leave but I have since stopped my payment and am about to delete my account.
-
Can anybody help me download my posts? Ideally I'd like all the threads I posted in - is that possible? I feel that hacking is justifiable.
-
It would be a great pity to lose the ACF library and post of the day. It has from time to time occurred to me you could make a saleable book, even on a small scale, out of the Library posts, interspersed with suitable posts of the day. Warrington did it with their crap cycle lanes.
-
Whilst you may try and retain other people's posts, purely for their use, on a free forum, and might possibly be OK in IP terms. If you tried to publish them in a book, without the authority of everyone whose posts you used, I think you would be on very dodgy grounds.
-
You would indeed, but permission is seekable (sic).
-
Beyond me, I fear.
Me too. It's a different galaxy for me.
No cyle pun intended.
-
The Management do not recommend leeching posts from ACF, but we can't stop you doing it.
Legally we think you're OK unless the traffic increase ends up doing a denial-of-service attack on the server, in which case you can be prosecuted under the Computer Misuse Act, should the BD complain. Morally...well, we think the posts are owned by the people who posted them, but it doesn't stop Google caching them without your permission. WANAL (WANL?), though.
Definitely don't take anything from Private Matters.