x
This website is using cookies. We use cookies to ensure that we give you the best experience on our website. More info. That's Fine
HPC:Factor Logo 
 
Latest Forum Activity

The Great Crash of 2007

1 2 3 4 5 6
C:Amie Page Icon Posted 2007-02-18 6:50 PM
#
Avatar image of C:Amie
Administrator
H/PC Oracle

Posts:
16,252
Location:
United Kingdom
Status:
On a scale only (just) surpassed by the great Wall St. crash of 1929. You will likely be aware by now that HPC:Factor has been off-line since the 10th February.

So I'm sure you are all wondering exactly what happened

As you will mostly be aware, the site is run on a trans-Atlantic, with our main operations being in Florida. The servers you are talking to right now are all based in Florida, however a lot of information and all of the development is located in the UK.

On Friday 9th my data storage server failed (no data loss)

This was followed the next day by the failure which you are all aware of. In this case the drives on HPC:Factors main HTTPd server failed (catastrophic data loss).

Finally the drive array on my test server got dropped in software at over the weekend (eventual data loss).

Factoring in the hardware losses, and the fact that the local and off-site database backup stores were lost in the space of a couple of days. The site has been off-line for the period. My data server was required to rebuild the other servers, my test server had the main repository of off-site backups on it and the HTTPd server in reality hasn't been fixed.

There was some relief in that all the content backups were safe, however it has been the database backups that have been the main victims in this affair. The last salvageable Forums backup being from Christmas Eve. Luckily the Central database, because of its fairly recent implementation was fairly up to date on my development system, so data loss there has been limited to a couple of days. The Forum and News/RSS databases are the worst hit, with no recoverable & recent backups being available/safe.

What isn't working?
This is a known list of everything that is known to have been lost from the public site at the time of writing.

  • Home Page Poll >= 14th January 2007

  • Central (HCL/Community/QLink etc) => a.m. 10th February 2007

  • Main News/RSS = 25th December (all else recovered)

  • Forum Posts/Threads >= p.m. 24th December 2006

  • Forums Private Messages (PM) >= p.m. 24th December 2006

  • Forum User Registrations >= p.m. 24th December 2006

  • Forum Attachments = ALL

  • Forum Avatars (stored locally) = ALL

  • Forum Personal Photo's (stored locally) = ALL



If you have cached information covering the missing 25th December posts, I would love to hear from you so we can get them back in. These could be in your browser cache, or in your RSS Aggregator if you have been subscribed to our feeds for more than 2 months. If you do find out that you have something and would like to send it along. I will keep the emergency contact mail address open for a little while longer: fema@hpcfactor.com
Yes; a little humour for our American Friends.


I and the other owners would like to extend specific personal thanks to Gary Emmert and Silvio Vernillo, without whom the reconstruction of the Main News and RSS feed databases would not have been anywhere near as successful as it has been with their gracious assistance.

I would also like to extend my thanks and surprise towards the warm hearted and passionate emails I have received from community members offering support, concern, motivation and kind words.

There are clearly some lessons to be learnt from this ordeal - perhaps a cynic would say I should never go away for a couple of days; and my bank manager might tell me I can't afford to take days off of work to fix servers. The sad truth is that what has happened is primarily the result of a series of unrelated, unfortunate errors which have nothing in common other than bad luck.
The surprising truth is what has turned out to be the size and scale of this website. Perhaps to my eternal detriment, I originally quoted an estimate of 6 hours of upload time to have enough information available to start the site up again. It actually took 45 hours to get there. I'm editing this post right now and the clock behind me is chiming the bells for 1am. A staggering 53 hours later, files are still being restored from my servers to the Florida ones - and I suspect there are a couple more hours yet in line!

I'm not entirely sure in my own mind of everything that has been lost, and also how that is going to impact you guys or the site as a whole. It has been a painful experience and the decisions to keep the site down were not easy; though necessary. Unfortunately the impact of this will be rippling out in gentle waves for some time yet.

Search engines have already started de-listing the site, various people have got themselves into various 'states', the data we have lost contains the thoughts and feeling of you, the H/PC Community and they can never be recovered; at least not in the same spirit. Though I'm confident we'll all be having lengthy and stimulating conversations for a very long time to come.
Technically speaking, the site will have to be placed into a state of flux as its emergency configurations at the moment have to be spanned out into stable production systems. We need new hardware in Florida, it takes time and money and the ability of the current system to cope with site load is a matter of some serious concern.


I would like to end on a personal note, a gripe if you will permit me.

I've served this community for some 7 years, Clinton and John around the same if not a little longer, and here I address a very small minority of those who so contemptibly vocalised their opinions to me. I have been appalled by the reactions of these certain insignificant minority of people throughout this last week.

That we would just flick a switch, and, without so much as a word walk away from all that has been achieved.
To those who sent those venomous emails and thought that after spending so much time and effort in keeping this site alive for so many years, that we would treat the members of this community so disdainfully. I suggest to them that they don't know us at all. I suggest to them that it is they who must learn some tact and decency. And I suggest to them; that they stay out of my way.

I am pleased to say, however, that we are indebted to all of those people - some I know well and others new faces - who did write to Clinton, to John and myself with words of support. They certainly made this headache a lot easier to deal with.

Now, on with the show! Lets get that post count back to 72,000 posts; post haste!

Chris
 Top of the page
wildbill Page Icon Posted 2007-02-18 9:34 PM
#
Avatar image of wildbill
Factorite (Elite)

Posts:
105
Location:
Waterloo, Ontario Canada
Status:
Just glad to see you are up and running-I really missed this site. Checked at least 3 to 4 times everyday. Computers are so much fun-especially when they crash!
All the best!
wildbill
 Top of the page
RTFM Page Icon Posted 2007-02-18 10:13 PM
#
Avatar image of RTFM
H/PC Elite

Posts:
635
Location:
Green Bay, WI
Status:
Although i've only been a member for a short time, i'm glad to see the site functional again. I didnt realize how often i checked the forums daily until i couldn't. I felt like what a drug addict might feel going through detox, and i'm glad that now i can have my daily fix.

Cheers.
 Top of the page
Ceya Page Icon Posted 2007-02-18 10:38 PM
#
Avatar image of Ceya
H/PC Elite

Posts:
703
Status:
CAmie,

Remember the info I gave before on the Printers downloads for the HP 450 cx and cxi as well as HP 990Cse and Cxi .

Thanks for the hard work from your team to get us back up and save as much as you could.

I will check my back up for more data and restore points on my other computer. My family member has it so it may be a couple of days.

S/F,
CEYA!
 Top of the page
CE Geek Page Icon Posted 2007-02-18 10:48 PM
#
Avatar image of CE Geek
Global Moderator
H/PC Oracle

Posts:
12,410
Location:
Southern California
Status:
I thought that's where you got "fema" in the e-mail address.

Geez, I don't even want to talk about the sites I ended up visiting to fill in the time while this site was down. But we did get an interesting new IRC channel going called - yep, you guessed it - #hpcfactor. (We didn't violate any copyrights with that name, did we? )
 Top of the page
Richard Plume Page Icon Posted 2007-02-18 10:54 PM
#
Avatar image of Richard Plume
H/PC Philosopher

Posts:
390
Location:
Toronto, Canada
Status:
I'm very glad you're back. I've missed both the site and the community!

Rick
 Top of the page
Yoldering Page Icon Posted 2007-02-18 10:56 PM
#
Avatar image of Yoldering
H/PC Vanguard

Posts:
2,577
Location:
The Lone Star State
Status:
Wow, I did not realize how much I would miss this site. I know how much work you all have placed into this site, and I am so glad you have it back up!!!

Edit: I might even have to catch you all up on my almost complete nervous breakdown… I may even blame it on HPC:Factor… Ha ha ha… Just kidding… No… I have had a rough week and a half.

Edited by Yoldering 2007-02-18 11:09 PM
 Top of the page
chasmanor
chasmanor Page Icon Posted 2007-02-18 11:09 PM
#
Status:
Gosh, I'm glad you are back.

Sorry you had to go through this and I appreciate all your hard work. This is the best site on the net as far as I'm concerned.

Now try to get a little rest!!
 Top of the page
H2OGuru Page Icon Posted 2007-02-18 11:51 PM
#
Avatar image of H2OGuru
Factorite (Elite)

Posts:
131
Location:
Memphis, TN USA
Status:
Hey guys,

Glad to have you all back again. Like others, I did not know I came here so much. I'm also thankful that just out of pure luck of having too many computers, that I had a laptop that had useful information. You guys are too kind. Thanks.

Gary
 Top of the page
gere Page Icon Posted 2007-02-19 12:51 AM
#
Avatar image of gere
H/PC Newbie

Posts:
10
Location:
United States
Status:
Hi -- sooo glad you're back!

My heart gave a little skip when I clicked on the bookmark and actually got to 'our' page!
Thank you ever-so-much for all your hard work.

Yes, we are all addicted! but ain't it nice!

gere
 Top of the page
rbsfou Page Icon Posted 2007-02-19 1:25 AM
#
Avatar image of rbsfou
Factorite (Elite)

Posts:
169
Location:
United Kingdom
Status:
Great that you are back online and thanks for all your hard work....Sorry you got those kinds of emails, but if it helps i'm pretty sure most of the H/PC community would agree with me in saying that these people are just jerks to have a) assumed it was malicious on your part b) that you could bring it all back at the snap of a finger. The kinds of people who are bosses, and cause many a nervous breakdown (in mine, and i'm sure many others experience!)...

I would have expected that kind of reaction on a big commercial site like eBay, not here however! Looks like not all the stupid arrogant suits dumped their keyboards for their trendy styli a couple years ago

Edited by rbsfou 2007-02-19 1:29 AM
 Top of the page
tomboyed Page Icon Posted 2007-02-19 3:44 AM
#
Avatar image of tomboyed
Factorite (Junior)

Posts:
41
Status:
I am very happy to see that the hpcfactor.com site is up and running again! Although I am a member for not very long. But I really missed the site and members of the community.

Sorry you had to go through this and I appreciate all your hard work. As for me this is the best site for handheld pc.

Thank you very much for all your hardwork.
 Top of the page
chiark Page Icon Posted 2007-02-19 4:09 AM
#
Avatar image of chiark
H/PC Sensei

Posts:
1,326
Location:
North of England
Status:
"An unfortunate scenario of events", to be sure!

Chris, it was a labour of love to get this thing back and working again - well done, and my sincere thanks for your input both in terms of effort and financially (losing days contractor rates is not small beans, let alone keeping this thing running) in getting this back.

HPCFactor: back for good
 Top of the page
Sch01ar Page Icon Posted 2007-02-19 4:26 AM
#
Avatar image of Sch01ar
Factorite (Elite)

Posts:
220
Location:
United Kingdom
Status:
Feels like I'm at home again. Thanks C, for all of your dedication and expertise.
 Top of the page
Half-Saint Page Icon Posted 2007-02-19 5:36 AM
#
Avatar image of Half-Saint
H/PC Philosopher

Posts:
468
Location:
Ljubljana, Slovenia
Status:
Good to see the site back on-line! I was really getting worried! Figured something bad must have happened but I had no idea the crash was so severe. Anyway, I'm glad the site is back up and running. Keep it up!

Cheers
SainT
 Top of the page
1 2 3 4 5 6
Jump to forum:
Seconds to generate: 0.187 - Cached queries : 62 - Executed queries : 18
Server Time now is: Thursday, 19 May 2022 - 8:54