IT Horror Stories and Things you can do to Prevent them from Happening to You

  • 0
  • October 9, 2013

With Halloween rounding the corner we here at Interface thought it would be fun to look up some horrific IT stories to share with our website visitors as well as what we would suggest to prevent this from happening to you.

Toy Story 2 was almost lost

In the age of tape backups of 1998 Toy Story 2 was almost completely deleted by an errant Unix command. Almost 90% of the files and sequences that Pixar had created had disappeared before someone pulled the plug on the server executing the remove command. Pixar did have a tape backup in place but unfortunately did not use best practices for the tapes, which resulted in an inconsistent age of files, parts of files were missing and metadata was off, their backups were unreliable, the movie seemed unsalvageable, 100 million dollars’ worth of work gone. What ended up saving the movie was luckily an employee that had gone on maternity leave had a slightly older but fully intact copy of the movie on her home machine so she could work remotely while taking care of her child.

Solution: Backup technology has progressed quite a bit in the 15 years since the creation of Toy Story 2 in 1998. The story of Pixar’s near disaster is a story heard in a smaller scale in businesses everywhere, make sure your company has a backup solution that’s reliable, redundant and secure. Head over to our remote backup page for more information.

Sasser Worm causes billions of dollars in damage

In 2004 an 18-year old German computer science student created a worm that took advantage of an exploit in Windows XP and Windows 2000 systems, the exploit allowed Sasser to spread through to other computers via network port. Any computer that did not have the latest Windows update or a well configured firewall was exposed to the Sasser Worm. While there was no malicious code built into the worm Sasser did cause slowdowns and crashing of computers causing high profile damage. The estimated cost damage of the worm totals approximately 18.1 billion dollars. It is estimated that the Sasser worm infected millions of computers. Finland’s Sampo bank had to close all 130 offices, Delta Air Lines had to cancel several flights due to their computer systems being bogged down by the worm, and these are just some of the many cases of which Sasser interrupted normal business.

Solution: Keeping your office computers updated is paramount to preventing large scale worms and viruses from entering your office network via exploits. Each individual workstation should also be running Anti-Virus software in the case of a virus getting into the system. On the network level your office should be behind an effective firewall. Even without malicious code Sasser caused a large amount of interruption, there are viruses out there with the intent of stealing data and a good security setup is a priority in keeping your data safe. We can take care of all of that on your business’ behalf, check out our Outsourced IT Support page for more details.

Duct Tape doesn’t fix everything

This one is taken from the excellent Nine Real-Life IT Horror Stories post from PC World. “This one still makes us laugh over beers,” says H. Foreman, an admin at a Midwest-based organization. “We were growing pretty well in 2004 and 2005, so we opened an office across the street,” Foreman says. To connect the two offices, they decided to buy two microwave bridges. “The setup is easy enough that we were able to do the job ourselves, though we had professional carpenters install the bridges to the walls of each building, just under the roof, pointing through double-paned office glass, so we would have no weather worries.” Success carried over into 2006, when the company decided to extend its leases. “As part of the deal, they get permission to put up a fancy sign near the top of both buildings — indoors but facing outward through the windows,” Foreman says. “The day the sign goes up, our network goes down for about 15 minutes. We’re still doing the basic set of troubleshooting diagnostics when it suddenly comes back up. Our guy shrugs, verifies everything again, and lets it go.” The next morning wasn’t as forgiving. The network went down and stayed down. “The basic software diagnostics aren’t working, so we go to physical link monitoring,” Foreman says. “Pretty quick, we see that one of the bridges isn’t responding anymore. Upstairs we go.” Apparently the bridges had been in the way of the signs. “The outfit that put up the signs just detached the bridges and moved them — outside,” Foreman says. “There was a balcony on the upper floor and they just moved both bridges out there and then duct-taped both of them to the railings. “What kills us is that the network somehow recovered the first time,” he says. “The duct tape across the street held, but the one on our side slipped off during the night and the bridge fell eight stories, bounced off the dumpster, and landed behind it. The sign installers apparently left a note explaining what they’d done with the receptionist across the street and she hadn’t passed it on.” Naturally, Foreman and company had fun pointing the finger at the sign company in front of the CEO, who then ran out to chew out the install rep. “But as soon as he left the room, the CIO, who is a really good ex-tech, pointed out that if we knew someone was going to be doing construction around a critical piece of network infrastructure, why the hell hadn’t we gone up there to check it? Especially after the network went down during the construction process,” Foreman says. “He had a point.” Solution: The basic network monitoring software this company was evidently using is as good a technology solution as you need in this instance. Without such software, however, this would have been a much nastier adventure.

Does your office currently have a pro-active network monitoring solution? If not, head over to our managed it services page for more information.

Ready to get started? Call us at 416-367-2500 or Request a Quote