HOW TO: Survive the Twitter Effect

“The Twitter Effect” – also known as The Mashable Effect or The Digg Effect – as become a topic of conversation lately due to a handful of blogs being taken offline from the amount of traffic instantaneously sent to them from Twitter.

The basics of the effect are this:  a link is posted by someone on Twitter, that link is “retweeted” (forwarded, in email-speak) again and again and again.  If it’s a popular or timely topic, this can cause a surge of traffic to that one page on your site almost instantly.

At Good Plum, we’ve been lucky enough to have a couple of these situations in the past week – both started by @Mashable‘s Pete Cashmore linking to a couple of the posts here, as well as guest posts I’ve written for Mashable.  The flood of visitors added up quickly and at one point, there was many hundreds of simultaneous readers on 1 or 2 pages of the site.  Server logs show a spike, but no where near peaking our processing power.  How did we survive this?

This amount of simultaneous visitors trying to access the same content on a blog can take many servers down temporarily.  We have a couple basic tools in place that help prevent this from happening and we’d like to share:

Tip 1 – Grow Up
Get OFF your shared (or grid) server.  I really don’t care much what technical people tell me about how their shared system grows with my site, etc. — I’ve had too many experiences where the accessibility is way too unpredictable.

We use a virtual private server (VPS) from ServInt here at Good Plum.  A virtual private server gives you dedicated resources (drive space, processor power) to keep your site and database running smooth.  It’s essentially giving you a full dedicated server for a fraction of the cost.

A good VPS from ServInt is only going to cost you about $49 a month to start – probably about double what you might pay for a decent shared server – but the long-term benefits are clear.  Flexibility, reliability, and performanace.  Who wants to lose out on hundreds and thousands of potential readers/visitors just because we have a cheap web host?

Tip 2 – Cache It
On your WordPress blog (which MANY people reading this are using), do you know the amount of stress a lot of visitors at once has on your database?  It’s amazing if you monitor database activity during a high-traffic rush.  WordPress performs so many queries that it’s difficult for it to perform those queries hundreds of time, all at the same time.  The effect is that it shuts down and you lose your traffic until you get the server reset (or, much worse, you actually go over some of your host’s imposed limits and they shut you down).

The ultimate tool to prevent WordPress overload is…..

WP Super Cache.  Super Cache is a WordPress plugin that actually caches (stores) copies of a dynamically generated page.  It then allows subsequent visitors to view that stored copy, instead of having to generate the page all over again for the next visitor, and the next, etc.

Once you activate the plugin and someone visits a page, the page pulls what it needs from the database just like it normally would.  But then it stores that page to a folder on your server.  Next time someone visits that page (or hundreds of people at once), it has the cached version of that page to display instead of having to pull it from the database again!  I can’t stress enough how much this saves your server.

Of course, if you update the page or someone comments on an article, the cache is cleared so the new version is available to the next person.

–
Are YOU prepared for the Twitter Effect?  You never know when someone with some social juice will start a retweeting avalanche that could either shut you down, or bring you a thousand new fans!

Posted on Feb 12, 2009
Filed under: Your Business and tagged with: digg, how-to, mashable, server, twitter, web hosting.