The website is designed from the ground up in Python and Django – completed in 5 weeks. It has a social media focus, integrating Facebook, Twitter and a number of other Web 2.0 technologies. The audience is quite large - the site was in the top 10,000 websites in the United States by traffic when launched and receives millions of hits. The social media footprint is also sizable - 313,289 Twitter followers, 131,253 Facebook Fans and numerous integration points with distributed social networks.
We began with a multi-server architecture, so the site could scale easily, and deployed the servers behind our own custom load balancer at Media Temple. We tried several different caching strategies, but settled on memcached in Apache as it let us handle the most users per server. We scripted as much as we could for migration, but with 30,000+ pages, there was a lot more done by hand than we would have liked.
With such a large amount of content, searchability was essential. For this we built a custom technology stack based on Solr, Lucene and Haystack. Then, to keep all the previous links pointing in the right direction, we created a customizable scheme of Django regular expressions and Apache mod_rewrites.
Full blog post on the development of the site here.