Hey all –

We just recovered from a couple hours of downtime. It wasn’t too fun, for you or for me. The database that was holding each set’s words and definitions had been crashing for the past few days, causing daily downtime (ack!). So tonight I decided just repairing it each time wouldn’t do, so I converted some of the database architecture to a (hopefully) better and more robust state. I also temporarily disabled the “online now” icons, which were causing some problems. I’m hoping to reinstate those as soon as possible - but right now I want to make sure the site just stays up, you know?

We’re in the process of hiring a sysadmin, so we should have a lot more expertise on this problem soon. Going down every evening is not acceptable. Sorry to everyone who couldn’t use it while the site was down.

Hey everyone. Sorry for the rough stretch today. I just got Quizlet back up after some pretty heinous downtime. Everything should be 100% back to normal, and none of your data should be lost. Feel free to moan in the comments.

UPDATE: Some users have reported finding their groups deleted. I’ve tracked this problem down to some code on the groups page that will delete a group when there are no members in the group. When the Quizlet database was up-and-down today, sometimes the users table was not working but the groups were. So groups that were visited when the users temporarily disappeared may have gotten deleted. Fortunately I have a back-up from early this morning with all of the groups. It looks like about 15 groups may have been deleted. I’m going to work on getting those back up as soon as possible. Sorry for the trouble.

UPDATE 2: Restoring the data from today’s early-morning backup went very smoothly. All deleted groups should be back up. Post in the comments if you’re still out. Sorry again.

Hey y’all - it’s been awhile. I don’t have any new features to announce tonight, but I’d just like to take this time to keep you in the loop about what’s going on inside Quizlet right now.

In the past few weeks - especially the past few days - Quizlet has become intolerably slow at times. The service is sometimes super speedy, but sometimes laggy or even unresponsive.

This morning I made some big changes to some of the database configuration. The reoccurring problem some of you may have noticed with the “Flagrant System Error” appears to be over with. I’ve taken care of the problem (too many mysql db connections) that was causing all of the previous database downtime.

Fixing that problem may have caused other problems - it’s too early to tell. What I can say is that Quizlet should be online a greater percentage of the time, but it may still be slow.

These problems are growing pains - which is a good thing in a way. The problems mostly appear in peak activity times, which is usually the middle of the day on weekdays. Quizlet is averaging about 120,000 page views per weekday - a lot of traffic for a single server.

Moving into the future, these performance problems are my first priority. In the immediate future, I’m putting all my Quizlet time into improving Quizlet’s performance over adding new features. One thing that’s coming in the near future is an overhaul of the Search page, which has a tendency of timing out of late. I use Quizlet all the time too, so it hurts me just as much as anyone when it has problems.

The problem is that while I’ve built quite a few websites in my day, none of them have ever seen the kind of high-performance needs that Quizlet has. So I’ve been feeling around in the dark to some extent up until now when it comes to performance. Now I’m being a lot more proactive about it, and I have several people helping with the site’s performance now. I could still use more help however, so if you are a mysql/apache/solaris expert and you’re interested in getting involved in Quizlet, send an email to andrew/quizlet.com.

Because I discussed it earlier, I thought I’d let you all know that my MIT application is all sent in and done with. I’ll find out. You can bet your boots that Quizlet factored heavily into my application - so thank you all for helping it grow and reach where it is today.

Hey all,

I just got Quizlet back online after an annoyingly long stretch of time - it was about 10 hours long. I hit a hard disk space wall, which is the same problem I had last time (the only other time) Quizlet went down.

I want to extend a sincere apology to all of you who tried to use Quizlet today. I know how frustrating it can be when something you depend on isn’t working. I’m sorry. Over the next few days I’m going to work on implementing some more preventative measures to make sure this sort of thing doesn’t happen again. And I’ll be investing in some sort of monitoring service to make sure I know the instant Quizlet tanks out.

It’s a good thing this happened on a Saturday, eh? Saturday is always the low point of the week for traffic, which is perfectly logical considering that no one studies on saturdays. Well, most people I mean. If you study on Saturday that’s cool too. OK bed time for me!

Hey all. It’s been awhile since I last posted - sorry about that. Just want to give a status update…

Perhaps you’ve been noticing some intermittent slowness when using Quizlet. I’ve been noticing it too - 30 second load times for each page are no fun. Sometimes it’s nice and snappy, sometimes it’s agonizingly slow. I believe this is an issue with Quizlet’s host, and not any “growing pains” within Quizlet. Their site is slow at the same times that Quizlet is slow.

Apart from that, I’ve been making changes here and there to improve performance and fix bugs. There was a fiasco with smileys a few days ago, but they’re back up and usable again. There’s nothing epic to say here in this post…Sorry

[UPDATE May 29th, 2007]
I just twiddled with some settings a bit and improved some mysql settings - these slowness problems should be fixed. You should notice some extra snap in the page loads now. I especially helped out the Quizmarklet, which is AT LEAST 15x faster now. It’s now very enjoyable to use. Yay.

Good morning! It’s been a rough day, with Quizlet experiencing its first major downtime ever. Quizlet is now back and rolling as normal. I believe the problem started around 6 AM, and has just been resolved at 12:45 PM. My sincerest apologies go out to everyone who couldn’t use Quizlet this morning.

Some errant multi-gigabyte server log files were running up my disk space and causing the MySQL server to not allow new data and eventually just shut off. The good news is that no data was lost, and everything can just pick up where it left off. This isn’t really my area of expertise, so I’m digging in and figuring out why all this happened and how I can prevent it in the future. It’s safe for the immediate future.

I arrived at school this morning and several friends informed me that Quizlet was down. I fortunately arrived early enough to run home and start figuring out the problem (contacting my host, Joyent) before school. I got back to school (we’re taking those stupid STAR standardized tests all this week) and fortunately I was in a room with plenty of computers so I could monitor the bringing-up progress. Then just a few minutes ago Joyent’s support staff came to my rescue and got everything back up. I’m on lunch break now, but I’ll be available later today to resolve issues. If you have any problems, please send me a message.

-Andrew