Some of you may have noticed some spotty service on Quizlet in the last few days. We’ve simply become overloaded with our popularity. Unfortunately for us, that means we have to bring the site down briefly to add more server capacity. We’re going to do this sooner rather than later — i.e. TODAY!

We’re expecting to go down around 5pm EST for as many as 5 hours (if all goes well, it’ll be less). We’ll be keeping everyone updated on our quizlet twitter feed. We know it’s short notice and extremely inconvenient, so thank you very much for your patience.

We just recovered from a couple hours of downtime. It wasn’t too fun, for you or for me. The database that was holding each set’s words and definitions had been crashing for the past few days, causing daily downtime (ack!). So tonight I decided just repairing it each time wouldn’t do, so I converted some of the database architecture to a (hopefully) better and more robust state. I also temporarily disabled the “online now” icons, which were causing some problems. I’m hoping to reinstate those as soon as possible – but right now I want to make sure the site just stays up, you know?

We’re in the process of hiring a sysadmin, so we should have a lot more expertise on this problem soon. Going down every evening is not acceptable. Sorry to everyone who couldn’t use it while the site was down.

UPDATE: Some users have reported finding their groups deleted. I’ve tracked this problem down to some code on the groups page that will delete a group when there are no members in the group. When the Quizlet database was up-and-down today, sometimes the users table was not working but the groups were. So groups that were visited when the users temporarily disappeared may have gotten deleted. Fortunately I have a back-up from early this morning with all of the groups. It looks like about 15 groups may have been deleted. I’m going to work on getting those back up as soon as possible. Sorry for the trouble.

UPDATE 2: Restoring the data from today’s early-morning backup went very smoothly. All deleted groups should be back up. Post in the comments if you’re still out. Sorry again.

In the past few weeks – especially the past few days – Quizlet has become intolerably slow at times. The service is sometimes super speedy, but sometimes laggy or even unresponsive.

This morning I made some big changes to some of the database configuration. The reoccurring problem some of you may have noticed with the “Flagrant System Error” appears to be over with. I’ve taken care of the problem (too many mysql db connections) that was causing all of the previous database downtime.

Fixing that problem may have caused other problems – it’s too early to tell. What I can say is that Quizlet should be online a greater percentage of the time, but it may still be slow.

These problems are growing pains – which is a good thing in a way. The problems mostly appear in peak activity times, which is usually the middle of the day on weekdays. Quizlet is averaging about 120,000 page views per weekday – a lot of traffic for a single server.

Moving into the future, these performance problems are my first priority. In the immediate future, I’m putting all my Quizlet time into improving Quizlet’s performance over adding new features. One thing that’s coming in the near future is an overhaul of the Search page, which has a tendency of timing out of late. I use Quizlet all the time too, so it hurts me just as much as anyone when it has problems.

The problem is that while I’ve built quite a few websites in my day, none of them have ever seen the kind of high-performance needs that Quizlet has. So I’ve been feeling around in the dark to some extent up until now when it comes to performance. Now I’m being a lot more proactive about it, and I have several people helping with the site’s performance now. I could still use more help however, so if you are a mysql/apache/solaris expert and you’re interested in getting involved in Quizlet, send an email to andrew/quizlet.com.

Because I discussed it earlier, I thought I’d let you all know that my MIT application is all sent in and done with. I’ll find out. You can bet your boots that Quizlet factored heavily into my application – so thank you all for helping it grow and reach where it is today.

I just got Quizlet back online after an annoyingly long stretch of time – it was about 10 hours long. I hit a hard disk space wall, which is the same problem I had last time (the only other time) Quizlet went down.

I want to extend a sincere apology to all of you who tried to use Quizlet today. I know how frustrating it can be when something you depend on isn’t working. I’m sorry. Over the next few days I’m going to work on implementing some more preventative measures to make sure this sort of thing doesn’t happen again. And I’ll be investing in some sort of monitoring service to make sure I know the instant Quizlet tanks out.

It’s a good thing this happened on a Saturday, eh? Saturday is always the low point of the week for traffic, which is perfectly logical considering that no one studies on saturdays. Well, most people I mean. If you study on Saturday that’s cool too. OK bed time for me!

Perhaps you’ve been noticing some intermittent slowness when using Quizlet. I’ve been noticing it too – 30 second load times for each page are no fun. Sometimes it’s nice and snappy, sometimes it’s agonizingly slow. I believe this is an issue with Quizlet’s host, and not any “growing pains” within Quizlet. Their site is slow at the same times that Quizlet is slow.

Apart from that, I’ve been making changes here and there to improve performance and fix bugs. There was a fiasco with smileys a few days ago, but they’re back up and usable again. There’s nothing epic to say here in this post…Sorry

[UPDATE May 29th, 2007]
I just twiddled with some settings a bit and improved some mysql settings – these slowness problems should be fixed. You should notice some extra snap in the page loads now. I especially helped out the Quizmarklet, which is AT LEAST 15x faster now. It’s now very enjoyable to use. Yay.

Some errant multi-gigabyte server log files were running up my disk space and causing the MySQL server to not allow new data and eventually just shut off. The good news is that no data was lost, and everything can just pick up where it left off. This isn’t really my area of expertise, so I’m digging in and figuring out why all this happened and how I can prevent it in the future. It’s safe for the immediate future.

I arrived at school this morning and several friends informed me that Quizlet was down. I fortunately arrived early enough to run home and start figuring out the problem (contacting my host, Joyent) before school. I got back to school (we’re taking those stupid STAR standardized tests all this week) and fortunately I was in a room with plenty of computers so I could monitor the bringing-up progress. Then just a few minutes ago Joyent’s support staff came to my rescue and got everything back up. I’m on lunch break now, but I’ll be available later today to resolve issues. If you have any problems, please send me a message.

-Andrew

There was a bug in learn mode this morning that was causing it to fail terrifically . Sorry, it was completely my fault.

Not that it matters to anyone, but the problem was one character in my 30,000 lines of code. I found the right character to fix though

It’s fixed now. Thank you to the 17 people who reported it. Sorry again!
-Andrew

Update: it appears I had it programmed right on my local copy, but there was a bug in the code that synced the files from local copies of Quizlet to Quizlet.com. So um, I just reuploaded EVERYTHING in case the problem happened to more than just the Learn page. If you’re still having problems, you can email me at andrew at the domain that this website is, or AIM me (sn: Harc Serf).

Sorry again.

While I’m here, I might as well give a quick update on Scatter. I’ve built it to work outside of Quizlet (as well as inside it), so you will be able to embed it in your own site as a sort of widget. I’ve been testing it on some other sites, and it works beautifully. I’m really looking forward to releasing it! Today I also completed the automatic saving of “Ignore” and “Prompt” preferences. These features will be released in the next major upgrade of Quizlet.

Even more tangential, I upgraded this blog to Wordpress 2.1 today. It’s a great update, I highly recommend it if you use WP for your blog.