I have always enjoyed data and problem solving, which has come in handy during my time as a computer science major at the University of North Carolina at Chapel Hill. When the opportunity to undertake a product analytics internship at Quizlet materialized, I quickly accepted the role, as I knew it would leverage my analytics skills and help me develop a sense for product.
I was excited to live in San Francisco and experience a more technical internship than my previous public health and operations internships. After discussing with my manager and analytics team, my role would be to analyze various A/B tests, learn about data pipelines, and support the other interns’ data needs in analytics office hours.
Some projects I was able to work on included looking at the increased traffic when Quizlet embedded flashcards onto the set page, working with a marketing intern to set up a data dashboard measuring the positive impact of Quizlet’s annual Unconference for teachers, and measuring the tradeoff between engagement and monetization through an A/B test focused on improving user study activity on Quizlet. What did I learn?
Prior to jumping on board at Quizlet, I had learned basic SQL and had taken a series of courses in statistics (meaning, I could pull data from a database and knew a handful of significance tests.) However, I had never learned about how data is stored, accessed and analyzed at scale.
Quizlet, like many other companies, collects lots of data, but they need a way to aggregate that data to use it to make decisions for the business. In order to do these summations, ETLs come into play. An ETL stand for extract, transform, load. Every night, data is extracted from the database, transformed so aggregations and manipulations can be performed, and loaded back into the database as a new table with useful information for company stakeholders.
QA or quality assurance was a concept I was aware of, but I did not have a grasp on the importance of sanity checking gathered data. After a few trials of calculating a statistic, asking a manger to check it, and realizing something was not right, I knew I had to do a better QA-ing my analyses. I learned to break down data I was gathering and check that I was not inadvertently dropping data in steps when combining data across tables. While this is simple in nature, it definitely took a few analyses before I started getting into this habit.
I also learned that when gathering data for a project, the first step is to understand the metrics available and then pose the right question. Initially, I would leap to the most challenging question I could think of … before another person on my team would say, “Let’s take that one step back.” (Which I definitely needed to do!) Starting with simpler questions is a better way to gut check the data and make sure the numbers seem realistic before moving to more complex questions.
A PM’s Role
At the start of the summer, I was also determined to learn about product management. As a rising senior at UNC, I am nervous about what career path to take and what jobs I should be applying to in the coming months. I know I love communication and technology, so I thought product management might be a good career for me to shadow during my time at Quizlet.
One of Quizlet’s values is “teach yourself something new,” so towards the end of my time I was able to partner with one of the product managers to learn about developing a product spec, creating a test plan, and working with a cross-functional team to get a project off the ground.
One of Quizlet’s product teams is working on matching students and teachers to relevant content. In its current stage, a lot of the work is about exploration and research. I learned the steps surrounding preparation for a project with this focus including estimation, meeting with the engineers for any features that need to be developed, and meeting with the analysts for test plans. I was able to compile the work into the product requirements document and test plan. The different areas of focus that a PM has to complete, including communication and research, makes me interested in pursuing this type of role.
An SQL cake to celebrate this summer's data interns.
Learning from My Mistake
I made my intern mistake of the summer about halfway through my internship. As a preface, it was fixable, but it did cause one of our most used analytics tables to be down for approximately five days.
The problem started when I misspelled a column name, and in trying to fix it, I inadvertently unpartitioned the table. To explain this in English, let’s take a step back. When using BigQuery, the application Quizlet uses to pull data, tables of data can be stored in what is essentially separate folders each day. The data is separated into folders by different partitions. These partitions can be a number of things, but most commonly the partition used is a date. These different partitions allow the data team to pull data only on the days we need rather than ALL of the data, reducing cost and time. When the data is unpartitioned, or no longer in these groupings by date, all of the data is stored in one folder, raising costs and time. The world did not end when I made this spelling mistake. I had a great and supportive community to help me pull everything back together after this mishap. Through this mistake, I was also able to learn about documentation that occurs when mistakes occur, called a post-mortem. We went through what we could improve on as a team and how to prevent from a mistake like this happening in the future.
Thank you to everyone who mentored me and shared their passion for education technology with me this summer. I know the analytical skills and sense of product I developed this summer will help me as I enter the tech field full time next year.