In part two of this review; we will examine the specific content delivery mechanics and interfaces that Coursera uses. This includes video lecture tools, how homeworks are presented, and how students interact with the course. We will also see that Coursera has provided a lot of flexibility, giving teachers a lot of power in how they design and manager their course.
Starting where we left off, the home room for each course acts as the navigation hub for all of the courses resources. This is where students will access the lectures, homework, syllabus, and all other relevant course information. When you first enter, students are greeted with the course announcements (center), the navigation bar (left), and a persistent bulletin for important info (right).
Course instructors have a lot of freedom in how they can customize this page. Each course can customize its own layout and design, within certain constraints. The teachers also have the freedom to build as many navigation links and sub pages as they need for the course.
The navigation bar contains the links to all of the relevant course information, so that students can quickly access any of the courses content. Rather then opening up a new page for each link, the info will be displayed in the center/right content pane (Where the Announcements and Upcoming Deadlines appear).
The problem this presents is that individual sections, like Video Lectures, won't have their own native navigation. Once you click on Video Lectures, you get a dump of all the released video lectures for every week in the course. The same holds true for the course assignments, quiz's, and so on.
You can collapse some of the content tabs, but these changes do not always persist when you refresh the page. Since the top of the list always starts with the first weeks material, this means you are constantly having to scroll though long list of content to reach the current week's material. Its a tradeoff between the teachers ability to customize the course page structure, and being able to have better navigation and control in sub pages.
The Coursera platform has developed its own video player, to which they have added a number of features such as variable play speed, keyboard interactions, and interactive quizzes. When I was first taking courses on this site six months ago, many of these features were not yet functional; but it seems a lot of work has been done since then to get most of it up and working.
The various keyboard controls are a nice touch, and the playback speed option is very much a welcome addition. Occasionally, the variable playback speed won't register when you select it from the drop down box, otherwise everything works fine.
Interactive quizzes are points in which the video pauses, and the user is prompted with a question related to the material they just watched. These questions are purely optional, but serve as a way to actively engage the student and help either reinforce the material or to think outside of the general concepts being presented. This is a very powerful tool when used well.
Unfortunately, I've seen very few Coursera classes use these question prompts to their full potential (beyond parroting back simple concepts), and their answer formats are fairly restrictive. I'll go into that further in the homework section though.
While the implementation of the advanced video features fixes one of the two major complaints I had with the video player, one other issue still remains...
Video Buffering. This was a constant curse inducing frustration in all of my courses. The source of this frustration; video buffering doesn't just affect you when you jump forward in the video, it also occurs when you move backwards in the video as well!
Often times, this resulted in me attempting to move back 5 seconds to repeat a section, only to have the video take 10 seconds to load. Every single time.
Being able to scan through a video is one of the major points of having online video lectures in the first place. It's used to take notes, repeat a section in order to reinforce the lesson, or replay a specific animation or demo. Constantly having that process interrupted by loading times that are twice as long as the content you want to replay is infuriating.
I don't understand the necessity for their video player to stream the content, instead of being able to pre-load it in like so many other video players. If the only reason for this is to support the interactive quizzes then I would say it isn't worth it. After having seen other MOOC's implement interactive quizzes without this constraint, I find it completely unacceptable.
Update (June/25/2013): In my latest classes, the video buffering no longer seems to be an issue. Moving forward or back in the video, once it has been loaded, happens in split seconds. There are still some quirks, such as the +/- speed controls not always acting consistently, but the system is much better now.
Coursera gave each course a great deal of freedom in how teachers choose to handle these assignments. They have the ability to set both soft and hard deadlines, provide optional 'late days' to extend deadlines, allow students to attempt homework's multiple times with optional penalties, and allow students to review previous attempts at the assignment. They also have a lot of variability in how they construct the problem sets, including different type of question formats and the ability to choose randomly from a set of questions.
With all of these options, course can end up differing from each other greatly. To start off with, lets take a look at the different question formats that are used.
For each individual homework there is a limited number of ways to present the questions. Most questions are made up of either multiple choice (radio buttons - select only one answer), multiple selection (check boxes - select all that apply), or text box answers.
All of the questions for the homework are submitted and graded at the same time, unlike other MOOC's where you can submit each questions 'final answer' separately. This shifts the focus from solving difficult stand alone questions, to solving a large number of easier problems (or maybe not always so easy...). Assignments are graded instantly, giving students feedback on their grade as soon as they submit their answers.
Asking the right questions
While there is some variability in the types of questions you can ask, this system is still limited in some ways; as I foreshadowed in the interactive quiz section.
Questions involving multiple choice or true and false are fairly limited in how engaging they can be to a student. Often times process of elimination or brute force can trivialize figuring out the answer, which negates the benefit of learning the actual method to solving the problem in the first place.
There is also the issue that having multiple attempts at multiple choice questions can trivialize the assignment. Teachers do have some ways to counteract this, such as being able to have the multiple answers from the question chosen from a random pool so that it is not the same every time. They can also apply grade penalty to repeated attempts at the homework, or make true/false questions worth only one point.
Ideally the student should be able to answer free form or text box answers to the question. Unfortunately the text box answer format is rather strictly restricted. Most of the answers have to be specifically formatted, including proper spacing and commas. The text boxes also lack the ability to deal with reduced/expanded forms of fraction or +/- 5% variability when dealing with precision questions.
This can result in the student knowing the correct answer to a problem, but still getting marked wrong because of answer format clarity. So far, the courses I've taken have dealt with this by primarily relying on the multiple choice format and only sparingly using free form answers. This is one of the reasons the interactive quizzes during video lectures aren't as useful as they could be as well.
Coming from another course which was able to identify both expanded and reduced forms of complex algebraic equations, this seems very limited. Probably, this is one point in which Coursera could gain a lot by collaborating with some of the other MOOC's to improve their system.
There are two other methods of submitting work to be graded. One of which was the peer review essay system mentioned in part 1 of this review. Students write an essay of some restricted length (no more then 500 to 1000 words), and then must grade five classmates essays before the deadline. The final assignment grade is an average of the peer review scores from other students.
The other method involves students submitting files or even programming code to be graded. I've seen this used effectively in several course on both MOOC platforms now; and it has been used to grade webpages, image files, and computer programs among other things. These systems are set up to run the students code, and then compare the output against a number of test, all of which are automated. The classes I've seen which have incorporated this type of lab or practical course work have been some of the most fulfilling to date.
Teachers also have control over a number of settings in how the assignments are actually graded.
One example of this is whether they allow students multiple attempts to complete a specific assignment. In one of my courses, I had up to four attempts to retry a homework and would receive the highest grade out of all attempts. During each attempt, it would switch the possible choices in the multiple choice questions.
Another course allowed up to ten attempts for each homework, but each subsequent attempt would have its score reduced by 10%, and gave you different equations to solve each time. Lets take a look at an example...
For this particularly difficult assignment, I received an effective score of 45/60 from my second attempt (No I will not tell you what I got on my first attempt). This means I got an actual score of 50/60, but because this was a second attempt I received a 10% penalty off of that 50.
I then went on to attempt the homework two more times, and in my last attempt I received a perfect 60/60. However at this point, that grade would have had a 40% penalty and only been worth 36/60. My final grade ended up coming from the second attempt since it was the highest.
Even though the penalty means that you eventually won't be able to beat your previous scores, it still allows you to get valuable practice in on the assignment.
There are a number of courses which have applied variations to this grading system, and a similar process can be applied to the exams and quizzes as well. Sometimes you only get one attempt, sometimes the penalty doesn't occur until the third attempt, some can even offer extra credit questions which won't count against your score. This allows each course to be tailored as necessary to its content and difficulty.
The immediate feedback students get for individual assignments is great. Unfortunately coursera lacks a way to track your overall grade or completion progress of the course. While you can see each of your individual scores for assignments, there is no indication as to how each assignment is weighted compared to your total grade. In the case of the class I had above, assignments ranged from a maximum of 50 to 85 points, and those assignments only made up 70% of my grade.
While the data is there and I could calculate it by hand if I wanted to, it is often spread out all over the place in the various homework, project, and exam subsections. It would be much better if there was one centralized place where you could track your progress for all of your grades, so you could easily measure your long term progress in the course.
Discussion and Community
The forums section for each course are a little bit lacking. There are some basic forums like General Discussion, Lectures, and Homework; but again they lack any subsections or organization to differentiate homework 2 related post from homework 10. The only quick filter options are primarily newest thread, most recent post, or most popular. Other then that you have to rely on a key word search, and whether or not a student used some kind of convention in tagging or naming their thread.
There is also nothing connecting individual lecture videos or assignment pages to their related forum post. The forums section is completely separated from any of the other course content sections. At best it is a bare minimum of features to create a functionally mediocre forum.
As for other community collaboration tools, it looks like they have recently added pages for a course wiki or organizing local meetups. Both of these sections seem relatively new, and are not necessarily fully flushed out yet. The course meetup page seems to be specific to location (aka Washington D.C., Baltimore, ect) but don't make any mention of the actual classes... which seems odd.
So what are my thoughts on Coursera, and how it compares to other similar MOOC platforms?
Coursera's most amazing asset is the speed in which it has added new courses to its platform. They have a huge course selection (hundreds compared to the dozens on other sites), covering a wide range of topics. One of the most likely reasons you'll find yourself on the site is to attend a course that isn't offered anywhere else. Coursera also does a lot to give a large amount of control to the individual teachers to design their course, both in the site design and in the assignment mechanics.
For all of that, it still feels like Coursera is a work in progress. Which it is to be fair, but this is in comparison to other platforms which haven't been around as long as Coursera and yet seem more polished. There are a number of convenience features that, once you've experienced them somewhere else, its hard not to think of them as being obvious. There are also a few usability issues such as the video player buffering, and mediocre forums that remain frustrating to deal with.
The verdict is still out on whether any of their premium services they might come up with will seem valuable to me; but it is hard to not resent it a little that they have intentionally watered down the sense of accomplishment from completing some of their courses (Unless you pay for their signature track).
More Open Education!
I like the idea of there being multiple MOOC platforms. I've even heard that a few new ones have opened up over in the UK and in Germany recently.
'Competition' in this area means diversity in design, such that different platforms will be willing to try new things, explore in different directions, and figure out what works and what doesn't in large scale online classrooms. As all of the current MOOC platforms are collaborating, it will be interesting to see what kind of technologies and solutions they can share with one another.
I for one, am looking forward to taking my next Coursera class (Cryptography II), this summer. Or maybe even picking up a new course before then. Hmm...