What's the Point of Coursera's Signature Track?

The question of whether or not to pay for the Signature Track option offered on many of Coursera's courses regularly crops up on the discussion forums. Even those who are already paying for the option are frequently unclear or mistaken as to its benefits. Here's my view, for what it's worth.

In my opinion, most students will see no benefit from the use of Signature Track and I'll explain why. Firstly, we should be clear what Signature Track is and what it offers. Signature Track is an identity verification service. Through the use of photo identification, web cam monitoring and keyboard profiling (basically recognising how you type) the operators can say that the person taking the course is the person named on the certificate. What the scheme does not offer is any accreditation of courses or certificates. Signature Track certificates share most features with the standard (free) Statements of Accomplishment; they are delivered electronically as pdf files, they do not indicate a grade and they are non-accredited. Unlike the standard Statements of Accomplishment, they carry both the Coursera logo and that of the content supplier (ie the college or university presenting the course). The free certificates carry only the Coursera logo although they do name the content provider.

Will colleges accept Coursera courses for credit? I've seen no indication that any college has accepted a course for credit with or without the benefit of Signature Track. There are, I think, a couple of reasons. Firstly, Signature Track, while verifying the identity of the person at the keyboard, does not replace proctored (in UK terms, invigilated) examinations. There is nothing to stop the student having a supporter 'off-camera' feeding them the answers or, indeed, from registering multiple accounts in order to access the examination in advance of their Signature Track attempt. Secondly, if I may be forgiven some cynicism, the colleges have a vested interest in recognising only paid courses for credit. If they were to routinely recognise free courses where would that leave the faculties formerly offering the courses? General Education courses, for example, seem particularly vulnerable as they are effectively standalone, so don't need to match later courses in terms of content, and seem to be relatively poorly valued by many students. These courses do, however, by dint of the large numbers required to take them, generate significant income for colleges. I would suspect that, for example, PSYCH101 or CS101 courses taken by many hundreds of students as a part of their Gen. Ed. requirement cross-subsidise more advanced Psychology or Computer Science courses taken by much smaller numbers.

So, will employers accept Coursera courses as adding a benefit to applicants or employee's CVs (resumes)? Here, in my experience, the situation is quite different from the world of academia. Employers are generally far less wedded to the concept of 'accreditation'. I've taken many professional development courses over the years, some costing up to £500 per day (say $750) but few have ever had any external accreditation. Indeed, I used to present such courses. But does Signature Track offer any additional benefit in this market? I don't think so. I've never been asked to confirm my identity on a professional development course and employers have never questioned this. Both in my former career as an engineer and in my more recent work as a teacher, employers are happy to take development claims at face value. It is important, however, to recognise that most employers are looking for evidence of vocationally relevant training rather than primarily academic courses so the direct benefits might be marginal.

There are some very limited circumstances in which Signature Track might have benefits. A small number of courses, for example, offer the option of proctored examinations in order to obtain credit for a course. In the two I have seen Signature Track is a prerequisite of registration for the examinations. I have also seen one or two students report that employers will only accept courses as fulfilling development requirements if they take the the Signature Track option (although I suspect this arises more from lack of understanding on the part of employers then from any inherent value).

The final point I would make is that this is largely academic (so to speak). Research indicates that most students of MOOCs are already graduates and a high proportion actually have post-graduate ('graduate' in US terms) qualifications. Few will have any intention of seeking credit for the courses undertaken here. From discussions, it seems that many view Signature Track as a cross between a vanity product and a way of offering financial support to the course supplier and Coursera. The profits from Signature Track are split between Coursera and the course providers. Those taking an altruistic view should, of course, remember that Coursera is a for-profit corporation, not a charity, and Signature Track is merely one of the ways that their operations are being monetized.

Coursera: The User Experience

I talked in very broad terms last time about Coursera. This time I want to look at how the user interacts with the platform and how well the various features of Coursera courses work. The advantage to course providers of using a platform like Coursera is that much of the technical work has been done and is (by this point) working meaning that the academics can focus on what they do best; teaching. The disadvantage can be that sometimes the fit between a course and the platform is not perfect - but that's something I'll come back to in a future  post.

So. let's take a look at a typical course. Once registered and logged in you will arrive in the main landing page. Here's one I am studying a the moment:

To the right you will see upcoming assignments and newly posted lectures. To the left you can see the main navigation strip which gives you links to all the sections of the course. The parts which you will probably visit most regularly will be the video lectures, which are the core of most courses, the forums and the assignments. In most cases there are both quizzes and peer assessments which will each have their own button. On this course, unusually, there is only peer assessment which, for some reason, the course providers have chosen to call Peer Responses - which was almost guaranteed to cause confusion when the first assessment came around.

The naming and arrangement of the navigation button is one of the most irritating features of Coursera. Clearly, the course providers are given free rein in naming the buttons and choosing the order in which they appear. What this means is that those who are taking more than one course will spend far too long looking blankly at the navigation bar wondering where, say, the Discussion Forum button has gone. Here are a few I took at random from current and past courses:

Sorry for the large images today. As you can, hopefully, see, there is very little in common between these courses - take a quick scan across and see how long it takes to spot the Discussion Forum link on each course. Now, I'm certain each course designer is sure that their layout best suits the material but perhaps this is an area where Coursera could reasonably lay down some stricter design rules.

Among the useful buttons there are also some which seem pretty much unused. There is always a Join a Meetup link although I've never heard of anyone actually doing so - I guess that Coursera has some arrangement with Meetup. There is usually a link to a Course Wiki although this feature is both rough around the edges and underused. The Wiki is still in beta, it often asks for a separate login (which requires the same credentials as the main site) and really does seem to be an afterthought bolted on to the structure. There is little evidence that the wikis are widely used. With a few honourable exceptions, most content on the wikis appears to have been posted by staff - not quite the point of a wiki.

The one thing you won't see anywhere is a Progress button because, for some reason, Coursera is convinced that nobody is interested in how they are doing points-wise. This means that working out your current percentage requires a tortuous collection of points from peer assignments, quizzes and exams. These then need to be weighted in line with the individual course policy - except that is often not given in detail (e.g, "the quizzes will account for about a third of the marks")  or, in some cases, not stated at all! For Coursera courses, the first thing I do when starting is to make up a simple spreadsheet to calculate my running total. I will then know when I've done enough to pass and also be able to calculate whether a 'Distinction' is still within reach.

Discussion Forums are undoubtedly one of Coursera's strengths (as is also the case with edX). Due to the very high numbers of students looking at the same material at the same time support is rarely more than a few minutes away. If you have a problem a post will normally see a quick response. There is, of course, some petty bickering, points scoring and randomly bad behaviour from the online equivalent of schoolyard bullies (and often with the same level of maturity) but overall it can be a major asset for those with the time and inclination to participate. Although it may not replace a night in a student union bar, it is undoubtedly more productive in addressing problems with understanding!

Take care when posting on the forums, particularly on a busy thread, as the default setting is to subscribe you for email updates. This can mean that, after posting on a couple of topics, you come home to find hundreds (literally) of email updates filling your inbox. This is exactly the opposite of Saylor.org where the default setting does not subscribe you where, given the quiet boards there, it would be sensible to do so.

Next time I'll look at one or two courses in more detail and see how they work.

Coursera: The Biggest MOOC Operator, But Are They The Best?

This time I’ll take a look at the biggest to the MOOC players, Coursera. Spawned from the same Stanford pilot that resulted in the formation of Udacity, Coursera is a commercial educational undertaking. Courses are available free of charge to students but there is a paid-for option, Signature Track, which I’ll discuss a little later.

Coursera has the greatest number of courses listed of any of the MOOC operators, 386 at the time of writing, produced by a number of university partners. Most of the partners are US-based although there are a growing number of international contributors. As you might expect, almost all courses are taught in English but there are small numbers presented in Spanish, French, Chinese, German and Italian.

So what are the courses like? All courses follow a fairly standard pattern based around a series of video lectures. Courses are normally offered on a ‘synchronous’ basis, that is to say, all the students start at the same time and follow weekly lectures with assignments having set deadlines for submission. Courses run from between 5 and 12 weeks and have an estimated 4-12 hours per weeks study load.

A very small number of courses are run on a ‘self-paced’ basis where all resources are available immediately and students can dip in and out as they like. The only ones I've seen to date are a couple of older courses from Stanford but it may well be a route pursued by other providers for courses they no longer wish to fully support.

Students undertake a range of assignments which can include auto-graded quizzes, peer evaluations and course specific activities such as machine-graded software submissions. Many courses also include one or more examinations. Overall grading varies from course to course with different weights being applied to each element.

Quiz-style assessments are multiple choice, single word or numeric questions – pretty much as you’d expect given the state of assessment technology. The advantage is that they are marked instantly and feedback can be given on how answers are derived. Most of the quizzes I have seen offer more than one attempt. There are disadvantages to the format which are mainly around the relative inflexibility but they can still be very challenging.

Peer evaluation is used very commonly on Coursera, often being the largest part of the overall mark for a course. This method of assessment allows the review of material which is outside the scope of machine grading. Such assignments might include written work, such as essays, graphical products like graphs or pieces of artwork. I won’t go into my views on peer evaluation here but will return to the topic in a future post.

Peer evaluated work runs in three phases. The first step is to submit your own work, in most cases the submission phase is open for about a week. Once the submission period ends work is allocated to peers for assessment. In most cases each student who has submitted is allocated five pieces for review. If a student doesn't complete their quota of evaluations then they won’t receive their own mark. One thing to look out for is self-evaluation; a few courses make this compulsory with a 20% grade penalty for non-completion! The final stage is to receive your feedback.

Exams, where they are used, use the same mechanism as the quizzes. Exams are timed (although this is usually fairly generous, for example 24 hours from beginning) and most only allow a single attempt. As with quizzes, marks are available immediately. 

Certificates – Statements of Accomplishment – are provided to students who meet the criteria. Usually that means getting 70% overall (although that varies) with some courses also offering a ‘with Distinction’ grading for those achieving the top grades (eg over 90%). Coursera also offer the option to pay for a ‘verified’ certificate through their Signature Track scheme. I’ll come back to Signature Track in a later post.

Next time I’ll look at the user interface and course experience on Coursera.

The Lifelong Learner's Ten Top Tips for Effective Study with MOOCs

From discussions on various courses it is apparent that many of those taking part have either been out of formal education for some time or never really put too much thought into the study process in the first place! Here are my top tips for effective study online. Some of these apply to any study and come from my own experience as a student and a High School teacher, others are specific to online learning and have been developed to take into account research findings and, of course, my own experience.

I make no claims to have any earth-shattering secrets but, if you apply these ideas, you should find that you make more effective use of  your study time and faster progress. As with all study, the more you put in, the more you will take out - so good luck.

1. Set up folders on your computer or online to keep your electronic files together and in order (I have one for each course which, in turn has a folder for each Unit or week). Keep your paper files in equally good order! Having an orderly filing system will mean you can find information more quickly which is important for revision and during exams (which are invariably 'open book'). These files will also be your main reference after the  course is finished so look after them - hopefully, you'll want to use the information again.

2. Set aside some undisturbed time, ideally on a regular schedule (I know, dream on) and try to work in the same place. Having a settled routine and a regular study environment helps to focus the mind and improves recall (honestly, it does). I know that one of the attractions of MOOCs is that you can study as and when time allows but, although this makes use of time which might otherwise be wasted, it really isn't as good as settled study. I should admit that I do about 25-30% of my study in odd short bursts between other tasks but ideally... 

3. Make notes. Depending on your preferences either buy yourself some new notepads and pens (or pencils!) or polish up your latest note-taking software. Make notes regularly as you work through the material. Especially for the video lectures, finding your way back to review something without notes can be quite difficult. 

4. Download videos and audio lectures rather than streaming so they are available for review even if you are offline. Remember that materials may disappear quite soon after a course completes and since most MOOCs are built around video lectures you could find yourself left with only pleasant memories... If your course doesn't offer the facility to download the lectures but uses YouTube videos then find yourself a downloader with which you are comfortable (I use YTD Downloader without any snags). 

5. Plan your time. Mark any due dates for assessments in your planner, either on your computer (or online) or in your desk diary. Be realistic about the time you have available and leave some 'contingency time' for when things go wrong. Working around any set dates, decide how much time each course needs and divide up the time and work to allow you to be ready in good time to submit your assessments. Don't forget to mark in your holidays or you may find yourself falling out with your loved ones! When you are planning your study time...

6. Don't try to work in big continuous blocks - it just isn't efficient. Normally no more than an hour is recommended on a single topic. Either take a break or switch subjects - one hour of Computer Science then half an hour of Psychology. A hidden benefit of the General Education requirement (should you be a Saylor student), is that you will always have 'something else' to study!

7. Review your notes regularly. Try to structure and restructure the information in different ways to help learning and understanding. I like to use visual representations (I would say mindmaps but Tony Buzan has registered that as a trademark!), you might just want to summarise or bullet point, or transfer to study cards (does anyone else still do that) or anything that requires you to change the format around. This process of synthesis (taking information and putting it together in a new form) is one of the key ways to strengthen learning.

  

8. Do all the exercises. Nobody's looking over your shoulder to make sure you do but we're all adults now and surely know who will lose out if you take short cuts. As well as helping to secure learning, these exercises can be a useful resource later in revision.

9. Don't put it off.  Leaving a difficult exercise until the last minute won't make it any easier. If your course has a more relaxed attitude to deadlines (or is self-paced like most Udacity and all Saylor courses) this doesn't mean you should let things drag on. If you don't make regular progress then you will waste much of your study time reacquainting yourself with the previous material. If it is self-paced then set yourself targets and plan to meet them. But...

10. Be flexible. It can be difficult to judge time requirements when you are studying new subjects or have been away from study for a while. The guidelines given by course providers are quite good - for average students. 

If you find yourself struggling to keep up then take stock. Is it a short term problem, for example a difficult new concept or an overlap of courses for a couple of weeks, or a major miscalculation of time requirements? The former you can just ride out with a bit of extra time squeezed out somewhere. the latter means you need to make a decision. 

While I encourage everyone to keep going when problems crop up, you need also to be realistic. There is no penalty for dropping a course and it (or another similar) will be along shortly. Decide which courses are most important to you, maybe a later course will rely on one you are studying now, and which you can postpone. Make a decision and move on. Another option pretty much unique to the current online offerings is to move to an 'auditing' mode. That means you 'sit in' on the lectures but don't worry about assessments or deadlines. This can be useful if you plan to retake the course later or maybe if you just want some background before going on to related study areas.

11. Because ten is never enough. Remember why you are studying. Enjoy it and don't get too stressed.

Next time I'll be taking a closer look at the largest of the MOOC platforms, Coursera.

Saylor - A Five Step Programme

I've written a little about the overall structure of Saylor and something about the courses so this is just a short practical guide to getting started.

All courses are self-paced and open to enrolment at any time so you can jump in whenever you like. In fact, as I said last time, you don't actually need to enrol or even register on the site, you can just turn up and start learning! So where do you begin? A number of students seem to get confused simply  by the open nature; it is as if you arrived on your first day at university to find the doors flung open and a note saying 'help yourself'. So this is my five step guide to getting started:

Step 1: Click on the ePortfolio link at the the top of the page and register. you don't  have to but it will give more options - which I'll discuss in a later posting.

Step 2: While in your new ePortfolio, click on the 'Course Catalog' link. You'll now see in the centre of the screen a list of every course offered in order of course code from ARTH101 (Art History) to STS203 (History of Technology) and on the left (in large blue lettering) a list of Areas of Study (ie 'Majors') from Art History down to Psychology. Click on one of the Areas of Study and you'll see just the courses which contribute to that area. Browse around, see what interests you and take a look at the first units for a couple of courses.

Step 3: Back to your ePortfolio for a rest... 

Step 3A: If you have decided on an area of specialisation then click on the 'Areas of Study' tab and choose your area. Click on the area and you will be taken to a list of the required courses. Select as many or as few as you like to register immediately.

Step 3B: If you haven't decided yet or just want to complete one or two individual courses, click on 'Add a course' (which takes you right back to the Course Catalog) and click on the 'Enrol' button next to the courses you want.

Step 4: Back in your ePortfolio, click on the 'Courses' tab, if you're not already there, then on the course that you want to study first.

Step 5: Start at Unit 1 and keep going....

Good luck with your studies and next time I'll offer a few hints and tips on how to make the most effective use of your time.

Saylor: Close Up on the Courses

As I promised, I'll look in a bit more detail at Saylor's courses this time. We obviously can't go through every unit of every course but I'll try to give a flavour of how learning works in this environment.

Getting started. There's no need to register (although the ePortfolio adds useful functionality), just choose the course you want and start. The courses are all laid out in the same way on screen so once you've found your way around the first course you are set! Each course starts with sections for the Purpose of Course, Course Information, Learning Outcomes and Requirements. You will hopefully have looked at these when deciding whether to take the course. Here you can find the time estimate, which is normally around 100-120 hours, and any pre-requisites. You may also find information about the core texts. Some courses make extensive use of one or two (electronic) texts which it might make sense to download as single complete volumes at the start rather than collecting endless separate chapters and excerpts.

Jumping into the first unit you will find that it is broken down into numbered sections and sub-sections. Click on anything which is blue and it will expand with further information or  instructions and links to resources. You might be told to read certain parts of a linked document, watch a video (in whole or part) or listen to an audio recording. Around about this point you begin to realise that somebody is taking this all very seriously. The work really is at college/university level. 

The materials. One thing you will notice straight away is that the resources used are generally hosted elsewhere and come from a wide range of sources. There are some Saylor originated pieces - mainly introductions, summaries or bridging pieces but the general policy is to use existing freely available resources. This can lead to a bit of a patchwork effect as you swap from book to book, site to site and lecturer to  lecturer. On the other hand it possibly reflects better the nature of  independent study and academic literature reviews at higher levels than would a course built entirely around a single book and a single lecturer (ie the typical undergraduate university pattern). I find it effective and stimulating, others might not like it so much. The quality of the materials is almost perfect; there are a very few pieces which really don't match the overall standard but Saylor are constantly reviewing courses so hopefully these will be improved as time goes by.

Let's look at POLSC101: Introduction to Politics. A nice gentle introductory course in the Gen Ed programme... It gets underway with a nine page reading from Jean-Marc Coicard on Political Legitimacy, followed by twenty nine pages of analysis of soft power under Obama then back to the 18th Century for twenty two pages from Jean Jacques Rousseau’s A Discourse on Political Economy followed up by two chapters from Harold D. Lasswell’s 1935 Politics: Who Gets What, When, How?. There are a few more readings and then a rest for an hour while you take in a video lecture. Finish off with two lengthy entries from the Stanford Encyclopedia of Philosophy and it's time to complete the first assessment. Official 'Time Advisory', 11.5 hours. It took me over twice that although I did get tempted off into some interesting sidetracks.

Assessment activities within a course are generally self-assessed against supplied rubrics or have an answer sheet supplied. There's nobody looking over your shoulder so you can cheat if you like - but it is only yourself you are cheating so why bother. If you can't get to grips with self-assessment or just want some feedback you can try posting on the course forum. I should warn you that these are very quiet at the moment but are seeing more activity as students start to find Saylor. There are auto-graded activities on some courses. It depends what others have prepared and made available. 

The Final Exam is the only assessment which counts for the overall grade. This is a two hour machine-graded multiple choice quiz and, despite all my doubts, they are tough exams which dig into every corner of the courses. Exams are 'open book' but as the questions often require the student to apply the learning rather than just recall answers, this isn't always helpful. The pass mark is 70%. 

Again, to give an example, despite being an experienced teacher, examiner, a normally rapid examinee and despite having been coached by my wife, a doctor, psychology graduate and soon to be Master, I took nearly the whole two hours to get through the PSYCH101 paper and was happy to pass with 78%. That's nearly two and a half minutes per question. I'd normally expect to crash through multiple choice papers at a rate two to four times as rapid.

So what are the courses really like? They are true representations of the academic standards of mainstream universities - so expect to be challenged. You can also expect to spend quite some time on the courses. I've found the time estimates to be pretty good (even if I've taken longer where interests lead me astray) so one hundred hours plus for most courses. As in real life you must get through extensive readings alongside video lectures and other materials. There is a decent amount of variety within courses and they are well put together.

What do I like? I like the challenging standards, the readings (because this is how I learn best) and the fact that the courses are part of an overall structure. I quite like the self-paced nature of the courses although this also leads to temptation... One of the major assets is the staff, both administrative and academic. Queries are usually answered quickly and Professors are prowling the boards on a daily basis.

What don't I like? The place is far too quiet - too many course forums have no postings at all! As you might guess from this blog, I like to talk! The nature of the site means that some resources disappear from time to time (these are highlighted as red text). I've not come across any in my own studies... yet! There are a few rough edges on the system software, such as the sporadic requests to sign in as you move between courses, ePortfolio and forums but nothing too drastic. 

What would I like even more? More courses (how about a Physics or Electrical Engineering major and what about the French, Spanish and Russian language courses that have been 'due shortly' for the last year), more students and more attractive certificates (I know, it's trivial and below me but even so, they look like they were designed in Microsoft Publisher over a coffee break - and they misspell 'verification').

Conclusion. To go back to my comment in the last posting, the courses are great. If you want real learning in a fairly traditional style then get down to Saylor.org and take a look at what they have to offer. Whether you want to sign up for a full four year 'degree' or just pop in to use part of one course then you are free to do so. The academic style may not suit everybody (but that is true of all university study) and the self-paced nature does mean you need more self-discipline but it really is well worth it.

Next time, a quick Five Step Guide to getting started on Saylor. After that I'll move on to some hints and tips for more effective studying whichever platform you choose.

Saylor.org - The Gem Hidden in Plain Sight

My topic for the next couple of postings is Saylor.org. A site that offers nearly 300 undergraduate courses and the unique opportunity to build a programme of study that emulates a full US-style four year degree - and all for free! The question that you should be asking (because it's the question I have asked) is why, if it is so great, has nobody heard of it?

The Saylor Foundation (more about this later) has been involved in free education for some years but only set out on its current mission about eighteen months ago. Their goal is to provide free, quality education to anyone with internet access.  Saylor has taken a quite different approach to this task than the better known platforms such as Coursera or edX. Rather than offering a haphazard assortment of courses supplied by existing universities, Saylor has recruited it's own academic staff (on a consulting basis) to construct a coherent set of courses based primarily on existing freely available online resources. 

The initial, and ambitious, target was to cover the most popular subject areas in US colleges. Rather than starting with a handful of courses and gradually growing, Saylor opened on day one with over two hundred courses (although, in fairness, not all were complete). There are now courses offered providing the opportunity to 'major' in fourteen different subjects from Computer Science to Art History and Mechanical Engineering to Psychology. There are also a range of Electives which basically cover odds and ends that don't fit into existing specialisations (Astronomy, for example) and General Education which provides for the requirement to study outside your own specialist area (this is a standard part of the US four year degree although unknown here in the UK).

So what is it like? Firstly, I should say that it is quite unlike the more 'flashy' MOOC platforms. It is rather conservative in appearance and has an beautifully uniform design across every course. Everything you see says that this is a serious academic venture and, as we shall see, that is borne out by the courses themselves.  Clicking on any of the Areas of Study (as the specialist subjects are known) will take you to a page detailing the courses necessary to build a degree majoring in that subject - these include mandatory core courses as well as electives which can take the degree down different specialist tracks. As I mentioned, to complete the full programme you will also need to undertake the Gen Ed element.

Saylor is totally open-source; materials can be used as the students wish (subject to whatever limitations the original suppliers had although most are issued under Creative Commons licences). Students need not even register to access the courses although you will have to do so if you want to take the exams and collect your certificates. Registering will also give you access the discussion forums and to an ePortfolio where you can manage your courses, showcase your work and form study groups.

So what about the courses? Well, I'll go into more detail about the courses in tomorrow's posting but for today I'll start with the conclusion - they're great! The courses are very much at the level I would expect for undergraduate study, unlike some other MOOC offerings which are rather watered down. They are heavyweight and satisfying with plenty meat to them (apologies to my vegetarian friends for the metaphor). Unlike virtually any other courses I've seen online there are substantial required readings as well as the usual video lectures. There are no concessions and no kid gloves - this is the real thing.

If you are serious about learning then you won't find much better than Saylor. Go and take a look then come back tomorrow for my next instalment where I'll look at some of the courses and report on my experiences as a student there.