Summary

This quarter, we added two major features to Calibrate: the beta release of Job Profile Validations and Job Profile Clusters. Job Profile Validations allow Job Profile Editors to invite Reviewers to validate and provide feedback about their Job Profiles. Job Profile Clusters are a way of bundling together several validated Job Profiles in order to identify the most common critical skills among them.

In addition to these two major features, many of the interfaces, screens and work flows of Calibrate were improved, making them easier to use and/or adding needed functionality. Furthermore, we continued our efforts to fix bugs and improve application performance and response times.

Job Profile Validations

Job Profile Validations enable Job Profile Editors to gather reviews of their Job Profile from industry and subject matter experts. A Job Profile Validation is a two-step process. First, editors invite Reviewers, then Reviewers Validate the Job Profile by providing feedback on the skills included and recommending additional ones. Two new sections were added to Calibrate and Job Profiles were updated to enable this functionality.

Job Profiles now include a state called "Review". This new state indicates that a Job Profile has been sent out for Validation.

Once in Review, Job Profiles display feedback from Reviewers as it comes in. New Skills added by Reviewers are displayed as Pending Suggestions, allowing Editors the option to accept or reject each Skill.

There are two sources of feedback for agreed-upon skills. First, the median ranking of all the Reviewers is displayed. Second, ranking discrepancies, where Reviewers and Editors disagree in terms of skill criticality, are highlighted. This feedback gives Editors an at-a-glance view of whether or not there is general agreement between what people who do the job think and what people who teach the skills for that job think.

Additionally, each skill includes a detailed breakdown of how each type of Reviewer  ranked the skill in an easy to read stacked bar chart.

The new Validate section in Calibrate allows Editors to create Job Profile Validations and manage Reviewers on those Validations. Both existing and new Reviewers can be invited to validate a Job Profile. Inviting new Reviewers only requires inputting their name and an email address.

The Reviewer section was created specifically for Reviewers to review Job Profiles. From this interface, Reviewers can perform three basic tasks:

1.   Rank an existing skill
2.   Mark an existing skill as irrelevant
3.   Suggest a new skill (along with its ranking)

Job Profile Clusters

Job Profile Clusters are created in a similar way to Job Profiles. Instead of adding Blueprints however, users add published Job Profiles to create a Cluster.

When a Job Profile Cluster is created, Calibrate smartly combines, sorts, and orders the skills so that it is easy to see the most common and critical skills. Isolating the most critical skills across a large number of Job Profiles allows curriculum editors to align their Calibrate Awards with only the most important skills. This ensures that graduates are equipped to succeed, no matter which job they accept.

Other Notable Features

Blueprint Search

Rather than scrolling through the list, it is now possible to search for Blueprints when editing a Job Profile by using the new search Blueprints feature.

More Detailed Award Alignment Reports

The Award Alignment Reports now include a new section for each Course, showing how the skills in that Course map to the Course Outcomes.