Tools
Simple Assessment
Assessment Tool for Scoring Evidence of Demonstrating CC2020 Dispositions
This simple tools suppoprts instructors in evaluating a student’s demonstration of the CC2020 dispositions. The tool applies the expectation for competence set out in ISO 24773: to apply knowledge and skills […] successful[ly] […] on an ongoing basis. This implies that each disposition must be demonstrated in multiple activities over the duration of the student’s course.
The tool is based on counting items of evidence that demonstrate particular dispositions. It follows that there needs to be some record of such items of evidence - whether an expolict portfolio, a lab notebook, a professional development logbook or whatever, which can be iin any reasonable format, either electronic or paper-based. The portfolio can cover a defined period ranging from an individual module to the entire program.
Each item of evidence (portfolio item) should be validated - that is, based on evidence, or corroborated, or confirmed by a supervisor or teacher, and should also be identifiable, by date, entry number, page number, a url or similar.
The tool then performs an overall evaluation by counting the number of portfolio item references entered against each disposition.
The outcome is “Pass” if the (configurable) thresholds have been met, otherwise “not pass yet” if the student still needs to develop some of the dispositions.
The thresholds are derived from the values set for the configuration parameters, which are defined on the next sheet.
Usage Notes
The configuration, input and output areas are denoted by shading:
- configuration
- input: references to the portfolio items
- output: *thresholds, scores, and outcomes
Non-input cells are locked.
Users who wish to reconfigure this tool for a different number of dispositions should contact the authors for a partially-unlocked version.
Go to the top of Tools
Enhanced Assessment
An assessment tool for scoring evidence for the demonstration of selected CC2020 dispositions
This enhanced tool supports instructors in evaluating a student’s demonstration of selected CC2020 dispositions.
The tool applies the expectation for competence set out in ISO 24773: to apply knowledge and skills […] successful[ly] […] on an ongoing basis. This implies that each disposition must be demonstrated in multiple activities over the duration of the student’s course.
The distinctive feature of this tool is that it allows the designation of each disposition as “essential” or “supplementary”. This is intended to support instructors of courses that focus on particular dispositions (labelled “essential” for the purpose of assessment) to provide feedback to students on how well they have demonstrated those dispositions.
Dispositions are labelled as Essential (“E”) or Supplementary (“S”) in the parameter entry area, B17:B27. NB - only “E”s are counted - any other character is effectively equivalent to “S”.
The tool is based on counting items of evidence that demonstrate particular dispositions. It follows that there needs to be some record of such items of evidence - whether an expolict portfolio, a lab notebook, a professional development logbook or whatever, which can be iin any reasonable format, either electronic or paper-based. The portfolio can cover a defined period ranging from an individual module to the entire program.
Each item of evidence (portfolio item) should be validated - that is, based on evidence, or corroborated, or confirmed by a supervisor or teacher, and should also be identifiable, by date, entry number, page number, a url or similar.
Each portfolio item may record evidence of the successful demonstration of (aspects of) zero or more dispositions. The assessment required is simply to read each portfolio item and decide which dispositions, if any, have been demonstrated. A reference to the portfolio item is then entered against each disposition; the format of the reference is irrelevant - as long as it can be de-referenced!
The tool then performs an overall evaluation by counting the number of portfolio item references entered against each disposition.
The outcome is “Pass” if the (configurable) thresholds have been met, otherwise “not pass yet” if the student still needs to develop some of the selected dispositions.
The thresholds are derived from the values set for the configuration parameters, which are defined on the next sheet.
Usage Notes
The configuration, input and output areas are denoted by shading:
- configuration
- input: references to the portfolio items
- output: *thresholds, scores, and outcomes
Non-input cells are locked.
The dispositions to be evaluated using this tool should be classified, in column B (rows 17:27), as Essential “E” or Supplementary “S”. Note that portfolio item references against any disposition - whichever its classification - will count towards the overall item score, S_ID.
Go to the top of Tools
SFIA-based Assessment
An assessment tool for scoring evidence for the demonstration of SFIA Level 3 responsibility characteristics and indirectly for the CC2020 dispositions to which they map
This tool supports instructors in evaluating a student’s demonstration of human factors expressed in terms of the Level 3 responsibility characteristics of SFIA v8 [5]. Sinced these responsibility characterisdtics have been mapped to the CC2020 dispositions4, the tool also supports indirect assessment against the dispositions.
Assessment of demonstration of SFIA responsibility characteristics
The tool applies the expectation for competence set out in ISO 24773: to apply knowledge and skills […] successful[ly] […] on an ongoing basis. This implies that each characteristic must be demonstrated in multiple activities over the duration of the student’s course.
There are 24 responsibility characteristics for SFIA Level 3 , and some may not be crucial in an educational setting. Hence, the tool allows the designation of each characteristic as “essential” or “supplementary”.
Characteristics are labelled as Essential (“E”) or Supplementary (“S”) in the parameter entry area of the SFIA sheet, B17:B27.
NB - only “E”s are counted - any other character is effectively equivalent
to “S”.
The tool is based on counting items of evidence that demonstrate particular charcteristics. It follows that there needs to be some record of such items of evidence - whether an expolict portfolio, a lab notebook, a professional development logbook or whatever, which can be iin any reasonable format, either electronic or paper-based. The portfolio can cover a defined period ranging from an individual module to the entire program.
Each item of evidence (portfolio item) should be validated - that is, based on evidence, or corroborated, or confirmed by a supervisor or teacher, and should also be identifiable, by date, entry number, page number, a url or similar.
Each portfolio item may record evidence of the successful demonstration of (aspects of) zero or more characteristics. The assessment required is simply to read each portfolio item and decide which characteristics, if any, have been demonstrated. A reference to the portfolio item is then entered against each characteristic which it demonstrates; the format of the reference is irrelevant - as long as it can be de-referenced!
The tool then performs an overall evaluation by counting the number of portfolio item references entered against each characteristic.
The outcome is “Pass” if the (configurable) thresholds have been met, otherwise “not pass yet” if the student still needs to develop some of the “essential” characteristics.
The thresholds are derived from the values set for the configuration parameters, which are defined on the next sheet.
Mapping of assessments to CC2020 dispositions to give an indirect assessment
The mapping built into the tool is that reported and validated in [4]. The mapping is many-to-many - that is, each SFIA responsibility characteristic is mapped to one or more dispositions, and each disposition has more than one characteristic mapped to it.
Hence, the scores for each disposition are normalised by both the number of characteristics mapped to each disposition, nch, and the number of dispositions to which each characteristic is mapped, n_d. Note that n_ch may be different for each disposition, and n_d may be different for each characteristic.
First, the number of references for each characteristic is divided by √nd to give a partial score for each disposition to which it is mapped; the partial scores for each disposition are then summed, and the sum divided by √nch to give the score for the disposition. That score is compared with the disposition threshold determined by the configuration to check whether or not the disposition has been demonstrated. Finally, the number of dispositions demonstrated and overall scores are compared with the high-level thresholds derived from the configuration to give an indirect assessment of the demonstration of the CC2020 dispositions.
The approach and calculations are described more fully in [1].
Usage Notes
The configuration, input and output areas are denoted by shading:
- configuration
- input: references to the portfolio items
- output: *thresholds, scores, and outcomes
Non-input cells are locked.
The SFIA responsibility characteristics whose demonstrate is to be assessed using this tool should be classified, in column B (rows 17:27) of the SFIA sheet, as Essential “E” or Supplementary “S”. Note that portfolio item references against any disposition - whichever its classification - will count towards the overall item score, S_ID.
The mapping between SFIA responsibility characteristics and CC2020 dispositions is built into the tool.
On the “CC2020 dispositions assessment” sheet, the SFIA characteristics contributing to each disposition are initially hidden: they can be exposed - together with indicators for the items entered against each, and the partial scores for the characteristics - by clicking on the “+” sign just below and to the left of the disposition name.
Go to the top of Tools