Psychometrician Austin Fossey writes about the question type report: "This report can provide a quick profile of the population of the item bank or a topic when needed, though more detailed item tracking by status, topic, metatags, item type, and exposure is advisable for anyone managing a large-scale item development project." Read the article for some potential use cases for this simple report.
If you work with assessment statistics or just about any branch of social science, you may be familiar with Simpson’s paradox—the idea that data trends between subgroups change or disappear when the subgroups are aggregated. There are hundreds of examples of Simpson’s paradox, but for the sake of illustration, here is a simple example from Psychometrician Austin Fossey.
The final step in item development is the psychometric review, which is designed to flag any items that may need to be removed before you build your assessment forms for production.
In this blog post, Psychometrician Austin Fossey digs into difficulty, discrimination, and bias to figure out what is wrong with items that return poor statistics.
LMS are not measuring metrics as expected! With the lightning speed at which we create and send data on the Internet, data analytics tools are a necessity.
Whether you work with low-stakes assessments, small-scale classroom assessments or large-scale, high-stakes assessment, Austin Fossey will help you understand and apply some basic principles of item development that will enhance the quality of your results. Check out part one of the series.
Assessment Report Design : should an assessment report report just one thing or multiple pieces of information? Austin Fossey looks at the intended use of the assessment results and how the stakeholder is using them in this blog article.
Do you want to survey multiple populations of stakeholders who may have different opinions about what content should be on the assessment? You can save time and money by providing demographic comparisons of aggregated JTA responses for each of the populations in the study. Psychometrician Austin Fossey explains how in this blog post.
The next issue of the eLearning Papers will examine the use of analytics to improve learning. How do we collect data on learner behaviours and transform it into useful information to improve learning experiences? A call for papers on this topic is now open. Submissions will be accepted until October 27, 2013.