Calculating scaled score using Item Response Theory (IRT, 3PL) in R with MIRT package

user_10822183 0
Viewed 440

I have an English Listening Comprehension test consisting of 50 items taken by about 400 students. I would like to score the test using the TOEFL scale (31-68), and it is claimed that TOEFL is scored using IRT (3PL model). I am using MIRT package in R to obtain three parameters, as I used 3PL model.

library(readxl)

TOEFL006 <- read_excel("TOEFL Prediction Mei 2021 Form 006.xlsx")
TOEFL006_LIST <- TOEFL006[, 9:58]

library(mirt)

sv <- mirt(TOEFL006_LIST,                # data frame (ordinal only)
       1,                                # 1 for unidimentional, 2 for exploratory 
       itemtype = '3PL')                 # models, i.e. Rasch, 1PL, 2PL, and 3PL

sv_coeffs <- coef(sv,
              simplify = T,
              IRTpars = T)

sv_coeffs

The result is shown below :

a b g u
L1 2.198 0.165 0.198 1
L2 2.254 0.117 0.248 1
L3 2.103 -0.049 0.232 1
L4 4.663 0.293 0.248 1
L5 1.612 -0.374 0.001 1
... ... ... ... ...

Then I calculated the factor score using the following codes:

factor_score <- fscores(sv, method = "EAP", full.scores = T)

The first five results are shown below:

head(factor_score)

[1,]  2.1839888
[2,]  1.8886260
[3,]  0.6791995
[4,]  1.2761837
[5,]  0.8195919
[6,] -1.5257231

The problem is that I do not know how to convert the factor score into the TOEFL score of 31 - 68 (as shown on the ETS website for listening score: https://www.ets.org/s/toefl_itp/pdf/38781-TOEFL-itp-flyer_Level1_HR.pdf).

Would anyone help me show how I can do that in R, please? Or maybe there are other ways of obtaining students' scores. Your help is much appreciated.

The data can be downloaded here: https://drive.google.com/file/d/1WwwjzgxJRBByCXAjdlNkGNRCtXjMlddW/view?usp=sharing

Thank you very much for your help.

r
3 Answers

I think you misunderstood something. It doesn't seem to be scored using IRT - according to wikipedia, it's just summed up, which is a quite normal way to score tests. Maybe I misunderstood it, but if it's done like you say, I think it uses a proprietary scoring model and you should likely contact and pay them to get you your scores.

So to get the scores, you would simply do TOEFL006_LIST$total <- RowSums(TOEFL006_LIST) and get the head using head(TOEFL006_LIST$total).

Also, I'm glad not to be a student of yours - you're putting their names out in the open in your linked dataset... If you're in Europe it would be illegal, possibly in other countries too.

user_67916990

I was looking for a solution to your question and I found that you can use this function to get a homologous score from a given model and theta value.

expected.test(model, as.matrix(theta))

theta should be specifically a matrix.

user_167435780

To calculate each respondents ability score use:

Theta <- fscores(mod, method = 'MAP', itemtype = '2-PL', stats.only = TRUE) head(personfit(mod, Theta=Theta))

Working through a similar project and found it in the mirt help files :)

user_22025800
Related