Christian Bokhove

…wonderful life

Journalclub at ResearchED

I think it’s a great idea to study articles in a journalclub setting. I read the article as well and made the following annotations:

See file attached to this message

File: 00220671%2E2013%2E807491 – annotated.pdf

Annotation summary:

— Page 2 —

Seems to me to be quite a limited view of Academic Performance, including only reading.

Must keep in mind: 2000 data. Unfortunately it is quite normal to use ‘older’ data. Some of the delay lies in publication mechanisms so PISA 2012 data released December 2013 is difficult but there are other instances.

I think multilevel analysis is useful but not everyone would agree.

PISA, so contextual effects. Differences countries?

— Page 4 —

Would be good to compare with TIMSS and PIRLS.

This is the ‘standard’ PISA sampling strategy.

I’m not sure if it already was the case for PISA 2000 but now most large-scale assessments need to take into account the complex sampling design. This means sampling WEIGHTS (because of non-response) and so-called PLAUSIBLE VALUES (because not all students make all test items). There is no mention if this in the paper so either PISA 2000 was done differently or they just ‘forgot’.

According to some (e.g. Willms) this is an appropriate substitute for the actual sampling design.

— Page 5 —

In general I would want more transparency regarding missing data and the ‘model building’, see Dedrick et al. (2009) for recommendations.

— Page 6 —

With large N results will be significant very quickly. Often ‘effect size’ is mentioned.

But see how much more is explained over the models: not much more, it seems.

— Page 7 —

So only the first PLAUSIBLE VALUE was used.

Not much ‘variance explained’

— Page 8 —

Stop with ‘thus’

— Page 9 —

Cognitive Load anyone?

Important: cause and effect

So this ‘significant’ marginally interesting because of large N.

Advertisements

One comment on “Journalclub at ResearchED

  1. Pingback: ResearchED 2015 | Christian Bokhove

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Information

This entry was posted on September 5, 2015 by in Education, Education Research.
%d bloggers like this: