Using open data for
research assessment
in the Netherlands

Open enough to get a foot in the door

Matthijs de Zwaan

m.c.de.zwaan@vu.nl

Vrije Universiteit Amsterdam

23 September 2024, Sorbonne

Context: Strategy evaluation protocol

 

…evaluate a research unit in light of its own aims and strategy…

…coherent narrative argument supported by factual evidence…

• Research units evaluated every six years
• Consortium of eight psychology departments in NL
• Psych @ VU project lead,
  asked team to support consortium analysis

Why use open information?

Why?

• Transparency, acountability, etc, etc
• Library pushes for Open Science,
  opportunity to ‘practice what we preach’

Solution looking for a problem?

• Does not solve a pressing problem for users

 

But are the data any good?

 

Questions about data validity

Our source: OpenAlex
Coverage very good
Validity not well known

Use trusted data to validate

Access to validated CRIS data
Basic information correct
Easier to accept

Back to the future

Community & Openalex did loads of work
e.g. CWTS open ranking

But can we do {x,y,z}?

{Classification, Citations, Cooperation}

• Classification, Citations:
    Available in Openalex
  
• Cooperation, field-norm citations:
    Can be constructed from CRIS data
    and/or Openalex

Takes (some) skill and effort

 

Back to the future

• Much improved topic classification
• Field-weighted citation index added

What did the assessees/-ors say?

Take-aways

Many points for discussion

• but no huge problems for open data
• Skillfully avoided? Just lucky?
• May be, (part of) the world is
  just ready for open data