Monday, 24 January 2011

Cross-College Impact Workshop, University of Aberdeen

Wednesday 19th January 2011

I signed up to attend the University's first cross-college impact workshop, not really knowing what to expect or if I should/shouldn't be there, but curious following the impact discussions during the recent National Coordinating Centre for Public Engagement (NCCPE) conference.

The full collection of slides from the workshop can be downloaded here.

The agenda set the scene...
"This workshop will explore how staff at the University of Aberdeen can enhance the societal and economic impact of their research. Staff will also have an opportunity to feed into the development of a series of training workshops for 2011 designed to enhance our impact by learning how to more effectively engage end-users throughout the research process."

Mark Reed (Acting Director of ACES) chaired the discussions and Dominic Houlihan (Vice Principal for Research and Innovation) stood in for the Principal, Ian Diamond. It's no secret that Ian Diamond is a passionate supporter of the case for impact with research.

Dominic talked about:
- the 'indicators' used for identifying high-quality impact case studies across the five research areas, i.e. fundamental research with non-academic, traceable, and significant impact, reaching beyond the local;
- the difficulties with submitting interdisciplinary (e.g. dot.rural) case studies when no interdisciplinary assessment group exists, and the need to tailor case studies to specific assessment panel(s);
- the "significant" and 'non fluffy' contribution that public engagement events can make to impact case studies;
- the committees or panels who judge these case studies not being strictly academic, but rather comprising ~40% academics and ~60% 'users';
- and referred us to the impact case studies on the REF website - take a looky here.

Before the break we heard from each of the colleges in turn, with impact case studies from Jane Geddes (College of Arts and Social Sciences), Wendy Graham (College of Life Sciences and Medicine), and dot.rural's John Nelson (College of Physical Sciences).

Jane Geddes' (Professor of History of Art) hilarious presentation of the CASS impact case study, entitled 'three impact case studies in History of Art', spanned:
- from medieval door hinges, to the the Dover Castle regeneration scheme and a special Channel 4 'Time Team' feature;
- from establishing a museum for pictish stones at St. Vigeans, Arbroath, to a personal voyage of discovery in a cave in Socotra, and the publication of a book entitled 'The Temenos Academy: Blackers, Beasts and the Bestiary', with a plethora of famous patrons... counting the Dalai Lama amongst them!;
- from the St. Albans Psalter - a previously inaccessible, rare book in Germany - and a worldwide exhibition, to it now being a core text on the U.S. school syllabus... not forgetting the case for 'Christina goes to Hollywood' (the big time!).

Jane's slides attempt to summarise the vast, non-academic and rather unexpected impact of the projects she presented, including: visitor awards, re-skilling and training craftsmen, increasing visitor figures, spin-off CD guides, outreach lecture tours, school visits, books, exhibitions, and, of course, publications.

Jane's retrospective 'top tips' for achieving impact were:
- select a rich topic;
- innovate;
- foster partnerships with sponsors;
- develop an outreach audience.

Wendy Graham (Professor of Obstetric Epidemiology) presented the Immpact project ("impact with two 'm's!") - a global research initiative for the evaluation of safe motherhood intervention strategies.

Wendy described the "impact onion", explaining that its not clear which unit of assessment the Immpact project falls into. Is it 'policy'? 'Medicine'? 'Economics'? 'Social Science'?...

She emphasised the need to start such potentially high-impact projects by asking the key, end-user stakeholders questions and proposed to add 'influence' to Dominic's indicators.

Outputs of the Immpact project - many of which are still ongoing - include: strengthening research capacity; secondment to DFID; publications; public engagement; a successful bid for the largest single award at the University (~£20m); circulation of a viral film (watch it here); high-profile conference invitations, e.g. from Ban Ki-moon; reaching the G8 and the UN general assembly; and employing ~250 people in developing countries.

Finally, Wendy urged us to consider the 'presentation' of impact and the potential value of associated, powerful and emotive imagery.

John Nelson (Professor of Transport Studies) then went on to present the COPS case study on 'supporting the development of local sustainable transport solutions'.

John referred to the Yellow School Bus Commission and the 'smarter choices' initiative.

Outputs from the former included media activity, follow-on funding, and a new perspective on peer review. Meanwhile, public engagement opportunities, productivity and employment, policy guidance, and safety, economic and health implications were all presented as outputs from the smarter choices Scottish Government initiative.

Questions - relating to the research timeline (i.e. how recent should the original research be? and how far back should researchers be looking?), the balance of project success and quality impact vs luck or coincidence (i.e. should researchers aim to be more reactive vs proactive?), value for money (i.e. an indicator employed by research councils, but not an impact metric?), the challenges of building a case for impact particularly for early-stage career researchers, and the potential for a 'template' to capture impact across the board - formed the basis of the discussions that followed. An additional discussion centred on the level of institutional support which may/must be provided.

After the break, we heard from Liz Rattray (Deputy Director, Research & Innovation, University of Aberdeen) and Paul van Gardingen (Strategic Advisor on Impact to the ESRC-DFID Joint Scheme for Research on International Development) on 'funding opportunities to enhance the impact of your research' and 'learning how to achieve excellence with impact', respectively.

Liz prompted us to consider the 'routes' to impact and referred us to the Research Council UK (RCUK) 'Knowledge Transfer Portal'. The routes fall under three categories: outputs (e.g. papers, patents etc), mechanisms (e.g. Knowledge Transfer Partnerships, exchanges, consultancy, engagement etc) and impact (e.g. policy change, jobs, health benefits etc).

Liz suggested that 'people and exchange programmes' are perhaps under-utilised and that this may be one particular area to focus and expand on.

She also pointed us towards the Scottish Economics and Social Science Knowledge Exchange Network event taking place on 23rd February 2011 in Edinburgh (contact for more information).

For more information on funding opportunities, Liz referred us to the Research and Innovation web pages here.

Lastly, Liz took questions on:
- the RCUK knowledge transfer portal;
- the previous NERC scheme that brought smaller projects together under one umbrella project, to have a bigger impact en masse;
- the potential to grow the ERASMUS student exchange programme;
- and the link between consultancy opportunities and REF.

Paul presented an Amazonian case study aligned with his research interests, explained that he believes "research excellence is the starting point - necessary but not sufficient" with a credible pathway to impact also needed, and declared "impact is about people" the take home message.

He also highlighted the importance of a "narrative, story approach to impact" as well as capacity building, recommended that there may often be more than one pathway to impact, and suggested that research councils are increasingly focusing on longer-term, larger and more collaborative research projects with impact.

With regards to 'telling the story' - potentially as a way of bridging the Arts and Sciences under the framework's 'communication' section? - Paul pointed us towards the Wellcome Trust website.

Paul responded to questions around balancing the economic and 'quality of life' impact metrics, the idea of establishing a quota of projects dedicated to 'blue sky' research, and the goal of producing an impact case study framework or template that's sufficiently broad to apply to all disciplines, and indeed across/within disciplines. It was also suggested that a forum to facilitate 'learning lessons' of unexpected/unwanted/negative research project outcomes may prove invaluable.

In terms of writing pathway to impact statements, Paul's tips include:
- being modest and focusing on what can be achieved and doing it well;
- acknowledging the potential for impact in ways which may or may not be realised.

Dominic closed the workshop, after which everyone was eager head to lunch.

Thanks to all those at ACES, the Centre for Sustainable International Development and Immpact for coordinating the day and facilitating lively debate.

It was really positive to see such a great turnout, particularly the support from senior members of University staff, attending the whole workshop: the university's commitment to impact is clear.

A quick reminder that the full collection of slides from the workshop can be downloaded here.

So, what did it all mean? What did I get out of the discussions?

Well, I left wondering...
-> is there a new/different role for public engagement officers, like myself, to play in the pathway to impact writing and review stages? e.g. c.f. The University of Bristol and UCL case studies here.
-> how can dot.rural further pursue knowledge transfer/exchange networks and opportunities?
-> can the University deliver the crucial capacity building and training programmes needed to support, identify and write high-quality pathway to impact statements? and how can dot.rural lead/support this?
-> is there one online central repository, capturing all available literature (e.g. the RCUK handbooks and the 'learning lessons' proposed forum) on impact and pathways to impact statements?
-> who would it be useful to see present case studies at future workshops in this new cross-college impact series?

Answers on a postcard*...


*or in fact in the blog comment section!


  1. Thanks for the great review Claire. I think there ought to be more of a role for people with your sorts of expertise in the grant writing process - especially given the importance of "pathways to impact" statements in directed funding programmes within RCUK. Paul van Gardigen admited that many funding decisions in his programme were made by non-academic panel members, as the academics were satisfied that all those short-listed were scientifically excellent and couldn't choose between them on academic merit. So it came down to a judgement about which were likely to have greatest impact on the basis of their "pathways to impact" statements. If it isn't possible to integrate people like you into the peer review process, then academics should at least be more aware of the skills that people like you have, and be able to contact you for help. If nothing else, I think it would be worth you advertising your skills in this respect to your colleagues in dot.rural as they prepare new funding proposals...?

  2. Thanks for doing this Claire, as someone else who was there, it is a very thorough run down of the event and the issues that were raised. Personally I found the CASS case studies inspiring and imaginative. I think we all need to accept that publicly funded researchers must face up to the need for accountability, and non-academic impact is one way of 'giving back', however imperfect the design of the metrics.
    As someone working on an interdisciplinary project I am well aware of the difficulties that causes for REF, but I am confident that the 'thinking it through' effort will be well worthwhile. I hope very much that people with your skills will be able to advise on impact statements for anyone writing a funding application. Mark's reminder about Paul van Gardingen informing us how important the impact statement can be in distinguishing between applications that are otherwise scientifically equal is timely.

  3. Nice to hear your thoughts Kate. Dominic and one of his research committees are preparing a series of events to help us prepare for the REF, some of which will focus on impact. I hope that the challenge of measuring impact from interdisciplinary research will be addressed head-on.

    In his introductory talk, Dominic suggested that we'd have to shoe-horn interdisciplinary impact narratives into the disciplinary shapes of the REF sub-panels (possibly submitting the same narrative to more than one sub-panel). This is certainly one (nice and simple) solution to the problem. Thinking of my own research though, I worry that you're then only going to be able to tell a quite narrow part of a much fuller story that can be specifically traced back to papers from a certain field.

    For example, in the RELU Sustainable Uplands project we've got a bunch of "interim" impacts linked to carbon offsetting through peatland restoration. You can trace that back to a hydrology paper we published in 2003 and tell a science story - but almost all the steps between that original paper and policy changes were linked to social science investigating for example the implications for property rights and understanding how such payments might influence wider land management decisions, and the knock-on effects of that. A hydrology panel probably wouldn't be that interested in those papers and the impact they had...?

  4. Thanks for that good example Mark. I have now started mining a rich seam of literature relating to the difficulties of interdisciplinarity that I have found from a search of SciVerse Science Direct. I'm also investigating Andrew Sayer's post-disciplinary perspective. All very interesting for the day job, but somehow still have to let go of the theory for the pragmatics of REF!