Evidence-based management: how to use research to make more informed decisions
In this article organisational psychologist Professor Rob Briner of Birkbeck College, University of London, considers the relationship between evidence and management and how the MBS Portal can facilitate evidence-based practice.
I've never met a manager who doesn't make decisions based on evidence. Whether it's about the best way to market a new product, manage a difficult situation with a member of staff, ensure the success of a merger, or maintain staff motivation during downsizing, managers use information and evidence to help inform their decisions. In this respect, managers are no different to practitioners in any field. From homeopaths to Home Secretaries to hairdressers, the interventions chosen by practitioners are usually explained in relation to some kind of evidence whether it be taken from research studies, statistics, case studies, or the lessons learned from years of experience. It would seem strange if any practitioner or professional could not or refused to justify their actions in this way and, when asked why they had taken a particular decision, responded 'just because'.
The simple idea that evidence is, can and should be used to help practitioners make better decisions is neither controversial nor complicated. However, like a lot of simple ideas, when you dig a little deeper it turns out to be rather less straightforward. What sorts of evidence do managers actually use? How reliable or accurate is it? What else, apart from evidence, guides the decisions managers have to take? Even where managers want to use evidence, does it exist and can they easily get hold of it?
How easy is it to get hold of and use evidence?
Using evidence and information to inform decisions is part of our everyday life. We use maps or navigation systems to help us know where we are and how to get where we want to be. We turn to websites reporting reviews and tests of consumer products to help us decide which to buy. We might pull together a huge range of information and data before deciding whether to move into a new neighbourhood. But we don't always operate in this way. Sometimes, there just isn't enough time to get hold of the information. Our sat nav might send us the wrong way. We choose to leave things to chance. The information we want just doesn't seem to exist. We can't be bothered to do the research. It's more fun just to decide on a whim.
While we all agree we should use relevant and good quality evidence in our decision-making, we also know that sometimes this doesn't happen or doesn't happen as much as we would like.
The emergence of evidence-based practice
Over the past few decades one profession, medicine, has tried to help medical practitioners make better use of evidence through developing what is usually described as evidence-based medicine. The first time people hear the term evidence-based medicine they are usually surprised as we all tend to assume and hope that medicine is based on evidence. While some medical interventions are based on evidence many others, perhaps a majority, are not.
There are many reasons for this. Patients demand particular treatments even though there is little evidence for their effectiveness. Senior medical practitioners may have lost touch with or failed to keep up-to-date with the evidence-based in their field. There simply may not be time to get hold of the evidence and, even when there is, the evidence may be difficult to understand and medical practitioners may not have the skills necessary to interpret and apply it.
Managers face exactly the same challenges in applying evidence to their work and for this reason there is growing interest in developing a parallel approach for managers called evidence-based management (EBMgt). So how can managers make better use of evidence in their research? One approach, outlined by Briner, Denyer and Rousseau (2009), suggests that EBMgt is:
'about making decisions through the conscientious, explicit, and judicious use of four sources of information: practitioner expertise and judgment, evidence from the local context, a critical evaluation of the best available research evidence, and the perspectives of those people who might be affected by the decision'.
Doing evidence-based management
Being conscientious in the use of evidence involves paying careful and sustained attention to the different, sometimes conflicting and difficult-to- interpret sources of evidence available. Being explicit refers to the idea that information is used in a clear and methodical way such that some form of audit trail could be documented showing how each part of the evidence was used in the final decision. Being judicious means judging carefully the reliability of the evidence and its relevance to the problem at hand. A key theme of EBMgt is that evidence and information must be critically evaluated and not simply accepted. Evidence can be shaky or poor quality. It can be biased or limited. It may in other ways be sound but just not relevant to the problem or question.
One way of thinking about this is in terms of how evidence might be used in a court of law. Lots of different types of evidence might be taken: witness statements, forensic evidence, expert witness statements, and so on. What the judge or jury needs to do is consider all this evidence collectively, put it together to build up a picture of events and the people in them, assess the reliability and credibility of the evidence, and use it to help answer the question of whether someone has or has not broken the law.
But what are the sources of evidence available in the work context? First, there is practitioner expertise and judgment. EBMgt does not suggest that evidence from other sources is necessarily more important than managers' own professional experience but rather it focuses on using it in combination with other sources of evidence. While experience can be a great teacher it can also be misleading when we fail to see what's different about the new situation and whether our experience is really relevant. Evidence from the local context refers to the data that often already exist inside the organisation such as sales figures, customer and staff satisfaction surveys, absence rates, and so on. Considering the perspectives of people who might be affected by the decision is not only important to help ensure we act ethically but is also likely to help the eventual acceptability and sustainability of decisions.
Doing evidence-based management with the British Library Management & Business Studies Portal
The fourth source of evidence and the one most relevant to the use of the British Library Management & Business Studies Portal is critically evaluated research evidence. Research evidence can be found in many places including academic or trade journals, articles written by researchers, books and book chapters and research reports published by government and other institutions. So how can you find and use this research evidence? Here are some suggestions:
- Formulate your question or issue as clearly as possible before you begin your search. What exactly is the problem or question you want to get evidence about?
- Once you have done this, try to make your search as specific as possible. For example, searching for 'small business effective growth strategies' is more likely to get you what you want than searching just for 'business growth'.
- Once you have started to find some research, it's important to be able to decide quickly whether it is relevant or not to your question or problem. Sometimes you will initially find a lot of research which - though interesting and partially related - is not directly relevant to you.
- EBMgt is about using the best available research evidence. Once you've gathered relevant evidence together you need to decide, in relation to your question, what you would rate as high quality evidence and what you would rate as poor quality evidence and give much more weight to the high quality evidence.
- The latest research is not necessarily the best or most relevant research. Don't be put off considering research just because it seems to you to be old. Judge it on its own merits relative to your question.
- Beware of any piece of research which claims to have definitely found 'the answer' to a management problem. Research almost never does this because management problems are complex and single pieces of research in themselves don't necessarily mean much. What is usually more important is the body or weight of evidence not a single piece of research.
- Keep up your guard of healthy scepticism. Management is quite dominated by fads and fashions which promise to deliver a lot but that very rarely do (except for those such as consultants promoting and selling the fad). Check out both the logic and evidence behind any idea, programme or product that is described as new, leading edge or cutting edge.
- Bear in mind that the main purpose of most academic writing is to communicate with other researchers. Academic research can therefore be difficult to read and seem obscure. Whether it's relevant to practical problems is not always clear and it's left to the practitioner to work out if and how it is relevant.
- Give yourself time. Finding, reading, critically evaluating and applying evidence to your work as a manager is a skill it takes time to develop.
- Don't expect a single and simple answer to your question. Very often and for important reasons the correct answer is 'it depends'. The art of EBMgt is to pull together various sources of information and evidence - including research evidence - to help inform your decision. In the end, it's you who makes and takes responsibility for the decision, and not the evidence.
Questions or comments
If you have questions or comments about this article or Evidence-Based Management, Professor Briner has kindly said you are welcome to email him at email@example.com
Key works by Rob Briner
Briner, R. and Rousseau, D. Evidence-based I-O psychology: not there yet. Industrial and Organizational Psychology: Perspectives on Science and Practice, Forthcoming
Briner, R. Evidence-based management and the idea of impact. Presentation to the Association of Business Schools Research Conference, 23 March 2010.
Listen: Evidence-based management and the idea of impact (MP3, 20 minutes, 14MB)
See: Presentation slides (PDF, 86kb)
Evidence-based management. [Video interview with Rob Briner]. Birkbeck College, University of London, 2009. Available online at http://www.bbk.ac.uk/manop/our-staff/academics/briner/evidence-based-management
Briner, R., Denyer, D. and Rousseau, D. Evidence-based management: concept clean-up time? Academy of Management Perspectives, 23 2009, pp.19-32
So where is your evidence? [Interview with Rob Briner]. Sunday Times Business, 21 March 2010. Available online at http://www.thesundaytimes.co.uk
Tried and attested [Interview with Rob Briner]. People Management, 1 November 2007, pp.32-35
Available online to People Management subscribers/CIPD members at http://www.peoplemanagement.co.uk/pm/articles/2007/11/triedandattested.htm
Key works by others
Tingling, P. and Brydon, M. Is decision-based evidence making necessarily bad? MIT Sloan Management Review, 51 (4) 2010, pp.71-76
Dane, E., and Pratt, M. Exploring intuition and its role in managerial decision making. Academy of Management Review, 32 (1) 2007, pp.33-54
Pfeffer, J, and Sutton, R. Hard facts, dangerous half-truths, and total nonsense: profiting from evidence-based management. Boston, Harvard Business School Press, 2006
Pawson, R. Evidence-based policy: a realist perspective. London: Sage, 2006
Rousseau, D. Is there such a thing as evidence-based management? Academy of Management Review, 31 (2) 2006, pp.256-269
Tranfield, D., Denyer, D. and Smart, P. Towards a methodology for developing evidence-informed management knowledge by means of systematic review, British Journal of Management, 14 (3) 2003 pp.207-222
Boaz, A., and Ashby, D. (2003). Fit for purpose? Assessing research quality for evidence-based policy and practice. ESRC UK Centre for Evidence-Based Policy and Practice Working Paper 11. Available online at http://www.kcl.ac.uk/content/1/c6/03/46/04/wp11.pdf
Tranfield, D., and Starkey, K. The nature, social organization and promotion of management research: towards policy, British Journal of Management, 9 (4) 1998 pp.341-354
Evidence-based management website
The evidence-cased management collaborative
This website was initiated by Denise Rousseau and is sponsored by the Academy of Management and Carnegie-Mellon University.