Which Research Method to Use
and When
7.36 It is clear that there are many methods available
for conducting research into healthcare interventions. The RCCM,
which has 16 years' experience in trying to develop CAM research
and getting its results accepted, said: "...there has been
a debate about the question of RCTs and their application to the
area of complementary and alternative medicine. We think that
this debate is unhelpful because, essentially, we need to begin
with what are the questions we want to ask and then design the
appropriate trial and use the appropriate methodology" (Q
26). We agree. The debate over which methods are applicable to
CAM and which are not is probably unhelpful; this dilemma has
consumed much energy and has produced strong divisions of opinion
in the CAM and conventional worlds. The more useful question is
which method is suitable for answering which problem.
7.37 Which method is most appropriate to use will
depend on the level of development within the therapy and on the
particular questions being researched. This was articulated particularly
well by the RCCM: "…the establishment of evidence-based
medicine requires evidence from both quantitative and qualitative
methodologies. Again, to reiterate, the method is determined by
the research question. We would suggest that a range of methodologies
should also be employed. Health service researchers are increasingly
using qualitative methods. And methods employed in the social
sciences should also be employed in the evaluation of CAM, depending
on the research question. So we may ask, for example, what it
is about complementary medicine that people feel is of benefit
to them? Is it a genuine therapeutic relationship or is it [related
to] where the needles are placed in acupuncture? They require
a different approach. One requires in-depth, qualitative interviews.
The questions of how does it affect a patient's quality of life,
and how does the therapy affect a patient's physical condition
would require a more quantitative approach, such as assessment
by using a disability scale or a health status measure. So that
the full range of methodologies ought to be applied, depending
on the research question"(Q 135).
7.38 FIM produces a useful table showing which methods
are suitable for which situations in the Discussion Document Integrated
Healthcare: A Way Forward for the Next Five Years?
7.39 Mr Michael McIntyre, a trustee of FIM, told
us that he believed this controversy over what research methodology
should be used was part of the reason why so few CAM practitioners
attempted rigorous research: "I think from the CAM side one
of the reasons why, perhaps, the amount of research and applications
is as low as you say, is that there is a general fear that there
is going to be a misunderstanding of the paradigm" (Q 91).
It is our hope that, as more CAM practitioners are trained in
research methods, and are made aware of the different types of
research design; and as more conventional scientific investigators
become aware of the intricacies of CAM research, this 'general
fear' will be overcome.
Expertise on Grant-Awarding
Boards
7.40 Given some of the complexities in designing
trials for CAM, it has been suggested by several of our witnesses
that there is a particular need for members of grant-awarding
bodies to understand the specific problems facing CAM research
in order to be able to make a fair and well-informed judgement
on the importance and quality of a proposal for funding. The British
College of Naturopathy and Osteopathy (P 31) also suggest that
to counteract the feeling that many Research Councils do not give
CAM proposals a fair chance, funding bodies should recruit CAM
members with the appropriate research qualifications to help determine
the validity of protocols.
7.41 We asked some of the main funding bodies whether
they thought that there was a need for them to appoint specialists
in this area on to their boards. The MRC told us: "We do
have boards that can judge the proposal. The Health Services Research
Board contains a wide range of people, not necessarily practising
complementary and alternative medicine but people who understand
research methodologies, research questions and can judge whether
those questions are answerable using the methodologies that individuals
can formulate. I think we do have the people who can judge those
things" (Q 1087).
7.42 The Wellcome Trust also defended the composition
of their panels: "The panel system that we have within the
Trust is a very strong one and is based on peer review. If within
the panel there is a lack of expertise we have the option to co-opt
an expert to deal with the particular application. If there was
a complex CAM application it could then be dealt with by bringing
a specialist on to the panel. The panel itself would make the
decision, taking account of that expert's advice. The Trust goes
to great lengths to get proper peer review. We employ 90 scientific
officers to send out all of the bids we receive for peer review.
We believe that that is important. To date, we have received 163
CAM applications. The indications are that 37 per cent of those
have been funded. That proportion is higher than we would expect
for our more orthodox applications, which routinely is about 30
per cent" (Q 1132).
7.43 We asked the Association of Medical Research
Charities (AMRC) whether they thought their member charities'
boards had the relevant expertise. They too felt that their current
provisions were fine and that there was no need to change their
system to give CAM proposals a fairer wind: "As an Association
we are committed to the use of peer review and believe that that
is the best way for charities to make judgements about the best
use of their funding. But peer review is quite a flexible system
and it should not be applied in a rigid way. The AMRC's guidelines
accept that there are certain areas in which one may need a different
review process, but the key principle is that there should be
an internal and external process of peer review. Where specific
expertise is not available on the panel we insist that those charities
must seek it externally and choose external referees in an open
way. We provide support and advice for charities in identifying
external referees. One possibility is for AMRC to draw up a list
of potential CAM external referees, although charities do not
indicate that they have any difficulty in identifying referees
through the normal process of literature researches, networks
and various other ways. We also advise charities to go overseas
so that questions about the status of organisations and the networks
in which particular individuals feature are diminished" (Q
1190).
7.44 The AMRC went on to describe one particular
initiative by one of their member charities to aid CAM applications.
"The Arthritis Research Campaign is about to introduce a
mentoring process of peer review for CAM applications. Even if
they are of lower standard initially, applications will be picked
up by a member of the panel and taken through with guidance by
specialists to try to raise the standard of specific applications.
Only very large charities with significant staff can take on that
mentoring role. It is an example of how the peer review process
can be used to give feedback and to raise the standard of an application
so that it can come back again. I do not believe that there is
anything inherently wrong with the peer review system for CAM
research. I hear criticisms of peer review from every speciality"
(Q 1190).
7.45 Overall it would seem that the majority of funding
bodies are now willing to ensure that CAM research proposals are
reviewed by well-informed individuals. To achieve equity with
more conventional proposals, we recommend that research funding
agencies should build up a database of appropriately trained individuals
who understand CAM practice. The research funding agencies could
then use these individuals as members of selection panels and
committees or as external referees as appropriate.
Environment and Infrastructure
for Research
7.46 There are currently a variety of different environments
in which CAM research is conducted in the United Kingdom. These
vary from university-based research departments which operate
as part of well established medical school research departments,
to projects based within charities and in clinical practice either
in hospitals or in primary care.
7.47 During the course of our Inquiry we visited
three different research environments. Two of these were university-based
research departments. The first was the Department of Complementary
Medicine at the University of Exeter, which is based within a
school of postgraduate medicine and supports the United Kingdom's
only CAM Chair. The second was based within a school of medicine — the
Complementary Medicine Research Unit of the School of Medicine
at the University of Southampton. The third research environment
we visited was an NHS clinical practice — the
Marylebone Health Centre, an NHS inner-London GP practice which
offers CAM therapies alongside conventional care and which supports
practice-based research. (See Appendices 3, 4 and 5.)
7.48 We heard much evidence in favour of establishing
and supporting a few centres of excellence in CAM research, such
as those at Exeter and Southampton, as opposed to spreading funds
and resources across many disparate projects. FIM's discussion
document Integrated Healthcare: A Way Forward for the
Next Five Years? suggests that "it would seem
appropriate to concentrate resources on establishing a number
of research centres linked with higher education institutes with
the capacity to conduct high-quality research into CAM".
7.49 This is the approach of NCCAM in the USA. Dr
Stephen Straus, the Director of NCCAM, told us: "The eleven
centres we fund to date are really intended, in part, to draw
those CAM practitioners and experts into the fold of a larger
research enterprise within an established community. Out of the
many hundreds of institutions in the United States we are creating
foci within only one dozen or so and we hope that we will see
leaders in the coming years" (Q 1734).
7.50 The Wellcome Trust also supported the idea of
centres of excellence and suggested these should be encouraged
to develop from existing centres of research excellence to avoid
the delays in generating high-quality CAM research. Dr Howard
Scarffe of the Wellcome Trust told us: "I had an opportunity
to visit one of the clinical research facilities that we fund
at a large university teaching hospital. I was excited that another
venture was to be undertaken by the Trust with Government under
the Joint Infrastructure Fund. Within 25 yards of that clinical
research facility the Wellcome Trust is to fund research laboratories…It
is very exciting that attached to a large university teaching
hospital campus is a clinical research facility in which all researchers
can work together. Adjacent to that is a purpose-built world-class
laboratory. We are also funding a director of the clinical research
facility so that he or she can give full attention to getting
it off the ground. It struck me…that if we had good facilities
and researchers we could begin to graft on other bits, of which
complementary medicine might be one. If one began to build capacity
from the ground level there would be a lag of between 10 and 15
years to train people up to a high level. Therefore, there is
a need for a system whereby the research can be grafted on to
what is already there and use made of the present expertise"
(Q 1141).
7.51 Although concentrating funding in a few centres
of excellence has many advantages this does not mean there is
no place for smaller practice-based research projects. As previously
noted, there are many different ways that CAM research can be
conducted and large-scale RCTs are probably best conducted in
centres of excellence; qualitative research may be ideal for practice-based
research.
7.52 At the Marylebone Health Centre (see Appendix
4) we heard from Dr Sue Morrison, one of the senior partners.
Dr Morrison explained that as a practice they favoured rigorous
clinical audit and have used such data to develop a manual of
integrated care for other practices to use. However, she described
some limitations to their data. For instance, some patients self-select
the Marylebone Health Centre in order to have access to CAM and
therefore wider information is needed from across the Primary
Care Group on what patients want from their healthcare and, within
this, the role of CAM.
7.53 Dr David Peters at the Marylebone Health Centre
described how research has the capacity to serve both practitioners'
and patients' needs. For example, audit ensures quality assurance,
research through qualitative methods increases understanding of
the patients' experience, action research promotes service and
professional development and case studies illustrate best practice
models. In this way practice-based research promotes quality and
understanding.
7.54 We received written evidence from the University
of Westminster Centre for Community Care and Primary Health (CCCPH)
(P 234) which is also run by Dr David Peters. As well as awarding
degrees in various CAM therapies and conducting research in this
area, the Centre runs a clinic. They explained the advantages
of an educational and research department having links straight
into a clinic: "The Polyclinic is creating unparalleled educational
and research opportunities where students in the BSc and Masters'
programmes will gain practical clinical and research experience
under the supervision of some of the United Kingdom's most experienced
and best-qualified practitioners…As a multi-disciplinary
complementary therapy teaching, research and service delivery
resource, the Polyclinic will be unique in Europe and the CCCPH
are looking to develop national and international education and
research partnerships" (P 235).
7.55 As we discussed in Chapter 5, several of the
newer universities now offer CAM courses. The aspiration is that
these courses will help establish more university-based CAM research.
The BMA told us that they expected these newer universities to
be "important players in the field of research in much the
same way that medical faculties have a role within medical research"
(Q 354). They added: "The question is whether those faculties
have sufficient experience yet in devising research protocols,
and clearly it is important that they work together to share that
experience. Again we believe that organisations such as the Medical
Research Council should also be able to offer their help and support
in these early stages in the devising of trials and protocols"
(Q 354).
7.56 We also asked the CVCP whether they thought
the newer universities had the infrastructure to support good
quality research. They told us they believed that they did: "The
quality assurance regime which the universities operate through
the Quality Assurance Agency, which is a tough, self-regulatory
regime, would expect every university to consider those issues
in respect of any programme: it would expect to ascertain that
each university has the appropriate infrastructure which, in certain
types of programme, would have to include a research base for
mounting a programme. The inspections which are done by the Quality
Assurance Agency would certainly cover those areas" (Q 281).
They also told us that the regulatory mechanism is there to ensure
that no university is left thinking that they will supported if
they are mounting programmes without the necessary infrastructure
(Q 281).
7.57 From the evidence we have received it is clear
that there has been a change of attitude of a few higher education
institutions towards CAM as a legitimate subject for both quantitative
and qualitative research. However, the small base and fragmentation
from which this research will have to be conducted would seem
to be a major barrier to progress. We recommend that universities
and other higher education institutions provide the basis for
a more robust research infrastructure in which CAM and conventional
research and practice can take place side-by-side and can benefit
from interaction and greater mutual understanding. A preferred
model would involve centres of excellence committed to establishing
a wider framework of conventional scientists, social scientists
and CAM practitioners. These would provide a basis for enhancing
research into CAM while ensuring it was of high quality, addressed
relevant questions and was integrated with conventional methodology.
Advantages would be gained by facilitating multi-disciplinary
research with access to medical, psychological, social-scientific
and pharmaceutical clinical trials. We recommend that a small
number of such centres of excellence, in or linked to medical
schools, be established with the support of research funding agencies
including the Research Councils, the Department of Health, Higher
Education Funding Councils and the charitable sector.
Research Education
7.58 An interest in science or clinical research
is not at present a requirement for all CAM training courses,
although some schools are introducing courses on research and
research projects. Our evidence has helped us construct a picture
of the attitude of CAM practitioners towards research. There is
an increasing number of CAM practitioners who believe research
is important and are willing to put their time and effort into
it, but very few appear to have sufficient knowledge or skills
in research to advance their interest.
7.59 In the USA one of the main ways that NCCAM is
hoping to improve the capacity to conduct CAM research is to improve
the education in research methodology of those involved in CAM.
This is done through the funding of career development awards
at various levels. Dr Stephen Straus, the Director of NCCAM told
us: "I believe that those awards need to be to individuals
who will be mentored by outstanding scientists and have a protracted
period of tutelage, a minimum of three years and ideally five
years, to cultivate their skills as an independent investigator"
(Q 1734).
7.60 We talked to various United Kingdom funding
bodies about the prospect of them awarding research fellowships
directed towards students with an interest in CAM in order to
invest in the long-term future of CAM research development.
7.61 Although the Government were eager to highlight
the importance of education and training in research, they do
not currently have a route for supporting research fellowships
in this specific area. When we asked Professor Sir John Pattison,
Director of NHS Research & Development, how he saw such fellowships
being supported, he told us: "I think we have a track record
of building capacity in areas of orthodox medicine. Primary care
is the first example we embarked upon. In collaboration with the
Chief Medical Officer we are about to embark upon similar research
training fellowships and career scientist awards in public health…If
one simply took those as models for how one would start to build
capacity in any area of research to health and health services,
then that would be the way to do it" (Q 1869).
7.62 Dr Howard Scarffe of the Wellcome Trust told
us he saw the Trust as having a role in supporting programmes
which provide training and support in research methodology in
the area of CAM. However, the Wellcome Trust's current policy
towards awarding PhD studentships is likely to disadvantage CAM
applications as applicants without conventional medical or scientific
training are likely to be excluded from applying. Dr Scarffe told
us: "At present our policy is oriented towards people with
medical or scientific qualifications" (Q 1161).
7.63 The Department of Health told us that they saw
encouraging research training as one of the best ways of improving
the research capacity within CAM: "The approach of the R&D
programmes to this and, indeed, all of medicine has been very
much to encourage access to training as well as access to research
funding. That is provided through a number of mechanisms, including
training in research design, training and methodology regionally
and centrally. There is a groundswell of movement towards a higher
standard of education for researchers. I see that move on the
part of the professionals as key to increasing the volume and
quality of research that is done" (Q 15).
7.64 The MRC also support research training. They
told us: "…our fellowship schemes include fellowships
in subjects allied to medicine, and people in complementary medicine
are open to apply for fellowships — as they are
in…Health Services Research" (Q 1087).
7.65 As well as attracting mainstream experts to
investigate CAM, the MRC also explained they have mechanisms for
advising people from all areas who need help in designing trials:
"...the new MRC Unit on clinical trials, which has a division
without portfolio - which is already giving advice to people in
trials in areas where traditionally we have been rather weak.
That includes advice to particular individuals on how to conduct
a trial on complementary medicine" (Q 1085).
7.66 Many CAM university degree courses now include
research modules and this is likely to catalyse a change in CAM
practitioners' attitudes to research and the need for evidence-based
practice. FIM summed up this change: "…we have a lot
of university degree courses and very good levels of training
being brought into being…and…in nearly every case there
is a research module. It does require undergraduates of CAM to…undertake
at least some training in research so that the culture of research
is encouraged…and that is a very important feature of the
education of CAM practitioners — so that we have
a common language between researchers in the orthodox field and
those in the CAM field" (Q 91).
7.67 Thus, while there currently exist some research
training opportunities available to CAM practitioners, none of
these is specifically directed towards CAM and very few, if any,
of these fellowships have been taken by individuals conversant
with the practice of CAM. There may be two principal reasons for
this. Firstly, if a practitioner has received no basic education
in research methodology he or she is unlikely to seek specialised
research training; this is why we hope that CAM regulatory bodies
will include research methods in their core curricula (as discussed
in Chapter 6) and why the new university courses in CAM represent
a promising development. Secondly, the reason why few CAM people
take up research training opportunities may well be that they
do not know about them. Bodies such as the Departments of Health,
the Research Councils and the Wellcome Trust should help to promote
a research culture in CAM by ensuring that the CAM sector is aware
of the training opportunities they offer. The Department of Health
should exercise a co-ordinating role. Limited funds should be
specifically aimed at training CAM practitioners in research methods.
As many CAM practitioners work in the private sector and cannot
afford to train in research, we recommend that a number of university-based
academic posts, offering time for research and teaching, should
be established.