View in EDS HTML Full Text PDF Full Text

Developing Definitions for a Knowledge Exchange Intervention in Health Policy and Program Agencies: Reflections on Process and Value

Saved in:
Bibliographic Details
Title: Developing Definitions for a Knowledge Exchange Intervention in Health Policy and Program Agencies: Reflections on Process and Value
Language: English
Authors: Haynes, Abby, Turner, Tari, Redman, Sally, Milat, Andrew J., Moore, Gabriel
Source: International Journal of Social Research Methodology. 2015 18(2):145-159.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 15
Publication Date: 2015
Document Type: Journal Articles
Reports - Descriptive
Descriptors: Intervention, Health Services, Foreign Countries, Definitions, Public Health, Public Policy, Outcome Measures, Research Utilization
Geographic Terms: Australia
DOI: 10.1080/13645579.2014.921457
ISSN: 1364-5579
Abstract: The development of definitions is an integral part of the research process but is often poorly described. This paper details the iterative development of five definitions: Policy, Health policy-maker, Health policy agency, Policy documents, and Research findings. We describe the challenges of developing definitions in a large multidisciplinary team and the important methodological repercussions. We identify four factors that were most helpful in this process: (1) An emphasis on fit-for-purpose functionality, (2) Consultation with in-context experts, (3) Our willingness to amend terms as well as definitions, and to revisit some methods and goals as a consequence, and (4) Agreement that we would satisfice: accept 'good enough' solutions rather than struggle for optimality and consensus.
Abstractor: As Provided
Number of References: 35
Entry Date: 2018
Accession Number: EJ1191066
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFAMwYzhB1DUEK45UPeqw_GAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDIHd6i2wGYc2FktIEwIBEICBmu6r2xXCtASdQFHrMBRHouO7cZdeGgx3_MbyX15rU-IJaAR-tKETHM4xBt007Uh5a47pmaqmnDQOgdDnH8mTeNqnoqvvQxdczHwJP5CkLJ0z7tjjm_E9Om8sDLVreYF8caXr23G686K2vMno7JAqx8PQ0joYt2q45ukRILrR5Iycn0Wif1IidLJi7DkDcu0gTkEX14uRu527VAw=
Text:
  Availability: 1
  Value: <anid>AN0100241117;9eb01mar.15;2019Feb13.15:35;v2.2.500</anid> <title id="AN0100241117-1">Developing definitions for a knowledge exchange intervention in health policy and program agencies: reflections on process and value. </title> <p>The development of definitions is an integral part of the research process but is often poorly described. This paper details the iterative development of five definitions: Policy, Health policy-maker, Health policy agency, Policy documents, and Research findings. We describe the challenges of developing definitions in a large multidisciplinary team and the important methodological repercussions. We identify four factors that were most helpful in this process: (<reflink idref="bib1" id="ref1">1</reflink>) An emphasis on fit-for-purpose functionality, (<reflink idref="bib2" id="ref2">2</reflink>) Consultation with in-context experts, (<reflink idref="bib3" id="ref3">3</reflink>) Our willingness to amend terms as well as definitions, and to revisit some methods and goals as a consequence, and (<reflink idref="bib4" id="ref4">4</reflink>) Agreement that we would satisfice: accept 'good enough' solutions rather than struggle for optimality and consensus.</p> <p>Keywords: definition; policy; policy-maker; knowledge exchange; research methods</p> <p>What's in a name? That which we call a rose by any other name would smell as sweet.</p> <p>William Shakespeare, <emph>Romeo and Juliet</emph> (c.1597)</p> <p>In real life, unlike in Shakespeare, the sweetness of the rose depends upon the name it bears. Things are not only what they are. They are, in very important respects, what they seem to be.</p> <p>Hubert H. Humphrey, Democratic politician and US Vice President (1966)</p> <p>'When I use a word', Humpty Dumpty said in rather a scornful tone, 'it means just what I choose it to mean – neither more nor less'.</p> <p>Lewis Carroll, <emph>Through the Looking Glass</emph> (1871)</p> <hd id="AN0100241117-2">Introduction</hd> <p>Ludwig Wittgenstein, the Austrian philosopher, illustrates the difficulties of defining terms in his exploration of the word <emph>game</emph>. He argues that definitions of <emph>game</emph> that focus on amusement or competition or rules each neglect import dimensions of the many activities we call games, and thus no single definition can be found. But, he suggests, we probably do not need a definition because we are sufficiently familiar with enough things which <emph>are</emph> games and enough things which <emph>are not</emph> games to recognize the difference between them (e.g. Canfield, [<reflink idref="bib8" id="ref5">8</reflink>]).</p> <p>This is true for most of us most of the time. We don't need to define concepts such as friend, beauty, or irony[<reflink idref="bib1" id="ref6">1</reflink>] because, like the famous US Supreme Court ruling on pornography (Jacobellis v. Ohio, [<reflink idref="bib24" id="ref7">24</reflink>]), we know it when we see it. We form views and make decisions based on comparisons and approximations. We cannot know that others perceive concepts in the same way, but we assume that in most circumstances it is close enough. However, the use of terminology in research and evaluation requires a more rigorous approach. Defining terms is one of the building blocks of research design. It obliges researchers to be exact about the character and parameters of concepts or phenomena under study, and to describe them transparently so that others can assess their validity. It also facilitates effective communication within and across fields of inquiry. This is particularly important in fields such as health research utilization which have emerged from a range of research traditions across diverse disciplines (Rabin, Brownson, Haire-Joshu, Kreuter, & Weaver, [<reflink idref="bib31" id="ref8">31</reflink>]).</p> <p>The development of a definition is an integral part of the research process. Consequently, it is usually iterative and informed by the very process that it is intended to support. Subjects of research become more complex and fragmented as they are better understood, requiring revised distinctions and ever tighter definitions. This increasing precision feeds back into the research process, provides a jumping off point for further investigation, and can also be a valuable output of research. Yet specificity presents particular challenges, particularly in social research. Operational definitions (constructs that define the tangible variables used as indicators in quantitative research) strain to describe socially situated concepts or phenomena. Measuring multidimensional constructs such as disability, anxiety, ethnicity, or intelligence is hard; indeed, some argue that it is impossible (Altman, [<reflink idref="bib2" id="ref9">2</reflink>]; Bernard, [<reflink idref="bib5" id="ref10">5</reflink>]). Such constructs defy universal application because they are embodied and perceived differently by different people in different contexts. Conceptual definitions (brief descriptions that tell us what a concept means) are more amenable. They refine constructs but can be formulated to cope with complexity and ambiguity, and they can incorporate situational dimensions. But their very precision frequently limits their utility to the conceptual lens and context of the study in which they were developed making genuinely universal definitions something of a holy grail. For example, Locock, Dopson, Chambers, and Gabbay ([<reflink idref="bib29" id="ref11">29</reflink>]) found that opinion leaders can be important mediators of research-informed organizational change initiatives, but lament the limitations of researching this phenomenon more broadly due to the improbability of capturing the multidimensional and contextual nature of an <emph>opinion leader</emph> in a single definition.</p> <p>Definitions also 'frame' their subject, asserting how it should be seen. As Hubert Humphrey points out in the quote above, the terminology we use to describe a thing affects our perceptions, including how we attribute value. This has critical implications not only for research, but for 'real world' responses too. For example, Huber et al. ([<reflink idref="bib23" id="ref12">23</reflink>]) note that the current WHO definition of <emph>health</emph> – a groundbreaking contribution to global health made in 1948 – now contributes to the medicalization of society. Laderchi, Saith, and Stewert ([<reflink idref="bib27" id="ref13">27</reflink>]) and Green ([<reflink idref="bib16" id="ref14">16</reflink>]) demonstrate that definitions of <emph>poverty</emph> affect policy agenda-setting, moral attribution, and resource allocation. Further, Hodges ([<reflink idref="bib22" id="ref15">22</reflink>]) argues that standardized definitions in ecology can impede progress by preventing important questions from being asked.</p> <p>Similar concerns have been raised about terminology in the field of research utilization. The term <emph>evidence</emph>-<emph>based policy</emph> has been criticized as naïvely ignoring sociopolitical context and the need for negotiated decision-making in a pluralist democracy (Greenhalgh & Russell, [<reflink idref="bib17" id="ref16">17</reflink>]). Terms that suggest one-way linearity such as <emph>knowledge transfer</emph> are increasingly supplanted by more fluid and participative terms like <emph>knowledge exchange</emph> (Graham et al., [<reflink idref="bib15" id="ref17">15</reflink>]). And, despite definitions of the term <emph>knowledge translation</emph> that include concepts of exchange and multidimensionality (see, for example, WHO 2005 & NIDDR 2005 in Oborn, Barrett, & Racko, [<reflink idref="bib30" id="ref18">30</reflink>]), Greenhalgh and Wieringa ([<reflink idref="bib18" id="ref19">18</reflink>]) argue that the metaphor embodied in the term constrains further study in the field by misrepresenting the socially constructed nature of how health knowledge is produced and used. It seems that terminology and definitions can provide precise in-a-nutshell conceptual syntheses that support critical debate and rigorous investigation, but they can also frame phenomena unhelpfully. The imperative is to select the best-fit terms and to define them clearly in relation to their context and use.</p> <p>Some of the more granular research utilization terms such as <emph>policy</emph>, <emph>policy</emph>-<emph>maker</emph>, and <emph>policy agency</emph> are used in a variety of theoretical, empirical, and commentary articles without defining them. The danger is that, rather like Humpty Dumpty, their meaning is known only to the people using them. For example, few articles that explore policy-makers' use of research explain the professional roles their study encompasses: are these policy-makers government employees? Political appointees? Elected politicians? Ministerial advisors? A mixture? It is important that we know because the power, constraints, and requirements of these roles differ considerably and are likely to affect how research is used (Haynes et al., [<reflink idref="bib19" id="ref20">19</reflink>]). But concepts which are politically volatile and strongly subjective such as <emph>policy</emph> are particularly resistant to definition (Barrett & Hill, [<reflink idref="bib4" id="ref21">4</reflink>]; Kemm, [<reflink idref="bib25" id="ref22">25</reflink>]). As Smith notes, 'the struggle to classify or define policy is itself at the center of political conflict' ([<reflink idref="bib33" id="ref23">33</reflink>]). Some authors offer procedural definitions, e.g.</p> <p>A long term, continuously used, standing decision by which more specific proposals are judged for acceptability in terms of means to be employed, ends to be pursued and time frame in which these proposals will have to fit. (Blum in de Leeuw, [<reflink idref="bib28" id="ref24">28</reflink>])</p> <p>Some opt for a less tangible approach which takes account of the inaction often associated with policy, e.g. 'Something that one group of actors wishes to see carried out by others' (Barrett & Hill, [<reflink idref="bib4" id="ref25">4</reflink>]) and 'Whatever governments choose to do or not to do' (Dye, 1984 in Fischer, [<reflink idref="bib13" id="ref26">13</reflink>]). While others focus on policy's diffuse and values-focused characteristics, e.g. 'Policy ... consists of a web of decisions and actions that allocate ... values' (Ham, 1993 in Kemm, [<reflink idref="bib25" id="ref27">25</reflink>]).</p> <p>Definitions of terms such as <emph>policy</emph>-<emph>maker</emph> will always require some contextual specificity because of the different systems in which policy work is done. Walton and Macagno ([<reflink idref="bib34" id="ref28">34</reflink>]) argue that definitions are not 'What a thing absolutely is,' but 'What a thing is commonly considered to be, based on evidential considerations pro and contra.' As such, they are works in progress, evolving to reflect our changing knowledge of the world and adapted for situation-specific use. Ultimately, the value of a definition lies in its functionality for particular purposes (Chan & Donovan, [<reflink idref="bib9" id="ref29">9</reflink>]): given our goals, how well does the definition capture what we consider this thing to be? In some cases, the goal is to develop universal definitions that traverse disciplines and contexts, in others it is to clearly unpack and boundary terms so that they can be investigated.</p> <p>In this paper, we will describe the evolution of five definitions that were developed for the purposes of an intervention study known as SPIRIT (Supporting Policy In health Research: an Intervention Trial). SPIRIT was designed to test strategies for helping government health agencies increase their use of research evidence in policy and program development. Broadly, the study comprises a multifaceted interactive program of tailored education and resource provision, six measurement periods, and a process evaluation. It is being rolled out to six health agencies in Sydney, Australia over two years using a stepped wedge design.</p> <p>We needed to define five terms for the purposes of this study: <emph>Policy</emph>, <emph>Health policy</emph>-<emph>maker</emph>, <emph>Health policy agency</emph>, <emph>Policy documents</emph>, and <emph>Research findings</emph>. We intended to use these definitions to: identify bodies of work from which policy/program documents could be selected for review; establish eligibility for an agency-level sampling frame and for individual participation in outcome measures; provide eligibility criteria for selecting documents to be reviewed in interviews; and to provide a frame of reference for participants completing outcome measures.</p> <p>We have three aims:</p> <p></p> <ulist> <item> To illustrate the value of fit-for-purpose conceptual definitions. We do not propose that others will necessarily find our definitions useful – they were developed solely for the purposes of our study – but we do urge research utilization investigators to explain how they defined their terms in published articles.</item> <p></p> <item> To provide an account of the challenging process of developing definitions. We hope this will counter depictions of apparently smooth and linear research processes that dominate the literature (Adler & Clark, [<reflink idref="bib1" id="ref30">1</reflink>]).</item> <p></p> <item> To share what we learnt during this process. In particular, the major considerations required to define these concepts, the value of the <emph>process</emph> of developing definitions, and the factors that facilitated this process.</item> </ulist> <hd id="AN0100241117-3">Methods and results</hd> <p>The definitions were developed to be fit-for-purpose, i.e. to serve specific needs within the study. The purpose of each definition is outlined in Table 1, and their development and functionality are explored in turn below.</p> <p>Table 1. Definitions and definitional purposes of terms used in a knowledge exchange intervention study.</p> <p> <ephtml> <table><thead><tr valign="top"><td>Term</td><td>Definition</td><td>Purpose</td></tr></thead><tbody><tr valign="top"><td>Policy </td><td>A formal statement or action plan developed by a government agency or statutory body in response to an identified problem. This includes state-wide or national legislation, policies, programs, directives, protocols, guidelines, and service models</td><td>Identification of bodies of work from which documents could be selected</td></tr><tr valign="top"><td /><td /><td /></tr><tr valign="top"><td>Health policy-maker</td><td>Someone employed in a policy agency who drafts or writes health policy documents or develops health programs, or who makes or contributes significantly to policy decisions about health services, programs or resourcing</td><td>Eligibility for agency sampling frame </td></tr><tr valign="top"><td>Eligibility for participation in outcome measures</td></tr><tr valign="top"><td /><td /></tr><tr valign="top"><td>Health policy agency </td><td>(A body within) a state or federal government department, or a statutory authority, whose focus is to develop policy which has an impact on state-wide or national services and programs intended to improve individual, family or community health</td><td>Eligibility for agency sampling frame</td></tr><tr valign="top"><td /><td /><td /></tr><tr valign="top"><td>Policy documents </td><td>A review, report, discussion paper, draft or final policy, formal directive, program plan, strategic plan, ministerial brief, budget bid, service agreement, implementation plan, guideline or protocol with a focus on health service or program design, delivery, evaluation or resourcing</td><td>Eligibility for selection of documents for targeted interviews</td></tr><tr valign="top"><td /><td /><td /></tr><tr valign="top"><td>Research findings</td><td>Analyses of quantitative or qualitative data, or theory, found in peer reviewed papers, technical monographs or books, or in grey literature such as internal studies and evaluations, and reports on authoritative websites</td><td>Inform the scoring of the outcome measures</td></tr><tr valign="top"><td>Provide a reference for participants completing outcome measures</td></tr></tbody></table> </ephtml> </p> <p>The process in all cases was iterative: two members of the team developed initial definitions based on broad searches in academic and government literature, and online reference sources. These were adapted in response to our developing study aims and constraints. Draft versions were reviewed by colleagues working in policy and program development and were revised accordingly except where the feedback did not align with the study design. Definitions were then reviewed by the whole investigator team, revised according to their advice, and signed off.</p> <p>Technically, that is what happened, but it did not happen smoothly. Despite being aware of the importance of definitions, we were slow to engage in detailed dialog about them. But as soon as we moved into action – testing draft instruments, constructing sample frames, building consultation networks, etc. – the need to be precise about who and what was being sought became pressing. At that point, we found that in some cases no definitions existed, and in other cases they were plentiful but divergent. Some of the richest sources were of uncertain quality, e.g. Wikipedia, blogs, online dictionaries. Then, as we started to consult with colleagues in policy and program work about the face validity of draft definitions, their feedback raised fundamental questions that caused us to critique and amend aspects of the intervention or evaluation design, so we bounced back and forth between the literature, the evolving study design and consultation advice. Our team comprised a large and diverse group of researchers and policy-makers located across multiple institutions, cities and countries making it difficult to find time and the appropriate forum to debate disagreements and find a resolution (see also Billings' [[<reflink idref="bib6" id="ref31">6</reflink>]] account of 'angst-ridden' definition development processes within a large research team). We recognized this as an essential and familiar developmental process, but there were differences of opinion about its duration: at what point would we draw a line in the sand? In fact, we never reached the fabled end point. Definitions that had been 'signed off' continued to change in response to shifting perceptions of the study design, goals, contexts, and participants right up to the moment we were obliged to give them to our participants. Some are still contested, as we describe below.</p> <hd id="AN0100241117-4">Policy</hd> <p>A formal statement or action plan developed by a government agency or statutory body in response to an identified problem. This includes state-wide or national legislation, policies, programs, directives, protocols, guidelines, and service models.</p> <p>Our primary purpose in defining <emph>policy</emph> was: (a) to identify streams of work from which policy documents could be identified and (b) to include the many types of policy and program work conducted by participant agencies because each of them had the potential to influence population health and health services in important ways.</p> <p>We arrived at our final definition through a process of plunder. We reviewed definitions of <emph>policy</emph> found in dictionaries and in the academic literature, chopped them up and rearranged them to synthesize aspects that aligned with our aims, and dropped or amended aspects that didn't. For example, consultations with some colleagues who were developing important population level initiatives indicated that they did not regard their work as a form of policy, so we supplemented the definition with examples to capture these: programs, protocols, guidelines, and service models. Given that our intention was to identify policy documents, the definition purposefully focused on formal statements, or action plans that would have written materials associated with them. The definition does not demand that policy includes action because our intervention focused on getting research into policy and program development processes, but did not extend to implementation or health services practice.</p> <p>The research team used this definition behind the scenes, often as a way of thinking about health policy agencies because these agencies were defined, in part, by their function and tasks. However, the definition appeared to have minimal utility for study participants. It was included with some study information, especially in relation to the selection of policy documents for review in structured interviews, but a definition was neither asked for nor referred to in any interviews or clarifying conversations. Presumably, policy-makers have a better idea than we do of what constitutes policy and do not require a definition. They know it when they see (or do) it.</p> <hd id="AN0100241117-5">Health policy-maker</hd> <p>Someone employed in a policy agency who drafts or writes health policy documents or develops health programs, or who makes or contributes significantly to policy decisions about health services, programs or resourcing.</p> <p>This definition was used to identify individuals within participating agencies. Specifically, to differentiate between staff who would be eligible or ineligible to take part in the outcome measures. Our primary goal was to ensure the intervention reached people who were in the best position to benefit from it – i.e. who had sufficient opportunity to apply research-related knowledge, skills, and resources in their work – so this was where we intended to look for measureable change.</p> <p>Dictionary definitions of <emph>policy</emph>-<emph>maker</emph> lacked the specificity we required, e.g. 'A person responsible for or involved in formulating policies, especially in politics' (Oxford Dictionaries) or 'Someone who sets the plan pursued by a government or business etc.' (The Free Dictionary). The academic literature did not present the required detail either since it tends to define the term by who it includes and excludes rather than describing it (e.g. DeRoeck, [<reflink idref="bib10" id="ref32">10</reflink>]; Feldman, Nadash, & Gursen, [<reflink idref="bib12" id="ref33">12</reflink>]; Haynes et al., [<reflink idref="bib19" id="ref34">19</reflink>]).</p> <p>Early consultation with policy colleagues alerted us to the breadth of roles involved in what we were calling policy-making. They advised us to include staff at different levels within each agency, and to ensure that staff who focused on population-level program development and resourcing were included. This advice, together with our observation that the people we were calling policy-makers did not use that term themselves, caused us to reconsider the term <emph>policy</emph>-<emph>maker</emph>. We liked Maureen Dobbins and colleagues' term <emph>decision-maker</emph> (e.g. Dobbins, Jack, Thomas, & Kothari, [<reflink idref="bib11" id="ref35">11</reflink>]) for its inclusivity, but recognized that this was advisedly broad so as to include health services managers – a group who were excluded from our study. We were also mindful of Weiss's ([<reflink idref="bib35" id="ref36">35</reflink>]) finding that few policy-makers are in the position to make final policy decisions. This is because policy development is a collective incremental process characterized by mutual adjustment, move and counter move, accretion and negotiation, thus policy-makers focus on affecting the shape and content of policy discourse rather than deciding on overarching policy. Consequently, given that the term was to be employed primarily for internal use (when we used the definition in agencies it appeared under the heading of <emph>eligible participants</emph>), and the term <emph>policy</emph>-<emph>maker</emph> was dominant in the literature, and we continued to use it.</p> <p>The consultation also raised the murky question of what policy-makers actually do – what component tasks make up the policy-making process and how could we identify those that are most amenable to research input? This included tasks such as refining/analyzing policy questions, reviewing data, conducting or commissioning research or evaluations, consultation with internal and external stakeholders, working with advisory committees, and information synthesis. The catchall phrase ... <emph>contributes significantly to policy decisions</emph> ... was added to capture that breadth of activity.</p> <p>We found this to be a functional definition for our purposes, and feedback from participating agencies confirmed that it provided clear guidance for assessing staff eligibility. For example, it was precise enough to exclude contractors from the study and staff who did not contribute to policy or program development such as those in administration and operations, but inclusive enough to capture many strata of policy and program developers and to allow pilot testing in an agency that funds state health services.</p> <hd id="AN0100241117-6">Health policy agency</hd> <p>(A body within) a state or federal government department, or a statutory authority, that focuses on developing policies or programs designed to improve state-wide or population level health.</p> <p>We needed to define what a health policy agency was in order to develop a sample frame for organizations that were to be invited to participate in the intervention. Rather like Wittgenstein's <emph>game</emph> example above, in the early design stages we knew enough organizations that were <emph>like</emph> our conception of a health policy agency and enough that were <emph>not like</emph> it to categorize organizations as potential participants or not, but we were unable to find any definitions in the literature from which to draw rigorous criteria that could be used to create a sample frame. Consequently, we developed a pragmatic definition that matched study eligibility requirements with the goals and in-development methods of the intervention. Our constraints (geographic and financial) were front of mind in this process.</p> <p>It was agreed that a health policy agency must develop population level health policy or programs as its core business. This excluded health agencies whose primary role is operational, e.g. accreditation, compliance, and benefits schemes administration. Given our intervention focused on public health and clinical research, we took a narrow perspective of health which excluded many health-related agencies such as community services. Non-government organizations could have been included in this definition; however, in our study we wanted to test the intervention with larger agencies that had most scope to directly influence population level health – so only state or national level government organizations and bodies with statutory authority were considered.</p> <p>This definition worked well enough for our purposes, particularly after discussion with agency staff in the early phase of the study led us to refine it to give greater weight to program development, and to remove some unnecessarily granular description.</p> <hd id="AN0100241117-7">Policy documents</hd> <p>A review, report, discussion paper, draft or final policy, formal directive, program plan, strategic plan, ministerial brief, budget bid, service agreement, implementation plan, guideline or protocol with a focus on health service or program design, delivery, evaluation or resourcing.</p> <p>The purpose of defining the term <emph>policy document</emph> was to provide inclusion criteria that would enable each participating agency to nominate four eligible documents which best represented their use of research evidence in policy or program agenda-setting, development, implementation, or evaluation within a given time period. Key personnel were then interviewed about the process of developing these documents which were considered to be proxies for the policies and program that they related to.</p> <p>The definition had to be broad enough to allow agencies to select documents that reflected their work, but contained enough to identify the sorts of documents that could usefully be informed by research. The consultation process involved colleagues in policy and program development kindly sending us examples of their documents as well as commenting on the draft definition. With each consultation our overarching definition diminished and our list of examples became longer until, eventually, only a list was left. The range and diversity of authors, content, aims, and intended readers were so great that we were unable to describe these dimensions meaningfully for all eligible agencies in a single brief definition. Happily, we were told that concrete examples were preferable so we honed these and, following a further round of consultation, we dropped <emph>press releases</emph> from the list (they lacked detail and were likely to be less 'authentic'), and added <emph>budget bids</emph>.</p> <p>In practice, as in development, this was our most problematic definition. From the start, agencies repeatedly required verbal clarification and expressed confusion. No specific problems with the definition itself were identified,[<reflink idref="bib2" id="ref37">2</reflink>] and so no improvements were made, but the concept of a policy document remained ambiguous with a need for locally specific discussion in each site.</p> <hd id="AN0100241117-8">Research findings</hd> <p>Analyses of quantitative or qualitative data, or theory, found in peer reviewed papers, technical monographs or books, or in grey literature such as internal studies and evaluations, and reports on authoritative websites.</p> <p>The purpose of this definition was to inform the design of the outcome measures (i.e. to identify what sort of research-related information was being used at each measurement point) and to provide a common reference for the SPIRIT team and participants, particularly during interviews that focused on research use.</p> <p>Definitions of research abound, but they tend to focus on the process of conducting research (systematic gathering and analysis of information to advance understanding) rather than the outputs of the process. Discussions of what constitutes research data provided a useful starting point (e.g. Australian National Data Service, [<reflink idref="bib3" id="ref38">3</reflink>]) as they drew attention to the important difference between raw and analyzed data. Given that most policy-makers do not have research qualifications (although a surprising number do), the intervention encouraged the use of analyzed data only, and the definition reflected this.</p> <p>The goals of the intervention were to increase the use of research, but we took a pragmatic view of what that might entail. Much so-called gold standard research is neither relevant nor applicable to policy and program development, or cannot be produced within the necessary timeframes, and grey literature (e.g. government white papers, internal program evaluations) is often particularly helpful. Therefore, we sought a definition that was broad enough to encompass this. The definition did not need to address research quality because that was explored conversationally during interviews conducted as part of the study's outcome measures.</p> <p>We considered including advice from researchers in this definition, but this was rejected because it was too amorphous a construct and hard to qualify without talking to the researchers themselves. If policy-makers sought or used advice from researchers, this was captured in our survey forms and interviews. The definition provided in those instruments includes the addendum: <emph>Advice from researchers is considered to be research</emph>-<emph>informed information, but not research</emph> per se<emph>.</emph></p> <p>Few participants asked how we were defining research or referred to it during interviews, but several mentioned their surprise at the inclusivity of the definition given that we were researchers (and thus, by implication, attached to academic hierarchies of evidence). So we concluded that the definition was helpful primarily as an invitation for policy and program developers to talk about the 'real world' range of research they used.</p> <p>The term itself – <emph>research findings</emph> – was contested. Some members of the team wanted to use the term <emph>research</emph> as a catch-all for the process of conducting research and for its outputs – the context made it clear which was which. But others argued for greater precision. We also debated using the term <emph>evidence</emph>. This debate was more challenging since it obliged us to explore our beliefs about the nature and status of human inquiry. Some argued that the term <emph>evidence</emph> is rhetoric (see Greenhalgh & Russell, [<reflink idref="bib17" id="ref39">17</reflink>]; Russell, Greenhalgh, Byrne, & McDonnell, [<reflink idref="bib32" id="ref40">32</reflink>]). Others argued that it was common parlance and that authors such as Head ([<reflink idref="bib20" id="ref41">20</reflink>], [<reflink idref="bib21" id="ref42">21</reflink>]) and Klein ([<reflink idref="bib26" id="ref43">26</reflink>]) had convincingly expanded the term to include other policy-relevant forms of information, but for the sake of clarity we should use the term <emph>research evidence</emph> or <emph>evidence from research</emph>. We agreed on the term <emph>research findings</emph> for use in this study but note that it was used inconsistently (as this very paper illustrates).</p> <hd id="AN0100241117-9">Discussion</hd> <p>We managed to arrive at five definitions or example lists that were largely successful for our specific purposes which were to: identify bodies of work from which policy/program documents could be selected for review; establish eligibility for a sampling frame and for participation in outcome measures; provide eligibility criteria for selecting documents for review during interviews; and to provide a frame of reference for participants completing outcome measures. The definitions also played an important role in the design of the intervention and outcome measures. Being explicit about the concepts we were studying obliged us to confront issues that had remained ambiguous in our day-to-day deliberations.</p> <p>Our draft definition of <emph>policy</emph> was a pragmatic amalgam of dictionary definitions. Following consultation, we supplemented it with examples of more specific state-wide population health and clinical initiatives. With the benefit of hindsight, a more inclusive response would have been to position program development work more explicitly within the term itself so that we were referring to <emph>policy and program development</emph> rather than <emph>policy</emph>. This would also have been a more process-orientated perspective which could have helped us to focus on the range of work practices we wished to influence rather than on outputs which we were already targeting with our definition of <emph>policy documents</emph>. A similar argument can be applied to our other terms. Each definition expanded to include program development, but the terms themselves did not. For example, having been advised by potential participants who we were calling <emph>policy-makers</emph> that they did not refer to themselves that way, we wrestled with alternatives to the term. We were unable to agree on a single term but, in the light of subsequent changes, <emph>policy/program developer</emph> may have been more suitable for our purposes. Likewise, the definition of <emph>health policy agency</emph> worked well, but if we were to use it with participating agencies we would consider revising the term to reflect the importance of program development: <emph>health policy/program development agency</emph>.</p> <p>Our definition of <emph>policy documents</emph> (which we would now call <emph>policy or program documents</emph>) was designed to help participants identify key documents produced in the course of policy/program work that would be discussed in interviews. Variation between agencies in work processes and document types precluded a description that could be applied to all; therefore, the definition became a broad list of examples. However, even this list remained problematic since there was no standard nomenclature: one agency's <emph>discussion paper</emph> is another agency's <emph>scoping brief</emph>. It is no surprise that, despite our changes to the instructions for applying this definition and the additional inclusion criteria, agencies continued to struggle with it. We recognized that a standalone definition that would apply to all agencies was probably not possible. Given that the process of presenting documents for assessment was an unfamiliar and potentially uncomfortable form of participation, we had an obligation to talk with the people nominating documents about how to apply the definition in their context, and to situate this within a broader conversation about our methods and aims and how the data would be used. Agencies had been given this information in written form, but more nuanced dialog that took account of local factors and concerns was also important.</p> <p>Some may disagree with our decision to include grey literature in the definition of <emph>research findings</emph>, but this reflected the real world use of investigative information within policy, and was acknowledged as such by participants. Including grey literature resulted in in-depth conversations about its use which, in turn, educated the research team about the variety and quality of innovative program designs and fit-for-purpose evaluations conducted by many government health agencies, most of which are never formally published. It was a reminder that much grey literature is grey not because of its merit, but because it is sometimes generated using expeditious methods, or is developed in contexts that do not prioritize academic publishing.</p> <p>The process of developing definitions – reviewing the literature and other likely sources, synthesizing and adapting definitions through the lens of the study methodology and goals, and consultation (testing) with colleagues in policy/program development – was messy but productive. We identified four factors that contributed to this productivity:</p> <p></p> <ulist> <item> <emph>An emphasis on functionality: our definitions were intended to be purpose specific so we were able to focus exclusively on our study aims rather than striving for generalizability</emph>. We believe that our definitions of <emph>policy</emph>, <emph>health policy-makers</emph>, and <emph>health policy agencies</emph> are broadly applicable in different contexts, but they should be reviewed for functionality when used in other studies.</item> <p></p> <item> <emph>Input from a broad range of people with in-context expertise</emph>. This helped us to understand the phenomena we were exploring, to refine the parameters of our inquiry and articulate the definitions so they reflected this. The appreciation that program development needed to be recognized explicitly within the field that we were calling policy-making led to changes in the definitions and, hopefully, to more sensitive communication with agencies who were participating in the intervention.</item> <p></p> <item> <emph>We asserted that conceptual definitions can and should be refined in response to data collection</emph>. Frustrating as it sometimes felt, the iterative revisions to the definitions obliged us to critique the terms we were attempting to define and, in some cases, to amend particular study aims and methods. We are confident that this resulted in more rigorous definitions, terms and methods. We would have preferred a clearer end-point – greater certainly about having arrived at the 'right' terms and definitions – but this will always be something of a judgment call since each application is liable to test them in new ways. Wading around in the marshlands of conceptual thinking and applied problem-solving is in the nature of social research and is critical for sound methodological development, but it must be balanced by the pragmatic need to arrive at an agreed (if imperfect) final decision. We probably underestimated how lengthy this process would be given the complexity of our study, its uniqueness, and the contexts in which it was to be applied. With this in mind, earlier consultation about our definitions and use of a more systematic and transparent revision and consultation process would have been valuable.</item> <p></p> <item> <emph>We agreed early on that the chance of reaching consensus on each of the definitions was, to quote one of the investigators, roughly that of a snowball in hell</emph>. The definitions would be intersubjective rather than objective, that is, they would reflect a variety of broadly agreed (or at least explicitly recognized) perspectives rather than providing a factual description (Gillespie & Cornish, [<reflink idref="bib14" id="ref44">14</reflink>]). The concept of deliberative dialogue was used where the goal is to explore different perspectives and to reach a shared understanding of the topic that can feed into solutions (Boyko, Lavis, Abelson, Dobbins, & Carter, [<reflink idref="bib7" id="ref45">7</reflink>]). This allowed us to move forward with some terms and definitions that were not favored by every team member. In fact, the debate about their merits and implications continues.</item> </ulist> <p>What is in a name? Returning to our opening quotes, we would argue that Hubert's advice is more persuasive than Romeo's declaration. The name that we give a thing frames how we see it. Defining a thing makes this frame more transparent and, certainly in our case, can lead to helpful critical consideration of the name itself. Although it is valuable, unpacking and delineating complex social phenomena is a challenging task, even within the confines of a relatively focused study. We take our hats off to those who tackle the fraught goal of developing universal definitions.</p> <hd id="AN0100241117-10">Conclusion</hd> <p>This paper explores the development of fit-for-purpose conceptual definitions of five terms: <emph>policy</emph>, <emph>health policy</emph>-<emph>maker</emph>, <emph>health policy agency</emph>, <emph>policy documents</emph>, and <emph>research findings</emph>. It illustrates the role of definitions as one of the building blocks in study design and provides an account of the process of developing them – reviewing the literature and other likely sources, synthesizing and adapting definitions through the lens of the study methodology and goals, and testing draft definitions with colleagues in policy and program development. It also describes our learning; in particular, our increasing appreciation of the value of the process itself, and the four factors that were most helpful: (<reflink idref="bib1" id="ref46">1</reflink>) An emphasis on fit-for-purpose functionality, (<reflink idref="bib2" id="ref47">2</reflink>) Consultation with in-context experts, (<reflink idref="bib3" id="ref48">3</reflink>) Our willingness to amend terms and definitions during their early stages of use and to refine some methods and goals as a consequence, and (<reflink idref="bib4" id="ref49">4</reflink>) Agreement that the team would 'satisfice': accept 'good enough' solutions rather than struggle for optimality and consensus.</p> <p>The development of definitions was a challenging, messy, and invaluable process. It allowed us to communicate more effectively with study participants, but it also obliged us to recognize our hazy understanding of key concepts and to hone and articulate them, to test our assumptions about shared meanings, and to wrestle with some fundamental questions about the nature of our inquiry. All of which, we believe, improved the overall rigor of the study. We ask others in the field of research utilization to define their terms when publishing.</p> <hd id="AN0100241117-11">Notes on contributors</hd> <p>Abby Haynes is a research officer at the Sax Institute in Sydney, and a research affiliate at the School of Public Health, University of Sydney. She is currently investigating the relationship between research and policy-making, striving to understand how interventions in complex systems can be improved, and pursuing her work using a variety of qualitative research and evaluation methods.</p> <p>Tari Turner is a senior health advisor at World Vision Australia and an adjunct senior research fellow at Monash University. She is passionate about translating knowledge into practice and, in particular, making sure women, babies, and children across the globe benefit from the findings of health research.</p> <p>Sally Redman is the CEO of the Sax Institute in Sydney. She is a public health behavioral scientist and her primary interest is in increasing the use of research evidence in health policy and practice. The Sax Institute seeks to assist health decision-makers to make better use of findings from research.</p> <p>Andrew Milat is an associate director of Evidence and Evaluation Branch in the NSW Ministry of Health, Australia. He is a policy-maker-come-intervention researcher with over 16 years' experience in the design, implementation, and evaluation of innovative health policies and programs at national, state, and local levels. He has overseen the development and implementation of numerous public health research programs and intervention trials, and is a strong advocate for the application of research evidence into policy and program development.</p> <p>Gabriel Moore is the principal analyst in the Knowledge Exchange Division of the Sax Institute. She is currently completing a PhD on the effectiveness of knowledge brokering in health policy settings. She is the lead author of <emph>What works to increase the use of research in population health policy and programmes: a review</emph> (2011).</p> <hd id="AN0100241117-12">Funding</hd> <p>This work was funded as part of the Centre for Informing Policy in Health with Evidence from Research (CIPHER) an Australian National Health and Medical Research Council Centre for Research Excellence [grant number APP1001436] and administered by the University of Western Sydney.</p> <ref id="AN0100241117-13"> <title> Notes </title> <blist> <bibl id="bib1" idref="ref1" type="bt">1</bibl> <bibtext> Some members of the research team argued that this point is poorly made given the (ironic) success of Alanis Morissette's 1996 hit single in which she erroneously claims that events such as rain on one's wedding day, dying soon after winning the lottery, and failure to take good advice are ironic. They are, of course, merely unfortunate.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref2" type="bt">2</bibl> <bibtext> We did identify problems with the instructions and additional criteria we gave about how to select the documents, but this was not an aspect of the definition and so is not discussed here.</bibtext> </blist> </ref> <ref id="AN0100241117-14"> <title> References </title> <blist> <bibtext> Adler, E. S., & Clark, R. (2011). An invitation to social research: How it's done. Belmont, CA: Wadsworth.</bibtext> </blist> <blist> <bibtext> Altman, B. M. (2001). Disability definitions, models, classification schemes, and applications. In G. L. Albrecht, K. D. Seelman, & M. Bur (Eds.), Handbook of disability studies (pp. 97–122). Thousand Oaks, CA: Sage.10.4135/9781412976251</bibtext> </blist> <blist> <bibl id="bib3" idref="ref3" type="bt">3</bibl> <bibtext> Australian National Data Service. (2011). What is research data? Retrieved March 21, 2013, from <ulink href="http://ands.org.au/guides/what-is-research-data.html">http://ands.org.au/guides/what-is-research-data.html</ulink></bibtext> </blist> <blist> <bibl id="bib4" idref="ref4" type="bt">4</bibl> <bibtext> Barrett, S., & Hill, M. (1984). Policy, bargaining and structure in implementation theory: Towards an integrated perspective. Policy & Politics, 12, 219–240. doi:10.1332/030557384782628291</bibtext> </blist> <blist> <bibl id="bib5" idref="ref10" type="bt">5</bibl> <bibtext> Bernard, H. R. (2000). Social research methods: Qualitative and quantitative approaches. Thousand Oaks, CA: Sage.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref31" type="bt">6</bibl> <bibtext> Billings, J. (2004). Towards rigour in qualitative health and social research across European partnerships. European Journal of Ageing, 1, 73–78. doi:10.1007/s10433-004-0012-310.1007/s10433-004-0012-3</bibtext> </blist> <blist> <bibl id="bib7" idref="ref45" type="bt">7</bibl> <bibtext> Boyko, J. A., Lavis, J. N., Abelson, J., Dobbins, M., & Carter, N. (2012). Deliberative dialogues as a mechanism for knowledge translation and exchange in health systems decision-making. Social Science & Medicine, 75, 1938–1945. doi:10.1016/j.socscimed.2012.06.016</bibtext> </blist> <blist> <bibl id="bib8" idref="ref5" type="bt">8</bibl> <bibtext> Canfield, J. V. (1981). Wittgenstein: Language and world. Amherst: University of Massachusetts Press.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref29" type="bt">9</bibl> <bibtext> Chan, D. J., & Donovan, B. (2005). What's in a word? Case definitions in sexual health medicine. International Journal of STD & AIDS, 16, 91–94.</bibtext> </blist> <blist> <bibtext> DeRoeck, D. (2004). The importance of engaging policy-makers at the outset to guide research on and introduction of vaccines: The use of policy-maker surveys. Journal of Health Population and Nutrition, 22, 322–330.</bibtext> </blist> <blist> <bibtext> Dobbins, M., Jack, S., Thomas, H., & Kothari, A. (2007). Public health decision-makers' informational needs and preferences for receiving research evidence. Worldviews on Evidence-Based Nursing, 4, 156–163. doi:10.1111/j.1741-6787.2007.00089.x10.1111/wvn.2007.4.issue-3</bibtext> </blist> <blist> <bibtext> Feldman, P. H., Nadash, P., & Gursen, M. (2001). Improving communication between researchers and policy makers in long-term care: Or, researchers are from Mars; policy makers are from Venus. The Gerontologist, 41, 312–321. doi:10.1093/geront/41.3.31210.1093/geront/41.3.312</bibtext> </blist> <blist> <bibtext> Fischer, F. (2003). Reframing public policy. Oxford: Oxford University Press.10.1093/019924264X.001.0001</bibtext> </blist> <blist> <bibtext> Gillespie, A., & Cornish, F. (2010). Intersubjectivity: Towards a dialogical analysis. Journal for the Theory of Social Behaviour, 40, 19–46. doi:10.1111/j.1468-5914.2009.00419.x10.1111/jtsb.2010.40.issue-1</bibtext> </blist> <blist> <bibtext> Graham, I. D., Logan, J., Harrison, M. B., Straus, S. E., Tetroe, J., Caswell, W., & Robinson, N. (2006). Lost in knowledge translation: Time for a map? Journal of Continuing Education in the Health Professions, 26, 13–24.10.1002/(ISSN)1554-558X</bibtext> </blist> <blist> <bibtext> Green, M. (2006). Representing poverty and attacking representations: Perspectives on poverty from social anthropology. Journal of Development Studies, 42, 1108–1129. doi:10.1080/0022038060088406810.1080/00220380600884068</bibtext> </blist> <blist> <bibtext> Greenhalgh, T., & Russell, J. (2009). Evidence-based policymaking: A critique. Perspectives in Biology and Medicine, 52, 304–318.10.1353/pbm.0.0085</bibtext> </blist> <blist> <bibtext> Greenhalgh, T., & Wieringa, S. (2011). Is it time to drop the 'knowledge translation' metaphor? A critical literature review. Journal of the Royal Society of Medicine, 104, 501–509. doi:10.1258/jrsm.2011.11028510.1258/jrsm.2011.110285</bibtext> </blist> <blist> <bibtext> Haynes, A. S., Derrick, G. E., Redman, S., Hall, W. D., Gillespie, J. A., Chapman, S., & Sturk, H. (2012). Identifying trustworthy experts: How do policymakers find and assess public health researchers worth consulting or collaborating with? Plos One, 7, e32665. doi:10.1371/journal.pone.003266510.1371/journal.pone.0032665</bibtext> </blist> <blist> <bibtext> Head, B. W. (2008). Three lenses of evidence-based policy. Australian Journal of Public Administration, 67(1), 1–11.10.1111/j.1467-8500.2007.00564.x</bibtext> </blist> <blist> <bibtext> Head, B. W. (2010). Reconsidering evidence-based policy: Key issues and challenges. Policy and society, 29, 77–94.10.1016/j.polsoc.2010.03.001</bibtext> </blist> <blist> <bibtext> Hodges, K. E. (2008). Defining the problem: Terminology and progress in ecology. Frontiers in Ecology and the Environment, 6, 35–42. doi:10.1890/06010810.1890/060108</bibtext> </blist> <blist> <bibtext> Huber, M., Knottnerus, J. A., Green, L., Horst, H. v. d., Jadad, A. R., Kromhout, D., ... Meer, J. W. (2011). How should we define health? British Medical Journal, 343, d4163.</bibtext> </blist> <blist> <bibtext> Jacobellis v. Ohio (No. 11). Supreme Court of the United States. 378 U.S. 184 (1964).</bibtext> </blist> <blist> <bibtext> Kemm, J. (2001). Health impact assessment: a tool for healthy public policy. Health Promotion International, 16, 79–85. doi:10.1093/heapro/16.1.7910.1093/heapro/16.1.79</bibtext> </blist> <blist> <bibtext> Klein, R. (2003). Evidence and policy: Interpreting the Delphic oracle. Journal of the Royal Society of Medicine, 96, 429–431.10.1258/jrsm.96.9.429</bibtext> </blist> <blist> <bibtext> Laderchi, C. R., Saith, R., & Stewart, F. (2003). Does it matter that we do not agree on the definition of poverty? A comparison of four approaches. Oxford Development Studies, 31, 243–274. doi:10.1080/136008103200011169810.1080/1360081032000111698</bibtext> </blist> <blist> <bibtext> de Leeuw, E. (1989). Health policy. An exploratory inquiry into the development of policy for the new public health in the Netherlands. Maastricht: University of Maastricht.</bibtext> </blist> <blist> <bibtext> Locock, L., Dopson, S., Chambers, D., & Gabbay, J. (2001). Understanding the role of opinion leaders in improving clinical effectiveness. Social Science & Medicine, 53, 745–757. doi:10.1016/S0277-9536(00)00387-7</bibtext> </blist> <blist> <bibtext> Oborn, E., Barrett, M., & Racko, G. (2010). Knowledge translation in healthcare: A review of the literature (Working paper series 5/2010). Cambridge, UK.</bibtext> </blist> <blist> <bibtext> Rabin, B. A., Brownson, R. C., Haire-Joshu, D., Kreuter, M. W., & Weaver, N. L. (2008). A glossary for dissemination and implementation research in health. Journal of Public Health Management and Practice, 14, 117–123.10.1097/01.PHH.0000311888.06252.bb</bibtext> </blist> <blist> <bibtext> Russell, J., Greenhalgh, T., Byrne, E., & McDonnell, J. (2008). Recognizing rhetoric in health care policy analysis. Journal of Health Services Research & Policy, 13, 40–46. doi:10.1258/jhsrp.2007.006029</bibtext> </blist> <blist> <bibtext> Smith, K. B. (2002). Typologies, taxonomies, and the benefits of policy classification. Policy Studies Journal, 30, 379–395. doi:10.1111/j.1541-0072.2002.tb02153.x10.1111/psj.2002.30.issue-3</bibtext> </blist> <blist> <bibtext> Walton, D., & Macagno, F. (2009). Classification and ambiguity: The role of definition within a conceptual system. Studies in Logic, Grammar and Rhetoric, 16, 245–264.</bibtext> </blist> <blist> <bibtext> Weiss, C. H. (1982). Policy research in the context of diffuse decision making. The Journal of Higher Education, 53, 619–639.10.2307/1981522</bibtext> </blist> </ref> <aug> <p>By Abby Haynes; Tari Turner; Sally Redman; Andrew J. Milat and Gabriel Moore</p> <p>Reported by Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib24" firstref="ref7"></nolink> <nolink nlid="nl2" bibid="bib31" firstref="ref8"></nolink> <nolink nlid="nl3" bibid="bib29" firstref="ref11"></nolink> <nolink nlid="nl4" bibid="bib23" firstref="ref12"></nolink> <nolink nlid="nl5" bibid="bib27" firstref="ref13"></nolink> <nolink nlid="nl6" bibid="bib16" firstref="ref14"></nolink> <nolink nlid="nl7" bibid="bib22" firstref="ref15"></nolink> <nolink nlid="nl8" bibid="bib17" firstref="ref16"></nolink> <nolink nlid="nl9" bibid="bib15" firstref="ref17"></nolink> <nolink nlid="nl10" bibid="bib30" firstref="ref18"></nolink> <nolink nlid="nl11" bibid="bib18" firstref="ref19"></nolink> <nolink nlid="nl12" bibid="bib19" firstref="ref20"></nolink> <nolink nlid="nl13" bibid="bib25" firstref="ref22"></nolink> <nolink nlid="nl14" bibid="bib33" firstref="ref23"></nolink> <nolink nlid="nl15" bibid="bib28" firstref="ref24"></nolink> <nolink nlid="nl16" bibid="bib13" firstref="ref26"></nolink> <nolink nlid="nl17" bibid="bib34" firstref="ref28"></nolink> <nolink nlid="nl18" bibid="bib10" firstref="ref32"></nolink> <nolink nlid="nl19" bibid="bib12" firstref="ref33"></nolink> <nolink nlid="nl20" bibid="bib11" firstref="ref35"></nolink> <nolink nlid="nl21" bibid="bib35" firstref="ref36"></nolink> <nolink nlid="nl22" bibid="bib32" firstref="ref40"></nolink> <nolink nlid="nl23" bibid="bib20" firstref="ref41"></nolink> <nolink nlid="nl24" bibid="bib21" firstref="ref42"></nolink> <nolink nlid="nl25" bibid="bib26" firstref="ref43"></nolink> <nolink nlid="nl26" bibid="bib14" firstref="ref44"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1191066
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Developing Definitions for a Knowledge Exchange Intervention in Health Policy and Program Agencies: Reflections on Process and Value
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Haynes%2C+Abby%22">Haynes, Abby</searchLink><br /><searchLink fieldCode="AR" term="%22Turner%2C+Tari%22">Turner, Tari</searchLink><br /><searchLink fieldCode="AR" term="%22Redman%2C+Sally%22">Redman, Sally</searchLink><br /><searchLink fieldCode="AR" term="%22Milat%2C+Andrew+J%2E%22">Milat, Andrew J.</searchLink><br /><searchLink fieldCode="AR" term="%22Moore%2C+Gabriel%22">Moore, Gabriel</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Journal+of+Social+Research+Methodology%22"><i>International Journal of Social Research Methodology</i></searchLink>. 2015 18(2):145-159.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 15
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2015
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Descriptive
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Intervention%22">Intervention</searchLink><br /><searchLink fieldCode="DE" term="%22Health+Services%22">Health Services</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Definitions%22">Definitions</searchLink><br /><searchLink fieldCode="DE" term="%22Public+Health%22">Public Health</searchLink><br /><searchLink fieldCode="DE" term="%22Public+Policy%22">Public Policy</searchLink><br /><searchLink fieldCode="DE" term="%22Outcome+Measures%22">Outcome Measures</searchLink><br /><searchLink fieldCode="DE" term="%22Research+Utilization%22">Research Utilization</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Australia%22">Australia</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/13645579.2014.921457
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1364-5579
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The development of definitions is an integral part of the research process but is often poorly described. This paper details the iterative development of five definitions: Policy, Health policy-maker, Health policy agency, Policy documents, and Research findings. We describe the challenges of developing definitions in a large multidisciplinary team and the important methodological repercussions. We identify four factors that were most helpful in this process: (1) An emphasis on fit-for-purpose functionality, (2) Consultation with in-context experts, (3) Our willingness to amend terms as well as definitions, and to revisit some methods and goals as a consequence, and (4) Agreement that we would satisfice: accept 'good enough' solutions rather than struggle for optimality and consensus.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 35
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2018
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1191066
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1191066
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/13645579.2014.921457
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 15
        StartPage: 145
    Subjects:
      – SubjectFull: Intervention
        Type: general
      – SubjectFull: Health Services
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Definitions
        Type: general
      – SubjectFull: Public Health
        Type: general
      – SubjectFull: Public Policy
        Type: general
      – SubjectFull: Outcome Measures
        Type: general
      – SubjectFull: Research Utilization
        Type: general
      – SubjectFull: Australia
        Type: general
    Titles:
      – TitleFull: Developing Definitions for a Knowledge Exchange Intervention in Health Policy and Program Agencies: Reflections on Process and Value
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Haynes, Abby
      – PersonEntity:
          Name:
            NameFull: Turner, Tari
      – PersonEntity:
          Name:
            NameFull: Redman, Sally
      – PersonEntity:
          Name:
            NameFull: Milat, Andrew J.
      – PersonEntity:
          Name:
            NameFull: Moore, Gabriel
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2015
          Identifiers:
            – Type: issn-print
              Value: 1364-5579
          Numbering:
            – Type: volume
              Value: 18
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: International Journal of Social Research Methodology
              Type: main
ResultId 1