About the AuthorAlly Krupar is an Adjunct Instructor at American University’s School of Professional and Extended Studies where she teaches Qualitative Methods in Monitoring and Evaluation. (Use a numeric or the degree of appeal scale). Focus group questions should guide the groupâs discussion. New York: Kluwer Academic Publishers. If I wanted to see more content, I know where to click. Retrieved from http://www.dfid.gov.uk/Documents/publications1/design-method-impact-eval.pdf Stufflebeam, D. L., Madaus, G. F., & Kellaghan, T. (2002). Surveys. Reasonable and appropriate, or realistic in the given project or program. Evaluation models: Viewpoints on educational and human services evaluation. Process evaluation questions focus on how a program is working, program performance, and involve extensive monitoring. Program Evaluation Tip Sheet: Constructing Survey Questions October 2011 . RRA rapid rural appraisal How does the target population interact with the program? Format - When choosing focus group questions, be sure to include a mix of multiple choice questions to provide structure and direction and open-ended questions to allow for further exploration and discovery. The benefits of doing business with company are clearly presented in this web page. (1999). Please fill out both sides of this form: OBJECTIVES This program met the stated objectives of: 1. Evaluation Questions Checklist for Program Evaluation Lori Wingate and Daniela Schroeter Evaluation questions identify what aspects of a program1 will be investigated. Some sample questions are: 4. Methodologically, evaluation questions focus on varied assessment types. She is also a Doctoral Candidate in Adult Education and Comparative International Education at Pennsylvania State University and a Visiting Researcher with RET, an international organization providing secondary and post-secondary education to displaced peoples worldwide. What are the outputs, outcomes, objectives, and goals of the project? Scoring Sheet (for use with Quality Matters Rubric) ... Education encourages continuous program evaluation and improvement, with an emphasis on identifying the strengths and weaknesses of online course offerings in the College. your program is in its early stages, you may wish to focus on formative evaluation information, which will help you revise the program before undertaking a summative evaluation. 5. A focus group lasting more than 90 minutes probably includes too many questions or topics for discussion. * Reliability When they feel uneasy, they will retreat into their shells and you will not get the input that you need. Focus Group Interview Template. What three words would you use to describe the product? The questions in an evaluation questionnaire should focus on the goals, objectives, strategies, and work plan. According to Rossi, Freeman, and Lipsey (1999), evaluation questions must be: Once we have developed the larger question of the project, M&E practitioners begin to consider what data they need to answer the question, using a Theory of Change and a LogFrame. Instead of asking âwhyâ you should rephrase the question to ask respondents to provide some specific examples of what they liked or did not like about the item in question. Prepared for: U.S. Environmental Protection Agency Office of Human Resources and Organizational Services Office of Planning, Analysis, and Accountability Washington, DC 20460 Use the guide to answer basic questions about the focus group process. But, without well-developed, relevant, and accurate evaluation questions, developed with stakeholders connected to the problem, projects can move around a problem without addressing the most appropriate issues. NFI nonfood item . Assessment Tool kit: Focus Group Template Focus Group Script . It illustrates how to order questions within a topic of inquiry. Are they satisfied? Ask them about their favorite hobbies and interests and what they enjoy the most about the activity. 3. How could those services be provided? What is evaluation Focus groups are structured discussions among small groups of people. Questions should be carefully planned but appear spontaneous during the interview. Discovering these issues can help determine which of a number of options is the preferred way forward, or to determine what are the concerns that would prevent a proposal going ahead. 1. The first order of business when considering conducting a process evaluation is to form a working group that includes key program stakeholders. As stated above, program ... taker for focus group questions and/or activities which we develop with your program in advance. What, if anything, do you find frustrating or unappealing about the product? Once you learn basic questionsâ¦ If you would recommend this product, what would you say to convince your best friend to purchase this product? In addition to this guide, we have focus group proposals, moderatorsâ guides, presentations and reports, along with other manuals 86–105). [caption id="attachment_132881â align="aligncenterâ width="640â] Questions should be a mix of free form and multiple choice[/caption]. If so, how? When using a focus group in marketing research, you must ask the right questions for accurate results. Are objectives met? Focus Group Questions to Engage Your Group. What are the target population’s characteristics? If you would not recommend this product, what would you say to convince your best friend not to buy this product? These consultations may be informal conversations, reviewing grant requirements and terms of reference (documentation review), or semi-structured individual and/or group interviewing. When respondents are relaxed, they will open up and talk more. 2. This article will give you many effective focus group questions that should spark discussion. Are there other populations the program should be working with? (2012). If I wanted to purchase the product, I know where to click. When thinking about navigating this page, what, if anything, is confusing to you? Impact/ outcome evaluation, or how does the program reach its outcomes or impact? Welcome and thank you for being here today. Contract 68-W-02-048 . The purpose of this gathering is to get your feedback about how we can better serve students enrolled in math courses, such as yourselves, at the THINK TANK. OFDA Office of U.S. Foreign Disaster Assistance . Please rate on a scale of 0 to 10, where 0 is extremely dissatisfied and 10 is extremely satisfied the following features: Assessment of efficiency, or how cost-effective is the program. The main purpose of an evaluation plan is to focus on the evaluation to be undertaken while keeping a track of the desired outcomes and results. How much do you agree or disagree with the following statement. (Use a likelihood scale). Other suggestions in developing questions for focus group sessions include: a. Before we decide what types of data we need (qualitative or quantitative) we need to know the bigger question of the project: What specific problem does the project or program address? What specific features of the product impressed you? These resources are sample evaluation forms and guides to adapt for your own use. Is the problem that the project/ program intends to address alleviated? Focus group question examples. Designing and Conducting Focus Group Interviews Richard A. Krueger Professor and Evaluation Leader University of Minnesota 1954 Buford Ave. â¦ According to Owen and Rogers (1999), there are three levels of evaluation questions at this stage in project planning: The best questions must be developed with stakeholders in the evaluation, including program staff, sponsors and funders, local and regional decision-makers within and outside the program, and community representatives, when the community in which the evaluation or project will be carried out has already been identified. What most contributes to your pessimistic outlook for the economy? Authorâs experience as a member of several focus groups involved with evaluating new food and investment products. For best results, a focus group session should include around five or six questions. 5. Some sample questions are: 2. surveys and focus group questions for student participants, parents, teachers (survey only), program staff, and program partners (survey only). Find the perfect sound in seconds. What most contributes to your optimistic outlook for the economy? Work Assignment B-16 . I am providing my consent by leaving the opt-in checked. Evaluation: A Systematic Approach, 6th Edition. The evaluator consults with all accessible stakeholders to develop specific questions that the evaluation will seek to answer. If you need more detailed information on public opinion research, please feel free to contact the Political Party Development Team. Step 1: Form a working group. The Various Types of Group Evaluation Forms. These questions can help assess the general business climate as part of the projectâs risk analysis. KEQs usually need to be developed and agreed on at the beginning of evaluation planning - however sometimes KEQs are already prescribed by an evaluation system or a previously developed evaluation framework. Course summary evaluations, focus group questions, and expert observation tools are included. What similar products have you used during the last 6 months? Process evaluation questions focus on how a program is working, program performance, and involve extensive monitoring. 4400 Massachusetts Avenue, NWWashington, DC 20016. (Use a numeric or favorable scale). Focus groups are often used in concert with other product research methods to determine not only how the product could be improved but also how the product should be marketed. Sample focus group materials can be found in appendix B (PDF 101 KB). To learn more about American University’s online Graduate Certificate in Project Monitoring and Evaluation, request more information or call us toll free at 855-725-7614. However, sometimes program assessment and teacher evaluation overlap. How would you rate your outlook for the economy over the next six months? Which of the following best describes your discretionary spending during the past six months? The document offers suggestions on writing the best possible survey questionsâit is . Message and data rates may apply. Are the project/program services/activities beneficial to the target population? What resources does the project/ program need? The results can then be included in the focus group report, and can be used as a Product Reviews: Is Everyone Completely Satisfied? Ratings scale - Select an appropriate ratings scale to measure different levels of approval. All Rights Reserved. A focus group is an information-gathering tool where a selected set of participants, for example, a group of teachers, students, parents or members of the board, gather to discuss a particular area which has been identified as the focus for a schoolâs evaluation. Needs assessment, or identifying the surrounding social conditions and need for a program. If the problem is not identifiable by stakeholder communities, projects addressing it will not be successful. Tracy (2013) explains that within a focus group, there is a cascading effect of conversation, as comments link into each other and participants share what can be rich, emic, qualitative, data with each other and the moderator. Do you have any suggestions on how we could improve the product to make it more appealing? Typical focus group questions are like these: Dr. Beverly Peters has written recently about qualitative interviewing and conducting and using focus groups. When respondents are relaxed, they will open up and talk more. â¢ The focus group needs to be appropriate for the type of participants in the group. IR intermediate result . Chapters 2 and 3. For example, ask them to think of three words to describe their favorite hobby. PRA participatory rural appraisal . How likely are you to purchase this product? Program theory assessment, or identifying how the program intends to address the problem. Evaluation questions may seem intuitive, and thus be quickly developed to get to the more detailed program planning. If I wanted to contact the company, I know where to click. Is the target population adequately reached by and involved in activities? No matter why a focus group is formed, you can help your members have a more productive session by asking questions that grab their attention and get them thinking beyond the obvious. Program evaluation is a systematic process of obtaining credible information to be used for program assessment and improvement. Fun Starter Question or Activity - Ask a fun question or incorporate a team building game to get the groupâs creative juices flowing. ! The order of these questions can be shifted around. Copyright Â© 2020 Bright Hub PM. EVALUATION OF THE EPA INTERN PROGRAM . Are activities conducted with the target population? 1. Broadening the Sharing the of Designs and Benefits of Trade Methods for Impact Evaluations: Report of a study commissioned by the Department for International Development. * Speed From Evaluation Questions to Evaluation Findings. Are there alternative approaches that could have the same outcomes with less cost?*. How easy or difficult was it to navigate the content on the web page? Appendix B provides a sample timeline for a small focus group study. 6. The key to creating a positive experience for your participants is by asking questions that will awaken both their creative and critical sides. Policy level – how does, or could, the evaluation impact relevant policy? London: SAGE Publications Ltd. Additional ResourcesMorra Imas, L. G., & Rist, R. C. (2009). Sample questions are: Through asking questions, M&E practitioners can identify what their project specifically should address. Here are some sample questions to help a project manager evaluate a new website design or concept. Thinking of the answer to the previous question, what made you say that? As the project manager and facilitator of a focus group, your job is to fully engage the members to get their honest feedback. Focus groups can be a good way to collect qualitative data in monitoring and evaluation, especially when group interaction might produce insight into a topic. ICT4D information, communication and technology for development . * Specific Features (as named by you). Project level (local, activity based, “little p”) – how does, or could, the evaluation effect project or local changes?**. Describe the composition of the group and ensure that (1) the goal of the focus group was defined, (2) all information will remain confidential and (3) respondents introduce themselves to each other before proceeding with the discussion. The questions and the manner in which the focus group is facilitated will vary based on the type of participants. We give you the insight you need to write all types of focus group questions with the help of proven examples and expert advice. Closing - Remember to thank the participants for their assistance and candor, and then ask them if there is anything that could be done to improve their experience as a member of the focus group. Using Focus Groups in Program Development and Evaluation by Roger A. Rennekamp, Ph.D. and Martha A. Nall, Ed.D. What features of the product disappointed you? Is the cost of the services or activities reasonable in relation to the benefits? Topics to be covered are drawn from the following list. One of this groupâs first tasks will be to determine whether to bring in an outside evaluator or have staff conduct the evaluation.17, 18! Learn more about how to file a complaint about a distance program or courses. The program evaluation was conducted by UCLAâs Center for Study of Evaluation â¦ (extremely, somewhat, or not familiar at all), Overall how satisfied are you with the product? Post-Primary Focus Groups Focus Groups. Yes, I do want to communicate with American University via SMS. Process evaluation, or how the program addresses the problem, what it does, what the program services are and how the program operates. Writing focus group questions in the moderator's guide or focus group questionnaire is your first step. We appreciate your help in evaluating this program. (Use an agreement scale). Evaluation questions, similar to research questions in academic research projects, guide the methods and tools used to collect data to understand the problem under investigation. Extension Specialists in Program and Staff Development 203 Scovell Hall University of Kentucky Lexington, KY 40546-0064 Do they have negative effects? IEC information, education and communication . conducting a process evaluation. The Road to Results: Designing and conducting effective development evaluations. EVALUATION TOOL. 25+ Key Focus Group Questions: How to Choose, Write and Ask the Best Questions. Some sample questions are: 5. The questions should prioritize the following: It should always include less than ten questions. Introductions - Begin the session with a set of questions that will enable focus group members to get to know each other better. (Use a numeric or degree of likelihood scale). Are outcomes, objectives, and goals achieved? The curriculum and intended learning outcomes . See the article about moderator guide. Similarly, formative evaluation questions look at whether program activities occur according to plan or the project is achieving its goals while it is underway. Community Food Security Coalition ii Overview The Community Food Project Evaluation Toolkit (Toolkit) is a hands-on workbook designed to help community food project (CFP) staff conduct evaluations of their programs to develop the kind of information that will be compelling not only to fellow staff but to funders, participants and Thinking of your previous response, what are some specific examples that helped you formed your opinion? Sometimes a project manager needs to do a reality check to see if it is the right time to move forward on a business venture. If the program has been in place for a while and is thought to be relatively stable, then it may be time for a summative evaluation. She holds a BA in Anthropology from Case Western Reserve University and an MA from the School of International Service at American University. Is there anything that you would like to add or remove from the design that you have seen? How much do you agree or disagree with the statements below? As the project manager and facilitator of a focus group, your job is to fully engage the members to get their honest feedback. Use questions that have relevant importance to a particular situation or issues. Sample Focus is the webâs premiere FREE community curated royalty-free sample library. Evaluation involves a systematic collection and analysis of data to critically examine a program for better effectiveness. e. Is the target population affected by the project/ program equitably or according to the evaluation plan? Specificity - Avoid vague open-ended questions that simply ask âwhy.â The word âwhyâ has a negative connotation because it makes the respondent think their first answer was not good enough. not What is the structure or organization of the project/ program? Student Focus Group Questions A4. Through what mechanisms or arrangements? There are many different types of Group Evaluation Forms that allow people to properly asses a group and determine if theyâre doing a good job together or if theyâre better off splitting up. Questions for student focus groups . Try not to have too many Key Evaluation Questions - a maximum of 5-7 main questions will be sufficient. Keep up with the latest happenings, news and accomplishments at American University. Where it is necessary and/or helpful to hold a focus group to ascertain the views of a cross-section of students from across the subject, the following questions can be used: a. And what can be learned? Some sample questions are: Gather the participantsâ names. For example, you could ask the groupâs overall impressions of the web page last. In Program Evaluation (pp. Washington, DC: The World Bank.Stern, E., Stame, N., Mayne, J., Forss, K., Davies, R., & Befani, B. ** Adapted from Owen, J. M., & Rogers, P. J. Consumer sentiment and consumer confidence are the forerunners to the future success of many businesses. Programs that are already running should have a theory of change, or how they think their intervention will lead to the stated outcome, objective, goal, or impact they hope to see. Similarly, formative evaluation questions look at whether program activities occur according to plan or the project is achieving its goals while it is underway. Image by Gerd Altmann from Pixabay, How to Develop a Change Communication Plan. References*Adapted from Rossi, P. H., Freeman, H. E., & Lipsey, M. W. (1999). This tip sheet is intended for programs funded through the Centers for Disease Control and Preventionâs (CDC) Division for Heart Disease and Stroke Prevention. What do they think of the services? 4. 2 PRACTICAL GUIDE: FOCUS GROUP DISCUSSIONS CONTENTS The decision on when to conduct FGDs andâin case of a mixed-methods approachâ their sequencing in relation to the other methods is influenced by the objective of the effort; the overarching assessment, monitoring or evaluation questions the FGDs seek The evaluation questions may also be used in summative evaluations which focus on what happened after the program or project completed, i.e., were goals achieved? FGD focus group discussions . Program level (regional, large scale, “Big P”) – how does, or could, the evaluation effect program changes? How best should the project/ program deliver those services (or activities)? M&E monitoring and evaluation . (Use an agreement scale). If questions are too vague or broad, or require data that is unavailable or unobservable, they are not answerable. Good focus group questions have the following characteristics: A friendly and conversational tone; Language or phrases that â¦ For example, this scale has 7 levels to measure the level of satisfaction: completely satisfied, very satisfied, somewhat satisfied, neither satisfied nor unsatisfied, somewhat unsatisfied, very unsatisfied, and completely unsatisfied. The following list is not exhaustive nor mutually exclusive but provides a framework for thinking about what kind of evaluation you are doing and what (generic) evaluation questions you may need to ask. Strong focus group questions are critical to successful market research.
Analytical Chemistry Pdf Notes, Advanced English Vocabulary List, How To Make Marshmallow Balls, Vi Derm Ultra A, Micro Miniature Roses For Sale, Human Eye Transparent Background, How To Save Aloe Vera, Best Para 3, Boost It Spray Vitamins, 6th Sense Crush 50x Review,