 |
 |
Home | Memo 1 | Memo 2 | Memo 4 | Promotional Materials
Memo 3Interpreting Your Printouts and Drafting Your Report When you have received the statistical printouts, showing the results of our evaluation of the Parenting the First Year instructional newsletter series in your county, you are ready for the next step in the evaluation process. Included with this memo you will find pieces on:
You will want to start by looking over
your printouts. Even if you've done statistical analyses
before, every computer system has its own way of displaying results,
so don't be surprised if you have to re learn how to make sense of these.
Included in Memo 3 are some sheets on reading
SPSSPC printouts. By the way, you are welcome to copy these results
for your local collaborators, if you think this is a good idea. Newly Constructed Variables At the end of the first printout, however, you will see several additional "constructed" variables. I will define the new constructed variables for you here as a quick reference:
I also constructed several alternative indices of risk status. These are useful for seeing if "at risk" parents benefit more from the newsletter series than do "non-risk" parents. This kind of information can be useful in targeting our programs for maximum impact, or showing that they have greatest impact where it is needed most. Here are explanations of the risk variables.
Next Steps I have kept a copy of your printouts in my office. As you page through your printouts, feel free to write notes all over them. I find that is a good way to begin the process of turning numbers into meaningful sentences of text. Your task now, after the initial excitement of these printouts wears off, is to turn these numbers into an interesting story-—a draft of a report. To facilitate this process, you should download a copy of a model report as a writing template, if you have not already done so. It will make your work easier to use the same basic form and sentences as the earlier report. You may still modify that basic report as much as you like. In fact you will have to modify it: your results will inevitably look somewhat different from others. For example, in discussing the most useful sources of parenting information reported by parents (questions 6 through 14), the source described as “very useful” by the highest percentage of parents might be different in your results than in Badger County. Modify accordingly. And, of course, you may choose to emphasize different findings than we did in the earlier report, or to include results that we ignored. As an initial step toward drafting your report, I recommend you modify the figures (charts) in the report first. Just cross off the numbers and write the correct numbers for your results in their place. Once you have done this, you can refer to your corrected figures when writing (or changing) the text. The Crosstabs by RISK Status I calculated cross-tabulations of the basic outcome variables (Questions 16-17, 19-20, and 22-23) using the different risk variables. I do this so we can select the index of risk that produces the simplest and clearest contrast of groups. I have also summarized these results on a one-page "crosstabs" summary sheet (you received this along with your printouts), which shows the percentage of each group who answered “yes” to each of the questions about behavior change. We will use these results to produce your own version of Figure 8. Look at the “crosstabs” summary sheet that was sent to you. You can see that similar contrasts were produced by the different indices of risk. Most of the contrasts produced the result we expected, namely that a greater percentage of the “risk group” parents would report positive changes in parenting due to the newsletter, as compared to the “non-risk” parents. Asterisks on the crosstabs summary sheet mark contrasts that were statistically significant for your data. I recommend using the comparison (set of contrasts) that is strongest for your Figure 8 (this is marked on your summary sheet). This will make a simple and powerful version of Figure 8 in your report. If the comparison that is strongest (for your data) is the contrast between “risk group" and “non-risk” parents, then you can modify the appropriate section that is included in the Badger County Report. If the comparison that is strongest (for your data) is the contrast between “first time” and “experienced” parents, a section is included that you can modify to use in your report (simply click on the underlined link). Lie Scale Items Two lie scale items (questions 18 and 21) were used to check for intrusion of social desirability or response set bias in our results. Each item asked about something that might have been in the newsletters, but wasn't. If a respondent reported having learned about both of these topics in the newsletters, then we strongly suspected that their answer to these and all other questions were biased by either response set or social desirability bias, or by both biases. If any respondent answered "yes" to both of these questions in your data, this respondent was deleted from the data file before conducting any of the statistical analyses (frequencies and crosstabs) to increase the accuracy (reliability) of the results. "Percent" versus "Valid Percent" You may have a question about whether to report the "percent" or "valid percent" from your printouts. Valid percent is the percent if missing data are excluded from the calculation. In general, I would use the "percent" for any question that had a "yes" or "no" answer, making the conservative assumption that people who didn't answer are really answering "no." I would report "valid percent" in the cases of socio-demographic variables (questions 24 to 32), where it is important that all the categories add up to100%. After you've looked over the results, you can email me with any questions or comments you have. It's always helpful to compare notes on how we are interpreting the results. Rounding off to Whole Numbers I recommend rounding off to integers. This is more honest: we don’t really have accuracy to the nearest tenth or hundredth of a percent! And it makes for a much friendlier report, especially for people who are “math phobic.” Here are the two rules I use for rounding. First, round to the nearest whole number, so 4.4 becomes 4, while 4.6 becomes 5. Second, if the number is exactly between two integers, round to the nearest even number. In that case 7.5 would become 8 (since 8 is even, while 7 is odd), and 8.5 would also become 8. By using this method, we ensure that numbers are rounded up or down about an equal number of times, without our being able to affect the number in a way that favors our own desired outcome. Quotations from Respondents Hopefully you have already transcribed the comments onto typed pages. These are very helpful to read, not to mention gratifying! After you edit the numbers and the basic text of the report, you can go back and insert quotations from the respondents in your own county. You will see from the earlier Badger County report the kinds of places where quotes fit well, but you can insert them in other places as well. I recommend no more than three or four in a row; otherwise, the reader may get impatient. These quotes do not provide “evidence” for our conclusions the way our data do, but direct quotes are a powerful way to provide “examples” of our main findings. They help our conclusions come alive. It is okay, indeed admirable, to include a few quotations that run contrary to our main conclusions. You can say something like, "A small minority of respondents disagreed strongly, as evidenced by this comment: . . ." Doing this is part of being fair and truthful, and it will raise your credibility with the reader. Next Steps after Drafting the Report Once you have written changes onto the earlier (Badger County) report,
so that it will fit your results, then send it to me through the mail
along with the corrected figures. Don’t forget the page of acknowledgments
to recognize any collaborators you have, your secretary, etc. If you prefer to have your office produce the report, just let me know. I will still want to review the correction draft. We should consider for whom the report is being written and what impact we hope it to have. The best option, in my view, is to write it for your local collaborating organization(s) and/or for a local agency or organization that might be convinced to fund the distribution of the newsletter for the future. We should hope for such an impact in every case. The results of our evaluation make a strong case for the impact of the newsletter. Who would be influenced by these results to budget some money? If you have any questions about how to proceed, Memo 4 discusses the "next steps" in more detail. There are a variety of ways you can use the final report as an educational tool. Memo 4 lists a few of these, but here is a preview. I always recommend that you use your results for presentations in your county. Some groups will be interested in hearing a report of your evaluation because of the topic (e.g. health professionals, parent educators), but many others will be interested simply because they are interested in their communities (e.g. Kiwanis, Lioness Clubs, etc.). To help in presentations like this, we include a draft copy of a 2-fold evaluation summary sheet. Use this as a sample of something you could produce in your own office. A sheet like this can be handed out to everyone who attends your presentations. After you receive the finished draft of the report go to Memo 4, which also includes some sample press releases. We always talk about how Extension educational programs are “research based.” With this project, you will be teaching based on your own research! If you have any questions as you proceed with drafting your report email us at: dariley@.wisc.edu or costergr@wisc.edu
Your Co Investigators,
|
Memo 1 | Memo 2 | Memo 4 | Promotional Materials
