The impact of the Czech Internet-Based Tobacco Control Program was assessed using qualitative or process measures as well as quantitative or impact indicators. The main evaluation indicators are summarized below.
Monitoring: The project maintained details of its own training, technical assistance and program dissemination activities which provided quantitative data on the type and number of events, participants/beneficiaries in each, use of the center and each of its programs including the web site, listserv, and resource directory.Process Evaluation
Diagnostic Profile-Baseline and Follow-up Data: A diagnostic profile containing more detailed entry-level information about every District Hygiene Station (DIT), health professional and NGO selected for training tracked progress in the key areas of technical skills, tobacco advocacy efforts and self-sustainability.
Internet Training Workshops and Tobacco-Control Conference Evaluations: Each workshop (Oct99, Apr 00, Jun 00) and the Tobacco-Control Conference was evaluated by the trainers and the trainees using an evaluation form to assess pre- and post workshop knowledge and attitude.
Internet and Email Surveys: The project hosted periodical pre- and post-e-mail and Internet surveys, based on campaigns such as the Quit and Win Campaign among the participating network member organizations to assess the outreach and impact of its training and technical assistance programs including the use of its web site, electronic bulletins, etc.
Quantification and qualitative assessment of Initiatives/Campaigns in Tobacco Control: Conducted by DIT's and others trained in this program the assessments were made on the basis of Internet-based informational materials, campaigns, strategies, documented community outreach efforts, Internet-based and other press releases and other possible evidence of campaign coverage, and by legislative or policy proposals submitted to government bodies or businesses.
MONICA Surveys: Patterned after the WHO, four longitudinal epidemiological assessments of the health of the Czech population conducted between 1985-1992 provided the baseline for plasma cholesterol, triglycerides, smoking prevalence and diet in the Czech population.
Process evaluation monitored and tracked the creation of the Internet-based Czech Tobacco-Control Training and Communications Program (TOB-CCP) in Prague, training and technical assistance to the tobacco-control community to enhance their capacity for communications, outreach and self sustainability, and fostering cooperation among Czech and international organizations through a nationwide electronic network for sharing resources, programs and services, and cooperation on tobacco-control advocacy. A comparison of planned and actual activities (below) indicates that the project fully accomplished its objectives.
Comparison of Planned and Actual accomplishments
|Needs assessment.Three information technology & skills transfer workshops.Conduct Tobacco-Control Conference.||First needs assessment
conducted Sept. 1999;
Assessment repeated February and May 2000.
Tobacco-Control Training and Communications Program to be established
at IKEM with high quality communications technology, and regional, national
and international linkage and outreach;
|TOB-CCP established at IKEM by December 1999; Electronic links with DIT's and NGO's completedRegional and international partnerships and linkages set up over 18 months|
|A resource service on tobacco control equipped with resource materials for Internet, computerized database research capabilities and access to the Internet.||TOB-CCP and NIPH capacity enhanced over 18 months or communications resources, technology, information dissemination.|
|Regular electronic-media tobacco-control communications, which is increasingly engaging Czech groups and as well as international partners in tobacco control.||Quit and Win Campaigns, 11th World Conference in Chicago and over 30 other events other on-going events exemplify an extensive series of communications and activities undertaken and continuing under the auspices of the project.|
|An electronic network of approx. 30 Czech organizations trained & engaged in tobacco control networking and others exposed to Internet-based communications technology.||An electronic network of nearly 40 Czech organizations-DIT's and NGO's are active members of the network initiated in October 1999 and completed by December 2000.|
|A WWW web site with a resource directory, monthly electronic bulletins and a list-serve in Czech in cooperation with UICC-GlobaLink, Geneva;||TOB-CCP website operational in March 2000. It includes a resource and membership, workshops and program description, events listing and communications with the public and professionals. Globalink posted a special Czech listserv for tobacco control in the Czech Republic on Globalink website|
|A Czech Stroke & Heart Association concerned especially with tobacco control formed & introduced to effective application of communications technology, exposed to international, sustainable models such as the American Heart Association, specializing in public health, especially tobacco control.||A Czech Heart Association (CHA) established in early 2000, formally launched in October 2000 with AHA assistance. CHA has a Board of Directors, planned agenda of priorities, resources and a time-table for the coming year|
|International partnerships with key American, and European organizations engaged in tobacco control and outreach to CEE-NIS organizations interested in extension of the program to their countries.||Dissemination of project and participation in extensive series of international and Czech events created a large number of international partners from the US, western Europe and the CEE-NIS. Several CEE-NIS partners have requested assistance replicating program in their countries.|
|Future Plans: No specific plans||Czech Heart Association and the TOB-CCP Network have a series of activities and events designed to insure long-term sustainability and enhance impact of the program.|
The impact of the program is demonstrated by the evaluation of the Training workshops (Oct99, Apr 00, Jun 00) and the Tobacco-Control Network as summarized below, and within the next 6 months will also become apparent in part from the results of the fifth Monica Surveys, to be conducted in 2001 to monitor this program.
Three MONICA Surveys were completed in 1985, 1988 and 1992 with a fourth comprising a study in 1992. Key results are that since 1985, plasma cholesterol levels have decreased significantly and smoking prevalence among men has decreased and remained stable. Changes in smoking prevalence were related to social and educational status and dropped substantially in subgroups of men with university education. The fifth MONICA survey to be completed in 2001, will also analyze diet using the same protocol used in the third MONICA survey in 1992, and allow further comparisons of project.
The impact of the three principal Training Workshops (Oct99, Apr 00, Jun 00) and the Tobacco-Control Conference was evaluated using pre- and post-survey methodology. In general, questions focused on the knowledge and skills of participants before the workshop and whether there were demonstrable improvements after participating in the workshop. In addition, participants were asked whether the training met their personal goals and the frequency with which the experience obtained will be applied in their jobs. Both the content and presentation by various workshop facilitators and presenters were also evaluated.
The Internet Training Workshop (Oct99, Apr 00, Jun 00) was rated as fully accomplishing, indeed exceeding the objectives of the course. Participants gained an increased understanding of a large variety of Internet and web concepts, ranging from email to listservers, bulletins, and research and participating in online discussions. They rated the workshop as extremely helpful in reaching their personal goals and in conveying technical skills and knowledge that would be applied frequently in their own professional work (Click the chart icon to see the chart).
Active participation in the media workshop produced ready-to-print articles and statements for the Czech media. Click on the corresponding chart icon for a chart summarizing the evaluation of the Media Relations and Advocacy Workshops and the Tobacco-Control Conference for usefulness, interest and relevance, showing that most ratings are at the upper end of the scale and thus demonstrate the success of the workshops.
Figure 5 is a pooled comparison of key representative sessions from the four major workshops- Internet Training, Media Outreach and Relations, Advocacy, and Tobacco Control - shows that participants rated all of them "extremely or very helpful and/or useful."
Summarized in Figure 6 are major activities of the Tobacco Control Network resulting from its participation in the TOB-CCP. About 25 of the 40 network members participated in a final network assessment to quantify the Internet communication activities - web page visits, research, tobacco control campaigns, e-mail for professional activities, workshop participation and activities, technological connectivity, and future activities. A large majority wrote health news stories and participated in the TOB-CCP coordinated "Quit and Win" smoking cessation campaign.
A sizeable group regularly used health websites and a budding group reported contributing to tobacco control legislation. Increase in knowledge and skills as well as contacts with each other and with other health professionals, especially Internet networking, for example with Globalink, rated the highest among the gains from the formation of the network and participation in the TOB-CCP.
Overall, Internet training helped network members become technologically sophisticated in networking with experts and health professionals, obtaining health-related information and using email for communications. While a significant majority of the network demonstrated technological connectivity through daily computer access, only two-thirds have daily Internet access. About the same proportion accessed health-related websites. This limited usage points to problems of accessing the Internet, currently via telephone and the cost is high. That a sizeable majority of network members participated in writing health news stories and contacting journalists demonstrates the success of the Media Relations Workshop. The network profile strongly indicates a reticence in legislative advocacy. These limited advocacy efforts reflect the nation's current history and lack of tradition in civic participation.
The evaluation of impact demonstrates that this comprehensive series of
communications and training activities fully met, indeed far exceeded the
project goals and objectives. While the specific, intrinsic outcomes
can benefit the public health community as well as the Czech public as a
whole, the overall project can also serve as a model program and its components
as templates singly or combined for future projects in other countries.
Questions? Comments? Concerns? E-mail CECHE at CECHE@comcast.net
Go back to the CECHE home page