In a world rightly pursuing evidence-based intervention, we started off with a handicap: we had no hard, objective evidence to validate the effectiveness of our program. S.T.A.R. was a new model, and our successes fell in the realm of “anecdotal”. The Psychology Department of Pacific University is changing this.
As part of Pacific University’s multi-year commitment to the program, since September 2013 year we have worked with seniors under the guidance of Dr. Heide Island, Associate Professor, Comparative Biological Psychology at Pacific University in Forest Grove, Oregon.
- Phase 1: Piloting Methodology for Evaluation and Interpretation of results (2013-14)
- Phase 2: Revising and Collecting Data of Revised Methodology and Interpretation of Results (2014-15)
- Phase 3: Implement Recommendations, Collect Data and Interpret Results (2015-2016)
- Phase 4: Submit completed evaluation (Spring 2016) WE ARE HERE – THE SUMMATIVE EVALUATION HAS BEEN PUBLISHED!
Phase 1: Piloting Methodology for Evaluation and Interpretation of results
Since September 2013 students have accompanied our S.T.A.R. teams to physically assist the handler, and to act as non-participating Field Observers who impartially collected data for outcome assessment.
The collected data is being used to create a program evaluation that will be used for grants, government certifications, and of course for their senior thesis presentations.
Equally important has been the weekly feedback provided by these scholars on pacing, terminology, and props. They have been valued allies in moving our program forward.
Phase 1 Findings
In April 2014 their findings were presented at multiple levels, from Pacific University to the Western Psychological Association Conference.
- CLICK HERE for a .pdf copy of the Power Point presentation where they describe how they solved the tricky issue of collecting meaningful, clinically relevant data in such an erratic environment.
- CLICK HERE for a .pdf copy of the final report of their findings.
- Conclusions in a nutshell: We encourage you to read their full report, but the following quote from the Discussion section at the end of the report contains a major take-away (bolding and use of color are for emphasis, and not part of the report):
This data demonstrates that brief bouts of Animal-Assisted Activities (AAA) can be evaluated in a meaningful, clinically relevant way. The within-block behavioral improvement can be used to compare differences in both cross-sectional and longitudinal designs. Data from individual points in time throughout the session were simultaneously compared to data from the beginning of a session to then end of a session. This methodology could be used for transient populations in clinical settings with brief bouts of animal assisted activities including hospitals, schools, and assisted living facilities. All of these settings would benefit more from shorter, less structured AAA’s, as opposed to the long-term, more structured nature of animal assisted therapy.
Results indicate that a child attending a single session with TLDL will show significant improvements in five of the six learning and behavioral realms (i.e., Engagement, Following Instructions, Concept Recognition, Attitude and Affect).
Phase 2: Revising and Collecting Data of Revised Methodology and Interpretation of Results
The students did some fine tuning of their observational methodology, and again accompanied our visits to two shelters. In May 2015 Dr. Island and the students prepared a formidable 73-page technical Formative Evaluation Report The Little Dog Laughed (TLDL) Animal-Assisted Therapy Program. The Summative (final) Evaluation Report will be published in July 2016.
- CLICK HERE for a .pdf copy of the full Formative Evaluation Report.
- CLICK HERE for a .pdf copy of the poster presented on November 19, 2015 at the Chicago Program Evaluation Conference. (This is for a large display poster)
- Vastly condensed from their conclusions (again, the bolding is not part of the report):
Since a mission statement is a formal summary of the values and goals of an organization, it was also necessary to evaluate how well the program actually met the objectives of its mission statement. The Little Dog Laughed, Animal-Assisted Therapy mission statement asserts, “Our volunteer dog/handler teams work with therapists/counselors/teachers in their effort to nurture empathy and non-violent problem solving skills in at-risk youths. We offer a carefully structured set of short classes that teach behavioral skills by engaging the children in actively training our dogs using positive training techniques.” From this statement, there were essentially three constructs we could measure: 1.) empathy, 2.) nonviolent problem solving, and 3.) teaching behavioral skills through reinforcement.
Based on the observational data, The Little Dog Laughed, Animal-Assisted Therapy Program did appear to achieve behavioral improvements among their target population in all three constructs associated with the mission goals… TLDL model is adaptable to a number of other contexts and target populations. It is conceivable that the seven behavioral outcome categories for child residents of domestic violence shelters could be modified for a variety of other populations. For example, TLDL model could be adapted for rehabilitation centers, to assist patients with motor damage, assisted living facilities to help residents with attention, memory, and loneliness, and inclusion facilities to help residents with developmental disabilities.
Phase 3:
Dr. Heide Island, Associate Professor of Comparative Biological Psychology at Pacific University reports that due to the hard work of the students, they will be able to wrap up what was expected to be a five-year effort in three! From her summary of their calendar:
November 19, 2015
New data – cumulatively evaluated with the old data was presented at the Chicago Program Evaluation Conference in November. [CLICK HERE for a .pdf copy of the poster]. You will note that the Concept Recognition category is now significantly different pre/post (this is what we were hoping to see after last summer’s formative evaluation), this is consistent with all behavioral outcome categories. So nice job in facilitating and promoting the behavior among the kiddos this last semester so the outcome could be better measured.
February 27, 2016
Students will be giving an oral presentation as the Oregon Academy of Science at Pacific the third weekend in February of these results as well.
April 27th, 2016: Senior Projects Day,
TLDL Seniors (all of the students) will be presenting the 5-year project, that through collaboration with TLDL in making small adjustments to program recommendations, was completed in 3!
dMay 1, 2016
Students will be presenting there poster on the final data set at the Western Psychological Association in California.
June, 2016
Sometime in July, I will have the final Summative Evaluation (much like last years’ but will be summative rather than just formative) for your review. This will be the culmination of our project, as you should have more than enough to demonstrate evidence-based programmings. There is no need to continue further since the Concept Recognition and Social Civility categories are now significantly different in the direction we would expect, from Block 1 to 4.
Phase 4:
As of July 31, 2016 The full Summative Evaluation Report has been published, and proudly posted on our site – CLICK HERE for a .pdf copy (warning: it’s 174 pages long!).
Here’s one of my favorite parts (bolding is mine):
Bottom Line: Based on the observational data, The Little Dog Laughed, Animal-Assisted Therapy Program demonstrated efficacy for child residents of two domestic violence shelters. Evidence was established through statistically significant behavioral improvement for each of the seven behavioral constructs (as represented by TLDL mission) within a single 15 to 20-minute animal-assisted activity session. TLDL Animal-Assisted Activity/Therapy model is adaptable to a number of contexts and target populations. It is conceivable that the seven behavioral outcome categories for child residents of domestic violence shelters could be modified for a variety of other populations, including but not limited to assisted living residents, youth and adolescent life skills training, and adults or children with physical or intellectual disability.
