Project planning, monitoring, evaluation and assessment.
Offering a Stance: An indigenous Celebratory Narrative.
We explore Planning, Monitoring, Evaluating and Assessment (PMEA) as evidence-based celebration of MARS practice and praxis. A celebratory narrative is not simply a series of self-congratulatory statements. On the contrary, this is a bold narrative that engages with change, with crises and mistakes, and with the learnings that have emerged from any of these – as part of an action learning cycle that is ongoing.
In this module, you will be invited to plan for such a narrative.
The PMEA narrative is firmly grounded in MARS actions, rather than in theory and ideas about MARS. It links together MARS values, a specific project’s aims and dreams, your intentions and assumptions, and how these have developed during the project's lifetime. In other words, the narrative is grounded in the practice and grows from people’s experiences and stories– rather than PMEA imposing a set of criteria and tools in order to explain / evaluate MARS work.
The PMEA narrative attends closely to the experiences of project participants through their stories, and to the specifics of the project place (history, geopolitics, culture, music, etc); as well as to the discourses of the audience / readers. The audience / readers might include project participants, funders, donors, beneficiaries, colleagues or policy makers.
Finally a celebratory narrative would be incomplete if it did not also contribute to the growth and enrichment of MARS practitioners and praxis.
Introduction to the module
Developing a stance - Planning, Preparation, and Panic-avoidance (3 Ps)
We discuss the pillars of understanding on which this module is built, which include:
Towards a Reflexive Agenda: No PMEA activity or narrative is value free: discerning our respective agendas: what story needs telling, to whom and for what purpose? What existing materials and what new materials might we need in order to assemble material for an agenda-driven narrative? (stories, films, photos, journal notes, session notes, random memories and moments, news items; etc)
Developing PMEA materials: What and when have we documented already? What might be useful for a PMEA narrative? What aspects of the fieldwork carry meaning and why and for whom; what would a rigorous documenting and description of our work look like? Are words the best? What about films? Photos? Sketches? ...
PMEA methods closely reflect, and emerge from the practices and from people’s experiences. Methods are not imported and imposed because they are considered to be high status or used internationally. Instead, MARS practitioners are encouraged to adapt, develop, and select aspects of existing methodologies on the basis that these ‘fit’ closely the project.
Celebrating MARS balances a narrative that conveys the best practice and that engages with project non-completion, impasses and disappointment. This is on the understandings that all experiences present a rich nexus for learning, reconsidering, and growing praxis. Celebrating missed opportunities and mistakes is as important as celebrating best moments.
Preparatory Reading Materials
- Barefoot Guide: a Threefold Theory of Change. Introduces ideas of emergent and transformative change as opposed to a Log Frame model that attempts a linear sequence starting with Aims and ending with Outcomes.
- The Action Learning Cycle Tools. Provides guiding questions for drawing together a PMEA narrative.
Additional introductory material
Assignment proposal: Finding your PMEA voice
This assignment is based on a practical exercise that involves role play. Here you are asked to develop a PMEA PLANNING narrative* (of a few minutes) based on 2 elements:
- a video document of the project in action;
- profiles and 'agenda' of various ‘stakeholders’, each of whom has in investment on hearing a particular narrative; we ask you to choose one of the stakeholders described, and develop your strategy with this person in mind.
Your task is to consider how you would address their agenda, based on the information you have. Tell us how you would go about preparing a PMEA narrative, given the MARS values, your experiences as a MARSian, the film and the agenda you are addressing. In your planning narrative, you need to convey an intention in response to a particular person and situation (described below). For your descriptive commentary, you need to select one or two moments in the video in order to help support your commentary.
The emphasis is on how you would prepare the PMEA narrative – what materials might be useful? How would you collect stories / anecdotes? What would be the main points you would want to make?
In other words, this is a planning narrative, discussing strategies.
We’re not looking for an instant 3 minute polished performance, and we don’t expect you to deliver an instant PMEA – that would be against the spirit advocated here.
2 very different scenarios are given below; the stakeholders are specific to each scenario.
A polished publicity film that conveys the work of the AULA children’s orchestra in Catalonia (work of MARS student Neus Carles).
- a youth counsellor who works with young people at risk and wants you to help him to convince his organisation that by playing in the orchestra, the children will be less inclined to lives of crime (nobody is convinced by his idea);
- the Parent of a marginalised / disabled child – who feel excluded from the orchestra and wants children like hers to join in (there is opposition to this idea as the orchestra has high musical standards);
- the Parent of a gifted child – who would like this to be a ‘proper’ orchestra where the children begin to play ‘real music’ (the parent is generally disliked but has political and economic influence);
- a Philanthropist who says she loves music, and appears both interested and distracted by this project – you know that the orchestra needs her financial support, and they depend on you to persuade the philanthropist;
- a local politician who wants funds diverted from the orchestra (too expensive, elitist, economically inefficient) to a multi-purpose leisure facility that benefits all children and young people.
An excerpt from a psycho-social music session for refugee children with behavioural/emotional problems, from NISCVT's Family Guidance Center, Beddawi Camp, Lebanon (work of Pòlestinian social worker Dalal Chahrour).
- the cultural adviser of a Palestinian NGO sponsoring this project, who is asking for 'success stories' to show to potential donors;
- the father of 1 of the children involved, sheikh representing an orthodox religious community of the camp for whom music is haram (prohibited), who is threatening to withdraw his child from the project and is asking NISCVT to stop the activity;
- the center's psychiatrist, who is asking that project funds are diverted into therapeutic treatment, which he considers more effective for these children;
- the director of one of the UNRWA schools in the camp, who may have funds available to develop similar activity in his school, and is asking for 'real evidence' of the efficacy of PS music intervention in this context;
- a European based NGO working with music therapy, who have offered funds on the condition that the project works more 'clinically'
The purpose of this exercise is to practice how a particular agenda (not our own) might set the evaluation frame. This is the frame through which you, as a MARS practitioner, need to thread together a convincing narrative that addresses and responds to the concerns of the person you’ve chosen. If you are working in a group, each of you could choose different stakeholders to respond to, so that you cover the range and get a good spread of planning narratives.
- Listening at Three Levels. How we listen when people tell us of their experiences – will influence how we weave their experiences into our narrative.
- Outside Story, Inside Story. The PMEA narrative needs to be warm hearted, persuasive and as rich as possible: there are outside and inside stories to be written.
More Conventional Texts on Evaluation
- Higgins, L. - Community music: In theory and in practice. Oxford University Press, 2012
- Tsiris, G., Pavlicevic, M., Farrant, C. - A guide to evaluation for arts therapists and arts & health practitioners. Jessica Kingsley Publishers, 2014
- White, M. - Arts development in community health: A social tonic. Radcliffe Publishing, 2009
Resonant writings on Evaluating and Celebrating: The Barefoot Guides (www.barefootguides.org)
- Doug, R. - A Threefold Theory of Social Change
Introduces ideas of emergent and transformative change as opposed to a Log Frame model that attempts a linear sequence starting with Aims and ending with outcomes. Also considers the value of crises and mistakes, and their role in learning.
- The Action Learning Cycle Tools
- Listening at Three Levels
This is a downloadable PDF called “Developing Deep Listening Skills - an Exercise and a Reading”; pp. 4-7 illustrate the concept of “Listening at Three Levels”
- Outside Story, Inside Story