There are a few tools used in evaluation (of designs in general, as of permaculture designs in particular) and one is simply to use well all of our Permaculture general design criteria, + your own particular & local ones :
1) to be clear of our design criteria at the out-set (when we start designing),
2) applying these design criteria in our design &
3) then observing which criteria we could keep to & why not others
What we hope this doesn't get mixed up too much with is Indicators ... a similar concept but quite a different animal in practice: & one which has been demonically effective at halting much real progress as it seems to easily seduce people into "paralysis by analysis" type cul-de-sacs.
Indicators, like Criteria, essentially are the visible parts of whatever model/s & values the observer is watching through, only Criteria can be 'fuzzy variables' and are applied more in changing things (could be said to be PRE-design), and Indicators need to be 'specific & measurable' as their purpose is just that: measuring things (& so could be said to be more POST-design).
You will, however, generally hear very little (if at all) about "Indicators" in permaculture design, essentially because it's a very pro-active science & usually the only indicators we are interested in are whether some system or design fulfills the permaculture design criteria, or not.
So when we talk of Design Criteria in Permaculture, we are essentially making explicit the 'tests' by which we judge whether something is (going to be) a good design or not: whether it's likely to be good Permaculture. And our design principles, ethics, basic design techniques & what our definitions of sustainability are all included in our basic design criteria. To those you will add your personal criteria: eg. has to be within x budget, within y time-frame, whatever is locally relevant & whatever personal preferences you bring to the mix, as the designer.
Criteria always exist whenever we set out to design something. The problem is (as always) when those criteria are unconscious, in conflict with each other and/or not sustainability-promoting - as is too often the case.
So making our criteria very clear & explicit is important & in fact totally key in evaluating your design.
Criteria (singular: Criterion), although as a concept very similar to Indicators, is much less enthusiastically covered by Wikipedia, and is defined as:
"A standard of judging; any approved or established rule or test, by which facts, principles opinions, and conduct are tried in forming a correct judgment respecting them."
There are lots of different types of indicators (see wikipedia's exhaustive list), & you will probably come across a lot of papers, articles, even books about Indicators, including now much discussion about how all our problems will be solved if we only change the types of indicators we measure & guide ourselves by.
As an example Environmental Indicators are described thus:
Environmental indicators are simple measures that tell us what is happening in the environment. Since the environment is very complex, indicators provide a more practical and economical way to track the state of the environment than if we attempted to record every possible variable in the environment. For example, concentrations of ozone depleting substances (ODS) in the atmosphere, tracked over time, is a good indicator with respect to the environmental issue of stratospheric ozone depletion ..
This can be useful for observation (&, arguably, very useful if observation is all you're interested in doing) but the main thing to note is that an indicator is, essentially, something you can measure only after some intervention is made (or not made).
It shows whatever the viewer considers most important & measurable about the current state of things, at any particular time.
What is the difference between Design Criteria & Indicators?
Not everything
that counts
can be measured.
Not everything
that can be measured
counts.
A. Einstein
Do make your own mind up about whether these many & varied Indicator efforts (many are 'alternative') are more likely to speed up or slow down progress in a permaculture direction, if used:
Full cost accounting (FCA) (with relevance to the environment)
Green gross domestic product (Green GDP)
Gross domestic product (GDP)
Happy Planet Index (HPI)
Human Development Index (HDI)
ISEW (Index of sustainable economic welfare)
In the Designers Manual the argument against getting too fixed on indicators (or measurables) is framed as an argument for the impossibility of using the scientific method in the biological sciences:
Rigorous scientific method deals with the necessity of rigorous control of variables, and in a life system (or indeed any system), this presumes two things that are impossible:
1) That you know all variables (in order to control some of them & measure others) before you start; or
2) That you can in fact control all of indeed any variables without creating disorder in the life system.
P. 11 of Pemaculture, A Designers' Manual, Bill Mollison
And goes on to give an excellent chapter on how we can know & design things.
(We have pointed out in the page on the Scientific Model that this understanding of "rigorous scientific method" is an old & inaccurate view of science, which was/is quite rightly criticized for only focusing on the external quadrants (when this is not even scientific, by definition). A great contribution of the Integral model has been to re-frame that issue in a more useful way).
However, having said our permaculture indicator is only whether we fulfill or not the design criteria, that one general indicator alone can cover a huge range of variables, which will change according to the situation (& may even be called "Indicators" - especially if there is a need to communicate in the language of bureaucrats & academics).
For eg. if we're interested in bio-remediation of contaminated land we might want to know exactly how many parts per million of contaminant it contains .. and indeed, precise figures can be interesting to have, if nothing else because it is satisfying to compare that to post-remediation or plant uptake measures - & you might have to prove a point to someone especially if there is public opinion or funding involved, etc.
But we already know that we would lots of other figures for that one to make any practical sense (eg. soil ph seems to govern how much of some contaminants are actually taken up by plants, then we know different plants take up different amounts... how much don't we yet know, even of such a complex system as soil?)
For a designer, in most (& "most" means "not all": flexibility is always key) cases the actual figures will change nothing or very little in the actual practical solution that we end up applying (and that can be the most scientific thing to do to!). Because in any case the landscape will speak volumes to a trained eye (or a common-sense approach), & the amount of organic matter, varied flora & / or fungus, water, time, etc. we will apply as permaculture designers to such a problem will follow more 'common sense' or 'fuzzy logic' rules - eg. quantities will be measured in "lots" or "as much & varied as you can manage", "whatever the soil can take / wants at the time", "whatever makes sense", etc., always observing as we interact with the dynamic system we're intervening in.
Taking Local Agenda 21 as an example:
huge amounts of resources (money, time, passion, hope, people, human creativity.. ) have gotten sunk into elaborating more and more 'sustainability' indicators, then figuring out ways of 'measuring them' and then measuring & writing lengthily about their analysis ... so they could end up in voluminous reports which are now gathering dust in forgotten basements ... & told the local people nothing they didn't already know about their locality.
Indeed, if a fraction of those wasted resources had been given to the local people (as, ironically, the very first indicator in the table below completely fails to address: although it tries to in spirit!), most areas would very probably be much advanced in terms of sustainability.
Because essentially... eg.
Who cares exactly how much rubbish a neighbourhood is producing per person, if we are clear that, whatever the figure, it just needs to be lowered?
And that, essentially, is the difference between using Indicators & Criteria:
Indicators all too often (not always) are used by bureaucrats & academics to spend massive resources in only describing the problem - with millimetric accuracy
whilst using permaculture design Criteria not only get straight into solving the problem but provide the creativity & essential flexibility to 'jump across categories' & in fact solve many problems at once (eg. because the waste will be seen as a resource usable by some other 'problem').
So enabling systemic thinking where the indicators-type approach constrains us to linear thinking, more often than not.
(article by Stella)
As a reflection exercise ... analise the "Sustainability Indicators" below, in the light of what you know so far of permaculture design criteria & the models we explored in Module 1.
Ideas, impressions, revelations... etc. >> in your design book!
( Local Agenda 21 )
6. Sustainability Indicators
The following indicators cover economic, social as well as environmental trends and will help us, over a period of time, to monitor whether or not we are moving towards sustainability. It is intended that baseline and historical data will be provided (subject to availability) at the time of the first annual review. There will also be a further opportunity, at that time, to review the goals and indicators chosen.