Archnet > Publication > Bringing Structure to Judging Success in Architectural Design: The TIOSE Qualitative Measure

Explicit measures of design quality are almost unknown in architectural design, particularly since judging quality in architecture, as in most design disciplines in general, resists quantification. Resistance in the judgement of design quality can be traced to requirements for evaluating complex intra- and inter discipline factor integration, as well as cultural knowledge which is tacit in nature. This paper aims to introduce the TIOSE Qualitative Measure, an outcome evaluation tool developed for a study on team interaction, cognitive style, team processes, and quality in design. The TIOSE contains five factors for judging architectural design success: thoroughness, informativeness, organisation, synthesis, and evocativeness. The methodology for the study involved evaluating the intra- and inter-rater reliability of six architectural critics who utilized the TIOSE to judge team project quality and success among 84 graduate students working in 12 teams. Analysis found high intraclass correlations within and between critics across three rounds of ratings, suggesting the TIOSE showed excellent intra- and inter-rater reliability, respectively. Study results provide evidence that evaluating the innovation and creativity of architecture and design projects can reliably extend past ratings of efficiency (time) and productivity (volume) as objective performance measures as the testing mechanism for success. In conclusion, this study demonstrates that qualitative measures of project success can be quantified in a reliable manner and can attain internalized agreement between architectural critics. The TIOSE measure provides a rating scale with explicit definitions for the qualitative measure of architectural design quality.

Type

journal article

Year

2012

Explicit measures of design quality are almost unknown in architectural design, particularly since judging quality in architecture, as in most design disciplines in general, resists quantification. Resistance in the judgement of design quality can be traced to requirements for evaluating complex intra- and inter discipline factor integration, as well as cultural knowledge which is tacit in nature. This paper aims to introduce the TIOSE Qualitative Measure, an outcome evaluation tool developed for a study on team interaction, cognitive style, team processes, and quality in design. The TIOSE contains five factors for judging architectural design success: thoroughness, informativeness, organisation, synthesis, and evocativeness. The methodology for the study involved evaluating the intra- and inter-rater reliability of six architectural critics who utilized the TIOSE to judge team project quality and success among 84 graduate students working in 12 teams. Analysis found high intraclass correlations within and between critics across three rounds of ratings, suggesting the TIOSE showed excellent intra- and inter-rater reliability, respectively. Study results provide evidence that evaluating the innovation and creativity of architecture and design projects can reliably extend past ratings of efficiency (time) and productivity (volume) as objective performance measures as the testing mechanism for success. In conclusion, this study demonstrates that qualitative measures of project success can be quantified in a reliable manner and can attain internalized agreement between architectural critics. The TIOSE measure provides a rating scale with explicit definitions for the qualitative measure of architectural design quality.

Plowright, Philip D. and Cole, Matthew L. "Bringing Structure to Judging Success in Architectural Design: The TIOSE Qualitative Measure," in ArchNet-IJAR: International Journal of Architectural Research, vol. 6, issue 3 (2012): 7-19.