Intraraters scoring and the average scores of the interraters for the various aspects.
In creative disciplines, “basic design” is offered as a foundation course to foster diverse thinking skills and creativity. The tasks are generally framed based on the principles such as “progressive transformation,” “borrowing,” and “deconstruction.” The emergent outcomes of such tasks are unique and very challenging to evaluate. In this context, this chapter aims to discuss a framework for assessing the creativity manifested in the emergent outcomes of generative tasks based on a puzzle. Three tasks based on “TANGRAM,” a dissection puzzle with slight variations, were formulated. The task was introduced as a practicum at a faculty development program conducted at the AMS School of Architecture in association with the Council of Architecture, India. Besides, the framed tasks were introduced as an assignment for a theory course and also as a basic design task at the Department of architecture, Sathyabama Institute of Science and Technology, India. The emergent outcomes are explored, decoded, and analyzed. The findings are triangulated and a framework is developed that can be suitably modified so as to investigate the degrees of creativity manifested in the emergent outcomes of an open-ended task.
- puzzle-based open-ended task
Design education is a process that promotes multiple solutions and diverse points of views where unique interpretation and expressions are encouraged  to provide different experiences . Basic design is introduced as a foundation course in various creative and design courses such as “architecture,” “interior design,” “visual communication,” “product design,” etc. It is stated that introducing problems with a wide gate of imagination unleashes the hidden abilities, encourages the emergence of original and creative ideas . One of the objectives of basic design course is to motivate the students to start questioning and exploring by stimulating sensibility and creative process . Broadly, the tasks are classified as experimental and conceptual approaches . Literature studies reveal that the assessment of the emergent outcomes is subjective. It is against this background, this chapter explores the emergent outcomes of an open-ended task addressing “puzzle-based learning.” The next section gives an insight into “puzzle-based learning” and the tasks framed based on a dissection puzzle.
2. Puzzle-based learning
Puzzle-based learning is a pedagogical experiment with the primary goal to foster domain independent reasoning and critical thinking skills , critical thinking and problem-solving skills . Puzzle-based learning encourages to reflect on “What are we learning?” “How are we learning?” and “How we are using what we have learned?” .
Puzzles are educational, instigating useful and powerful problem-solving rules  and fostering problem-based learning . Puzzles are effectively redrafted appropriately in “science, technology, engineering, and mathematics” context ; engineering and computer science [12, 13]; human anatomy and physiology lab ; and architecture domain . With a focus on architectural education, the next section discusses the formulation of a “puzzle-based open-ended task” revolving around “TANGRAM,” a dissection puzzle.
2.1 An insight into “Tan-A-Morph”
“Tangram” is described as the most ingenious and imaginative puzzle, formed by dissecting a square in to seven or five pieces termed as “tans.” Its uniqueness lies in composing the tans as charming, elegant, sophisticated, and sometimes, paradoxical two-dimensional figures . The seven pieces or “tans” are comprised of 16 unit triangles, and the relative edges of all edges are powers of √2 . The pieces are dissected at the geometrical angles of 45 and 90° only .
“Dissection” or “put together puzzle” was identified to frame an open-ended task for the participants at the faculty development program; to foster thinking skills as part of a theory course, “Theories of thinking,” offered in the fifth semester; and as a basic design task introduced to the students pursuing first semester architecture. Among the various types of two-dimensional dissection puzzles, “Tangram” was identified to be appropriate for framing a unique task integrating “design” and “arts,” “composition with planes,” and “forced perspective.”
The framed task was about exploring “forced perspective,” the manipulation of human visual perception through illusions on an area of 0.11 square meters. The silhouette for painting was created by the respective groups by sticking the planes on the given base at right angles to both horizontal and vertical planes. The students created planar composition with all the seven pieces of the dissection puzzle which comprised of five similar triangles, a square, and a rhomboid. For the teachers, three similar triangles, a square, and a rhomboid, a variant of the “Tangram” comprising of five “tans” as shown in Figure 1 were provided. The students worked with the seven pieces of the dissection puzzle as a basic design task.
3. Assessment of creativity
Evaluating the degree of creativity in the emergent outcomes of open-ended or generative task is challenging. Assessment in studio is also widely debated by many . The assessment involve parameters such as identification of goals and purposes, selection of procedures, methods, procedures and measures, time management, analysis of data, interpretation of results, and formulation of responses to the results . Assessment of creativity need to be both “product- and process-oriented” . With respect to art, architecture, and design, the evaluation revolves around the “product, process, hard and soft skills” .
Dorst and Cross developed a method to identify the various factors that played a significant role in analyzing the aspects associated with degree of creativity in industrial design by adopting Pearson’s correlation coefficient . Even though literature studies reveal that quantitative techniques have potentials to explore the emergent outcomes, the authors posit that collection, analysis and integration of both quantitative and qualitative data yield rich findings. It is against this background, mixed methods research design is effective to explore, analyze and decode to construct rich knowledge about the emergent outcomes of generative tasks [24, 25].
3.1 Mixed methods approach
Mixed methods research is defined as “the class of research where the researcher mixes or combines quantitative and qualitative research techniques, methods, approaches or language in to a single study” . A mixed methods approach focuses on the pragmatic grounds addressing the collection of both qualitative and quantitative data . The choice of the design depends on research question, purpose, and context  as well as the research domain .
The reasons for mixing methods include the need to construct different and multiple perspectives or more complete understandings, need to confirm quantitative measures with qualitative experiences and need to explain the qualitative measures . Based on the framed research question, the level of integration is observed at three levels such as “design, methods, interpretation, and reporting” for “confirmation, expansion, and discordance” .
Triangulation, embedded, exploratory, and explanatory are the various types of designs either planned sequentially or concurrently . Integration of data requires a clear rationale and is always a matter if innovation . Mixed methods approach in academics research produces richer insights in to the phenomenon being studied, enhance the body of knowledge to arrive at robust conclusion and probe new questions for future studies . For assessing the degree of creativity manifested in the emergent outcomes, multi-method triangulation design has been adopted.
The term triangulation refers to the practice of using multiple sources of data or multiple approaches to analyze data for enhancing the credibility of a research study. It gives a holistic understanding of specific topics  and enhances the internal validity in qualitative studies on complex studies . Four types of triangulation such as method triangulation, investigator triangulation, theory triangulation, and data source triangulation are identified [37, 38].
Among the four types, this chapter focuses on methodological or multi-method triangulation. Methodological or multi-method triangulation entails the gathering of information or data addressing a phenomenon through more than one method, primarily to determine the convergence . As discussed earlier, qualitative and quantitative data were collected to investigate the degree of creativity embellished in the emergent outcomes of the framed puzzle-based open-ended tasks.
4. The emergent outcomes
The emergent outcomes of the framed task “Tan-a-morph” as discussed in Section 2.1 are as shown in Figure 2 . The first five outcomes were done by the participants at the Faculty Development program conducted in November 2016 at the AMS School of Architecture in association with the Council of Architecture and National Institute of Advanced Studies in Architecture. The last five outcomes were completed by the students of Architecture at the Sathyabama Institute of Science and Technology with seven “tans.” The sixth and seventh were done as an assignment for the theory course, “Theories of thinking,” by the fifth semester students offered from June to November 2015. The last three were the outcomes of a basic design studio conducted from August to December 2016. With an intention to assess the emergent outcomes of the framed task “Tan-a-morph,” this chapter discusses the issues related to the assessment of creativity.
With respect to the investigation of creativity manifested among the emergent outcomes of an open-ended task, the next section focuses on the aspects that need to be probed. Further, the various ways to analysis the outcomes quantitatively are highlighted. However, the authors identified the mixed methods design as a potential tool to assess the degree of manifested creativity.
4.1 Data collection
The emergent outcomes as shown in Figure 2 were selected for investigating the degrees of creativity. The outcomes were analyzed qualitatively and quantitatively. For the qualitative analysis, the authors investigated the emergent outcomes based on the rotation about the imaginary vertical and horizontal axis positioned at any one of the vertex as well as along one edge of the base respectively. The various aspects identified by the authors are listed in Table 1 .
|The emergent outcomes||Overall impression score on five point-scale||Rotation||Composition of planes|
|Vertical axis||Horizontal axis (o−1800)||Parallel planes||Skew planes||Edge to edge||Edge to face||Vertex and planes||Vertex and edge|
|00 to 3600||00 to 1800||00 to 900||Overlap||Non-overlap|
|Bird with skew planes||2.5||4.3||4.7||3.0||3.0||3.0||2.7||4.0||3.0||2.7||3.3||2.7|
|Pyramids and mountains||3.9||4.7||4.3||3.7||3.7||3.7||3.7||3.3||2.7||4.2||3.3||3.0|
With respect to the quantitative data, three architects with minimum 12 years of experiences were identified as the skilled assessors. The skilled assessors evaluated the “forced perspective” on a seven-point scale. Besides, 10 images of the outcomes were shown to the assessors for 15–20 s for rating. The images were shown again for 5 s so that the intraraters were given a second chance to reconsider the scoring.
Secondly, three architects with a minimum 5 years of experience were identified through convenience sampling. A structured questionnaire as in Appendix A with five-point Likert scale for rating the identified aspects was framed. The numbers 1–5 were associated with “strongly disagree,” “disagree,” “neutral,” “agree,” and “strongly agree” on various aspects based on the “views” and “composition of planes.” The images were shown to the interraters till the three interraters completed the rating.
4.2 Data analysis
The authors decoded the various aspects based on “rotation” and the “composition of the planes.” The average scores of the interraters for the identified aspects were calculated as shown in Table 1 . Pearson’s correlation coefficients were determined between the overall impression score and the average scores of the interraters.
The emergent outcomes that scored more than six points on the seven-point scale were narrowed down for further analysis. Such outcomes included “nature,” “abstract wolf,” “kaleidoscope,” “scorpion,” and “dog.” However, the frequency of average scores more than four points by the interraters including the intraraters were only considered to interpret the degree of creativity in incorporating the “forced perspective.”
The Pearson’s correlation coefficients between the overall impression scores by the intraraters and the interraters were determined as shown in Table 2 . The aspects such as “0–3600” and “0–1800” along the vertical axis display strong and very strong relationship, whereas “0–900” along both the vertical and horizontal axes, including the “overlapping and non-overlapping of parallel planes,” shows only moderate relationship. The relationship between the overall impression score and the “skew planes” was observed to be the weakest. With respect to the composition of the planes, “edge and edge” has shown a moderate relationship, whereas “edge to face,” “vertex and plane,” and “vertex and edge” display strong relationship.
|Aspects||Pearson’s correlation coefficient||Degree of relationship|
|Views||0–3600||Vertical axis||0.80||Very strong|
|Composition of planes||Parallel planes||Overlap||0.55|
|Skew planes||0.08||Very weak|
|Edge and edge||0.56||Moderate|
|Edge and face||0.60||Strong|
|Vertex and plane||0.72|
|Vertex and edge||0.78|
As discussed in the previous section, the five outcomes appreciated by the intraraters were decoded as in Table 3 . Among the five outcomes, one was done with five “tans,” whereas the rest were done with seven “tans.” Around two fifths were observed to be “metaphors” and the remaining three fifths to adopt the principles of “symbolism.” Even though the outcomes were noted to be manifested with diverse views, the visual texture was observed to be on both the sides of the vertical planes as well as on the visible side of the horizontal planes either wholly or partly. It was recognized that the ideas with respect to theme adopted for exploring “forced perspective” as well as the “composition of planes” to be evolved simultaneously.
|The outcomes||Symbolism/Metaphor||Theme/Visual texture||Synchronous approach|
|Nature||Symbolism||On all the sides of the vertical planes and on the horizontal plane||Ideas for the composition of the planes and the theme adopted for painting the planes evolved simultaneously|
|Abstract wolf||Metaphor||On one side of the vertical planes and partly on the horizontal plane|
|Kaleidoscope||On all the sides of the vertical planes and on the horizontal plane|
|Dog||On the horizontal plane and partly on one side of the vertical planes|
However, among the five outcomes that were decoded as shown in Table 3 , it was recognized that the frequency of scores more than four points were observed to be the maximum for the “abstract wolf” and minimum for the “kaleidoscope.” The emergent outcomes such as “nature,” “scorpion,” and “dog” were recognized to fall under the maximum and the minimum frequencies as shown in Table 4 .
|Frequency of scores >4||The emergent outcomes||Views||Forced perspective|
|Five||Kaleidoscope||Vertical axis||0–3600||Horizontal axis||0–1800 (along both the sides)||Rich|
|Dog||0–1800||0–900 (along the shorter side)|
It was observed that the forced perspective has been manifested richly in the emergent outcome “kaleidoscope.” “Abstract wolf” was interpreted to be the richest even though the diverse views were restricted to the rotation along the vertical axis from “0 to 180o” only. With respect to the rotation along the horizontal axis was limited to the longer axis of the horizontal plane with a rotating angle of “900.”
The process adopted to explore the emergent outcomes of a “puzzle-based open-ended task” portrays the mixed method research to be rational. The entire process is mapped in Figure 3 . This process entails the collection of both qualitative and quantitative data from three perspectives. The authors decoded the emergent outcomes qualitatively to identify the appropriate aspects. The intraraters assessed the overall impression score. Besides, the interraters assessed the identified aspects individually for each of the outcome.
The qualitative data gathered from the three interraters were converted to quantitative data. The correlations between the overall impression scores and the identified aspects were determined. The relationships were analyzed to examine the appropriateness of the identified aspects by the authors. Frequencies of scores for other aspects greater or equal to the value “four,” including the overall impression score, facilitated the identification of outcomes with unique ideas exploring “forced perspectives.” Further, the qualitative analysis of the five shortlisted outcomes facilitated the coding of the salient features, giving an insight into degree of content as “rich,” “richer,” and “richest.” From the study, it is observed that the “triangulation model” adopting the “mixed methods approach” is effective in exploring the unique characteristics of the emergent outcomes in identifying the best ones too.
As a concluding remark, the authors reinstate that creativity plays significant roles in identifying the appropriate aspects that need to be considered during the assessment phase. Numerous other quantitative techniques shall be adopted to explore and investigate the emergent outcomes of an open-ended or generative task. Besides, confirmatory factor analysis and structural equation model’s latent variables have been adopted to analyze the emergent outcomes of a basic design studio in depth . Directions to integrate such analysis shall be incorporated in the mixed method analysis to construct a thick description about the unique outcomes.
|Contact no.||E-mail id|
|One (tick your opinion)||Strongly disagree||Disagree||Neutral||Agree||Strongly agree|
|Relationship between planes and vertices||Strongly disagree||Disagree||Neutral||Agree||Strongly agree|
|Overlapping of parallel planes|
|Non-overlapping of parallel planes|
|Edge to edge|
|Edge and face|
|Vertex and plane|
|Vertex and edge|
|Signature with date:|