UT Tyler CE program outcomes
According to the International Monetary Fund (IMF Staff, 2008), globalization “refers to the increasing integration of economies around the world, particularly through the movement of goods, services, and capital across borders.” This integration can be positive or negative depending on one’s point of view. The current trade deficits and job losses for the United States as companies outsource their work to countries where the labor costs are extremely low can be seen as negative (e.g., The North American Free Trade Agreement or NAFTA), while the economic development of these poorer nations can be seen as positive. Friedman’s (2005) Book, The World is Flat, highlights how technology has allowed many countries to include China and India to become part of the global supply chain for manufactured goods as well as services such as engineering. Of course, globalization not only affects economics, but there are also broader political, environmental, and cultural dimensions which have resulted in globalization becoming a major focus for many companies and universities as well as nations.
What are the skills needed to compete in the global economy? How does one gain these skills? The European Union is currently studying how to not only improve the workforce skills through “New Skills for New Jobs” (European Commission, 2011), but to ensure they have a unified employment strategy that builds upon successful cooperation between higher education and businesses to modernize educational preparation of new graduates with skills for the global markets of the future. Many schools point toward their International Studies programs as the gateway to a “broad, interdisciplinary program with a flexible curriculum that emphasizes the development of knowledge and analytical approaches needed to understand the contemporary global system.” (UW-Parkside, 2011) Others require a semester abroad or the completion of a team project abroad. Some schools require the faculty to travel abroad in hope they will pass along their global perspectives to their students. One Dean has stated that “international study shouldn’t be elective.” (Goinglobal, 2011)
Career counselors highlight the need to be not only strong in one’s discipline expertise using cognitive skills such as problem-solving, decision making, and knowing how to learn, but also through the use of social skills such as working in multi-cultural teams and personal skills such as being flexible and innovative. (Peters-Behrens, 2011) Work experience during summer internships as well as semester abroad experiences provides real-world opportunities for applying these skills. Even the prime minister of the United Kingdom called for expanded apprenticeships to aid in Britain’s “global skills race” to remain internationally competitive. (Brown, 2008)
These same questions have been the source of discussions among US engineering firms and societies for many years leading to new ABET EC2000 (adopted in 1997) criteria supported by the engineering societies. (ABET, 2011) A quick look at these criteria depicts the importance of professional and global skills by their inclusion within ABET Criterion 3 a-k. The generally recognized ABET professional skills are: ability to function on a multi-disciplinary team (3.d), understanding professional and ethical responsibility (3.f), ability to communicate effectively (3.g), broad curriculum (3.h), recognition of need by an ability to engage in life-long learning (3.i), and knowledge of contemporary issues (3.j). Investigation of program criteria provides insight into additional professional skills being required within the Civil Engineering program criteria – leadership, business practices, public policy and administration, and asset management. All of these professional and global skills point to the growing importance of the management of engineering services over a focus solely on number crunching work. Are these not some of the same skills desperately needed within engineering companies as they move toward international markets?
If these professional skills are possibly a globalization equalizer and initial understanding and skill development occurs within university curriculum, then academic programs must provide the correct emphasis of these professional skills within their programs. (Steering Committee, 2006; Downey et.al., 2006) Based on the author’s experience teaching senior design courses and as an ABET program evaluator (PEV), students applying professional skills for the first time in the curriculum during the senior design will not attain the level of performance desired in the professional skills (ABET Outcome 3d, f, g, i, j). These topics are sometimes relegated to a senior seminar during the last semester before graduation. If they are important enough to be elevated to outcomes, and some feel they are difficult to master and assess, then why are they sometimes addressed and demonstrated only once in the curriculum? PEV experience highlights that programs tend to focus only on what they understand.
1.1. Professional outcomes
What is the current emphasis of professional outcomes within curriculums? The ABET criteria are based on input from the numerous professional societies that are continuously evolving the required skills of future employees. The Engineering Accreditation Commission (EAC) works with the professional engineering societies to establish the skills required for future engineers. After many years of discussion and negotiation, EC2000 Criterion 3 a-k were adopted in 1997. The ABET EC2000 professional skills are: ability to function on a multi-disciplinary team (3.d), understanding professional and ethical responsibility (3.f), ability to communicate effectively (3.g), broad curriculum (3.h), recognition of need by an ability to engage in life-long learning (3.i), and knowledge of contemporary issues (3.j). Even though the EC2000 criteria and the professional skills have been part of the assessment process for two assessment cycles for many programs, programs are still experimenting with how best to assess and demonstrate accomplishment of these professional skills. The difficulty arises in how students demonstrate or fully understand what these outcomes are asking when the demonstration is occasionally left to a single event such as a seminar within the senior year.
These professional outcomes have been referred to as “soft” outcomes by many when they were first presented as part of ABET EC2000 Criterion 3. (ABET, 2011; Shuman et. al., 2005) Others have used the word “squishy” when considering how difficult it is to assess professional skills compared to the “hard” skills of engineering. (Welch et. al, 2005) “Soft” or “Squishy”, the professional skills are not as easily assessed and many times require multiple assessment methods, multiple activities within the academic setting as well as possible activities like Engineers Without Borders (EWB) or other service activities to be able to properly demonstrate accomplishment of the outcome. (Shuman et. al., 2005) However, not every student will participate in EWB or other service activities unless a program requires participation as part of their graduation requirements. Another example of the difficulty is that students may properly assess a situation based on proper ethical reasoning, but there is no assurance that they will actually act ethically. Some define professional skills as how we perform in professional settings, but how do educators develop and assess such skills? Does the educator possess professional experience and a professional engineer (PE) license? Even a PE needs a methodology understood by all to properly and continually assess professional practice skills.
Even though EC2000 was newly adopted and programs were investigating how best to demonstrate accomplishment of all new criteria, ASCE recognized the lack of certain knowledge and skills among civil engineering graduates. ASCE took the lead in education reform and formed a committee to study and develop a Civil Engineering Body of Knowledge (BOK) (ASCE 2004) to document the requisite knowledge, skills, and attitudes necessary for future civil engineers. This new civil engineering BOK included the ABET EC2000 3 a-k outcomes. Two key factors associated with the BOK were: 1) a lack of a clear definition of expected performance levels by these new engineers even though the wording was consistent with ABET definitions which are generally difficult to define clearly, and 2) the addition of four new outcomes focused on additional professional topics and discipline depth.
Very quickly it was determined by most programs and ASCE that the discipline depth could only occur at the Master’s level with larger breadth occurring at the undergraduate level. The addition of professional skills above what even ABET EC2000 requires reflects greater recognition of the importance of the development of professional skills at the bachelor’s level. How/Where are these supplementary professional topics to be included in current curriculums? How are these additional topics to be covered when some programs are required to decrease credit hours for faster graduation rates without decreasing the required Core courses? The engineering programs within the University of Texas System decreased their credit hours to 128 or less in 2007 based on legislative requirements to decrease time to graduation. Currently, there are discussions to possibly decrease the number of credit hours to 120 credit hours for all programs within publically supported programs to decrease the amount of funding being sent to schools of higher education as part of state-wide cost cutting measures. Many universities are facing similar conditions while needing to meet global skills of future engineers.
ASCE also realized that adjustments to the first edition of the BOK (ASCE, 2004) were needed based on the comments that arose from the ASCE Curriculum Committee (ASCE, 2007) investigating how to implement the outcomes. The central issue was the lack of a methodology to determine how well and at what level a program needed to demonstrate accomplishment of outcomes. Some of the schools on the ASCE Curriculum Committee working towards implementation of the outcomes decided after reviewing numerous taxonomies that Bloom’s Taxonomy (Bloom et. al., 1956) should be used to define the proper level of demonstration of outcomes. One of the first orders of business of the new BOK II committee (ASCE, 2008) was to apply Bloom’s Taxonomy (read action verbs) to define the level of demonstration of each outcome at not only the undergraduate level, but master’s and experience levels as well. This level of definition of demonstration was applied to ABET Outcomes 3 a-k as well. Another focus as a result of the earlier ASCE Curriculum Committee’s work was the splitting of ABET outcomes into a larger number of smaller outcomes. For example, ABET Criteria 3.a is “an ability to apply knowledge of mathematics, science, and engineering,” (ABET, 2011) which became in BOK II outcomes 1 Mathematics, 2 Natural Sciences, 5 Materials Science, and 6 Mechanics. (ASCE, 2008)
|1. Can apply knowledge of traditional mathematics, science, and engineering skills, and use modern engineering tools to solve problems.|
|2. Can design and conduct experiments, as well as analyze and interpret data in more than one civil engineering sub-discipline.|
|3. Can design systems, components, and processes and recognize the strengths and areas for possible improvement of their creative designs within realistic constraints such as regulatory, economic, environmental, social, political, ethical, health and safety, constructability, and sustainability.|
|4. Can work independently as well as part of a multidisciplinary design team.|
|5. Can identify, formulate, solve and evaluate engineering design problems using engineering models in the four of the six sub-disciplines civil engineering: structural engineering, transportation engineering, construction management, geotechnical engineering, hydrology and/or environmental engineering.|
|6. Can analyze a situation and make appropriate professional and ethical decisions.|
|7. Can demonstrate effective oral, written, and graphical communication skills.|
|8. Can demonstrate a commitment to learning and continued professional development outside the classroom, incorporate contemporary issues and historical issues during problem solving, and determine the impact of engineering solutions in a global and societal context.|
|9. Can explain professional practice attitudes, leadership principles and attitudes, management concepts and processes, and concepts of business, public policy, and public administration.|
|10. Can demonstrate the importance of humanities in the professional practice of civil engineering.|
|11. Can demonstrate the incorporation of social sciences knowledge into the professional practice of civil engineering.|
|12. Can use the knowledge of material sciences to solve problems appropriate to civil engineering.|
|13. Can analyze and solve problems in solid and fluid mechanics|
|14. Can apply principles of sustainability to the design of traditional and emergent engineering systems.|
|15. Can apply the principles of probability and statistics to solve problems containing uncertainties and risk assessment.|
The previous discussion surrounding the addition of new outcomes to BOK I, when reviewing skills needed by graduating engineers, resulted in the addition of outcomes to BOK II when necessary entry level skills were readdressed. The new outcomes are: sustainability, historical perspectives, globalization, humanities, social sciences, risk and uncertainty, and breadth in civil engineering areas. Some of these new outcomes may be considered professional practice skills such as humanities, social sciences, historical perspectives, sustainability and, of course, globalization. The ASCE Fulfillment Committee (Fridley, 2010) is currently studying implementation techniques for the new outcomes as well as implementation of adjusted levels of outcome demonstration at the undergraduate level using Bloom’s Taxonomy. It is only a matter of time before these new outcomes and levels of demonstration occur within ABET general program criteria as is being called for by some (Ressler, 2010) or CE program criteria as the new outcomes in BOK I did in 2008 just four years after BOK I was in print. Some even hint that changes every six years are reasonable to consider. (Ressler, 2010) This current BOK II list of professional skills is a mixture of the skills gained through many international studies, management, business, and engineering programs.
The current CE program criteria professional outcomes are leadership, public policy and administration, business practices and asset management (adopted in 2008). What are the best methodologies and location for demonstrating these outcomes? Can programs successfully demonstrate these professional outcomes at the Bloom’s Taxonomy levels indicated in the BOK II?
As a case study, this chapter will provide an in-depth look at the CE program at The University of Texas at Tyler (UT Tyler). The professional topics emanate from Program Outcomes (PO) 4, 6, 7, 8, 9, 10, 11, and 14 (Table 1 (Bold)) which are derived from ABET Outcomes (3.d), (3.f), (3.g), (3.h), (3.i), and (3.j) (Table 2 (Bold)) and the CE program criteria with additions and adjustments based on BOK II. PO 9 specifically incorporates the new undergraduate professional requirements located within the current CE program criteria (from BOK I). Outcomes 10-15 in Table 1 are the new program outcomes emanating from BOK II. Like a number of programs, UT Tyler has added the new outcomes and adjusted others to begin experimenting with how best and where to implement and demonstrate them. (Fridley, 2010)
|Demonstration (incl. Process & Measurements) that Graduates have:|
|(a) ability to apply knowledge of math, engineering, and science|
|(b) ability to design and conduct experiments|
|(b) ability to analyze and interpret data|
|(c) ability to design system, component or process to meet needs within realistic constraints such as regulatory, economic, environmental, social, political, ethical, health and safety, constructability, and sustainability.|
|(d) ability to function on multi-disciplinary teams|
|(e) ability to identify, formulate, and solve engineering problems|
|(f) understanding of professional and ethical responsibility|
|(g) ability to communicate effectively|
|(h) broad education|
|(i) recognition of need by an ability to engage in life-long learning|
|(j) knowledge of contemporary issues|
|(k) ability to use techniques, skills, and tools in engineering practice|
1.2. Alumni and employer assessment of professional skills
How important are current professional outcomes to recent graduates working within the civil engineering profession? The importance of professional skills to engineering is clear through the inclusion of professional outcomes within ABET EC2000 and the civil engineering profession through the inclusion of professional outcomes within the program criteria in 2008. An available data source for investigating the importance of professional skills is surveys from recent alumni of UT Tyler and their employers within East Texas. As expected, the return rates were much better for the alumni (82 percent) than the employers (33 percent). Using these results presented in Table 3 (professional outcomes are bold), it is clear that the alumni felt they had improved their professional skills through their one year of experience and that the demonstration of the professional skills as worded was highly appropriate. The employers also rated highly the alumni professional skills and felt that even in East Texas that the assessment of these professional skills as worded was highly appropriate.
|1a. Can apply knowledge of traditional mathematics to solve problems||4.8||4.8||4.9||4.5||4.5|
|1b. Can apply knowledge of traditional science (calculus-based physics, Chemistry, additional science) to solve problems||4.4||4.8||4.6||4.5||4.5|
|1c. Can apply knowledge of traditional engineering skills to solve problems||4.8||4.6||4.8||5.0||4.5|
|1d. Can use modern engineering tools (excel, MathCAD, etc.) to solve problems||4.7||4.5||4.8||4.5||4.5|
|2. Can design and conduct experiments, as well as analyze and interpret data in more than one civil engineering discipline||4.6||4.4||4.5||4.5||4.5|
|3a. Can design systems, components, and processes||4.5||4.5||4.1||4.0||4.0|
|3b. Can recognize the strengths and areas for possible improvement of their creative designs within realistic constraints such as economic, political, social, constructability, sustainability, public health and safety, environmental, and ethical.||4.5||4.6||4.8||4.5||5.0|
|4. Can work independently as well as part of a multidisciplinary design team||4.7||4.8||4.8||4.5||4.5|
|5a. Can identify, formulate, and solve engineering design problems using engineering models in the discipline of structural engineering||4.4||4.6||4.8||4.5||4.5|
|5b. Can identify, formulate, and solve engineering design problems using engineering models in the discipline of transportation engineering||4.2||4.1||4.4||4.5||4.5|
|5c. Can identify, formulate, and solve engineering design problems using engineering models in the discipline of construction management||4.3||4.6||4.6||4.5||4.5|
|5d. Can identify, formulate, and solve engineering design problems using engineering models in the discipline of hydrology and hydraulic design||4.2||4.4||4.8||4.0||4.5|
|5e. Can identify, formulate, and solve engineering design problems using engineering models in the discipline of environmental engineering||3.1||3.4||4.0||4.0||4.5|
|6a. Can analyze a situation and make appropriate professional decisions||4.6||4.6||4.6||5.0||5.0|
|6b. Can analyze a situation and make appropriate ethical decisions||4.4||4.6||4.8||5.0||5.0|
|7. Have effective oral, written, and graphical communication skills||4.6||4.6||4.9||4.5||4.5|
|8a. Demonstrate a commitment to learning and continued professional development outside the classroom||4.7||4.8||4.9||4.5||4.5|
|8b. Incorporate contemporary issues during problem solving||4.3||4.8||4.4||4.5||4.5|
|8c. Determine the impact of engineering solutions in a global and societal context||4.2||4.5||4.8||4.0||4.0|
|9a. Can explain professional practice attitudes||4.3||4.3||4.6||4.0||4.5|
|9b. Can explain leadership principles and attitudes||4.4||4.5||4.5||4.0||4.5|
|9c. Can explain management concepts and processes||4.4||4.6||4.6||4.5||4.5|
|9d. Can explain concepts of business practices||4.4||4.5||4.6||4.5||4.5|
|9e. Can explain public policy and public administration||4.7||4.8||4.6||4.5||4.5|
The employers did feel that the alumni did not have the same skill development as the other outcomes in demonstrating the impact of engineering solutions in a global and societal context, professional attitudes, and leadership principles and attitudes. The local focus of the firms and the limited leadership opportunities for new Engineering-in-Training (EIT) engineers would severely limit rapid development through on-the-job experiences. The results did highlight for the program that these outcomes may require more emphasis within the curriculum such that alumni may more aggressively take on leadership roles when opportunities arise. The employers also did not feel that demonstrating the impact of engineering solutions in a global and societal context was as appropriate, and from the East Texas focus of the firms the result is expected. As with any survey, the perception of those being surveyed must be considered.
The transition to a need to understand and demonstrate the impact of engineering solutions in a global and societal context within all engineering firms is somewhat similar to the transition by structural engineers from Allowable Stress Design (ASD) to Load Resistance and Factored Design (LRFD) for steel and reinforced concrete design a number of years ago. A major portion of the transition work was completed within universities through their preparation of the next generation of employees. (ASCE, 2004) The universities researched the methodologies, developed new techniques, and taught the new LRFD techniques that are now the standard in most structural firms. The firms were able to make the necessary senior level hires to manage new hires through a successful transition to the new structural analysis and design techniques. Could a similar process that provides some level of stability within a company be used during the development of required global skills in all employees?
A key point from the data in Table 3 is that the employers did not feel that an outcome such as demonstrating the impact of engineering solutions in a global and societal context was inappropriate at all, just not as important for their firm. On the other hand, the employers provided a maximum score not only on the alumni demonstration, but the appropriateness of demonstrating proper analysis of a situation and making appropriate professional and ethical decisions. A large step towards the desired global skill set. Many programs are still investigating, applying, and assessing how, where, and how much demonstration of each professional skill is required within engineering problem solving.
2. A possible process to improve professional skills
Can professional (global) skills be taught? Should ABET use Bloom’s taxonomy to elevate and demonstrate the professional outcomes? What techniques are best to assess professional outcomes? Every program is challenged with how best to teach and then assess the demonstration of each outcome, but especially professional outcomes. As a PEV, the data and assessment of professional outcomes were the hardest to interpret and evaluate for proper demonstration based on the wording used by ABET (ability, recognize, understand). The use of action verbs with each outcome would greatly decrease the uncertainty of whether a program has demonstrated accomplishment of an outcome.
The UT Tyler program used the following process to teach and then assess the professional outcomes. First, action verbs were associated with an appropriate Bloom’s level, and more recently with the levels proscribed within the BOK II (Table 1). The ABET professional outcomes (3.d) multi-disciplinary teams, (3.g) communicate effectively, (3.i) life-long learning, and (3.j) contemporary issues are being addressed in multiple courses and the two-semester senior design (capstone) courses (CENG 4115/4315). ABET outcome (3.h) broad education is addressed within the confines of the healthy Core (44 credit hours) program at UT Tyler, but are students making the connection between the humanities and social sciences that they are taking and how these disciplines influence engineering design? This type of questioning most likely led to the adjustments from BOK I to BOK II where humanities and social sciences were elevated to an outcome. The BOK II has led many programs to begin the process of demonstrating all newly proposed outcomes. ABET Outcome (3.f) professional and ethical responsibility is addressed in the senior design and an Introduction to Ethics Course (sophomore year), but could students miss fully understanding and demonstrating this outcome when completing an engineering design since all of the work within the senior design is completed in teams?
The UT Tyler Civil Engineering Program decided to add a required course to the curriculum to ensure coverage of the professional topics in the title as well as integration of the other professional skills. The course is CENG 4341 Leadership, Public Policy, Business Practices, and Asset Management (Figure 1, second semester, senior year) which provides coverage of the three new outcomes in the BOK I (represented by PO 9) as well as demonstration of parts of PO 6, 7, and 8 (Welch, 2010a; 2010b; CENG, 2009). Formal assignment of embedded indicators to courses based on ability to demonstrate an outcome as well as annual assessment of these embedded indicators collected not only provided BOK compliance, but also accreditation through ABET (Welch, 2009a). CENG 4341 synergistically assisted students in seeing the big design picture and the nuances of teamwork, leadership and management required in internships prior to the senior year, and within the two-semester senior design (CENG 4115/4315) by analyzing and discussing solutions to leadership issues seen during the one credit hour CENG 4115 in the fall and current public policy affecting engineering projects in Texas, business practices that could constrain their capstone project design as well as how well they have been managing their personnel assets within their team during CENG 4315 in the spring. Even though some public policy is being covered in environmental engineering, the in-depth coverage of public policy and administration within CENG 4341 along with leadership, business practices, asset management and other professional skills provides a synergistic meshing of these multiple professional practice issues as they occur within practice. The course also provides demonstration of ethical responsibility, contemporary issues, and life-long learning.
2.1. Leadership, public policy, business practices, asset management, CENG 4341
CENG 4341 is comprised of four modules each with papers and an exam leading to a comprehensive final exam. The original idea was to draw upon the expertise across campus and have the individual teaching a course covering the topic to develop a 10 lesson block with an exam (i.e., team teaching). When the issue of how to give credit for one-fourth of a course became a stumbling block, the author decided to develop each block himself. Upon reflection of the order of topics to develop a richer experience as well as provide linkage to additional professional topics, it was obvious that all of the professional topics focus on the engineer being a leader within their firm - so it is covered first. Public policy affects a firm’s business practices and it takes leaders within government for public policy to be enacted. So it was taught next followed by business practices. The key focus in small business practices (a large number of civil engineering firms are considered small) is the development of a business plan which is affected by public policy and the leaders within the firm. Once the business plan is in place, leaders manage firm assets to accomplish assigned tasks and missions to meet the desired level of business performance. Sometimes a single focused non-technical course is more synergistic for the learning process than inserting single or multiple lessons into existing technical courses.
2.2. Senior design, CENG 4115/4315
The senior design experience (CENG 4115/4315) within the UT Tyler CE Program (many programs refer to this as a ‘capstone’ course) is a 4 credit, 2 semester Senior Design course that has been conceived to do the following:
Give students a real-world, design office design experience tackling an open-ended design scenario that encompasses high level (Bloom et. al., 1956) cognitive thinking across seven CE sub-disciplines
Ensure students wrestle with professional issues such as communication with engineers, policy makers and the public
Ensure students are faced with issues associated with typical design constraints such as regulatory, economic, environmental, social, political, ethical, health and safety, constructability, and sustainability in a multi-disciplinary design.
Provide a platform where student performance against the ABET general criteria for engineering programs (3 a-k ) (ABET 2011) and civil engineering program specific criteria can be assessed (basically BOK I now and BOK II in the future).
The senior design experience was tailored to ensure coverage of the appropriate program material – items (1) – (3) in the above list suggests this. In some ways, constructing the appropriate assessment vehicle(s) was a more considerable challenge. The open-ended nature of realistic design does not always lend itself to concrete assessment methodologies. So the creation of a time effective assessment scheme that forces an experience that includes coverage of all outcomes to include professional skills was a focus.
The UT Tyler CE Program’s senior design experience is a two course sequence, with CENG 4115 a one credit fall offering that introduces the students to the year’s project through activities up to 35% design completion, and CENG 4315 a three credit spring offering that takes the project to 100% design completion. The experience centers on a multidisciplinary design project. Ideally, this project is aligned with an actual project being designed or constructed in the local area, so that at the end of the project students can review parallel plans and designs that have been professionally produced. The 2007-2008 project was a new UT Tyler art complex (completed in 2009), 2008-2009 project was renovation and expansion of the University Center (completed in 2010), 2009-2010 project was an outdoor multi-use amphitheater at UT Tyler, the 2010-2011 project is a realignment of Lazy Creek and Patriot Avenue at the south-west corner of campus, and the 2011-2012 project will be the development of a parcel of university property for an intramural complex and campus shops and eateries.
CENG 4115 begins with a review of the 9 step engineering design process (Woodson, 1966) and primarily revolves around targeted submittals at typical early project milestones – 10% and 35% submittals. A discussion of nine primary constraints to engineering design (sustainability, environmental, constructability, economics, ethics, political, social, technology, and public health and safety) follows, and these are emphasized throughout the experience such that the program requires consideration of all nine. In preparation for the 10% submittal, the course involves client meetings during which students gather needs, functionality requests and client driven constraints. A site orientation visit follows, and the class becomes fully enmeshed in the design project for the rest of the academic year. Each
senior design course (CENG 4115/4315) includes lessons, assignments and other activities supplemental to the design project. These are additional advanced design topics that prepare students to complete the design project (such as activities around wetland identification, ESA development, etc.) or that bridge sub-disciplines covered in other courses (such as parking lot material and roadway section design that bridges structural, geotechnical and transportation concerns). These advanced topics also close holes in the program identified through other assessment vehicles or the previous year’s senior design assessment. Other successful features of the experience are ‘fact finding missions’ (FFM) that task student design teams with finding information (e.g., local code requirements regarding architectural features, ADA building requirements, ESA interviews, traffic signaling requirements, etc.) on their own that has not been covered elsewhere in the curriculum. FFM simulate real-world design practice and give an opportunity for self directed learning (lifelong learning 3.i.). (Welch & McGinnis, 2010) CENG 4115 culminates with the submission of a 35% design package that is graded over the semester break.
The students are immediately enmeshed back in the project when they receive feedback on the 35% design package on the first day of CENG 4315 the next semester. They are required to present this design to faculty and clients within the two weeks, after making necessary changes and developing an architectural and/or site model, a K’Nex structural model if a bridge, building, etc. is required, and a stand-alone presentation board. CENG 4315 continues to 100% design completion with an intermediate review at 65% that is structured to simulate a desk-side review with an engineering supervisor in a design office. The 65% review gives students practice in identifying the critical parts of a project to relay to a supervisor – there will usually not be time to cover every detail in a design review meeting, and identifying what is important is a critical skill. At 100% design completion, students submit their design package and deliver a final oral presentation. At this time they field questions from working engineers (evaluation panel) regarding their design, receive feedback from clients and faculty, and review actual plans from existing designs if they exist and time permits. (Welch & McGinnis, 2010)
2.3. Demonstration of outcomes
A portion or the entire assignment in every course can be used as an embedded indicator for an outcome if the assignment is properly designed. In CENG 4341 the use of action verbs (Table 4) ensures that the assignment focuses the students work to demonstrate their knowledge, skills and attitudes at the given Bloom’s level desired by the program even prior to the levels being defined within BOK II. As a departmental team, embedded indicators are assigned to each course based on first the perceived ability of the course projects to demonstrate the outcome, and after annual assessments based on the need to spread the coverage of outcomes across the curriculum. (Welch, 2009b)
Grading of each major milestone in the senior design (10% design report, 35% design report, 35% oral presentation, 65% Review Reports, 100%, and 100% oral presentation) is done using ‘cut sheets’ – rubrics that incorporate the key features of the design. Figure 2 (professional skill - asset management, construction management schedule) shows a portion of the grading cut sheet from the 35% design submission as an example (modified from rubrics used at the United States Military Academy (Meyer & Bert, 2007)). The grading of the senior design is accomplished by the assigned instructor and the discipline expert in the department for each discipline section. Industry partners review the design using the rubrics for comment during the final presentation and submission.
|Paper 1 – define your leadership skills and attitudes.||9b - Leadership|
|Paper 3 – define your leadership strategies for ensuring your Capstone team is successful.||9b - Leadership|
|Paper 4 – Define public policy while considering T. Boone Pickens and West Texas Water rights||9e – Public Policy|
|Paper 5 – Define and provide the pros and cons for wetland public policy||9e – Public Policy|
|Paper 6 – define a business plan through the eyes of a junior engineer||9d – Business|
|Paper 7 – explain how you would improve the efficiency of assets used within the company you worked for this past summer||9c – Management|
|Exam 3, Question 1, define and explain business practices||9d – Business|
|Exam 3, Question 4, Define and explain asset management though lean six sigma and waste limitation||9c – Management|
3. Assessment activities and results
3.1. CENG 4341
The students feel that their understanding of professional topics improved due to in-class discussions defining these skills, how these skills were applied in their internships with in-depth analysis of their supervisors and companies, and then applying their understanding to their senior design teams using the newly acquired skills. Figure 3 displays the Fall 2007 (071S) and 2008 (081S) and Spring 2010 (092S) data of the students perception of how well they understood and demonstrated the new professional skills within the CE program criteria. Student comments from Fall 2007 pointed to the need to increase the number of papers of smaller length to allow greater specificity for each paper when studying the numerous topics within the course. They also asked for case studies and guest speakers. (CENG, 2009) The suggestions were applied during the next offering of the course with the resulting improvements (081S, Figure 3). Additionally, it must be noted that teaching the course the second time could have had some effect on the results. Student comments on the leadership section highlighted the limited ability to reflect on what was happening within the capstone teams (CENG 4115) with the leadership portion at the beginning of the fall semester, so CENG 4341 was moved to the spring semester of the senior year (Figure 1) and the second environmental engineering course was moved to the fall to improve environmental content coverage prior to the fall FE exam. Even though the course was moved to the spring and the instructor taught three courses during that semester, the assessment remained fairly consistent with the ability to apply leadership principles being rated the highest (Figure 3).
3.2. Senior design
As noted, the UT Tyler CE program has embraced the ‘embedded indicator’ approach to assessment (supplemented by several other methods of assessment such as capstone, external exams, internal gateway exams, and surveys). However, in a large, open-ended design project, a dilemma results if too many separate submissions are required. Too many assignments can cause the problem to be over-defined. A key part of open-ended design problems is that students must develop skills to properly define the scope and natural breaking points of the problem themselves. Assessment methods requiring subdividing the project into multiple predefined problem sets were thus eliminated. Furthermore, grading, managing and providing feedback to multiple student design teams as they pursue different solution strategies is already a time consuming endeavor for the course director. The program administrator strongly desired an assessment method that did not add an undue administrative burden on to the course director.
|9c||10%R Long Lead||0||5||0.0||10%R||8||0||14||57.1|
|10%R Cost Estimate||8||9||88.9||35%||5||/||5||100.0|
|35% Cost Estimate||5||5||100.0||65%|
|35% OP Prep||30.0||40.0||75.0||35% Oral||72.0||90.0||80.0|
|35% OP Method||42.0||50.0||84.0||100% Oral||47.5||55.0||86.4|
|100% OP Prep||21.3||25.0||85.0||Total||137.5||/||169||81.4|
|100% OP Method||26.3||30.0||87.5|
|10%R ESA1 not used in 2010||0.0||0||#DIV/0!||65%|
|10%R Life Safety||8.0||10||80.0||100%||34.2||0||38.2||89.5|
|10%R Constraint (Politics)||3.0||4||75.0||35% Oral|
|10%R ADA||5.0||5||100.0||100% Oral|
|35% Life Safety||4.0||5||80.0|
|35% Green Buildings||11.0||15||73.3|
|35% Constraint (Politics)||4.6||5.0||92.0|
|35% ESA 1||23.0||25.0||92.0|
|100% Life Safety||3.0||3||100.0|
|100% Env. Green Bldgs||11.0||15||73.3|
|100% Asbestos Not Used 2010||#DIV/0!|
|100% Constrain (Politics)||5.0||5.0||100.0|
|100% Social, Politica||15.0||15||100.0|
Fortunately the faculty were able to evaluate a process followed at USMA for assessing their capstone course that focused heavily on four CE sub-disciplines. (Meyer & Bert, 2007) Inspired by the USMA approach, the UT Tyler CE Program created a matrix that tied specific portions of the grade to specific program outcomes while covering seven CE sub-disciplines. Items on each cut-sheet (Figure 2) were linked in the matrix to one or more program outcomes. As each item in a cut-sheet is graded, these grades accumulate in ‘buckets’ for each outcome. Table 5 depicts an excerpt of the matrix showing the activities accumulating in the 9c and 9e bucket (construction management and public policy and administration). The benefits of this approach are numerous:
The overall weight of the grade that is ultimately assigned to each program outcome can be readily identified. Like most of us, students typically focus their efforts where they identify the greatest impact on their grade can be made (i.e., the activities with the most points assigned). Thus, a review of all of the points assigned to each program outcome can reveal if areas without enough focused attention exist within the capstone. Of course, this must be done in conjunction with other methods of assessment that look at the whole program. Sometimes areas that are covered extensively in other areas of the program can be lightly covered in the capstone to allow effort and the limited time within the capstone to be expended properly.
Areas where students achieve below an accepted standard of performance can be readily identified. Within the UT Tyler CE Program the performance standard is currently set as 70% - scores above the threshold are considered to demonstrate successful completion of an outcome. This allows a simple determination of which program outcomes need more attention within the program.
Table 6 shows the assessment matrix from 2007-2008 and 2008-2009. It is shaded in several ways to show key features related to the two points noted above (color scheme within titles). The overall points allocated to a particular outcome is coded to show outcomes below 50 and 100 points (note that the exact number of points for this threshold is somewhat arbitrary and can vary from year to year depending on the overall points for the senior design project) to clearly show areas with minimal points assigned that need more emphasis in future years.
3.3. Assessment results
Since the senior design experience is being used to try and assess all program outcomes, points are shifted to reflect the proper distribution or desired effort across the outcomes. Additionally, the percentages earned by each design team and overall are shaded differently for scores below and above 70% and above 80%. This methodology provides a visual determination of areas where students do not achieve an acceptable standard on an outcome and areas where students are excelling in achieving outcomes. Note, the format changed in Table 6 from 2007-2008 to 2008-2009 because of the program’s assessment of the assessment process and how the data should be displayed (i.e., closing the loop).
The strength of this approach is perhaps best illustrated by discussing the changes made as a result of this assessment vehicle. Several major changes to the course sequence from year 1 (2007) to year 2 (2008) to year 3 (2009) resulted, but only the one for professional skills is provided here: more Emphasis Needed in Professional Practice Skills (Outcomes 4, 6-9); faculty not totally pleased with overall professional skills development in 2008-2009 so increased total points in outcomes 4 and 6-9 and moved CENG 4341 to spring semester.
Table 6. Assessment matrix: (a) 2007-2008; (b) 2008-2009
With the possible addition of new outcomes in the future, the program assessment is looking for more efficient methods of teaching and assessing outcome demonstration. Experience teaching CENG 4341 has provided insight into how much coverage is needed to actually demonstrate at the required BOK II level and with the possible requirement to decrease the number of credit hours even further due to budget constraints, the program is analyzing the possible conversion of the current ten hours on leadership in CENG 4341 to five hours with three hours in CENG 4115 and two hours in CENG 4315. Public policy could be completely covered in the two course environmental sequence with one hour in CENG 4315 to focus the senior design solution based on the current public policy issues. The program could convert ten hours on business practices in CENG 4341 to five lessons culminating in each team developing a business plan as part of the 100 percent submission in CENG 4315. The program could convert eight hours on asset management to four hours culminating in each team developing a construction management plan as part of the 35 and 100 percent submissions in CENG 4315 that includes management of people within the team. However, individual assignments from CENG 4341 could be used within the senior design courses to ensure each student can demonstrate accomplishment of the professional outcome before the team uses their combined skills to accomplish a team product within senior design. The increased requirements within the senior design to provide coverage of the professional outcomes will be reflected with increased point totals and requirements, both individual and team.
|Outcome 9c: Can explain management concepts and processes.|
|Direct Measures||Tab||Standard||2009-10 Performance||Historical Average|
|CENG 4339, HW 3, Pr 2||1||80.0||81|
|CEG 4339, Final Exam, Question #26||2||80.0||99.0||100|
|CENG 4341, Paper 7||3||80.0||89.7|
|CENG 4341, Exam 3, Question 4||4||80.0||88.8|
|CENG 4341, Final Exam Question||5||80.0||94.5||81.7|
|CENG 4315, 100% Submittal||80.0||87.2||86.9|
|Indirect Measures||Standard||2009-10 Performance||Historical Average|
|Question A22. Senior survey||4/5||4.4/5||4.35/5|
|Question A22. Faculty survey||4/5||4.2/5||4.0/5|
|Question A22. Alumni survey||4/5||4.6/5||4.2/5|
|Question A22. Employer survey||4/5||4.5/5||3.75/5|
|Curriculum Measures||Standard||2009-10 Performance||Historical Average|
|Completion of CENG 4341, CENG 4339||5||5||5|
|2009-10 Assessment:4+ ( out of 5)|
Table 8 presents a representative excerpt from the annual program assessment for professional outcomes where the results of each assessment method are collated to allow an aggregate review of each outcome. As can be seen the outcome assessment within CENG 4341 and senior design (CENG 4315) are critical pieces demonstrating accomplishment of the professional outcomes. Note also the use of survey results as part of the multiple assessment methods used to provide a clearer picture. Due to the age of the department, the survey data is from alumni and employees within one year of graduation. The alumni and their employers will be assessed at the four and eight year marks as well.
The program presented has been able to adequately demonstrate knowledge, skills and attitudes associated with professional topics demonstrated within CENG 4341, the senior design experience and other senior year courses. The program continues to fully integrate professional topics across the curriculum and has already seen improvement in student skill development within each group of seniors as compared to the previous graduating classes. As can be seen from Table 6, there were a number of professional outcomes not being assessed and a number below 80 percent in 2007-2008, while more were being assessed and only two with below 50 points in 2008-2009 (PO 6b & 8b). 2009-2010 results are not provided in this chapter due to the table’s size; however, there was only one outcome with a result barely less than 80 percent and two with less than 100 points. The students are simply more comfortable with using their new skills due to more required repetition and focus within their undergraduate curriculum. The CE program has determined that the best method to integrate and assess demonstration of professional skills is to insert embedded indicators in appropriate courses. Requirements to consider professional skills are needed within the sophomore and junior years, while the seniors must address the professional skills within the 10, 35, and 100 percent design submittals.
The key has been identification of the required embedded indicators for each course before the assessment cycle begins and for each faculty member to establish which assignments before the semester begins will include demonstration of program outcomes, but especially professional outcomes such as leadership, the impact of engineering solutions, and consideration of contemporary issues. Adequate thought is required to craft assignments that not only demonstrate course objectives that feed seamlessly into demonstrating the “hard” outcomes like design and use of modern tools, but especially to demonstrate “professional” outcomes such as ethics, consideration of contemporary issues, leadership, business practices, and asset management. (Welch, 2009b)
4. Current actions
The faculty are experimenting with Public Policy coverage in the two course environmental sequence (CENG 3371 and 4371) since much of environmental activity is grounded in past and present public policy. With the current level of Bloom’s taxonomy for the new outcomes within the CE program criteria (leadership, business practices, public policy and administration, and asset management) at Explain (PO 9, Table 1) (ABET, 2011; ASCE, 2004) which could be covered by 2-3 lesson blocks rather than the current use of a 10 lesson block, the program is studying if the minimal coverage of leadership, business practices, and asset management within the senior design two-course sequence as listed in Section 3.3 of this chapter as well as if possible additional embedded indicators within sophomore and junior year courses will be satisfactory. The consideration of modules of professional outcomes throughout the curriculum allows the program to consider accreditation of its master’s program as well. (Welch, 2009b; 2010a) Without it, any student wanting to graduate from an accredited program would need to complete the undergraduate program before starting the graduate program to ensure coverage of all of the outcomes. Requiring accomplishment of modules will also allow students without an accredited undergraduate degree to take the accredited graduate degree and complete modules for outcomes not covered within their previous undergraduate program. This methodology will allow modules to be moved between courses without affecting the overall course structure or focus, while providing the program flexibility to meet defined outcomes as the technical body of knowledge advances and faculty skills improve. Since many non-accredited degrees do not require a capstone experience, the senior design will be ideal to provide demonstrated accomplishment of
|Outcome||Course or Module|
|Communication||English I and II, paper and presentation in each graduate course|
|Public Policy||Module in CENG 4341 Leadership, Public Policy, Business Practices, and Asset Management (moving to CENG 3371 Intro to Environmental Engineering)|
|Business and Public Administration||Module in CENG 4341 Leadership, Public Policy, Business Practices, and Asset Management (moving to CENG 4315 Senior Design II)|
|Globalization||Senior Design, Module in ENGR 4109 Senior Seminar (to be completely covered in CENG 4315 senior design)|
|Leadership||Module CENG 4341 Leadership, Public Policy, Business Practices, and Asset Management, CENG 4115/4315 Senior Design (Moving completely to CENG 4115/4315 Senior Design)|
|Teamwork||CENG 4115/4315 Senior Design, projects in senior level design courses (CENG 4412, CENG 4351, CENG 4371, CENG 4381, etc.) (remain the same)|
|Attitudes||Module in CENG 4341 Leadership, Public Policy, Business Practices, and Asset Management, CENG 4115/4315 Senior Design (Move completely to CENG 4115/4315 Senior Design)|
|Lifelong Learning||Projects in senior level design courses (CENG 4412, CENG 4351, CENG 4371, CENG 4381, etc.), CENG 4115/4315 Senior Design (remain the same)|
|Professional and Ethical Responsibility||Module in CENG 4341 Leadership, Public Policy, Business Practices, and Asset Management, CENG 4115/4315 Senior Design (Move completely to CENG 4115/4315 Senior Design)|
many outcomes with the insertion of individual assignments; thereby, decreasing the number of leveling courses and modules required by graduate students from non-ABET accredited undergraduate programs. Many of the outcomes currently not covered by non-ABET accredited programs are the professional skills.
The initial work to develop the matrix of coverage of professional skill outcomes for graduate leveling is shown in Table 9. The level of detail is whether a course or a module within a current course is required. As can be seen, the movement of condensed modules from CENG 4341 into the senior design experience would be efficient; especially with a decrease in technical lessons needing to be covered in the senior design with the maturing of other senior level design courses. However, there will be a need for students to complete individual assignments on each professional outcome prior to applying in a team setting to ensure each graduate can demonstrate each outcome.
Many programs experience problems within the design experience and the recent ABET visit noted that the coverage of seven CE sub-disciplines and the assessment process working toward coverage and assessment of all outcomes within the senior design was the best the PEV had seen in all of his numerous visits. The fact that the students must address demonstration of their skills for each outcome truly brings the entire process together and reinforces what skills they must have at the time of graduation. The comments have been reinforced by the engineering firms who have reviewed the senior design documents and provided comments on the design experience by the UT Tyler CE students. Increasing coverage of professional skills within the senior design will only strengthen the senior design since professional skills are critical during the design process and the profession has been active in demanding more professional skill development as evident with the additional outcomes generated from BOK I (ASCE, 2004) and the increased level of Bloom’s for professional skills within BOK II. (ASCE, 2008)
5. Conclusions/lessons learned
The faculty team must decide the courses that should be able to provide an embedded indicator for each program outcome. However, this is usually not enough; each professor must also develop a plan before the semester starts as to which assignments within the course will contain an embedded indicator. If the plan is not fully developed prior to the start of the semester, the normal day-to-day activities may (usually will) inhibit quality development of embedded indicators or prevent collection of embedded indicators that leaves some program outcomes without adequate demonstration of accomplishment. This is especially true for some of the professional outcomes which appear harder to demonstrate than other outcomes.
Currently the UT Tyler Civil Engineering program has successfully implemented a single synergistic course CENG 4341 Leadership, Public Policy, Business Practices, and Asset Management which includes multiple modules to demonstrate many of the professional outcomes. The offering of this course along side of the senior design allows for the students to study the concepts using their senior design experience as a focus. The single course provided a solution to properly demonstrate multiple outcomes that do not naturally align with traditional technical courses. The experience has also enlightened the faculty to the possible use of modules to successfully level graduate students desiring an ABET accredited graduate degree without graduating from an ABET accredited undergraduate program.
The current investigation to move accomplishment of the professional outcomes from CENG 4341 into the senior design would allow for the removal of CENG 4341 as the program investigates the affect of a possible mandated decrease in credit hours from 128 to 120. Additional embedded indicators for the professional outcomes in sophomore and junior level courses would be required to ensure multiple opportunities to apply skills and develop confidence before the senior design since one of the opportunities maybe deleted (CENG 4341). The senior design is a part of the solution and as shown above can assess more outcomes than just multi-disciplinary design alone.
The senior design is already being taught and each assignment is being assessed. Once developed, the evaluation rubric only needs to be modified each year to improve the balance of points between outcomes. Many faculty within the UT Tyler CE Program are now part of the senior design grading since the design usually includes all seven traditional sub-disciplines of CE. Where better to demonstrate through the actions of the faculty as well as the design products of the students that professional skills are integral to a successful design. Since design products and construction of these designs is the heart of the globalization issue for engineers, the active demonstration of professional or global skills during the senior design will develop new engineers who are ready to meet the globalization challenges as firms adjust to a global perspective while completing day-to-day company missions, especially as companies move toward international markets to remain competitive. Professional skills are the globalization equalizer.