Gloria Rogers
Program Chair
Associate Executive Director
Professional Services
ABET, Inc.
(410) 347-7700
Judy Houghtalen
Symposium Coordinator
(812) 872-6241
Kathy Gregory
Administrative Assistant
(812) 877-8816
Fax Number
(812) 877-8931
|
|
|
April 13, Friday 1:15 - 2:35 p.m. |
Turning Data into Info to Facilitate Continuous Improvement and Closing the Loop
Karen Tarnoff, East Tennessee State University
Dr. Donald Sanderson, East Tennessee State University
Olin Hall O157
Session Description
Schools have spent so much time and effort developing outcomes, objectives and measures and then gathering data that many are now left asking, "How do we now use the data for continuous improvement as we close the loop"? Faculty resistance to assessment is likely to increase if they are required to work with raw data or results. Once data is in hand, it is worthwhile to consider how that data can be transformed into information that is easily used in closing the loop and in continuous improvement of student learning. This session will present methods by which data and results can be transformed into information that is easily processed and that readily allows users to see trends, ask insightful questions, and quickly move to identify improvements and closing the loop. Methods of transforming data into information to be discussed include graphic design, spreadsheets and databases.
Key Words
Reporting Results, Results for Improvements
Creating an Assessment Plan for a New Engineering Program
David Pape, Central Michigan University
Olin Hall O159
Session Description
This session will focus on the development and implementation of an assessment plan for a newly created engineering program. Knowledge of and experience with existing programs is used to create an assessment plan that efficiently uses resources while at the same time providing valuable input to the continuous improvement process. Application of this plan to existing programs is discussed. After attending this session, attendees should be able to do the following: 1. describe the elements of the assessment plan presented in relation to ABET criteria; 2. engage faculty in the assessment process while simultaneously reducing the burden on their time; 3. use the presented assessment tools and techniques at their own institution; and 4. modify the presented assessment plan and tools to meet individual program needs. Session will be structured as follows. First, the general framework of the assessment plan will be presented, followed by details of individual elements. Questions and comments will be welcomed at any point. Examples of various tools and forms will be distributed. Next, we will do an exercise in which participants will create an assessment tool using program outcomes and course learning objectives from their own institution. Participants should bring a copy of their program outcomes and the course learning objectives from an upper level course (other than senior design). Alternatively, they can create the tool using provided sample materials. Finally, we will wrap up the session with discussion.
Key Words
Assessment Tools, Embedded Assessment, Faculty Involvement, Measuring Outcomes
Developing and Assessing Professional Skills in Senior Project Courses
Sharon Hall, University of Houston - Clear Lake
Olin Hall O167
Session Description
In preparing for the second round of accreditation visits, we have placed particular emphasis on developing and assessing students' professional skills within the senior project courses of our three accredited programs. This session addresses three of our most important assessment activities: advisory board and faculty involvement at the senior presentation day, faculty assessment of project deliverables, and instructional modules for developing and assessing professional behaviors. At the end of each semester, advisory board members observe and evaluate the final presentations for each project team, and individually interview graduating seniors. A faculty team also observes and evaluates the presentations, assesses each team's project documentation and deliverables, and assesses individual samples of work that demonstrate students' professional capabilities. Instructional modules with accompanying rubrics are shared between programs to ensure that students' professional skills are addressed in all project courses.
Key Words
Advisory Boards, Faculty Involvement, Measuring Objectives, Results for Improvements
An Online Assessment System that Promotes Participation: M.E.E.T.
John Wise, Penn State University
Dhaneshwar Lall, Penn State University
Olin Hall O169
Session Description
The M.E.E.T. system is especially designed for higher education, recognizing that faculty are by nature independent and unlikely to participate in activities that don't make sense to them. If your goal is to go beyond just satisfying ABET to building a culture that supports performance-based learning, M.E.E.T. may be just what you need. The online system involves every faculty member in the use of existing embedded assessments to measure learning outcomes at the course level. It is scalable, allowing for data collection for all students, all instructors, and all courses, or any subset of interest. This session will describe the M.E.E.T. system at three levels: Administration, Faculty, and Student. A demonstration of each will be presented, with ample time for questions and answers. Participants will be able to try out the system for themselves if they have a networked computer; Otherwise, the demonstration will be projected.
Key Words
Embedded Assessment, Faculty Involvement, Measuring Outcomes, Use of Technology to Support Assessment
Using Communication for Effective Assessments of Capstone Design Courses
Alan Cheville, Oklahoma State University
Olin Hall O257
Session Description
Many engineering programs use capstone design courses to cover multiple ABET outcomes. There is the potential for these courses to become "a mile wide and an inch deep" with little lasting impact on student learning. This session will introduce methods of assessing student learning in capstone design courses using self-reporting by both individual students and student teams. Because of the unique format and broad spectrum of capstone design courses, the presentation focuses on how to measure multiple program-specific outcomes by combining a variety of complementary assessment tools. Participants will learn: - Methods to measure the contributions of individual students to a design team; - Techniques for using graduate students for rubric-based student portfolio evaluation; - How to design report templates and rubrics to address program-specific outcomes; and - Ways of recording the progress of student teams through the design process.
Key Words
Assessment Tools, Qualitative Methods, Quantitative Methods, Use of Technology to Support Assessment
Direct Quantitative Assessment Process for Educational Objectives & Outcomes
Paul Biney, Prairie View A&M University
Olin Hall O259
Session Description
A direct assessment process and analysis methodology for educational objectives and outcomes developed and being used at Prairie View A&M University will be presented and discussed. The process is based on direct measurement of students' performance in program outcomes taught, tested, and evaluated in the various courses. Methods for documenting assignments used to assess outcomes and for preparing outcomes based binders are discussed. This procedure provides an accurate determination of students' competencies in the outcomes, and methodology for addressing any perceived problems. Comments from attendees will also be sought.
Key Words
Assessment Tools, Measuring Objectives, Measuring Outcomes, Quantitative Methods
Creating Clear and Concise Visual Displays of Information
Kay C Dee, Rose-Hulman Institute of Technology
Glen A. Livesay, Rose-Hulman Institute of Technology
Olin Hall O203
Session Description
Say "program assessment", and most people think of collecting and analyzing information but few people think through to the critical step of visually presenting that information and the assessment results. This interactive session will present fundamental principles that can be used to create clear, concise, visual displays of information based on the works of Tufte, McCloud, Few and others. We will especially focus on the kinds of information often presented in accreditation reports, critiquing example tables, process schematics, dashboard charts, and plots. To immediately put ideas from this session into action, small groups of participants will work to create effective presentations of sample accreditation-related information, using only standard Microsoft Office software. With an emphasis on using technology to meet your demands rather than altering your plans to match automatic functionalities of technology, this session will give you key tools for communicating your assessment results.
Key Words
Reporting Results
|
April 14, Saturday 8:00 - 9:20 a.m. |
Comprehensive Evaluation System for an Undergraduate Multidisciplinary, Team-based Project Program
Margaret Huyck, Illinois Institute of Technology
Daniel Ferguson, Illinois Institute of Technology
Sherine George, Illinois Institute of Technology
Mohammed Z. Mahmoud, Illinois Institute of Technology
Olin Hall O157
Session Description
During the past three years of the IPRO program we have 1) clarified and redefined the core learning objectives, 2) developed measures to assess cognitive knowledge and self-perceived competence in these domains, 3)modified the evaluation forms used by judges for IPRO Day to reflect the learning objectives, 4) provided feedback to faculty team facilitators about the performance of their team, 5) developed a system for evaluating proposed IPRO projects that involves students and faculty, and 6) systematically evaluated interventions designed to enhance the learning objective accomplishments. The IPRO program has been modified on the basis of our research, and modifications are evaluated continuously. We have used technology in assessment by conducting on-line surveys of alumni, students, and faculty; and by developing special on-line systems for storing archival data from all the IPRO projects, and by providing communication within and between IPRO teams. We will describe processes, outcomes, pitfalls and accomplishments.
Key Words
Assessment Tools, Measuring Objectives, Measuring Outcomes, Use of Technology to Support Assessment
Yes You Can: Reduce Faculty Burden and Enhance Assessment and Continuous Curriculum Improvement
Thomas Brumm, Iowa State University
Steven Mickelson, Iowa State University
Larry Hanneman, Iowa State University
Olin Hall O159
Session Description
Some engineering faculty members' experience is that outcomes assessment is burdensome, requiring large amounts of faculty time and resources. In particular, gathering assessment data can be onerous. However, such data collection can be centrally implemented and managed, involving little or no faculty time. Faculty members' activity should be focused on data evaluation and interpretation, drawing conclusions from the assessment data, and implementing change as part of their programs' continuous curricular improvement processes. This session demonstrates a systems approach to collecting large volumes of direct, quantitative assessment data on-line with little faculty time or resources. College-wide and program-specific results will be presented, showing successful integration into curricular improvement process and ABET self-study reports, second-cycle results, and how quantitative assessment data facilitates qualitative understanding.
Key Words
Assessment Tools, Faculty Involvement, Measuring Outcomes, Use of Technology to Support Assessment
Relieving the Pain of Assessing Lab Courses
Richard Layton, Rose-Hulman Institute of Technology
Patsy Brackin, Rose-Hulman Institute of Technology
Olin Hall O167
Session Description
In this interactive session on lab assessment, participants learn an approach for assessing program outcomes supported by laboratories. Participants apply our rubrics to sample student work to illustrate the approach and develop similar materials for their own lab courses. Activities include: modifying existing grading rubrics to include questions that address accreditation-related outcomes; writing assessment questions suitable for lab courses; summarizing and reporting assessment results; and describing methods for improving courses based on the assessment results.
Key Words
Assessment Tools, Measuring Outcomes, Reporting Results, Other: Laboratory Assessment
An Effective Online Assessment Tool: EvalTools™
Fong Mak, Gannon University
Olin Hall O169
Session Description
In this session, we introduce an online assessment tool, EvalTools ™ , that allows not only creation of mixed quantitative and qualitative survey instruments for classroom settings, but also allows creation of survey sites for any non-classroom environment. The tool supports classroom instruments like student and faculty course exit surveys, and the non-classroom instruments such as senior-exit, alumni surveys, and employers' surveys. This is a commercial-grade outcome assessment tool with expanded capabilities to support external surveys, on-line syllabi, trend analysis, and more effective linking of course and program outcomes. We would like to illustrate features of the tool that help faculty do the following: (a) assist in collecting, maintaining, and tracking objective evidence, (b) automate parts of the faculty member's ABET documentation process, (c) facilitate the reporting effort on survey results and FCAR (Faculty Course Assessment Report feature), and (d) improve on student's feedback rate relevant to program goals assessment
Key Words
Assessment Tools, Quantitative Methods, Reporting Results, Use of Technology to Support Assessment
Using Electronic Portfolios for Program and Institutional Assessment: 11 Years Experience
Julia Williams, Rose-Hulman
Olin Hall O257
Session Description
Electronic portfolios appear to hold a utopian promise for effortless data collection and easy institutional assessment. Many of the electronic portfolios currently offered on the market make such pie-in-the-sky promises, while few identify what real benefits and drawbacks come with the use of portfolios. At Rose-Hulman, we have been developing and implementing an electronic portfolio system since 1996, and our experiences can serve other institutions that may be considering a similar decision. This presentation will focus on design parameters that guided development of the RosE Portfolio, the assessment methodology that is the foundation of the system, and the strategies used to implement portfolio assessment at the institutional and program levels. The presenter will also discuss the role of electronic portfolios in institutional change and the impact of ABET.
Key Words
Assessment Tools, Measuring Outcomes, Use of Technology to Support Assessment
IT Assessment Tools
L. K. Williams, Data180, LLC
Olin Hall O259
Session Description
This session provides a demonstration of an online faculty activity reporting system created by Data180, LLC. The Web-based application is called Faculty180 and uses an institution-tested process for collecting each faculty member’s teaching, research and service activities. Faculty180 also provides reports for internal decision makers, accrediting bodies and other external users. Faculty members can create personal assessment reports for self-evaluation, as well as reports for their annual evaluations and tenure/promotion reviews. Administrators benefit by receiving standardized documentation from faculty, which promotes a more efficient and effective evaluation process. The presentation will also discuss other online solutions that will be developed from Faculty180. Those applications include a course-embedded assessment tool, a student exit survey, and a student activity reporting system that supports electronic student resumes, student portfolios and co-curricular transcripts.
Key Words
Assessment Tools, Use of Technology to Support Assessment
Build Your Own Database for Your Assessment Data, Its Not That Hard
Donald Sanderson, East Tennessee State University
Olin Hall O203
Session Description
There are several good tools available to track assessment data; the problem is that an existing tool might not track your data exactly the way you want it to. It would be great to be able to "roll your own" but that task can be daunting. There are, however, almost certainly database "gurus" on your campus who could aid in the process if you could just talk to them. The leader of this workshop is bilingual being both his department's assessment coordinator, and lead database instructor. The workshop will give you a basic database vocabulary, and some hands on training using a small assessment database. This should give you the vocabulary you will need to work with your local database experts. We will also look at the ways having your data in a database can save you from hours of tedious work reformatting the same data for different audiences.
Key Words
Assessment Tools, Reporting Results, Use of Technology to Support Assessment
|
April 14, Saturday 10:30 - 11:50 a.m. |
How Can We Assess and Promote Reflective Judgement Thinking in Undergraduates?
Mike Cama, Illinois Institute of Technology
Margaret Huyck, Illinois Institute of Technology
Daniel M. Ferguson, Illinois Institute of Technology
Olin Hall O157
Session Description
During the past three semesters of the IIT IPRO program we have implemented a reflections pilot to encourage and evaluate reflective thinking based on the Reflective Judgment Model (King and Kitchener, 1994). Reflective thinking is important because it is a way to deal with uncertainty and confusion when attempting to solve problems with no clear solution. King and Kitchener used 1-2 hour interviews to assess reflective thinking. While this method has demonstrated reliability and validity, it is not a practical assessment system for wide use. We are developing a written version of reflections along with a rating system to assess reflective thinking in the undergraduates involved in the IPRO program. We will provide examples of what we interpret as different levels of reflective judgment among our undergraduate students, and invite participants to make their own evaluations. We will provide a list of “reflective judgment” questions used in the IPRO program, along with current evaluation rubrics.
Key Words
Increasing the Efficiency of an Effective Outcome Assessment Process
Lisa Shay, United States Military Academy
Robert Sadowski, U.S. Military Academy
Peter Hanlon, U.S. Military Academy
Olin Hall O159
Session Description
We will use the framework from the Electrical Engineering Program at the U.S. Military Academy at West Point to describe a systematic process to increase the efficiency in assessing program outcomes while maintaining the effectiveness of the assessment results. First we describe applications of a course-outcome crosswalk matrix to reduce the time required by faculty to accomplish outcome assessment. Then we describe a cross-correlation matrix that eliminates redundancy of assessing a program multiple times for different accreditation sources. These approaches not only effectively monitor graduate abilities, but also provide mechanisms to monitor individual course contribution and serve as a troubleshooting instrument for deficient outcome results. This process can also be extended to satisfy other institutional assessment requirements and encourages increased faculty interaction which results in improved course linkages.
Key Words
Assessment Tools, Embedded Assessment, Measuring Outcomes
ABET Assessment Process: Moving from "Good to Better" Problem Solving Skills within the Curriculum
Greg Boyd, Southeast Missouri State University
Olin Hall O167
Session Description
This session will present steps that the Department of Industrial and Engineering Technology at Southeast Missouri State University went through to identify and improve student problem solving skills within the overall curriculum as a result of the ABET accreditation process. The goal of this session is to share with participants the department's experience using the ABET accreditation process, starting with our Industrial Advisory Committee's involvement in developing program objectives, through the development and use of individual course outcomes with assessment rubrics, and finally the evaluation and use of assessment results to move the teaching of "problem solving" from an ad hoc approach to a more formalized one.
Key Words
Faculty Involvement, Measuring Outcomes, Reporting Results, Results for Improvements
Web-based Quantitative Analysis and Reporting of Program Outcome Coverage and Student Performance
Paul Van Halen, Portland State University
Olin Hall O169
Session Description
A web-based tool for course and program assessment has been developed and deployed in the Electrical and Computer Engineering Department at Portland State University. Using XML/XSLT documents, numerical course outcome to program outcome mappings, normalized by credit hours, are dynamically combined into program outcome coverage. Course grade information, entered in a spreadsheet, is processed to generate course outcome coverage and distribution and statistical information on course outcomes performance. Using an online form, course instructors can review this data and add observations and action items. Groups of courses, user selectable, are then combined to provide distribution and statistical information on program outcome coverage and student performance. The data collection flow and processing will be described in detail. This will be followed by a live demonstration. The presentation will conclude with a summary of faculty observations and comments on faculty participation.
Key Words
Measuring Outcomes, Quantitative Methods, Reporting Results, Use of Technology to Support Assessment
Building an Assessment System that Doesn't Overload your Faculty: How to Successfully Leverage Resources
Karen Tarnoff, East Tennessee State University
Linda Garceau, East Tennessee State University
Olin Hall O257
Session Description
The prime, often deadly, source of faculty resistance to assessment is the fear that participating will be time-consuming and burdensome. It is, in fact, possible to build an assessment system that leverages financial, human, and technical resources and doesn't overload faculty. The central question that must be answered is "What is required to build a sound assessment system in terms of time, money, technology, and people?" The answer is, of course, dependent on the school and the system. However, there is a set of questions that should be considered in the design and continuous improvement of assessment systems which, if properly addressed, can decrease resistance and increase involvement. Those questions will be presented as will successful strategies for resource utilization. Presenters will share their approach to building a system that spans seven departments and multiple accreditors. Participants will engage in discussion sharing their challenges and approaches to addressing these questions.
Key Words
Faculty Involvement, Other: Use of resources to support assessment
Enabling the Feedback Loop: Four Ways to Skin a Cat?
Kevin Huggins, U.S. Military Academy
Christa Chewar, U.S. Military Academy
Olin Hall O259
Session Description
We recently redesigned our assessment strategy to avoid several pratfalls (commonly known pitfalls). One pratfall is addressed by our explicit goal to "close the feedback loop" and drive continuous improvement. We will present aspects of our assessment strategy enabling this goal--the notion of Course Monitoring Teams (CMTs) and performance indicators, organized by learning levels. When our four CMTs began their evaluation cycles, we observed each team embrace a different guiding philosophy and develop a unique set of organizational tactics. Through role play and group work, workshop attendees will gain familiarity with the mechanics and rationale of each approach. Further interaction will probe the benefits and drawbacks of these four approaches. We will also discuss how the evaluations of each CMT are effectively synthesized at the Steering Committee level. If desired, attendees will be able to implement an assessment strategy similar to ours for their own programs.
Key Words
Assessment Tools, Embedded Assessment, Faculty Involvement, Measuring Outcomes