Using Academic Analytics to Support and Catalyze Transformation
University of California Davis
University of Indiana Bloomington
The objective of RAC3 is to facilitate faculty use of learning analytics tools and technology in order to maximize student learning gains. Our hypothesis is that improvement of curriculum, instruction, and assessment practices at the course and program levels requires the ability to measure change in a useful and appropriate way. We cannot improve what we cannot measure. Currently, access to student data is limited and fragmented as are the tools and know-how needed to analyze the data. Once a critical mass of individuals and/or departments is supplied with and use data and analytic tools to measure change and inform decisions about student outcomes, evidence-based decisions and practice will become normalized. This shift to a culture of data and self-reflection is a necessary (but not sufficient) condition for systemic change to occur.
Some early aspects of RAC3 were funded by an AAU grant. However, development and dissemination of a comprehensive suite of academic analytic tools and strategies for employing them will require significant investment of additional time and resources. Toward this end, RAC3 includes two complementary and linked initiatives.
Tools for Evidence-Based Action (TEA)
University of California Davis
The mission of the TEA community is to enhance evidence-based teaching methodology and policy by developing and sharing new technologies. With funding from the Helmsley Foundation we have developed a visualization system to understand the pathways students take through a university (Ribbon Tool) and a classroom observation tool (GORP) used for researching teaching methodology and teacher development training.
Preliminary tests in which faculty were given access to data and analysis tools resulted in quick and positive shifts in the course planning and developing process toward evidence-informed practices. Such rapid and high-level integration into program culture suggests that these resources are meeting a significantly underserved demand, and that the potential to affect broad and long-term change is great.
Measures of intervention impact will include (but are not limited to):
- Tracking faculty use of analytic tools including the type and quality of driving questions
- Changes to course curriculum, instruction, and assessments informed by evidence – and resulting performance outcomes
- Changes to program-level curricular planning, TA training, and/or faculty reward systems related to instruction
- Faculty engagement in curricular pilot projects
Methodologies include interviews of faculty and students, online analytics, and qualitative analysis of course content, instruction, assessment, student assessment responses, and iterative course revision.
The work will focus on:
- Continued construction of the tools by the various partner development teams,
- Implementation of tools and training at each partner campus, and
- Assessment of impact including characterization of how data are used and if/how tools and data are impacting faculty culture and student outcomes.
An exploratory RAC3 workshop was held at UC Davis in December, 2013 attended by several BVA partner universities. In January 2015, UC Davis hosted the first TEA workshop. Professors, administrators, education developers and data analysts from 28 different institutions came to learn about the tools and begin collaborations on how to use them at their home institutions. A second face-to-face meeting was held in February 2016.
How to Engage with TEA
Please contact Marco Molinaro at firstname.lastname@example.org
Student Learning Analytics Fellows Program
Indiana University Bloomington
The University of British Columbia
University of California Davis
University of Kansas
Queens University of Canada
University of Saskatchewan
The Bay View Alliance Learning Analytics Fellows program is a Community of Transformation that has embarked upon a multi-year project to use data analytics to improve student learning, retention and success. The Fellows program is being led by Indiana University Bloomington, which began its own Student Learning Analytics Fellows Program in 2015 (Rehrey et. al., 2018), and will be joined by the University of British Columbia, the University of Kansas, Queens University of Canada and the University of Saskatchewan.
The BVA Learning Analytics Community of Transformation is a collaboration between six research institutions from Canada and the U.S.
According to Kezar and Gehrke (2015) Communities of Transformation have several distinctive characteristics that quite often distinguish themselves from similar Communities of Practice (Wenger, McDermott, & Snyder, 2002) and Faculty Learning Communities (Cox & Richlin 2004). Communities of Transformation (CoT) create and foster an innovative space that did not already exist, rely on philosophy more than practice, and adhere to that philosophy as core to their community engagement and action.
In our case, the Learning Analytics Community of Transformation (LACoT) is comprised of individual distinct Learning Analytics Fellows programs from each campus (Macfadyen et. al). Each campus is engaging faculty in scholarly research about their students, their behaviors and the choices they make on their pathway toward graduation. Generally speaking, each Fellows program consists of an administrator, a facilitator, and a group of faculty members, who often times herald from different disciplines, departments or programs.
Each Fellows Program engages faculty in research about the impact of existing course transformation projects, expands interest in evidence-based instruction, and collectively creates an interdisciplinary and multi-institutional community around the broader issues of student success. By developing culture that values and empowers individuals and units to make data-informed decisions, the program will help build capacity and bring awareness to the use of learning analytics in order to transform teaching and learning and improve course design and assessment.
Thus, faculty will harness the growing collection of student data on their individual campuses to engage with a data-driven learning community. Working in conjunction with data stewards and other relevant units they will help to enhance and expand the availability of data, develop a fuller understanding of the value of an undergraduate educational experience, and create new models to be used for institutional decision making. We anticipate that these activities will encourage faculty to make connections between student performance in their own classes with the pathways students take on their journey toward success during their college years.
LAK Conference News Release
Cox, M. D., & Richlin, L. (2004). Building faculty learning communities. San Francisco: Jossey-Bass.
Kezar, A., & Gehrke, S. (2015). Communities of transformation and their work scaling STEM reform. Pullias Center for Higher Education.
Macfadyen, L. P., Steinwachs, M., Groth, D., Rehrey, G., Shepard, L., Greer, J., … Molinaro, M. (2017). Developing institutional learning analytics “communities of transformation” to support student success (pp. 498–499). Presented at the Learning Analytics and Knowledge Conference, Vancouver, BC: ACM Press.
Rehrey, G., Groth, D., Shepard, L., & Hostetter, C. (forthcoming, 2018). The scholarship of teaching, learning and student success: Big data and the landscape of new opportunities. In J. Friberg & K. McKinney (Eds.), Conducting and applying SoTL beyond the individual classroom level. Indiana University Press.
Wenger, E., McDermott, R. A., & Snyder, W. (2002). Cultivating communities of practice: A guide to managing knowledge. Harvard Business Press.