Measuring DEI Outcomes
How do we know we have made progress on our DEI goals?
Collecting data is important, but equally important is making sense of the data that is collected. The University of California collects a massive amount of data; research questions help narrow the focus of our search through the many dashboards and resources currently available. Asked during the design of activities, a good research question can focus a project's priorities. Asked after the completion of work, research questions can drive the presentation of findings and the further refinement of data dashboards and reports. Following is a list of broad research questions based on the strategic vision plan or HSI Taskforce report. Note that these are simply a jumping off point toward more specific questions that are aligned with more focused efforts.
Diversity and Inclusion Strategic Vision Research Questions
Defining Diversity for Measurement
Wherever differences are disproportionally represented or underrepresented, it is important to understand whether structural barriers are preventing proportional representation. No system of sorting or categorization is without controversy; neither is it acceptable to assume that a blindness to all categories is equivalent to correcting for biases.
For students, we reflect on the categories of ethnicity, race, gender identity, sexual orientation, disability status, veteran status, age, income, postal code (rural or urban), educational attainment of parents (“first-generation college students”) and residency status. For our workforce, we also reflect on the categories of ethnicity, race, gender identity, sexual orientation, disability status, veteran status, age, education level, rank, salary, duration of employment, citizenship, nation of bachelor’s degree, and postal code (urban or rural).
For undergraduate students, underrepresented status requires a comparison with California and national demographics and an understanding of the pipeline of students from early in the K–14 pipeline. For graduate students, we must understand how our graduate student population compares to the undergraduate student population, as well as the demographics of national M.A., M.S., Ph.D., and professional students. Faculty can also be benchmarked against the national pool of graduate and professional students, composition of their undergraduate population, and hiring at peer institutions. For staff, the campus looks at designated availability pools depending on the position and title—some positions pull candidates from the local area, while others are part of recruitments from a larger—even national—pool of candidates. We also look internally, across our own structures of schools, colleges, departments, divisions, and administrative units, to understand where we can learn from each other on the challenges and benefits of enabling a diverse composition of people and ideas.
- Students
Is enrollment growth reflected equally/proportionally across all groups?
Are certain schools/colleges/majors more diverse than others?
What schools, colleges, and departments are growing, and is that growth making an impact on demographics?
Where are Rising Scholars falling out of the pipeline to graduation? Where are graduate and professional students falling out of the pipeline to completion of their degree?
Where are undergraduate and graduate/professional Rising Scholars coming from?
What is the success and progress gap for Rising Scholars, and where on campus is the difference statistically significant?
Is the social and academic environment of UC Davis conducive or detrimental to emotionally and physically healthy, successful students?
How is student financial support distributed for Rising Scholars?
- Workforce
Are workforce increases reflected equally across all job groups?
Are certain types of positions more diverse than others?
Are colleges and schools growing, and is that growth/hiring enough to make an impact on demographics?
What can retention rates, turnover rates, duration of hire, vacancy rates and turnover quotients tell us about who is leaving and when? Where are there opportunities for providing greater stability in the workforce, particularly among underrepresented or historically marginalized groups?
Who is advancing, how quickly, and in what positions? How equitable are financial resources distributed?
Might UC Davis be able to create an algorithm to identify and intervene in climate hotspots?
- Peer Benchmarking
- How does UC Davis compare to UCs systemwide, the Comparison 8 and with the graduate student pool?
Hispanic Serving Institution Taskforce Research Questions
UC Davis is not the first organization eager to measure its impact in terms of equity contributions. Many nonprofits and corporate entities are beginning to develop metrics that measure the impact of initiatives attempting to make broad social change. Guidestar and the Social Progress Index are just two examples, and their work—along with higher education scholars such as Robert Teranishi at UCLA or Estela Mara Bensimon (The Equity Scorecard) at USC—are shaping the way we might measure the social impact of our HSI initiative. It is important to note that The Equity Scorecard is a useful assessment tool for “the self-reflective process that is at the foundation of continued change” but follows as a planning tool, after team members have had an opportunity to review a core set of institutional data.
In the simplest terms, we hold to the principle that individuals who are educated have the capacity to live better lives, and by improving themselves, influence their families, their communities and society. While individual units, departments and programs might use a variety of metrics to drill down on the success of their programs, we recommend that these evaluations be grounded in the following important research questions. Following each question is a list of possible methods, metrics and tools to use in this work:
- Are we attracting a broad profile of incoming students that reflects communities most in need of access to opportunity and social mobility?
Proportion of Rising Scholars, low-income and first-gen enrolled.
Proportion of students from those under-resourced regions as identified in the Regional Opportunity Index issued by the UC Davis Center for Regional Change. https://interact.regionalchange.ucdavis.edu/roi/.
- Are our students healthy, with adequate access to health care and mental health care? Are our students food- and housing-secure?
National College Health Assessment data (NCHAACHA).
On food security: UC Global Food Initiative and local campus surveys regarding food security.
On affordable housing: A 30% ratio between half the rental cost of a Davis market-rate two-bedroom apartment and a Teaching Assistant (TA) salary at 50% FTE; a 4-5% rental housing vacancy rate in the City of Davis; a 0% or minimal gap between the Davis housing market average annual rent increase and the average annual inflation as measured by the Consumer Price Index; less overcrowding in rental units as measured by a yearly student housing survey; evidence that rental rates and conditions for low income students are improving.
On mental and physical health services: utilization and availability of Student Health and Counseling Services; hiring and staff development to promote culturally relevant and sensitive counseling and therapy practices.
- Are our students persisting in and completing their educational programs at normative rates?
Retention within major, institution and higher education, particularly first year retention.
Graduation and completion: (a) Low Income Graduation/completion rate compared to all other students (b) First Gen Graduation/completion rate compared to all other students, (c ) Chicanx/Latinx compared to all other students.
- Are our students learning what they need to succeed in their chosen careers and at our institution?
Academic units are engaged in a reiterative, rigorous and thoughtful process for the adoption of learning outcomes and the means of assessment in response to changing student needs and best practices.
Existence of inter-departmental collaborations that ensure academic coherence and coordination in related programs of study or courses (e.g., gateway courses).
- How much opportunity and resources does the institution make available for faculty development of excellence in teaching skills?
Number of faculty and staff engaging in professional development in relevant areas.
Recognition of teaching excellence in the tenure and promotion process.
- Are students able to access equitably all opportunities the campus has to offer, and does that engagement come through positive and inclusive campus experiences that promote a sense of belonging and inclusion?
Student participation in internships, leadership roles, organizations and other sense of belonging measures, with nuance across intersectional identities (specific ethnicities within Chicanx/Latinx, income level, first gen, school of origin, major).
Qualitative (UCUES, Campus Climate Report) measurements of student engagement and inclusion, levels of respect, experiences with exclusionary (e.g. shunned, ignored), intimidating, offensive and/or hostile conduct (bullied, harassing).
- Is UC Davis a campus that is preparing all students to thrive in a pluralistic society?
Representation of faculty and staff at all levels at UC Davis that reflects the rich diversity of California, including Chicanx/Latinx communities.
Majors provide culturally responsive and relevant curricula that reflect the history and experiences of Rising Scholars, or respond to their needs and aspirations.
Faculty are employing methods of assessment that recognize multiple contributions and distinct learning approaches of all students, especially those that employ proven practices to promote the success of Rising Scholars.
Strength of ethnic studies programs at UC Davis, including Chicano and Chicana Studies (e.g., resources, faculty retention and research support).
Results of climate surveys that take into account how race and ethnic difference influence how students, faculty and staff experience UC Davis.
An operationalization approach to the Principles of Community that recognizes the consequences of hate speech and adequately supports and trains faculty, students and staff experiencing racial and other forms of bias, discrimination and harassment.
Assessment and implementation of plan to develop strategic actions to deal with campus climate issues.
- How are we fulfilling our mission as a public, land grant institution?
Ability of Rising Scholars to build wealth and financial well-being upon graduation or completion; e.g., debt-to-earning ratios 2- and 5- years after graduation under 10%.
Increases in graduating students in industries where there are critical workforce shortages (need to clearly identify and have method for revisiting and revising list).
Increases of Rising Scholars going into graduate school.
Number of students and alumni working in underserved communities.
Production of public scholarship and public service by faculty, staff and students and institutional recognition.
Continuous Improvement Model
We offer several models of assessing the initiative as a whole, each employing a continuous improvement model that is consecutively innovating, evaluating, iterating and documenting the work. An advisory group—perhaps including a separate or integrated student advisory group—has been proposed to maintain and update the vision for the HSI initiative. Another proposal is to do an assessment in 2-5 years, using a UC-based assessment team, formed from administrators and faculty from other University of California campuses, who would visit the campus to determine whether any progress has been made toward our goal of becoming a premier R1 land grant institution and make recommendations on next steps. A central coordinating group might also work with various units across campus to continue to understand what projects are moving UC Davis toward its HSI vision and how those projects can be coordinated, evaluated and documented.