LEVERAGING AI FOR ‘DIFFICULT-TO-MEASURE OUTCOMES’ IN OBE

Leadership, critical thinking, teamwork and ethical judgement are among the most important capabilities expected of an MBA graduate. Yet these are also among the most difficult outcomes to assess reliably.

Behavioural capabilities cannot be developed through conceptual understanding alone. They require practice, feedback, reflection and repeated opportunities to demonstrate behaviour—often in increasingly complex situations. Traditional teaching-learning processes provide some of these opportunities through classroom discussions, case studies and simulations, supplemented by activities such as managing events, participating in committees and clubs, and undertaking projects. But much of this behaviour is either not assessed or is assessed only indirectly.

This raises an important question:

Can Artificial Intelligence help us make some of these difficult-to-observe capabilities more visible?

The answer is potentially yes—but with an important qualification.

AI can generate and analyse evidence of behaviour. It cannot, by itself, certify leadership, teamwork or character.

The real opportunity lies in using AI to create multiple situations in which students have to demonstrate capabilities, systematically capturing evidence, and triangulating that evidence with human observation and authentic performance.

1. LEADERSHIP: FROM KNOWING LEADERSHIP TO DEMONSTRATING IT

Consider a conventional assessment question:

“Explain the different styles of leadership and discuss their effectiveness.”

A student may produce an excellent answer without necessarily demonstrating the ability to lead.

AI creates a very different possibility.

Imagine a simulation in which the student is asked to lead a cross-functional team. Sales wants an aggressive product launch. Finance is concerned about margins. Operations says that capacity is inadequate. The CEO wants a decision within 24 hours.

AI can enact the roles of the CFO, Operations Head, Sales Head and a dissatisfied team member.

The student must now listen, question, negotiate, prioritise, decide, communicate and handle resistance.

The assessment is no longer simply about what the student knows about leadership. It is about what the student does when placed in a leadership situation.

A rubric could examine:

This is arguably much closer to demonstrating leadership than answering a question about leadership styles or analysing a leadership case.

But there is an important distinction: AI has not measured leadership in its entirety. It has generated evidence of leadership behaviour in a particular situation.

That distinction matters.

2. TEAMWORK: MAKING INTERACTION OBSERVABLE

Teamwork presents an even more interesting possibility.

AI could be used not merely as an interviewer but as a simulated team. A student could be placed in a virtual team consisting of AI agents with different personalities and agendas:

• one highly analytical but resistant;
• one creative but impractical;
• one dominant and impatient; and
• one quiet member who possesses useful information.

The student must work with the team to solve a business problem. AI can then analyse observable behaviours:

• Does the student listen?
• Do they invite quieter members to contribute?
• Do they build on others’ ideas?
• Can they disagree constructively?
• Do they resolve conflict?
• Do they share information?
• Do they take disproportionate control?

This could reveal much more about individual teamwork behaviour than a conventional team presentation, where one student may do most of the work while the performance of others remains largely invisible.

Yet again, the caveat is important:

AI has not “measured teamwork”. It has generated and analysed one piece of evidence of teamwork behaviour.

Greater confidence would come from triangulating:

AI simulation + peer assessment + faculty observation + project contribution

This moves us towards a genuine 360-degree view of teamwork.

3. ETHICAL JUDGEMENT: THERE MAY BE NO “CORRECT” ANSWER

Ethical judgement may be the most difficult of these outcomes to assess.

A conventional ethical case often presents a relatively obvious right-versus-wrong situation. But real ethical decisions are rarely so neat. They involve competing values, stakeholders, consequences and pressures.

AI can create precisely such ambiguity.

Consider this situation:

You discover that a highly successful sales executive has been manipulating customer data. Reporting the matter could result in the loss of an important client and potentially affect 200 employees.

AI could then probe the student's response:

AI: What would you do?

Student: I would report it.

AI: What if reporting it causes 200 employees to lose their jobs?

AI: What if your CEO asks you to remain silent?

AI: What if the manipulation is technically legal but clearly misleading?

The objective is not necessarily to discover whether the student has selected a predetermined “correct” answer.

Instead, the interaction can provide evidence of:

• ethical reasoning;
• stakeholder consideration;
• consistency of judgement;
• recognition of competing values;
• willingness to challenge authority; and
• consideration of consequences.

Here AI can play an especially valuable role: it can keep probing the reasoning behind the decision.

But ethical behaviour in a simulation should not be confused with ethical character.

A student may demonstrate sound ethical reasoning in an AI-generated dilemma and behave differently when confronted with real organisational pressure.

This brings us to perhaps the most important philosophical distinction in this discussion:

AI can observe behaviour; it cannot directly observe character.

4. CRITICAL THINKING: AI - A “PROVOCATEUR”

Critical thinking is among the top 10 skills demanded of MBA graduates in the 21st century. Here, however, there is often confusion between analytical thinking and critical thinking.

Analytical thinking involves breaking down information systematically and examining relationships, patterns and evidence. Critical thinking goes further. It requires questioning assumptions, evaluating competing claims, recognising ambiguity and uncertainty, considering alternative explanations and being willing to revise one's conclusion. In a VUCA environment, therefore, critical thinking becomes particularly important.

AI can deliberately create situations containing:

• incomplete information;
• contradictory evidence
• hidden assumptions;
• misleading statistics; and
• plausible but false conclusions.

For example, AI might present three analysts’ recommendations and ask:

Which would you choose?

It can then challenge the student:

• What assumptions are you making?
• What evidence would change your conclusion?
• What is the strongest argument against your position?
• What information is missing?

The important feature here is not simply that AI asks questions.

AI can become a “provocateur”—not merely asking questions, but deliberately provoking the student's thinking.

This makes the thinking process more visible.

And that is potentially far more valuable than simply awarding marks for the final answer.

COMPARING THE AI LEVERAGED APPROACH WITH CONVENTIONAL ASSESSMENT

The difference between conventional assessment and an AI-enabled approach can be expressed simply.

Conventional assessment: Question → Answer → Marks

AI-enabled assessment: Situation → Student response → Challenge → Reasoning → Counterargument → Revision → Reflection

The second approach produces richer evidence of the learning process. More importantly, AI can potentially provide multiple opportunities for students to demonstrate and improve a capability.

A student who performs poorly in the first leadership simulation need not be labelled “not competent”. The student can receive feedback, practise, confront another situation and demonstrate again. This is much closer to the development of a capability.

A simple proficiency framework could be:

0 – Not Demonstrated → 1 – Developing → 2 – Proficient → 3 – Advanced

For example, in decision-making:

The objective is therefore not merely to assign a score. It is to determine what the student can actually demonstrate, what remains deficient, and what the student needs to do next.

The Need for Triangulation

The greater the complexity of the outcome, the greater the need for multiple sources of evidence.

A possible assessment architecture is:

The evidence can then move through:

Triangulation → Rubric → Level of Demonstration → Feedback → Re-demonstration

This is fundamentally different from allowing a single assessment instrument to determine whether a complex capability has been attained.

AI Should Create More Opportunities—Not Simply More Assessments

This leads to a broader question about the role of AI in OBE.

Should we use AI primarily to automate assessment? OR

Should we use it to create more opportunities for students to demonstrate learning?

The second is far more interesting.

One of the challenges of assessing leadership, teamwork, ethical judgement and critical thinking is scale. A faculty member cannot realistically create dozens of individual simulations for every student, observe every interaction and provide repeated feedback.

AI potentially changes that equation.

It can act as a stakeholder, colleague, customer, team member, opposing negotiator or provocateur. It can create different situations, challenge the student's reasoning, record the interaction and provide criterion-level evidence for subsequent human judgement.

But AI should not become the final authority.

AI can simulate. AI can probe. AI can analyse evidence. Humans must still judge demonstrated capability in context.

The more complex the outcome, the more important this distinction becomes.

Returning to Spady: Expanded Opportunity

This brings us back to a fundamental principle of Outcome-Based Education articulated by William Spady, namely:

“All students can be successful learners, but not necessarily at the same pace or in the same manner”.

If that principle is taken seriously, assessment cannot simply be a mechanism for declaring whether a student has succeeded or failed at one particular point in time. Students need opportunities to practise, receive feedback, improve and demonstrate again.

AI could make such expanded opportunities considerably more feasible.

A student may encounter one leadership situation today, another next week and a more complex one later. Each can generate evidence, feedback and another opportunity to demonstrate improvement.

This shifts the role of assessment from certifying a moment of performance to supporting the development of capability.

CONCLUSION

We should stop trying to measure complex outcomes such as leadership, teamwork, ethical judgement and critical thinking through a single instrument.

AI gives us an exciting new possibility: we can create situations in which students actually have to demonstrate these capabilities.

But we should also resist the temptation to allow AI to become the assessor of everything.

AI-generated evidence is not the same as proof of character.

AI can observe behaviour; it cannot directly observe character. Hence, AI evidence needs to be triangulated with faculty observation, peer assessment, self-reflection, authentic projects and real-world performance.

The more complex the outcome, the greater the need for triangulation.

The real promise of AI in OBE, therefore, is not that it can mark more papers or automate assessment.

It is that AI can create vastly more opportunities for students to demonstrate capabilities that were previously difficult to observe at scale.

Perhaps that is the real opportunity before us:

Not to automate assessment, but to make learning more visible.