SPEAKER_00 0:46

The RTO Superhero Podcast. Episode 30, Training Innovation and Alignment, where defensibility is won or lost. Welcome back to the RTO Superhero Podcast. I'm Angela Connell Richards and this is Episode 30, the sixth driver episode in our eight critical drivers to RTO Success Series. Last week in episode 29, we installed Driver 5, Systems and Innovation, the infrastructure layer underneath everything. We built the evidence control architecture, the integration gate, the data integrity stack, and the spreadsheet dependency limit. If you ran the evidence retrieval test, you now know whether your systems

Why Driver Six Matters

SPEAKER_00 1:32

produce governance grade certainty or whether you are manufacturing certainty after the fact. Today we are moving to driver six, training innovation and alignment. And I said at the end of last week's episode that this is where RTO defensibility is won or lost. Let me explain what I mean by that. Driver 6 is not about the governance pack. It is not about the risk register. It is not about the compliance calendar. It is about what actually happens between the training and assessment strategy and the issued credential. It is about what happens in the delivery room. And for most RTOs, what happens in the delivery room is drifting quietly in a direction nobody is measuring. Here is the line that defines this driver. Delivery only governs when consistency holds before it is tested. Drift in training and assessment is gradual. It is locally rational. Every individual adaptation a trainer makes to manage workload or learner need makes sense in the moment. Collectively, across cohorts and over time, those adaptations create version drift, assessment inconsistency, and portfolio misalignment that only becomes visible when scrutiny tests it. By then, the option to correct quietly has already passed. Before we dive in, your reminder that my new book, The Eight Critical Drivers to RTO Success, is available for pre-order at 8-critical drivers-book.vivacity.com.au. It releases in July and gives you the complete system behind everything we cover today, including the assessment integrity gate, the qualification level P and L framework, the skills portfolio map, and the full product retirement pathway. The companion workbook has the fillable forms, but the book is where the architecture lives. Right? Let me start with the scenario. An audit request arrives for a sample from the largest qualification on your scope. Your training manager starts pulling files. She finds three versions of the assessment tool in the shared drive. She cannot immediately confirm which version applied to which cohort. Two trainers adapted the sequencing to manage their respective workloads. Each adaptation was reasonable. Neither was documented as a TAS change. The regulator asks, did conditions remain consistent across cohorts? The answer is probably. But we cannot demonstrate it without reconstruction. The compliance manager then runs a product performance check. The qualification has a 61% completion

Audit Scenario And Version Confusion

SPEAKER_00 4:42

rate. Margin is 9%. It has been that way for three quarters. Entity level reporting showed stable revenue throughout. The product is failing, the portfolio is carrying it. And nobody connected it to driver six because driver six was never operational. Now driver six has two distinct failure modes that often exist simultaneously and reinforce each other. The first is drift. Trainers adjust delivery and assessment under operational pressure. Tools are copied to local drives. Sequencing is modified. Assessment timing shifts. Each change is explainable. None is governed. Over time, different cohorts receive materially different assessment conditions. And the organization cannot demonstrate that the principles of assessment were consistently applied. This is not misconduct. It is the structural consequence of delivery without governance. The second

Drift And Portfolio Blindness

SPEAKER_00 5:49

is portfolio blindness. Products remain on scope because they have always been there. Enrolments may be declining. Margins may be compressing. Completion rates may be falling, but entity level reporting aggregates these signals until the product is already financially damaging. By the time the qualification level problem is visible, the portfolio has been carrying the cost for quarters. Both failure modes share the same root cause. Governance that reports at the entity level, when the risk lives at the product level. This is where the flywheel either compounds or slips. When training products are governed, when validation is risk-based, when the TAS is cohort specific and version controlled. When products are assessed against margin and completion thresholds quarterly, the flywheel turns. Consistent delivery produces defensible credentials. Defensible credentials sustain employer confidence. Employer confidence supports enrolments and placements. The system compounds. When products are not governed at the qualification level, drift accumulates until it becomes a regulatory or financial event. That is the doom loop entering through driver six. There are four named models in driver six. Model one is the delivery control architecture, the governing system for how training products are designed, delivered, validated, and retired. Model two is the assessment integrity gate. A five question check before any new delivery mode, innovation or significant product change is deployed at scale. Model three is the assessment economic stack, the three metrics that tell you whether your training portfolio is a controlled asset or a compounding liability. Model four is the product retirement limit. The board approved thresholds that trigger mandatory review, redesign, or product retirement. Let me start with the delivery control architecture. Most RTOs have training products. They do not have a product governance system. The delivery control architecture, which I will call the DCA for this driver, is the system that governs how training products are designed, delivered, validated, and retired. It has six components. Component one is the skills portfolio map, a quarterly view of which qualifications are on scope, their alignment to national shortage occupations, their margin and their risk rating. This is the governance lens through which product decisions are made. Not entity level revenue. Product level

Delivery Control Architecture Basics

SPEAKER_00 8:52

performance. Component two is the TAS governance framework. Every training and assessment strategy is cohort specific, volume justified, industry reference and version controlled. The TAS is a risk control document, not a template. Component three is the assessment integrity gate. Every new delivery mode, innovation, or significant product change passes a five question gate before it is deployed at scale. Pilot before scale. Evidence before commitment. Component four is risk-based validation. Validation frequency is determined by product risk. New products, high volume products, products with complaint patterns, and products with trainer turnover are prioritized. Independence is applied where required by Outcome Standard 1.5. Component five is the assessment economic stack. Three metrics product margin ratio, completion integrity rate, and validation systemic findings rate. Tracked monthly and reported quarterly. These tell governing persons whether the training portfolio is generating sustainable outcomes or concealing structural problems. And component six is the product retirement limit. Board approved thresholds that trigger mandatory review, redesign, or retirement. Three consecutive red indicators on any product activates the retirement pathway. Not a discussion, a defined process. The DCA works because it governs the training lifecycle at the product level, not the entity level. Every component produces product specific data that flows through the same operating sequence. Skills scan, then TAS design, then assessment gate, then delivery, then validation, then product performance, then review or retire. At each stage, the threshold is defined. At each stage, the owner is named. At each stage, the evidence is produced by the system, not reconstructed for audit. When this sequence holds, drift is caught before scrutiny tests it. When it breaks, the organization explains variation after outcomes have already hardened. Now, before we get into the models, let me talk about the skills portfolio map because this is the governance lens that most RTOs are missing. Before any qualification is added to scope or retained, it should be assessed against four dimensions. Scope alignment ratio. The percentage of your qualifications link to national shortage occupations. The threshold is above 60% for green. Below forty percent triggers a portfolio review. Revenue at risk. The percentage of revenue coming from non shortage qualifications. Below twenty five percent is green. Above forty percent triggers CEO escalation.

Skills Portfolio Map That Governs

SPEAKER_00 12:08

Product margin ratio revenue minus direct cost divided by revenue at the qualification level. Above twenty five percent is green. Below ten percent triggers a review or retirement pathway. And completion integrity. Completions divided by commencements at the qualification level. Above seventy five percent is green. Below sixty percent triggers immediate intervention. These four numbers calculated for every qualification on your scope, give your governing persons a view they almost certainly do not have today. Product level performance not entity level aggregation. Installing the DCA follows the same four week pattern. Week one, build your skills portfolio map. Pull margin and completion data for each qualification. Map each to national shortage data. Calculate scope alignment ratio and revenue at risk. Week two, build your assessment integrity gate. Week three, build your assessment economic stack. Calculate product margin, completion integrity, and validation systemic findings rate for each qualification. Where are the red indicators? Week four, set your product retirement limit. Define the threshold combination that activates a retirement pathway. Get it board approved. Now let us go deep on the assessment integrity gate. The rule no new delivery mode, no innovation rollout, and no significant product change is deployed at scale until it has passed the assessment integrity gate. Here is why this matters. An RTO introduced blended online delivery without piloting it first. Completion dropped from 81% to 63%. Trainer support hours doubled. The rollout had to be paused and redesigned. The assessment integrity gate would have required an equivalence test before scale, at a fraction of the cost of the remediation. Five questions.

The Five-Question Integrity Gate

SPEAKER_00 14:25

Everyone must be answered yes before the change is deployed at scale. Question one. Has a pilot cohort been run and equivalence tested against the baseline? Before any delivery innovation goes to scale, it must be tested on a controlled pilot cohort. The pilot is not a trial run for the new model. It is a test of equivalence. Do students in the new delivery mode achieve comparable outcomes to students in the established model? The measure is the competency equivalence rate. New delivery completion rate divided by baseline completion rate times one hundred. The threshold is ninety five percent. Green is ninety five percent or above. Equivalence confirmed. Scale with monitoring. Amber is ninety to ninety four percent. Investigate before scaling. Additional pilot cycle required. Red is below ninety percent. The innovation does not scale. It returns to design. The pilot must be a true controlled comparison. Same qualification, same period. Comparable cohort. Question two. Is the TAS updated and version controlled to reflect this change? Any change to delivery mode, assessment sequencing, or volume of learning must be reflected in an updated version controlled TAS. The TAS is the governance document that justifies the delivery conditions. If the TAS does not reflect what is actually happening in delivery, the organization cannot demonstrate compliance with Outcome Standard 1.1. The version number, issue date, and change log must be current before the change is deployed. Not after the next annual TAS review. Before deployment. Question three. Has industry input been captured and linked to this change? Under Outcome Standard 1.2, industry engagement must inform training relevance. Where a delivery change is driven by industry signal, new technology in the workplace, changing assessment requirements, updated licensing obligations, that signal must be documented and traceable to the change. Gate three is where that trace is confirmed before deployment. If the change is not driven by industry signal, gate three asks a different question. Has the industry advisory body been informed and have they raised any concerns about the new model? Question four. Is trainer capability confirmed for the new delivery mode or tool? A delivery innovation that outpaces trainer capability is not an innovation. It is a quality risk. Gate four requires confirmation that every trainer delivering the new model has the capability to do so, including any digital tools, platforms or assessment methods introduced by the change. Capability confirmation is not a verbal agreement. It is documented, either as a completion record for targeted professional development or as a signed capability declaration with specific competency evidence. Question five, is there a rollback trigger defined? At what point does this change get reversed? Every deployment of a delivery change must have a defined rollback condition. What indicator at what threshold requires the change to be reversed? If completion drops more than ten percentage points below baseline after full rollout, what happens? Who decides? By when? A rollback trigger is not a sign of pessimism about the innovation. It is a governance control that makes the commitment reversible. Innovations without rollback triggers are commitments without exits. And in a regulated environment that is a governance liability. Five questions that is the assessment integrity gate. The fillable form is in the companion workbook that comes with the book. Now let us move to the assessment economic stack. Here is the question that most RTO executives cannot answer without a financial modeling exercise. Which qualification on your scope has the lowest margin and do you know what is causing it? Entity level margin hides product level failure. A strong performing qualification can carry a poor performer for years before the aggregate numbers move enough to attract attention. By then, the poor performer has consumed resources, generated complaints, and created regulatory exposure that the entity level view concealed. The assessment economic stack replaces entity

The Metrics That Reveal Truth

SPEAKER_00 19:44

level reporting with three product level metrics that tell the truth. Metric one product margin ratio. The formula is qualification revenue minus direct delivery cost divided by qualification revenue times one hundred. This tells you whether each product on your scope is financially viable or whether it is being cross-subsidized by stronger performers without anyone knowing. A margin below 10% means the qualification is consuming resources without adequate return. Below 10% for three consecutive quarters with no improvement plan is the first indicator that triggers the product retirement limit. This connects driver 6 directly to driver 7. A qualification that cannot achieve margin sustainability is a financial governance risk, not just an academic one. Metric 2 completion integrity rate. The formula is completions divided by commencements times 100 tracked per qualification. This tells you whether the training and assessment model for each product is enabling learner success, or whether structural weaknesses in TAS design, delivery or assessment are creating avoidable attrition. When this rate is below 60%, the qualification requires TAS redesign, not just additional student support. Low completion is not primarily a support problem. It is a delivery design problem. Metric three, validation systemic findings rate. The formula is systemic validation findings divided by total validation reviews times one hundred tracked per qualification. This is the most important and least commonly tracked metric in the stack. A systemic finding means that an assessment problem is not isolated to one assessor or one file. It is a pattern. Patterns are not correctable with individual remediation. They require tool redesign, TAS update, and often trainer professional development at scale. When this rate exceeds 15%, the validation system is telling you that assessment integrity is structurally compromised, regardless of what the completion rate is showing. Calculate your stack right now. Pull revenue, direct delivery costs, commencements, completions, and validation findings for each qualification for the last 12 months. Any product with two or more metrics in red, go to the product retirement limit immediately. Now let us talk about the product retirement limit because this is the model that stops your portfolio from carrying dead weight. Products remain On scope long past their usefulness because there is no defined trigger to retire them. The product retirement limit changes that. Let me give you a scenario. A metropolitan RTO had 35 qualifications on scope. They had been delivering eight business and management qualifications for nine years. Enrollments had been declining for three years. Margins were compressing. Completion rates were below 65% across four of the eight. National labor data showed

Product Retirement Triggers And Pathways

SPEAKER_00 23:22

sustained shortages in healthcare and trades. Not in business and management. Entity level revenue remained stable because the trade and healthcare products were strong. The board saw stable revenue. They did not see that eight qualifications were carrying structural losses. With a product retirement limit, four of those eight would have been flagged after the first year of decline. The retirement pathway would have activated 18 months earlier. Portfolio margin would have improved from 12% to 27%. Instead, the RTO carried the cost for three years and then made reactive decisions under pressure. The product retirement limit is a board-approved set of thresholds that, when a combination is met, activates a mandatory product review pathway. The five indicators are product margin ratio, completion integrity rate, validation systemic findings rate, skills alignment to shortage occupations, and industry relevance based on employer feedback trend. The retirement trigger rule works like this. One red indicator, monitor, assign owner, bring to next monthly executive review. Two red indicators, product review panel activated within 30 days, redesign plan required. Three red indicators, retirement pathway activated, CEO and board notified. No new enrolments in that product without a board decision to continue. When the retirement trigger fires, the product enters one of three pathways. The redesign pathway. The product has market demand, but the delivery or assessment model is failing. TAS is redesigned. Assessment tools are rebuilt. Validation is accelerated. The product returns to scope under the redesign model, which must pass the assessment integrity gate before full relaunch. The wind down pathway. The product has declining market demand or is misaligned with national shortage data. Existing cohorts are completed with full support. No new enrolments are accepted after the trigger date. The product is formally retired from scope with a documented decision record. Or the strategic hold pathway. The product has strategic value that outweighs current performance. For example, it is required to maintain a specific employer relationship or funding contract. The board formally accepts the performance shortfall with documented awareness of the risk, a defined review timeline, and a performance improvement plan with measurable milestones. Now, let me share what high performers do with these models. Serena Russo Group treats portfolio alignment as an ongoing governance obligation. Products are assessed against labour market demand as part of the normal planning cycle, not when enrolments have already fallen. Qualification level performance data is visible at governance level, and products that fall below defined thresholds enter formal review. Lifetime training, when they went through their quality challenge, the recovery centered on driver six controls. Assessment tools were redesigned. Validation was accelerated for high risk products. Independent validators were brought in where independence was required. Crucially, the validation findings were linked directly to trainer performance reviews, removing the disconnect between assessment quality signals and workforce decisions. UTI tracks program return on investment directly to graduate employment metrics. A program is not assessed as successful because students completed it. It is assessed as successful because graduates are employed in roles that the program prepared them for. This makes curriculum relevance a financial governance metric. And CNI designs training products against industry task clusters, not against qualification units alone. The delivery structure is justified by the complexity of the occupational competency. When industry practice changes, the product changes. The evidence chain from industry input to curriculum update is built into the product governance cycle. What all four have in common? Training products were governed at the qualification level, not obscured by entity level aggregation. Validation was risk-based and connected to redesign. Delivery innovations were piloted and equivalents tested before scale. Products were retired when performance thresholds were breached. And assessment integrity was treated as a governance obligation, not a training department responsibility. Now, the key thresholds and escalation protocol. Product margin ratio. Green is above 25%. Amber is 10 to 25%. Red is below 10% for two or more consecutive quarters. At red, a product review panel is activated within five business days. Completion integrity rate. Green is above 75%. Amber is 60 to 74%. Red is below 60% for two or more consecutive quarters. At red, TAS redesign is initiated and the product retirement limit trigger is assessed. Validation systemic findings rate. Green

Cadences Thresholds And Escalation

SPEAKER_00 29:26

is below 10%. Amber is 10 to 15%. Red is above 15%. At red, an assessment system review is triggered within five business days. An independent validator is appointed and the board is notified. Validation closure time. Amber is 30 to 60 days. Red is above 60 days. TAS version currency rate. Green is 100%. Amber is a gap with a plan active. Red is below 90% or no plan. Reassessment burden rate. Green is below 12% of total delivery hours. Amber is 12 to 20%. Red is above 20%, which signals that assessment tools or conditions are creating unnecessary failure and rework. And the critical trigger, three red indicators on a single product. This breaches the product retirement limit. The retirement pathway is activated. The CEO and board are notified. A board decision is required within ten business days. No new enrolments in that product without the board actively deciding to continue. The execution rhythm for driver six follows the same three cadences. The weekly operational review. Any validation findings requiring action. Any assessment integrity gate applications for proposed delivery changes. The monthly executive review, CEO chairs it, assessment economics, stack on the table, per qualification, product margin, completion integrity, and validation findings for every active product. Any products approaching retirement trigger thresholds. Reassessment burden rate. The quarterly strategic review. Product retirement limit assessment across the entire portfolio. Industry relevance check are employer feedback trends, positive, neutral, or negative by product. Validation schedule integrity is the risk-based validation plan on track. Under the revised standards, governing persons must demonstrate that training and assessment was governed at the product level, not just reported at the entity level. The DCA satisfies that test. The assessment integrity gate is evidence that delivery changes were governed before they were deployed. The assessment economics stack is evidence that product performance was visible to governing persons continuously. The product retirement limit is evidence that underperforming products were identified, reviewed, and acted upon through a defined governance process. So here is your action step for this week. Three things all doable before episode 31. Action one, run a qualification level P and L on your top five products by enrollment. Revenue minus direct delivery cost divided by revenue for each one. If any qualification has a margin below 10%, you have a product that needs governance attention this week. If it has been below 10% for two or more quarters, the product retirement limit has been triggered. Action two.

Three Actions Before Next Week

SPEAKER_00 33:50

Action three, check your validation systemic findings rate for each qualification validated in the last 24 months. If any qualification has a systemic findings rate above 15%, assessment integrity is structurally compromised in that product, and the escalation protocol should be activated. And if you want the complete model, the full qualification level P and L framework, the skills portfolio map template, the equivalence testing methodology, the product retirement pathways, and the ninety day implementation plan. The book gives you everything. Pre-order the eight critical drivers to RTO. Success at eight-critical drivers dash book.vervasity.com.au It releases in July. Three actions, all doable before next week. Do them. Next week, in episode thirty one, we move to driver seven. Financial sustainability and growth. I am going to walk you through the viability control architecture, the investment gate, the margin economic stack, and the runway limit. We are going to talk about what happens when stable revenue figures conceal completion slippage, margin compression, and cash runway erosion. And I am going to give you the single most important financial governance tool for an RTO, the 13-week rolling cash forecast. That is next week. For now, go run your qualification level P and L. Go check your assessment tool version currency. And go audit your validation systemic findings. The system does not need to be perfect before you start running it. It needs to start running. I will see you next week. You have been listening to the RTO superhero podcast with Angela Connell Richards. If this episode was useful, share it with another RTO leader who needs to hear it. Pre order the book at 8-critical dash drivers dash book.au or find us at vivacity.com.au and comply hub.ai.