Major upgrade to Gooroo Planner – advanced forecasting

Major upgrade to Gooroo Planner – advanced forecasting

08/12/2020
by Rob Findlay

We have today released a major upgrade to the statistical power of Gooroo Planner, raising it from a traditional ‘projection’ model based on averages, into an advanced ‘forecasting’ model that is responsive to recent changes in demand and performance. This will be critical during the post-covid recovery as demand and performance shift to new levels.

Other enhancements include:

Downloading week-by-week planning profiles for all services at once (previously this was only available via the Planner API).
Automatic measurement of trend demand growth, using statistical tests to filter out spurious trends.
Uploading literal week-by-week activity plans, and automatically basing the future activity plan on that.
New patient scheduling simulator that works in the browser (not Flash, which is being retired), with clear designer graphics.
Ability to cancel and delete stalled uploads.
We are also ready for average waiting time targets, which will be useful if these ill-advised targets become a reality in England.

Advanced forecasting

We have been working since early 2019 with a Professor of Statistics at the London School of Economics, to test a broad panel of advanced forecasting techniques against millions of rows of detailed NHS demand data going back 15 years. So now we know which techniques show the greatest forecasting skill, and in which situations, and we have built those techniques into the heart of Gooroo Planner.

I will go through the details of the new forecasting methods in a separate post and research paper, and the documentation and standard SQL queries have already been updated. The online wizards for data upload and report creation will guide you through the process.

There are three things that you may want to think about now, if you want to use the new forecasting method. (You always have the option of sticking with the existing ‘average’ method if you prefer.)

Firstly, if you want to use the new forecasting method then you will need to be uploading patient-level activity and additions data (as opposed to statistical data), because the forecasting involves the construction and analysis of time series data which only the patient-level uploads can provide.

Secondly, you will want to make sure that your past period doesn’t end at an unusual point in time (e.g. the Christmas & New Year holiday at the end of December), because the forecasting model will respond to the latest data when forecasting the level of demand. Similarly you will need to avoid artefacts such as the data tailing off (e.g. because the activity date is based on the admission date, but the data is filtered to include only patients who have been discharged and not those still in hospital).

Thirdly, you may want to increase the duration of historical activity and additions data you upload to Gooroo Planner. It is best to provide at least five years of historical data to measure seasonality properly (and a minimum of three years). If you can’t get that much data, then don’t worry because the forecasting method can switch to a non-seasonal approach if the data is shorter; our research shows that this also performs well, and – because it is responsive to recent changes in the level of demand – it should outperform the traditional ‘projection’ method that the classic ‘average’ method relies on.

Ready to give it a go? Just follow the prompts in the wizard when you upload your dataset and create your report on-screen. Or, if you’re using a Goorooconfig.xml file to automate it, see p.50 of the latest documentation for the new <PlanMethod> tag (and p.52 for the optional <SetSimpleExpFcastUpToWks> and <SetMinTimeSeriesWksHW> tags).

Downloading all the week-by-week planning profiles

You are already accustomed to downloading week-by-week (and month-by-month) planning profiles, for one service at a time, from the Profiling tab in Planner Professional. But until now, the only way to get those profiles for all services at once was by using the Planner API. The reason was that Planner Professional was designed to be interactive, so if you edited a Report, then any pre-calculated profiles would be out of date. That problem does not arise with the Planner API.

When we created the Planner API, we thought it would mainly be used to automate the planning process and integrate it with business intelligence. In practice, however, many people have been using the Planner API interactively just to get all the planning profiles at once. So clearly this is something very useful that should be easier.

For the new Planner Horizon licence we have therefore combined the web-based interactivity of Planner Professional (which it upgrades) with the comprehensive all-services profiling of the Planner API. In the Report Manager screen, when you click the “Download Report” icon beside a report, you will now be offered a menu of things to include in the download package, and one of the items on the list is the weekly and monthly profiles. If you select this option, those profiles will be calculated afresh to allow for any edits to the report since it was created, and included in the download package.

Measuring trend demand growth automatically

We are often asked whether Gooroo Planner can measure trends in demand growth automatically, and until now the answer has been ‘no’ for the simple reason that most people are only uploading one year of historical data, which is far too short a period to sensibly measure trend growth over.

The introduction of advanced forecasting will change all that, and historical data covering longer than 3 years will become more common. So we have introduced the automatic measurement of trend demand growth as part of the ‘forecasting’ method.

Measuring growth is not as simple as just fitting a trend line. There is randomness in the data, and this can create the appearance of a trend where none really exists. Even worse, this randomness will cause a naively-calculated trend to change every time the data is refreshed, leading to noticeable changes in the results and a lack of confidence in them.

So the method we have used is to fit an exponential trend line to the data, and then calculate the probability that the trend is an illusion caused by random variation. If that probability is small then it was unlikely to have arisen by chance and the measured trend is accepted. We decided to go with a relatively strict test (p < 0.01) to reduce the number of false trends being measured, bearing in mind that a typical whole-hospital Gooroo Planner model may contain hundreds of services.

In forecasting mode, Gooroo Planner will now automatically attempt to measure the trend growth in demand, and store the result in the Growth field. The calculation settings will still cap Growth at reasonable levels by default, and you can of course control how that happens. And if you want to override the measured Growth and always set it to zero (or some other value) then you can do that by resetting Growth in your high-level statistical assumptions table.

Uploading literal week-by-week activity plans

A year ago we released operational profiling, which allows managers to use the Profiling tab to edit the literal number of patients they plan to see or treat in each future week.

This has been well received. But once you have created a literal week-by-week activity plan, it hasn’t been quite as easy to use it again the next time you refresh the data. The reason is that the total amount of activity done in the future period was defined by the ‘future activity’ field FutActiv, not the week-by-week activity you were uploading.

The upgrade resolves this issue. There is a new special value of FutActiv (-1) which automatically calculates FutActiv by adding up the relevant week-by-week activity in the fields FutActivWkProfile01-53. So FutActiv can be driven by the weeks and part-weeks in your future period, and the week-by-week activity you have specified, giving you a more exact way to manage operational planning every time you refresh your data.

Average waiting time targets

NHS England and NHS Improvement have been piloting the use of average waiting time targets. Not only would this be pointless and a backward step for patients, but it adds a lot of complexity when it comes to demand and capacity planning.

So we have done the hard work for you (it required calculus), and average waiting time targets are now included in Gooroo Planner.

If you set the percentage within target (TgtMaxWaitPC) to the new special value of -1, then Gooroo Planner will interpret your waiting time target (TgtMaxWait) as a mean average waiting time target. You will probably want to set the target type (TgtMaxWaitType) to ‘s’ for snapshot, because that is how NHS England and NHS Improvement have been trialling this target.

Then all the waiting time results calculated by Gooroo Planner will be on an average basis, including waiting times and the estimated number of long waiters beyond that average wait.

And this works for all activity scenarios, whether you are calculating the activity and capacity required to achieve the target, or working out the consequences of some other amount of activity.

The new Planner Horizon licence

Both advanced forecasting, and the ability to download profiles for all services at once, are major enhancements which will be available in the new Planner Horizon licence. But don’t worry: we are giving all existing Planner Professional licence holders a free upgrade to the new Planner Horizon licence until at least your next renewal date, delivering all the benefits of this upgrade immediately at no extra cost.

Information and support

If you have any questions then we are here to help. Please contact us at support@gooroo.co.uk.

Return to Post Index