Inspection targeting model
A model trained on the utility's own field inspection results that ranks premises by their likelihood of an irregularity, so each inspection campaign visits the premises most likely to have theft or a faulty meter, with a share of random visits to correct for where inspectors used to go.
Supporting approachTechnique: Supervised Learning
The model scores each premises on the data available before an inspection, such as its consumption history, meter events, customer class, and location, and ranks premises for the next campaign.
It is trained on the outcome of every past inspection, including those that found nothing, and evaluated by the hit rate inspectors achieve with it against random visits in the same campaign.
Endesa developed such a model on smart meter data for all its industrial and commercial customers, trained and tested on about 57,000 inspections, and found gradient boosted trees the strongest of the classifiers it tried.
A study of a Brazilian utility’s 820,000 inspection results shows the risk: past inspections clustered in certain neighbourhoods and customer classes, so a model trained on them is least reliable for everyone else, which is why a share of every campaign should be random.
This is the wrong starting point for a utility without a large, recorded inspection history, and the wrong tool for illegal connections that never pass through a meter.
Strengths for this problem
- Combines consumption history, meter events, customer class, location, and past inspections into one ranked list per campaign.
- Learns the utility's own patterns of theft and faults from its own inspection outcomes, as Endesa's model did from about 57,000 inspections of industrial and commercial customers.
- Sized to the number of inspections a campaign can do, it raises the hit rate of the visits the utility can afford.
Limitations for this problem
- Learns from premises that were inspected, which past campaigns concentrated in certain neighbourhoods and customer classes, so it is least reliable for the rest.
- Cannot see theft that leaves no trace in the premises' own data, such as an illegal connection with no meter.
- Degrades as customers, meters, and theft methods change, and the cause of the decline is hard to see without explanations.
- Mixes theft, faulty meters, and billing errors unless findings are recorded separately.
Data needed
- Every inspection, including those that found nothing, with the date, the reason the premises was chosen, and the finding split into theft, faulty meter, and billing error.
- Consumption history and meter events for each premises up to the date it was inspected, and nothing recorded afterwards.
- Customer class, activity, contracted capacity, tariff, location, and connectivity to transformer and feeder.
- Results from random inspections, to measure and correct the bias in targeted ones.
How it is measured
- Hit rate of model-ranked inspections against random inspections in the same campaign.
- Energy and revenue recovered per inspection, split into theft, faulty meters, and billing errors.
- Hit rate on premises in neighbourhoods and customer classes that were rarely inspected before the model.
- Share of findings upheld when the customer disputes them.
Pitfalls
- Training only on targeted inspections, so the model ranks premises by how much they resemble where inspectors used to go.
- Using data recorded after the inspection, such as a corrected bill, as a feature, which looks excellent offline and fails in a live campaign.
- Measuring the model by accuracy on historical data rather than by hit rate in campaigns against a random sample.
- Running the model for years without anyone looking at why premises are ranked, which the team behind a long-running system at a Spanish utility says is how shift and bias go unnoticed.
Human oversight
A revenue protection analyst reviews each campaign's list and the explanation for the top-ranked premises before inspectors are sent. Field inspectors confirm every finding on site. Revenue protection managers decide back-billing, disconnection, and referral to the police, and take account of vulnerability and safety. The model never triggers a disconnection or a bill on its own.