Contact-reason analysis for product fixes
Classifying every ticket by the reason the customer had to contact support, such as a defect, a confusing setting, a missing feature, or a billing question, and sending the largest and fastest-growing reasons to the product and engineering teams that can remove them.
Supporting approachTechnique: Language Model Text Classification
A classifier reads each ticket and conversation and assigns the reason the customer had to ask, distinguishing a defect from a confusing setting, a missing feature, or a billing question.
Every month the support lead sends product and engineering the largest and fastest-growing reasons, with the number of customers and plans affected and links to example tickets.
When a reason is fixed in the product or the documentation, the same report shows whether its tickets fell.
This playbook’s view is that a fine-tuned language model earns its place here only when agents’ manual tags are too inconsistent to use, and that a small team can do the same work by having agents choose from a short reason list on every ticket.
This is the wrong choice where product and engineering will not act on support data, since the report then changes nothing.
Strengths for this problem
- Classifies the free text of every ticket and conversation, where agents' manual tags are inconsistent or missing.
- Turns support volume into a ranked list of product problems, with the number of customers and plans affected.
- Is the only option that reduces the number of tickets rather than the time spent on each one.
Limitations for this problem
- Needs a set of contact reasons that product and support teams agree on, and labelled examples of each to train or tune the classifier.
- Its value depends entirely on product and engineering acting on the report, which is an organisational problem, not a modelling one.
- Drifts as the product and customers' language change, so the reason list and the classifier need regular review.
Data needed
- Ticket and chat text with product area, plan, product version, and resolution.
- A list of contact reasons agreed by support and product leads, with a labelled sample of tickets per reason from experienced agents, large enough to train or tune the classifier.
- Known defects, incidents, and release dates, to link reasons to the change that caused them.
How it is measured
- Tickets per active customer for each top contact reason, before and after a product fix.
- Share of tickets the classifier assigns a reason with confidence, and agreement with experienced agents on a reviewed sample.
- Number of top contact reasons with a product owner and a planned fix.
Pitfalls
- Producing a monthly report nobody owns, so the same reasons top the list every month.
- Letting the reason list grow until each reason has too few tickets to matter.
- Classifying by the feature named rather than by why the customer had to ask, which hides whether the cause is a defect, the design, or the documentation.
Human oversight
Experienced agents label the training examples and review a sample of classified tickets each month. The head of support and product managers agree the reason list and decide which causes to fix. Product managers own each top reason and report back when it has been addressed.