Outsyd Sentinel
Machine learning that sends each traveller review to the right team — and an honest look at a perfect score.
100%
accuracy on the test set — and why I don’t trust it.
01 — The idea
Every complaint has an owner. The trick is getting it to them first.
Hotels and travel apps collect a constant stream of free-text feedback, and most of it lands in one inbox. Sentinel reads each review and makes two calls: which department should handle it — front desk, food & dining, services & Wi‑Fi or location — and whether the guest is happy. The person who can fix a cold shower sees the complaint about it, instead of everyone or no one.
02 — How it works
- Data800 balanced, synthetic reviews in the same shape as Outsyd’s comments — id, user, text, timestamp. No personal data.
- PreprocessingLowercasing, punctuation stripped, a bag of words built with
CountVectorizer. - Two modelsLogistic regression for the department (4 classes) and for sentiment (2), trained on 80%, tested on the other 20%.
- DashboardStreamlit: analyse one review with a confidence score, batch-process a CSV, inspect the metrics.
Why not a neural network? 800 short sentences don’t need one. Logistic regression trains in milliseconds, its weights can be read and explained, and a small team can retrain it on a laptop — for a business process, that matters more than a few points of accuracy.
03 — Try it
This is the trained model, running in your browser.
I exported the vocabulary and the regression weights from the scikit-learn pipeline into a 6 KB JSON file. The page tokenises your text exactly like CountVectorizer does and does the maths in JavaScript — the numbers match Python’s to the fourth decimal. The model learned Italian, so write in Italian or start from an example.
The last three are there to break it.
—
—
04 — The 100% problem
Both models scored 100% on the held-out test set. That should make you suspicious.
The test set came from the same generator as the training set: the same 86-word vocabulary, the same sentence shapes. Separating them perfectly proves the pipeline works end to end — not that it understands reviews. Give it a sentence the generator never wrote and the sentiment confidence slides towards a coin flip: “the sea view was stunning” comes out slightly negative, because it has never seen the word for stunning.
What it would take to deploy for real: labelled reviews from actual guests; TF-IDF with word pairs, so “non funzionava” reads as one negative idea; a multilingual transformer if the budget allows; and a Cloud Function that labels each comment before it is written to the database.
Department — predicted vs actual, 160 test reviews
Sentiment — predicted vs actual
05 — Fit with Outsyd
Sentinel was designed around data Outsyd already has. The app’s comments are stored with the same fields the generator produces, so the classifier can sit in a Cloud Function between the user and the database and enrich every comment with a department and a sentiment before it’s saved.
The thesis covers the full journey — requirements, an iterative build, the evaluation and its limits — and ties each step to the courses behind it: probability and statistics, algorithms, databases, and organisation and marketing.