What a review count cannot tell you

Review count measures exposure, not craft

A Google review count is a running total of how many customers were both asked and willing. Three things drive it, and none of them is workmanship. The first is age: a company that has been trading for two decades has had two decades of customers to ask. The second is volume: a company running eight crews across three counties generates ten times the opportunities of a company running one. The third is process: some companies have a person whose actual job is asking for reviews, with a text message that goes out the afternoon the dumpster leaves, and some companies have nobody doing that at all.

Put those together and the ranking becomes readable. The top of the ledger is a list of the oldest, largest and most systematic. That is genuinely useful information. It is simply not the information most people think they are reading.

The average is worse than the count

Star averages compress badly. A company with nine reviews and a five star average and a company with four hundred reviews and a 4.8 average are not comparable quantities, and the smaller number is the less trustworthy one in almost every case. Nine reviews is a handful of friendly customers. Four hundred reviews at 4.8 means roughly twenty people were unhappy enough to say so in public, and their words are sitting there for you to read.

Across the whole ledger the median star average is 4.9. When almost every company in a market scores 4.9, the score has stopped distinguishing anything. It has become a floor: below about 4.5, something is genuinely wrong, and above it the number is noise. The interesting information moved into the text of the reviews years ago.

Five star reviews tell you a job went fine. Three star reviews tell you what a company does when a job does not.The reading rule for this ledger

How to read reviews once you stop reading the score

• Sort by newest, not by relevanceCrews change. Owners sell. A company's reviews from four years ago describe a company that may no longer exist in the same form. The most recent twenty are the only ones describing the business you would actually hire.
• Read the three and four star reviews firstThese are written by people who were not angry enough to be unfair and not delighted enough to be vague. They are the most accurate paragraphs on the internet about any contractor.
• Look for the owner repliesNot whether they exist, but what they say. A reply that argues with the customer about the facts tells you exactly what a dispute with that company feels like. A reply that names the manager who will call tells you something else.
• Watch for clusteringForty reviews in one week and nothing for eight months is a pattern worth noticing. So is a run of reviews from accounts with one review each and no photographs.
• Search the reviews for the words that matterDeck, decking, leak, callback, warranty, gutter, cleanup, nail, and the name of the person who sold the job. If a company has a recurring problem, it will be in those words in more than one review.

The things no review platform captures at all

There is a category of information that decides whether a reroof goes well and that never appears in any review, because the customer never learns it. Whether the crew are employees carrying the company's workers compensation or a subcontracted crew hired that morning. Whether the person who sold you the job sets foot on the property while it is being done. What the company does when it opens the roof and finds the decking has gone soft, which is a question with two answers: stop and call you with a number that was already in the contract, or keep going and produce a change order afterwards.

None of that is in a star rating and none of it is in a count. All of it is answerable in a fifteen minute conversation before you sign, which is why this desk spends more pages on the conversation than on the ranking.

Why run the ranking at all, then

Because it is a shortlist generator and it is honest about being one. Two hundred and twenty two rated roofing contractors is an unusable number of options. Sorted by review count, the ledger gives you a manageable set of companies that have been around long enough and worked enough houses that there is a body of public evidence to read. Then you go and read it, and then you ask the questions that the evidence cannot answer.

What would be dishonest is running the same list under a headline that says best, with a badge graphic, no methodology and no pull date. That page exists in this market several times over. This one is the same data with the arguments against it printed alongside.