Science & Tech · Big Tech & Internet · 2 hrs ago

HyperLogLog estimates unique visitors using far less memory

HyperLogLog estimates unique visitors using far less memory

Counting page views is easy, but counting unique visitors is harder because one person may open a page more than once.

A precise count requires a website to keep and check a list of visitor IDs, which can grow very large across many pages and ads.

HyperLogLog estimates the count instead of tracking every ID, using much less memory.

Its central idea is compared with flipping coins and recording the longest run of tails before heads appears.

Because long runs are rare, that record can give a rough idea of how many rounds took place; the algorithm applies a similar idea to visitors.

Google Analytics uses an augmented version of HyperLogLog to estimate visitor metrics, and the method is also used in genomics and cybersecurity.

The trade-off is that the result is an estimate rather than an exact count.

1 source

Related

What people are saying

Scientific American article narratorArticle narrator
It settles for close estimates instead of exact counts, and in exchange, it uses only a fraction of the memory required by full lists of IDs.
via Scientific American
Google Analytics, which tracks traffic on nearly half of all websites, relies on an augmented version of it to estimate visitor metrics.
via Scientific American
A streak of 10 tails turns up only about once in every 1,000 rounds.
via Scientific American