LogloupeComing soon

Find out why your site got hammered

Logloupe reads your web server’s access logs and shows you when the traffic spiked, who sent it, which bots were faking their names and what they were stuck on. Then it writes a block list that leaves the real search engines alone.

  • Free and open source
  • Runs on your computer
  • Desktop app and command line
A pagination loop, and the bots following it. Each one checked against its operator’s published addresses.Invented example

Example log: requests for /catalogue/ with page= appended again and again. Most come from addresses claiming to be Googlebot that are not in Google’s published list, so Logloupe marks them fake. A request from 66.249.66.1, which is in Google’s list, is marked verified.

See when it happened, and who sent it

A list of top IP addresses tells you who was busy across the whole log. It doesn’t tell you what happened between 13:21 and 13:25. Logloupe finds the minutes when one address, one network, one hosting provider or the whole site ran far above its normal rate, and groups them into incidents with a start, an end, a peak and the paths that were hit.

From any incident you can open the individual requests behind it.

Incident: 12 November, 13:21–13:25 UTCExample log
13:05requests per minute, whole site13:39
LevelTargetPeak / min
Whole siteall traffic1,078
└ Network198.51.100.0/24980
Address198.51.100.30100
Address198.51.100.2398
Address198.51.100.2297
Address198.51.100.2492

Top path for the incident: /catalogue/, 4,020 requests. Incident figures from the app’s example log; the per-minute chart is an illustration. All traffic is invented.

Then find out what they were stuck on

Logloupe groups requests by the shape of their query string, and the cause often jumps out. Here it’s a pagination link that appends ?page= to itself, so every crawler that follows it finds an endless supply of “new” pages, and the server builds each one.

Response codes over time show the cost, with the URLs behind the errors. Fix the link and the loop goes away for everyone, polite bots included.

Query strings by shapeExample log, 8–14 November
PathQuery shapeRequestsAddressesErrors
/products/page=…&sort=…16,8362,361710
/catalogue/page=…1,02212411
/catalogue/page=…?page=…99512408
/catalogue/page=…?page=…?page=…98312362
/catalogue/page=…?page=…?page=…?page=…1,02012393

The same 12 addresses keep requesting /catalogue/ with one more ?page= each time. A normal page with sorting, above, comes from thousands of addresses. Invented traffic.

Know which bots are real before you block anything

Anyone can write “Googlebot” in a user agent, and plenty of scrapers do. Block a fake one and nothing is lost. Block the real one and your pages start dropping out of search.

So Logloupe checks every claim against the IP ranges the operator publishes, and by forward-confirmed reverse DNS where they publish none, then marks the request verified, fake or unverifiable. A bot that can’t be checked either way is marked unverifiable, not guessed. Old logs are checked against the lists as they were at the time.

Checked today

Published IP ranges for Google (including its special-purpose and user-triggered crawlers), Bing, OpenAI, Anthropic, Perplexity, Apple and Common Crawl; announced network routes for Meta; forward-confirmed reverse DNS for Amazon, Yandex and Baidu. Reverse DNS is optional, because it sends addresses to your own DNS resolver.

Bots in an example log, 8–14 November11 bots · 6,853 requests from fakes
BotRequestsVerifiedFakeUnverifiable
GooglebotGoogle42,99838,9784,0200
BingbotMicrosoft Bing32,73432,73400
GPTBotOpenAI28,08826,6511,4370
OAI-SearchBotOpenAI5,8895,88900
ChatGPT-UserOpenAI3,7473,74700
ClaudeBotAnthropic22,80421,4081,3960
Claude-UserAnthropic1,8281,82800
Meta-ExternalAgentMeta70970900
AmazonbotAmazon16,65616,65600
YandexBotYandex12,51012,51000
AhrefsBotAhrefs8,982008,982

An illustration with invented traffic, not a real site’s results. Amazonbot and YandexBot are shown as they appear when reverse DNS is switched on. Bots whose operators publish neither an IP list nor a reverse DNS domain, such as most SEO tools, are shown as unverifiable rather than guessed.

Then write a block list that leaves the real ones alone

Tick the addresses or networks and Logloupe writes them out as a plain list, nginx deny lines or Apache rules, ready to copy. Verified bots and your own server’s addresses are always left out.

It doesn’t apply anything. Busy is not the same as hostile, so you read the list and decide.

Fake bots, by addressExample log · try it
Add to block listAddressClaims to be
203.0.113.8Googlebot Fake
203.0.113.4Googlebot Fake
203.0.113.5Googlebot Fake
203.0.113.2Googlebot Fake
203.0.113.3Googlebot Fake
203.0.113.1Googlebot Fake
198.51.100.23GPTBot Fake
198.51.100.30ClaudeBot Fake
66.249.66.1Googlebot Verified

5 entries. Read it before using it.

deny 203.0.113.2;
deny 203.0.113.3;
deny 203.0.113.4;
deny 203.0.113.5;
deny 203.0.113.8;

Including the AI crawlers

AI crawlers can fetch a lot of pages, and their names are popular to fake. Logloupe shows each operator’s crawlers separately (training, AI search, and fetches made when someone asks an assistant about your page), with the pages they fetched and how many of their requests were real.

Visits that arrive from ChatGPT and other assistants are counted too.

AI crawler requests by operatorTrainingAI searchSomeone asked an assistant
  • OpenAIGPTBot, OAI-SearchBot, ChatGPT-User
    37,724 requests
    1,437 fakesusing its name
  • AnthropicClaudeBot, Claude-User
    24,632 requests
    1,396 fakesusing its name
  • AmazonAmazonbot
    16,656 requests
    no fakesusing its name
  • MetaMeta-ExternalAgent
    709 requests
    no fakesusing its name

Logloupe also lists the pages each crawler fetched. Invented traffic from the app’s example log.

The rest of the log, too

Busiest addresses and networks
Top addresses, networks, hosting providers and countries, for the whole log or any time window.
Every request behind every number
Open the individual log lines behind any filtered view.
Ordinary traffic
Requests and pages over time, sources, UTM campaigns, landing pages and referrers, straight from the log.
Command line
The same engine as a command-line tool with JSON output, for servers and scripts.
Large files
Logs are read as a stream rather than loaded whole, so big files are fine.

How it works

  1. 1

    Get your access log

    Download it from your server. Logloupe reads the standard format nginx and Apache write by default, and more formats are coming.

  2. 2

    Open it in Logloupe

    Everything runs on your computer and nothing is uploaded. It fetches the bots’ published address lists and an optional location database when you ask, and reverse DNS checks, if you switch them on, go through your own resolver.

  3. 3

    Find the cause

    Start from the burst, follow it to the network, the paths and the bots, and copy a block list if you need one.

The Logloupe desktop app showing the Bursts report: a sidebar of reports grouped into Traffic, Bots, Abuse and Manage, and a table of incidents on 12 November.
The desktop app, on Windows. Generated log, invented traffic. Scroll sideways to see all of it.

Coming soon

Logloupe is being tested now. It will be free and open source under the MIT licence: a desktop app for Windows first, with macOS and Linux to follow, plus the command-line tool.

Good to know

  • It writes block lists but doesn’t block anything itself.
  • A log records requests, not people, so it isn’t a replacement for analytics.
  • Behind a caching CDN, the log only shows requests that reached your server.
  • Location and network data is approximate.