Product · Tools
Product · Intel loops · recurring, in build
Solutions · By problem
Solutions · Online stores
Resources · Learn
Resources · Guides and ideas
Resources · Company
Name the rival sites that matter. On each run the monitor will read their sitemaps, the prices they publish in structured data and the offers on their homepages, then list what is new, what has gone and what changed since last time.
In build · not on sale The engine for this loop is built and tested on a development branch. It does not run for customers yet and is not on sale.
What changed since the last run
A loop that re-reads the rival websites you choose and reports what changed since the last run: pages added or removed, published prices and on-page offers.
You note a rival's prices and offers once. The note is out of date the next time they run a sale or launch a range. The website monitor repeats the same read on a cadence you set and compares each run with the last good one. You see what changed.
It reads three things. Pages: each rival's sitemap, found through the Sitemap lines in robots.txt and the usual sitemap addresses, including nested sitemap indexes. Prices: the schema.org Product and Offer data on the homepage and on a handful of product pages picked from the sitemap in a stable order, so the same pages are compared each time. Offers: promotional language on the homepage, with guards against the obvious false positives, so that "free quote" is not counted as a promotion and a banner repeated three times counts once.
It does not report what it did not see. Every run records which areas it actually reached for each rival. A page only counts as removed if this run read that rival's sitemap in full; a price only counts as changed if the product page was readable both times. When a read is incomplete the change is marked "not checked" with the reason, instead of being reported as a launch or a withdrawal that never happened.
This loop is in build. The engine, the comparison and the internal interface are written and tested on a development branch, but the database tables it needs are not applied in production and the scheduled job is switched off. Nothing on this page runs for a customer today, and it is not on sale.
Reads each rival's sitemap and compares the list of addresses with the last complete read. New service pages, new product ranges and retired pages show up as changes.
Reads the prices rivals publish in their product structured data, on the homepage and on a small, stable set of product pages, and reports which ones moved.
Detects offer language on the homepage (discounts, free delivery, bundles) and reports when a promotion starts, stops or changes.
Each run is recorded as observed, partial, empty or failed, with the reason. A failed run is never used as the baseline, so a bad read cannot make everything look new.
A history of what each competitor's site said on every run, so you can go back and see when a price or offer changed.
Per rival, the pages added and removed, prices changed and offers started or stopped since the last good run.
The plan is a digest by email on your cadence, built in the app that will host the loops. That part is not built yet.
When a rival's site refuses a read or a sitemap is partly unreadable, the run says so, so you know which gaps are ours and which are theirs.
Add the websites you want watched, and choose pages, prices, offers or all three.
Choose how often the read repeats, in whole days. The first run records a baseline.
Each later run is compared with the last good one and the changes are listed per rival.
The website monitor is being built. When it runs in production the pricing page will say which plans include it; until then nothing is charged and nothing is promised.
No. The website monitor is in build and does not run for customers. Join the loops list and we will email you when it does.
It reads robots.txt first and obeys it, spaces out requests to each site, identifies itself by name, and only retries on server errors or rate limits. A site that refuses the read is reported as not reached.
The run is recorded as partial or failed, with the reason. A partial read never produces a false "removed": a page only counts as gone if that rival's sitemap was read in full both times.
The report is a one-off read of how your site compares with rivals today. The monitor repeats a narrower read of the rivals themselves and reports the changes between runs.
Nothing is priced yet. When it runs in production, the pricing page will say which plans include it.
The engine for this loop is built and tested on a development branch. It does not run for customers yet and is not on sale.