TMCnet News

The Web Spiders That Help You Save Money
[June 28, 2016]

The Web Spiders That Help You Save Money


OXFORD, England, June 28, 2016 /PRNewswire/ --

A new website is helping Brits score the best deals on the high street with an automated algorithm that actively tracks down rock bottom prices. Written and developed over the course of eight months, Everyday is Black Friday sees a trio of 'spiders' actively scan the internet for price drops on over 250 major retailer websites. The software currently processes over 2 million products every day, sourced from prominent UK retailers including Argos, Currys, John Lewis and many more.

A three step 'spider' process powers the core of the software. Spider 1 extracts all links from the retailer website. Spider 2 then sorts the links, checks whether the URL is a product page or not, then puts the product pages in a queue. Spider 3 checks the price, and continues to re-visit the produc every 24 hours. If it finds that today's price is lower than yesterdays, it displays that product on the front end of the website as a discount. Deals are pushed down as new ones arrive, which means browsers always receive up-to-the-minute offers. The entire process is autonomous, with no human intervention.



"Thanks to the incredible search power of our spider bots, Brits can shop like every day is Black Friday," says Ferencz Kaszoni, founder of Everyday is Black Friday

Every product is accompanied by a 30 day price history chart. This innovative feature allows shoppers to determine if the deal is real, or if the price has been artificially inflated for a few days, then dropped back and labelled a "discount." As well as finding deals users can also use the 'All Products' button to compare prices. This allows them to choose the best retailer based on location, loyalty and of course, price.


With the current power and resources, the spiders can execute more than 1600 price checks/minute. As retailer friendly bots they only visit two or three pages per 60 seconds per website. This 'crawler politeness' ensures that retailer servers won't undergo strain, and shoppers can enjoy lighting fast load times.

In 2016 Everyday is Black Friday will be expanding its reach, with a target of 100 million products per day. The site will undergo significant improvement, with further improvements to functions, features and usability. This includes email alerts, a premium paid feature and other unique advanced functions.

Fery Kaszoni - 0800 246 1001 - [email protected]


[ Back To TMCnet.com's Homepage ]