Have you ever you looked for a product online and then been recommended the exact thing you need to complement it? Or have you been thinking about a particular purchase, only to receive an email with that product on sale?

All of this may give you a slightly spooky feeling, but what you’re really experiencing is the result of complex algorithms used to predict, and in some cases, even influence your behaviour.

Companies now have access to an unprecedented amount of data on your present and past shopping and browsing preferences. This ranges from transactional data, to website traffic and even social media posts. Predictive algorithms use this data to make inferences about what is likely to happen in the future.

For example, after a few times visiting a coffee shop, the barista might notice that you always order a latte with one sugar. They could then use this “data” to predict that tomorrow you will order the same thing, and have it ready for you before you get there.

Predictive algorithms work the same way, just on a much bigger scale.

How are big data and predictive algorithms used?

My colleagues and I recently conducted a study using online browsing data to show there are five reasons consumers use retail websites, ranging from simply “touching base” to planning a specific purchase.

Using historical data, we were able to see that customers who browse a wide variety of different product categories are less likely to make a purchase than those that are focused on specific products. Meanwhile consumers were more likely to purchase if they reached the website through a search engine, compared to a link in an email.

With information like this websites can be personalised based on the most likely motivation of each visitor. The next time a consumer clicks through from a search engine they can be led straight to checkout, while those wanting to browse can be given time and inspiration.

Somewhat similar to this are the predictive algorithms used to make recommendations on websites like Amazon and Netflix. Analysts estimate that 35% of what people buy on Amazon, and 75% of what they watch on Netflix, is driven by these algorithms.

These algorithms also work by analysing both your past behaviour (e.g. what you have bought or watched), as well as the behaviour of others (e.g. what people who bought or watched the same thing also bought or watched). The key to the success of these algorithms is the scope of data available. By analysing the past behaviour of similar consumers, these algorithms are able to make recommendations that are more likely to be accurate, rather than relying on guess work.

For the curious, part of Amazon’s famous recommendation algorithm was recently released as an open source project for others to build upon.

But of course, there are innumerable other data points for algorithms to analyse than just behaviour. US retailer Walmart famously stocked up on strawberry pop-tarts in the lead up to a major storm. This was the result of simple analysis of past weather data and how that influenced demand.

It is also possible to predict how purchase behaviour is likely to evolve in the future. Algorithms can predict whether a consumer is likely to change purchase channel (e.g. from in-store to online), or even if certain customers are likely to stop shopping.

Prior studies that have applied these algorithms have found companies can influence a consumer’s choice of purchase channel and even purchase value by changing the way they communicate with them, and can use promotional campaigns to decrease customer churn.

Should I be concerned?

While these predictive algorithms undoubtedly provide benefits, there are also serious issues about privacy. In the past there have been claimsthat companies have predicted consumers are pregnant before they know themselves.

These privacy concerns are critical and require careful consideration from both businesses and government.

However, it is important to remember that companies are not truly interested in any one consumer. While many of these algorithms are designed to mimic “personal” recommendations, in fact they are based on behaviour across the whole customer base. Additionally, the recommendations or promotions that are given to each individual are automated from the database, so the chances of any staff actually knowing about an individual customer is extremely low.

Consumers can also benefit from companies using these predictive algorithms. For example, if you search for a product online, chances are you will be targeted with ads for that product over the next few days. Depending on the company, these ads may include discount codes to encourage you to purchase. By waiting a few days after browsing, you may be able to get a discount for a product you were intending to buy anyway.

Alternatively, look for companies who adjust their price based on forecasted demand. By learning when the low-demand periods are, you can pick yourself up a bargain at lower prices. So while companies are turning to predictive analytics to try to read consumers’ minds, some smart shopping behaviours can make it a two-way street.