Recommending Content For The Cultural Visitor

Going back only 10 years ago we would have thought it crazy and potentially slightly disturbing to be recommended new tv shows or songs based on our previous viewing history. It would feel kind of spooky! In order to discover new stuff you would have to put some work in; read reviews, speak to friends… Now we don’t need to do any of that if we don’t want to. Everything is personalised, put right in front of us and in general we like it. As a result we watch, listen and buy far more online than before and and the companies that have created these algorithms are worth billions of dollars (think Netflix, Facebook, Amazon, Spotify… the list goes on).

These algorithms are made possible by observing patterns in user behaviour and then predicting what users will want to view next based off what other users with similar patterns did. In data science or machine learning parlance these are known as recommendation engines and one of the most common is a method known as collaborative filtering.

In its simplest form, this is a ratings table with users as rows and the product to be recommended, e.g. film or song, as the columns. Users rate (either explicitly e.g. give number of stars or implicitly e.g. completion rate) the products they have used and the task of the recommendation engine is to take these ratings as inputs and predict the user-specific ratings of other products and finally to recommend the highest rated for that user.

One of the shortcomings of this method is it suffers from the problem of a “cold start” i.e. what do you recommend to a first time user? What if the majority of users are first time as your product is young etc. In this case you do not have enough user ratings to predict anything useful and will likely recommend the same highest rated product to each user. This doesn’t feel particularly personal.

We started exploring whether there was a way to recommend related content through connections in the content without the need for user ratings.

The aim is to build a tagging system such that if we tag a piece of content e.g. an exhibit, artist or artwork, we can then surface other bits of content on the platform based purely on the similarity of their tags without the need for any ratings. This means that users and organisations will be able to tag content and the platform will surface relevant content without the cold start problem.

Classical tagging systems require the same tags in order to surface the same content. The advantages of our approach are; first, we can suggest relevant content even if no tags are not exactly the same based off of the “similarity” of the tags. Secondly, we have a numerical value for the similarity of tags and so can set thresholds for how relevant we want recommendations to be.

In the last post we went through building a language model and then used that as a pre-trained model as a starting point to build a sentiment model predicting whether art reviews were good or bad.

We would use a similar approach this time. We took a collection of several thousand art reviews as our corpus of art-related words (total corpus was over 25,000 unique words) and aimed to build a language model using LSTM. The result of this was a language model for culture and the arts; a numerical representation of words trained in an art-specific context.

Now, given that the words can be represented as vectors it is fairly trivial to find the top 3 “closest” words for each word in the corpus using a method known as Euclidean distance. The final result is a graph (in the mathematical sense, connections/friends on Facebook) of words taken from cultural contexts where we define connections to be only between the “closest” words.

This is a similar output to what we experimented with in a previous blog post however the data and architecture are different here.

Now we have a better understanding of the language of culture, we can make connections between tagged content. But how do we come up with the right tags in the first place? In a classical tagging system the user inputs the tags they think best correspond to the content but we suggest a more complete way of tagging content.

The final method, “image-based suggested tags”, has been something that we have been experimenting with as a way to tag images and surface content automatically during the process of uploading an image. It incorporates a multi-class image recognition model which takes an image and outputs multiple suggested tags based on the image data.

A short demo video of our prototype in action can be seen below:

From the video we can see the advantages of being able to produce tags automatically based on a relevant image. We demonstrate its ability to surface relevant articles from Wikipedia and also produce further relevant tags based off of our graph.

There has not been a huge amount of work done in the cultural sector that leverages machine learning techniques to organise and surface relevant content to users.

Much of the feedback that we get is that visitors struggle to find out about upcoming exhibitions. Many are unsure about going to see exhibitions they know nothing about because they don’t know whether they will like it or are worried it will not be accessible enough for them. Given that the cultural sector is one of, if not, the most content-rich sectors there is a huge opportunity to connect visitors to new institutions with relevant artists, artworks, exhibits, exhibitions, festivals or whatever it may be. This will drive interest and traffic to the content producers (artists, galleries, etc.) and will be mutually beneficial for all involved in the cultural sector.

As we have seen in many other industries; discoverability, personalisation and access from anywhere has been hugely disruptive and, in general, has been great for the end-user. It would be hard to argue that Netflix and Spotify have not increased access and variety of films and music across the world. This is what good recommendation systems can do.

Related posts:

When I was 7 my Grandad Bob built me a beautiful doll’s house for Christmas — so much love went into it. It took the best part of a year to make and even to this day it is one of my most precious…

The Amarna period is one of the most exciting and studied times in ancient history. The radical and highly polarizing Pharaoh Akhenaten, during his 17-year reign in 14th century Egypt, instituted a…