Creating a Product Recommendation Neural Network in the Web Browser

Neural networks have demonstrated their power over the last few years, creating major breakthroughs in computer vision, speech recognition, bioinfomatics, and many other problem spaces.

Although training a neural network requires massive amounts of data, once the network is trained, the code required to run the network function is quite small. For simple feed-forward neural networks, these functions can even be run right in the web browser.

Note: Source code for the following demo will be available at this GitHub repository.

Running a Neural Network in the Browser

As with everything on the web, there are many options when looking for a library to help you train and run your neural network in a browser.

TensorFlow.js is probably the most robust option at the moment. Created by Google, TensorFlow.js can import models from

TensorFlow or Keras, meaning you don’t have to do the training step in the browser. Although TensorFlow.js can use the GPU to accelerate training, it’s still faster to train a network in regular TensorFlow and import it to the browser afterwards.

Another very nice option is an open source library called

Brain.js. Brain.js greatly simplifies the process of creating a neural network. With Brain.js, you can work at a higher level of abstraction than you would with TensorFlow. In this case, we trade granular control over the shape of our network for a much more simplified coding experience. This is an ideal way to explore solving problems with a neural network, without having to understand neural networks at the PhD level.

Using Brain.js

Brain.js can be run in Node.js or in a web browser. Networks trained in Node can be saved and loaded into a browser and vice versa. Instructions for using Brain.js can be found in the GitHub repository.

The simple example from the GitHub repository creates an XOR function. Historically, early “perceptron” based neural networks couldn’t account for the non-linearity of an XOR function. And since the training set for this function is quite small (only 4 possibilities), it makes for a very good (as in short) example for setting up a simple neural network.

Here’s the XOR example from the Brain.js page:

const config = {

    binaryThresh: 0.5,

    hiddenLayers: [3],

    activation: ‘sigmoid’

};

const net = new brain.NeuralNetwork(config);

net.train([{input: [0, 0], output: [0]},

    {input: [0, 1], output: [1]},

    {input: [1, 0], output: [1]},

    {input: [1, 1], output: [0]}]);

const output = net.run([1, 0]); // [0.987];

Lets take a look at this piece by piece. We’ll start with the config object.

The config object has three properties:

1. binaryThresh — (binary threshold) a value between 0 and 1 that defines the point at which a single neuron in the network will switch from being off to on. By using a value of 0.5, when the value of the neuron is below 50% it will be considered off. When the value is over 50% it will be considered on. This binary threshold can be adjusted when tuning your network. Adjusting this value is similar to a volume control. Turning this up or down will cause your network to make decisions with greater or lesser vigor. A starting value of 0.5 is the most reasonable until you have some reason to adjust.

2. hiddenLayers — this is an array of integers describing the size of the hidden layers within the network. For example if you wanted to have three hidden layers with 25 neurons in each layer, your hidden layer array would look like [25, 25, 25]. In this example, the author is using just one hidden layer with 3 neurons in it, so their array contains only one number: [3].

3. activation — a function describing how a neuron goes from an off state to an on state. A ‘sigmoid’ is a mathematical function that has a smooth “S” shaped curve. We use this shape to help the neural network generalize and explore the problem space. By having the “S” shaped curve, rather than a square binary value of on and off, the network will be able to use all of the decimal values between 1 and 0. If we were using a binary type of activation, all neural values would only be 1 or 0. There would be no in between.

After the config object, the example instantiates a new brain.NeuralNetwork and passes this config object into the constructor.

The next step is training. In the example, the author defines the four possible input options and expected output result from the neural network. In this case, the input is always an array with two binary values and the expected output is going to be a single binary value.

Note: although the training data uses absolute binary 1’s and 0’s for the training data, the actual output from the network will be a floating-point value approximating 1 or 0.

When the network is run, and given an input of [1, 0] the expected result should be near 1. The actual value coming out of the network will be closer to 0.987 (close to 1).

Framing the Problem Space

Framing the problem space is the core task for the person creating the neural network. You have to give meaning to all of the inputs and outputs. The way you frame that problem will greatly impact your chance of success.

If we change the training data, we change the whole function of the network.

This simple example of an XOR function in Brain.js is actually the exact same architecture we need to create a product recommendation system. The only thing specific to XOR in the example is the training data. If we change the training data, we change the whole function of the network.

In our case, we have historical data for customer’s shopping carts. Meaning we know what products people have historically purchased together. So, if we use this data to train the network, it should be able to tell us the statistically most likely next product a shopper may want to add to their cart, based on the existing contents of their shopping cart.

Transforming Data

To create a recommendation system, I first took our sales history and simplified it. We don’t need cart quantities and prices. We just need to know what items are in the cart.

Normally we reference products by item number, but to make this even simpler for the neural network, we’re first going to convert the data from four digit product numbers to an index in an array representing all possible products in the cart.

For example, if we have a product list of 10 items, and items at index 3 and 7 exist in the cart, the input set for our neural network would look like this:

[0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0]

As an output result, we want an array of this same 10 item length representing the recommendations. A result may look something like:

[0.012, 0.014, 0.012, 0.001, 0.9834, 0.011, 0.732, 0.001, 0.32, 0.243]

This output would indicate a 98% recommendation for the item at index 4 and a 73% recommendation for the item at index 6.

Training the Network

To create the training set, I take a list of about 100,000 orders, removing orders that only had a single item in the cart, we’re left with around 70,000 orders that contain multiple items.

We then loop through this list, removing a single item from the cart, and using that item as our input value, then using the remaining items in the cart as the expected output value.

What we want the neural network to train on is looking at a single item, and have it guess which other items are likely to be purchased with that one item.

During training, the network will only ever be given a single product as an input, but during actual use, we will be feeding it multiple items in a cart, expecting it to compose a recommendation list based on these multiple inputs.

Feed Forward Neural Networks

What we have here is called a Feedforward Neural Network. This basically means that as data moves through the network, it never feeds back into the system. It’s a simple one-way process illustrated in this top-down form:

When we pass in a product to be evaluated, we are essentially activating a neuron at the top input layer. Through training, the connections between each layer has been adjusted with weights and biases to from pathways through the neural network, resulting in the activation of the output neurons indicating which items the network recommends.

As we activate multiple neurons at the input layer, the inner layers will compose these paths together to create the output array of recommendations.

Tuning

With the plan of running the neural network in the browser, it’s important that the state data of our neural network not be too large in terms of file size. Experimenting with more or less hidden layers and adding more or less neurons in those hidden layers will greatly affect the size of the network state used to reload the network in the browser.

You want to try to accomplish the task with the minimum number of hidden layers and neurons per layer. The only way to find the right balance is to train and run the network multiple times with different settings.

On the high end, I had three hidden layers consisting of around 100 neurons per layer. This configuration had great results in the output values, but the data needed to create this network in the browser was over 50 megabytes. This is way too big for our simple little recommendation app.

I pruned the system down to a single hidden layer with just 3 neurons in that layer. That would take our network data to less than 5 kilobytes. Plenty small! But, at this level of small, you can easily see that the network isn’t ‘playing with a full deck’. It’s making overly broad recommendations and not really responding well to different inputs. This is to be expected when there literally aren’t enough neurons in the network to compose a whole opinion about the inputs.

After playing around with these values, the optimum region for our recommendation system was to have 2 hidden layers, the first with 15 neurons and the second layer with only 10 neurons. This would get us into the 25 kilobyte range, which will work fine in the browser.

This may seem like a very small number of neurons for a recommendations system that will be picking between hundreds of items. But after testing, this seems to be the optimum compromise between network data size and the richness of its performance.

Saving Your Trained Network

Once trained, you can get a JSON object representing your trained network by calling the toJSON() method of a Brain.js network:

var json = net.toJSON();

You can then save this JSON string to a file that you will load into the browser.

To load your JSON data back into a new instance of Brain.js you can do the following:

var brain = require(‘brain.js’);

var net = new brain.NeuralNetwork();

net.fromJSON(json);

Known Issues

I had one minor issue when trying to load the data into the browser. If you have other code in your web application that has done any kind of mutation to the Array.prototype, you may have issues getting your model to load.

For an unknown reason, Brain.js will throw errors if the Array.prototype is not a virgin. So, don’t mess with it. If you have prototype functions, you may have to go another route with those if your models refuse to load.

Output to Function

There is a very nice method included with Brain.js that would allow you output your fully trained network as a single function. This would allow you to embed the function in your browser, and you could run the network without having to load the model, or even without adding the Brain.js library.

net.toFunction();

While awesome in theory, and interesting to see in practice, outputting this file can be VERY memory intensive. In the larger hidden layer configurations I wasn’t able to output this function before running out of system memory in Node.js.

Also, the resulting function is HUGE. While our trained network state is around 25 kilobytes, the function output creates a file that is many megabytes in size.

So, while it’s cool that you can output your whole network as a single function. In practice, I found it to be impractical for use in the browser.

Conclusion

Using Brain.js for this recommendation system worked really well in practice. We used this exact system to create the product recommendation system for a large nutritional supplements company.

Seeing Brain.js perform with real world data provided a lot of product purchase insights that aren’t always apparent when looking at aggregate sales numbers.

By running the network in the browser, we can provide new product recommendations every time the users add another item to their cart.

Source Files (GitHub repo)

Source code for this demo is available at this GitHub repository.

Need Professional Help?

If you’ve got an AI or web project and you are looking for professional help, you can contact us at Preese Interactive Media.