Topic Modeling and Data Visualization with Python/Flask
Templates
Base.html
First, we’ll want to make our base template. I like to include all of these templates in a templates folder, as you can see from our tree structure earlier. Here’s the basic skeleton for our base.html file:
<!DOCTYPE html><html> <head> <meta charset="utf-8"> <link href="{{ url_for('static', filename='styles.css') }}" rel="stylesheet"> <script type='text/javascript'> </script> <style type="text/css"> </style> <title>Trump Search</title> </head> <body> <div id="navBar1" style="display:flex;flex-flow:row wrap;justify-content:center;align-items:center;width:100%;background-color:#e1e9f7;height:45px;"> </div> {% block content %}{% endblock %} </body></html>If you’ve written some HTML before then this should look pretty familiar. You may notice towards the bottom, but still in the body tag, we have this line:
{% block content %}{% endblock %}In our subsequent bubbles.html file, we’ll include this skelton:
bubbles.html
{% extends "base.html" %}{% block title %}i{% endblock %}{{ super() }} {% block content %} ---- content goes here -----{% endblock %}The way that these html templates work, is that the bubbles.html simply extends the base. Since we have the navbar in the base file, the bubbles.html, will extend that navbar. The navbar will still appear when bubbles.html is rendered, and all of the bubbles.html content will be rendered within the {% block content %}{% endblock %}
Back in app.py
In our app.py file, we included the line return render_template(‘bubbles.html’, result=output). Now, if we want to access that data on the front end template, we could simply add a script tag to our bubbles.html file, and set a variable that would hold the return data. We can do that like this:
{% extends "base.html" %}{% block title %}i{% endblock %}{{ super() }} {% block content %}<style type="text/css"> </style>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script><script type='text/javascript'> $(document).ready(function(){ var graphData = {{ result|tojson }}; console.log(graphData);</script><div id="display1"></div>
{% endblock %}Here, I’ve include a <div> tag, a <script> tag and <style> tag. The return data from our function can be stored in the graphData variable, so that we can access and display as we please, and it can be rendered inside the <div> tag if we so desire.
From here on out, it’s down to basic css and javascript. And because this blog post has gone on for far too long, I’ll wrap this up shortly, after giving a few key ideas about the Data Visualization side of this equation.
Data Visualization
1 ) Font Size
If you remember earlier, the final ‘data’ key we used, and sent to the front end is set equal to the current index divided by 2. Since we’re only using the first 80 results, if there are 80 results, this means that the data key will be a number between 0 and 40 maximum. On the front end, I used this value as my font size. I created a template expression and then passed in the ‘data’ key value as a variable to be used as the font size. However, we obviously don’t want a font size of 0, so I wrote a conditional that if the index is ever less than 12, then the font size will be 12. Since I’ll bring the values into the front end in order, they will have a consistent font size, that decreases gradually. I found that this worked the best on the front end, to visually express to the user that some words are more important than others. Otherwise, you would often end up with all the words being almost identical in size, which is of little help.
2) Spacing on page
Because I wanted to keep the words of most importance, or more closely related to the search term closest to the center of the page, and the other words towards the edges of the page, I used a position:fixed css styling for all the words. The first term, or search term, I gave a top:50% value and a left:50% value, centering it on the page. I know there has to be a better way to do this, but what I ended up doing was using a for loop, and gradually changing and spacing the words out depending on the index value:
for (var i = 0; i < graphData.length; i++) { let fontSize = graphData[i].data; if (graphData[i].data > 40){ fontSize = 40; } if (graphData[i].data < 12){ fontSize = 12; } if (i == 80){ top = 40; left = 50; } if (i == 79){ top = 46; left = 65; } if (i == 78){ top = 46; left = 35; } if (i == 77){ top = 57; left = 47; }This was fairly cumbersome, since I had to do this for all 80 indices. However, the advantage is that, they are always consistently spaced on the page, and if there are ever less than 80 words, then the words are automatically spaced further and further from the center, visually indicating to a user that the current search term yielded few and loosely related results.
The Mouseover Effect
Probably the most important piece of this aside from the data itself is the mouseover. To achieve this, I wanted to highlight the related topics whenever a word is moused over. I started by creating a mouseenter and mouseleave event. If you remember, our data that we now have on the front end, looks like this for each word on the page:
{'word': 'deal', 'data': 31.5, 'matches': ['deal', 'promise', 'seek', 'will']}So, on the mouseenter event, I get the html value of the current word being moused over. Then, I loop through my data returned from our Python function, and find the matching word. This word will contain an array of words in the ‘matches’ key, which I’ll want to test against the other words on the page. I start by storing all the elements on the page that belong to this class in a variable. This variable, “allElements” is now an array of HTML elements. I’ll loop over this data, still inside my mouseenter event, and if any of the html() values in this array match any of the elements in our ‘matches’ key, I change the css class of that element to a new class, the hightlighted class.
Then, on the mouseleave event, I simply revert any of the elements that were changed, back to the original class.
var allElements = $(".hoverBtn");$( ".hoverBtn" ).mouseenter(function() { var currentWord = $(this).html(); var currentList = []; for (var i = 0; i < graphData.length; i++) { if (graphData[i].word == currentWord){ currentList = graphData[i].matches; } } $(this).removeClass('hoverBtn'); $(this).removeClass('hoverBtn2'); $(this).addClass('hoverBtn3'); for (var i = 0; i < allElements.length; i++) { if( currentList.indexOf(allElements[i].innerHTML) > -1 || allElements[i].innerHTML == currentWord) { allElements[i].className = 'hoverBtn3'; } else { allElements[i].className = 'opacity'; } }});
$( ".hoverBtn" ).mouseleave(function() { for (var i = 0; i < allElements.length; i++) { if (allElements[i].className != 'hoverBtn2'){ allElements[i].className = 'hoverBtn'; } }Styling:
Last, but not least, styling. If you visited the live site, you’ll notice that each time you search, even for the same terms, the experience is slightly different.
To achieve this look I did the following:
First, I created an array of fonts I wanted to use:
var arrFonts = [‘Slabo 27px’, ‘Sawarabi Mincho’, ‘Baloo Tammudu’, ‘Nunito’, ‘Anton’, ‘Bitter’, ‘Fjalla One’, ‘Lobster’, ‘Pacifico’, ‘Shadows Into Light’, ‘Dancing Script’, ‘Merriweather Sans’, ‘Righteous’, ‘Acme’, ‘Gloria Hallelujah’];
and imported those fonts in a link tag from google fonts:
Then, inside each iteration of the for loop, I pick a random font from this array of fonts:
var randFont2 = arrFonts[Math.floor(Math.random() * arrFonts.length)];
And finally, in the template expression, I pass in this variable as the value of the font, via inline styling, and append it to the page:
var plot = `<a class="hoverBtn" style="Font-Family:${randFont2},sans-serif;position:absolute;top:${top}%;left:${left}%;font-size:${fontSize}px;color:${color9}">${graphData[i].word}</a>`;$('#display1').append(plot);As you can see, I passed in similar variables for the color. I used the same process by grouping sets of color schemes together based on hex values. Then, I pick an array of colors at random. Inside each loop, I pick a random color from the randomly chose array of colors, and this color is passed in as the color variable.
var greens = ['#DAA520', '#EEB422', '#CD950C', '#E6B426', '#CDAB2D', '#8B6914', '#8B6508', '#EEAD0E', '#FFCC11', '#FFAA00', '#DC8909', '#ED9121']; var purples = ['#38B8B', '#5F9F9F', '#2F4F4F', '#528B8B', '#388E8E', '#53868B', '#50A6C2', '#00688B', '#6996AD', '#33A1DE', '#4A708B', '#63B8FF', '#104E8B', '#344152', '#26466D', '#7EB6FF', '#27408B', '#1B3F8B', '#1464F4', '#6F7285', '#0000FF']; var yellows = ['#2E0854', '#7F00FF', '#912CEE', '#71637D', '#BF5FFF', '#4B0082', '#BDA0CB', '#68228B', '#AA00FF', '#7A378B', '#D15FEE', '#CC00FF', '#CDB5CD', '#4F2F4F', '#DB70DB', '#EE00EE', '#584E56', '#8E236B', '#B272A6'];
var colorArr = [blues, reds, oranges, greens, purples, yellows, skinTone1, griffindor, bluegray, downtown, lightblues, darkTones1, summertime, nightroses, coolcolors, winered, dustybeach, saltlamp, junkCarLot, autumnColors, cherrySoda]; var randColors = colorArr[Math.floor(Math.random() * colorArr.length)];
var color9 = randColors[Math.floor(Math.random() * randColors.length)];
var plot = `<a class="hoverBtn" style="Font-Family:${randFont2},sans-serif;position:absolute;top:${top}%;left:${left}%;font-size:${fontSize}px;color:${color9}">${graphData[i].word}</a>`;$('#display1').append(plot);Thanks for reading.
If you have any thoughts, or suggestions, or criticisms, feel free to leave them here. Thanks for reading.
