My experience working with an unconventional web development framework: Plotly Dash
To those who has never used a Python framework for web development, I hope this article encourages you to give it a shot!
Now, onto the sharing.
First things first, about me…
GIF taken taken from Yarn
Cleanrooms, rows of bulky noisy machines, and people fully clad in smocks and mini ear plugs, that’s pretty much what I experience every day at workplace for the first 5 years of working life. How come?! Well, I was a Process Engineer in Semiconductor industry. Come 2020, I see colleagues in casual wear coding away and at break time around 3–4pm, some went to the couch to have a short game of FIFA on Nintendo Switch or have a match at the Foosball table at GovTech cafe. It was a pretty nice change.
I started my developer journey at GovTech with designing and developing mechanical mounts and enclosures to house sensors for SmartGym project. At the time, SmartGym was at its infancy stage where not a single sensor is deployed for public use. Once we have done the heavy lifting of sensor development, parts procurement, and retrofitting gym equipments at 3 ActiveSG gyms, I volunteered to work on SmartGym Analytics Dashboard. Finally! Gonna get my hands dirty with frontend web development!
Dashboard use case
Now that sensors have been deployed at the gyms, we needed a way to share information with the whole team. The analytics dashboard aims to
- track growth in product usage to motivate the team and unite everyone around the same goals
- aims to boost the team’s morale when there’s growth
- gives a sensing if the team is heading in the right direction
To kickstart the exploratory data analysis, these are some of the questions which we brainstormed as a team:
- How many users signed up and how many actually use it?
- What is our user retention in number and percentage?
- What is the age and bmi population of SmartGym users?
- Time and day breakdowns of when users are using SmartGym
- What are the machines that are commonly used?
- Why are some machines not frequently used?
- Which gym has the most users and why?
- How many SmartGym transactions have been carried out?
Here are the key metrics that I show on the dashboard: Active Users, Recorded Workouts, Recorded Health Metrics. Period of data shown would be daily, weekly, and monthly.
For the dashboard to be self-explanatory, the terms used had to be clearly defined and standardized. For example, what defines an active user? This definition step is one of the most crucial steps of all. We went through this process as a team so that everyone is aligned and on the same page when using terminologies in future discussions. Going back to the question, an Active User is defined as someone who, over a period of time (e.g. a day, a week, a month) either recorded a workout, or recorded a health metric, or simply sign up with SmartGym.
Frontend Development
What I enjoy about my role is that I have the freedom to use any technology for development. It was like, “get out there and explore!”.
I did research on web development frameworks that are trendy in the industry. “ReactJS” pops up almost every time and it’s recommended to understand Javascript first. With zero knowledge in that, I decided to look for python alternatives and found Plotly Dash.
In the course of development, I’ve learnt a lot on creating intuitive design that is simple and easy to use. A lot of empathy has to be placed on the users of the dashboard. What makes sense to me may not necessarily makes sense to the user. My lecturer used to say, “Simple is the most difficult”. Now I get what he meant.
We practice agile development where we build fast, get feedback, and then iterate. Like a mantra, I had to remind myself consistently to seek feedback from users and team members, making design an iterative process.
These are the technologies I used in the course of development:
- HTML and CSS, for web layout positioning and styling
- MongoDB, this is where raw data is being stored. Pymongo library is a great API to use when talking to MongoDB in python.
- Plotly, I used this to plot graphs. It’s an open-source, python plotting library that is interactive, and supports a variety of chart types that cover a wide range of use-cases such as statistical, financial, and geographic.
- Plotly Dash, a python framework for creating interactive web applications. It’s great for pythonistas looking towards working with a React frontend ecosystem in Python!
- Pandas, a python data manipulation and analysis library. Prior to using pandas, the dataset must first be converted to a 2-dimensional labeled data structure with columns known as a dataframe.
Features and User Interface
In terms of user interface, I wanted the users to be able to interact with the plots and determine the range of data to be presented in those plots. The interactivity I had in mind should allow users to do the following:
- set date range (Last 30 days, Month to Date, Year to Date, etc.)
- set summary level (by gym location, type of event, age groups etc)
- basic viewing tools on the plots e.g. zoom in/ zoom out
- save graph option so that plots can be shared in presentations...
- navigate pages easily
- refresh the page when new data comes in
thinking from web design perspective, I’m going to need buttons, navigation bars, sidebars, date pickers, dropdowns, and a way to organize the graphs on the pages.
- Plotly graphs come built-in with tools such as zooming in/ out and save JPEG. The toolbar appears upon hovering over the graph.
- By having a Date Picker on the sidebar (essentially a Navbar), user is able to set the dates of interest and filtered data will be reflected on the graphs almost instantly. Date picker is a dash core component and Navbar is a Dash bootstrap component.
- To organize the graphs in terms of daily, weekly, and monthly, I made use of Tabs to separate the graphs for clarity. Tab is a dash bootstrap component.
The dashboard is multi-page and interactive that was made possible via Plotly Dash components. Below are some of the dashboard capabilities which I’d like to share:
I. Multi-page web
The app structure looks something like this:
- assets
- index.py
- some_helper_functions
- pages
| — homepage.py
| — sgym_overview.py
| — sgym_campaign.py
Dash pages is a recently released feature by Plotly Dash which handles URL routing and offering an easy way to structure and define the pages in the web app. It removes the need for boilerplate code when creating multi-page app. From the structure above, a page of the app is placed in a .py file inside a folder called pages. With a single line of code, a .py file is declared as a page and Dash will handle the routing. Interesting? Find out more here: Multi-Page Apps and URL Support | Dash for Python Documentation | Plotly
II. Refresh Data
From the users outreach programs and user feedback sessions we’ve had in the past, we can expect data to come in on a daily basis. To facilitate fresh data being pumped into the graphs, I set a dynamic layout on every page load where a call is made to the database to retrieve data (and then run it through a processing pipeline) upon clicking the Refresh Button.
III. Use of states to store data
The data presented on some of the pages came from the same dataset (consisting of processed data). I couldn’t afford to repetitively generate the dataset each and every time a user moves onto another page. By using states (a dash core component), I enabled shared data capability across pages where data is being pulled and processed only once and that any page could have access to it.
I’d definitely recommend building a frontend web app with Plotly Dash for you Python lovers out there. It’s well documented and tons of tutorials are available on YouTube (my personal fave would be uploads by Charming Data) and Udemy.
That’s pretty much my sharing. There were times when weird bugs appeared after deploying a new version and to ensure a working version is deployed timely, I had to time block myself on troubleshooting and ask for help from peers when it’s not getting resolved. I cannot say thank you enough to my RO and peers for the feedback and guidance they have given me throughout my development journey.
Author: Shasha Khairy
Originally published on Sensors & IoT, GovTech.
For more articles, visit Sensors & IoT, GovTech.