How to detect the unexpected? Is the behaviour of some measured value normal or did something unexpected happen?

To answer these questions we need to detect anomalous behaviour in a time series. In this article I want to show you how we can do this with prophet.

Prophet is a library written by Facebook in python and R for prediction of time series.

So for anomaly detection we train our model according to the known values except the last n. Then we predict the last n values and compare the predictions with the truth. If they differ we call them an anomaly.

Here’s an example with some data of website usage.

At work I’m using Tableau as a tool for visualizing data with the possibility for the user to filter the data he’s looking at. So the user doesn’t need to program. He can use just is mouse which is great for people who don’t know exactly how to work with data. All this typical data mangling is hidden below the glamorous surface.

Unfortunatly for me the creation of these dashboards is done also by mouse clicking. When you’re used to program it’s a little bit annoying to click here and there to do something you could do with five lines of code…

An example is changing the datasources of a Tableau workbook. Let’s say we’ve created a workbook with lots of worksheets which are composed to several dashboards.

Now we want to apply all these calculations and visualizations to another set of datasouces with the same structure but with different content.

I sometimes have a function which takes some parameters and returns a data.frame as a result. Then I have a data.frame where each row of it is a set of parameters. So I like to apply the function to each row of the parameter-data.frame and rbind the resulting data.frames.

There are several ways to do it. Let’s have a look:

Update 2020-11-28: I’ve updated the resulting image with data until Nov 27, 2020.

In August I published first an animation of the spreading of SARS-CoV-2 in Germany.

Some days ago I found a similar visualization for Austria by Matthias Schnetzer. But he uses a different scale. His worst colour is for 150 incidents per 100,000 residents during the last 7 days. (I used 50 as last value of the scale.)

He also provides a link to his code He uses a little different way for plotting. So I thought I should update my animation using his way.

I used shinyapps.io for my own shiny app. It’s a great service. You can deploy your app for free, test it and show it to other people. But there’s also a downside: The memory an app can use is limited.

So I was looking for another way to deploy my app. So I took a look at Docker.