Reactive programming is a paradigm that has been around for a while in software engineering and is based on the concept of values which change over time in response to certain actions, and calculations that use these values (and therefore also return different outputs over time). Reactive programming is extremely common and the most widely-used example is a spreadsheet program. In a spreadsheet (e.g. Excel), you define relationships between cells, and if a cell's value gets updated or overwritten then everything that has a relationship with that cell also updates.

In shiny, reactive programming is what makes apps interactive. Users can change values and anything which relies on those values will be recalculated. The power of shiny is in the way it a) implements reactive programming in a language (R) that is inherently not reactive and b) how it determines the optimal way to execute recalculations involved in reactivity.

The concept of reactivity in an R shiny app is almost magical, in the sense that it makes you believe R works on a push basis. When you click a button, it feels like that button sends a notification to R and then R executes something (e.g. a popup that says 'Success!'). In reality though, this is not how R works at all. Consider the following code:

a <- 1
b <- a + 1 # (2)
a <- 5
         
         Copy
         
         

The value of b does not automatically update so as far as R is concerned, it is still 2. To update is simple though, we simply run the code again

a <- 1
b <- a + 1 # (2)
a <- 5
b <- a + 1 # (6)
         
         Copy
         
         

So values are not automatically updated when their dependencies change, we have to tell R to recalculate the value instead. R can retrieve the value of 'a' to calculate a + 1, but it will not know if 'a' later changes value.

As mentioned above, this isn't what we see in shiny where the flow of information seems to be reversed, so how is it doing this?

To best understand reactivity, consider the '4 maxims of reactivity' (see the RStudio overview of reactivity for more info):

1. R expressions update themselves, if you ask
   • So all we really need for reactivity, is to check every so often whether something needs updating and to recalculate it if it does.
   • Although clicking a button can't send a message to tell a value to update, we can keep checking whether the button has been pressed or not, and execute the relevant code if it has.
2. Nothing needs to happen instantly
   • Human perception isn't quick enough to notice a delay of microseconds.
   • To make things look instantaneous, you can just recalculate everything every few microseconds.
   • If the inputs haven't changed then the app will remain the same but if they have, then the new values appear after an imperceptible delay.
   • This is essentially how shiny works, but recalculating everything every few microseconds would be hugely inefficient and lead to a very slow app, so shiny also uses an alert system to identify which calculations need to be rerun.
3. The app must do as little as possible
   • Without a supercomputer, calculating everything every few microseconds would lead to long delays and unresponsive apps.
   • So instead, every few microseconds shiny will check all the expressions to see which are out of date and rerun only those.
   • If the inputs to a value have not changed, the expression doesn't need to be rerun and so this saves a lot of computation time.
   • Every few microseconds, if inputs have changed then any expressions which use those inputs become invalidated and shiny recalculates their values.
   • This recalculation is called a flush.
4. Think carrier pigeons, not electricity
   • This alert system uses two special object classes, reactivevalues and observers.
   • What is special about these is that if an observer uses a reactivevalue, it registers a reactive context with the reactivevalue which contains a callback for that observer.
   • All a reactive context is, is a notification that says 'every time you (the reactivevalue), change your value, I will tell the server to execute this callback', where the callback is just a piece of R code. In a shiny app, the callback is always the command: 'recalculate this observer'.
   • The analogy RStudio use is of carrier pigeons. When an observer registers a context with a reactivevalue, it gives the reactivevalue a carrier pigeon. Then whenever the reactivevalue changes its value, the pigeon (context) flies to the server and delivers a message (the callback). The server then reads the callback 'rerun this observer', and recalculates the value of the observer.
   • So if a reactivevalue has changed, its callback is placed in the server queue and every few microseconds, the server just executes anything in the queue. This saves the server having to recalculate everything every time or even decide what needs to be recalculated.
   • Another important point is that reactive contexts are one-shot. Once a callback has been sent to the server, the context that sent it is destroyed. Then whenever an observer is recalculated, it creates a new context with the reactivevalue. This is important in making sure shiny does as little as possible (see the box on the right).
   • It is also important to note that a single reactivevalue can have multiple contexts, if multiple observers use it. Then when its value changes, it puts all of those callbacks in the servers queue. Note that the order of these is more or less random, which is why observers must be independent of each other.

So a simplified example of how reactivity in shiny works is:

1. An observer is created (e.g. a bar chart) which relies on a reactivevalue (e.g. input$number_of_bars).
2. On app startup, shiny tries to create the bar chart but realises that it relies on input$number_of_bars, so it retrieves this value, which is 3 to begin with, and plots the chart.
3. At the same time, a reactive context is registered with the reactivevalue (the renderPlot() function will do this), which contains a callback that simply says 'plot the bar chart again'
4. The shiny server starts checking every few microseconds whether anything has changed and it needs to initiate a flush.
5. After a while, the user changes input$number_of_bars from 3 to 5.
6. The reactive context associated with input$number_of_bars then places the callback for the bar chart into the server's queue (and any other contexts associated with input$number_of_bars do the same for their callbacks).
7. On the next check a few microseconds later, the server sees there is something in the queue and so initiates a flush.
8. This flush involves executing the callback 'plot the bar chart again', so it does this.
9. It realises the bar chart depends on input$number_of_bars, and so it retrieves that value and sees that it is 5, then plots the chart.
10. At the same time, a reactive context is registered with the reactivevalue, which contains a callback that simply says 'plot the bar chart again' (same as step 3)
11. The new bar chart appears some microseconds after the user changed input$number_of_bars to 5 and so the change appears almost instantaneous.
12. On its next check a few microseconds later, the server sees that the queue is now empty and so returns to a resting state where it continues to check the queue every few microseconds.

The d3 animation to the right shows how we can visualise the reactive graph of a shiny app. It also gives some insight into how shiny knows what to recalculate and when.

For a simple app like the one given, it is not too difficult to picture this and understand what is going on, but for a much larger or more complex app this can quickly become too much to work out in your head. That is where the reactlog package comes in.

Using reactlog is very easy. The code below is a copy of the example app given on the right, but with a couple of lines of code included to enable reactlog.

#install.packages('reactlog')
library(shiny)
library(reactlog)

reactlog_enable()

ui <- fluidPage(
    numericInput('a', 'A', 1),
    numericInput('b', 'B', 1),
    numericInput('c', 'C', 1),
    textOutput('g'),
    textOutput('h')
)

server <- function(input, output) {

    d <- reactive({ input$a + input$b })
    e <- reactive({ input$b * 2 })
    f <- reactive({ input$c * 3 })

    output$g <- renderText({ paste0('A + B is ', d()) })
    output$h <- renderText({ paste0('(2*B) + (3*C) is ', e() + f()) })
}

shinyApp(ui, server)
         
         Copy
         
         

Try running the code above, and changing the value of 'B' to 2, before closing the app. Then execute the following function:

reactlog::reactlog_show()
         
         Copy
         
         

You will see something that very much resembles the d3 visualisation of the app's reactive graph (see 'Graphing It' on the right), but with some more information on names and timings. You can use the arrows in the top bar to step through each stage of reactive reasoning in much the same way.

As mentioned above, this is a simple example and the image below shows what one small part of the reactlog looks like for a far more complex app. It is these situations that reactlog was designed for, to allow you to reason about what is happening at each stage of the reactive graph in your app.

The boxes on the left gives one way to understand reactivity. Another important concept which may help to contextualise the process is to realise that shiny apps use declarative rather than imperative programming.

• Imperative programming means that you tell the code to do something and it does it. For example when you run print('Hello, world') you are issuing a specific command that is carried out immediately.
• Declarative programming on the other hand, involves specifying constraints and more high-level 'recipes' for how to do something, and then relying on something or someone else to decide when to translate that into action.

When we create an output in shiny, for example using a render*() function, we are not issuing a command that is carried out immediately. We are creating a 'recipe' that tells shiny 'this is how you create this output'. That recipe may be followed once, twice, many times or even never. The point is, we leave it up to shiny to decide when and how to do this.

The benefit of using declarative programming is that shiny can be as lazy as possible and only calculate something if and when it is required. You can see this in action in the example app below:

library(shiny)

ui <- fluidPage(
    textInput('first_name', 'Your name:'),
    textOutput('my_name')
)

server <- function(input, output) {

output$myname <- renderText({
    print('This is executing now')
    paste0('My name is ', input$first_name)
})

}

shinyApp(ui, server)
         
         Copy
         
         

Note that there is a typo, the output is called 'myname' in the server instead of 'my_name'. This means that the code in the renderText() call is never run. You can verify this because you will see no 'This is executing now's in the R console either.

Now try fixing the typo and changing it to 'my_name'. You will see that it works as expected now and every time you type a name, the print statement is executing and the text output changes correctly. Note that the renderText code is not run every time you type a letter - which is what we'd expect based on everything we learned in the '4 Maxims' box. This is by design and another example of how shiny uses laziness to improve efficiency (more on this in the next tab 'Controlling reactivity').

Another example of designed laziness lies in a key part of shiny that often surprises people but works extremely well. It is often overlooked that shiny won't calculate an observer if it is not visible. This means that, by default, if your app has multiple tabs then shiny will only 'flush' the outputs on the currently visible tab.

This drastically improves the speed of shiny apps, as there may be situations where a tab has an extremely long running output which is invlidated often, but if it is never viewed then it will never be calculated - meaning the rest of the app can run much faster. Note that this function of reactivity is not visualised in the animation below, but can easily be conceptualised as follows: at the calculation stage (e.g. Step 2) if the output G is on a hidden tab, it simply won't calculate and shiny will move on to the next output. This also means that any reactive expressions which are only used in G (i.e. F) won't be calculated either. You can see how - in a large, complex shiny app - this would make a huge difference.

One consequence of this designed laziness is that reactive dependencies are dynamic . In other words, shiny tries to minimise unnecessary dependencies by working out the minimum number of times it can get away with recalculating something.

Consider the two observe() calls below:

observe({
    if (input$a > 1) {
        print(input$a + 5)
    } else {
        print(input$b)
    }
})


observe({
    a <- input$a
    b <- input$b

    if (a > 1) {
        print(a + 5)
    } else {
        print(b)
    }
})
         
         Copy
         
         

While these observers look equivalent, there is a subtle but important difference. The second observer will always depend on both input$a and input$b, whereas the first observer will only depend on input$b if input$a <= 1. Consider the situation where input$a is set to 5 and then input$b is changed multiple times. The second observer will recalculate (the same value) several times, because it depends on input$b, whereas the first observer will not have registered the dependency on input$b because the value of input$a is greater than one. This means that the first observer will not recalculate unnecessarily if input$b is changed multiple times and will only recalculate when input$a is changed. If input$a is changed to 0, then a dependency on input$b will be created. This rendering of reactive dependencies on the fly is an important design feature as it allows even more laziness and improves shiny's efficiency drastically. Understanding this feature can make a big difference in speeding up your own apps, if you try to ensure that dependencies are only established when they are needed.

Reactive dependencies are dynamic (see box 'Laziness FTW'), but what are reactive dependencies? Well they are more or less what it says on the tin - if you have a variable or object which depends on a reactive value, it has a reactive dependency on that value. Take the below example:

output$my_plot <- renderPlot({
    plot(runif(input$number_of_points))
})
         
         Copy
         
         

The my_plot output has a reactive dependency on input$number_of_points. If input$number_of_points changes, my_plot will have be to re-built

Visualising how reactive objects and observers interact can help improve our understanding of reactivity in a shiny app, as well as provide a method for debugging reactivity. Consider the following trivial app:

library(shiny)

ui <- fluidPage(
    numericInput('a', 'A', 1),
    numericInput('b', 'B', 1),
    numericInput('c', 'C', 1),
    textOutput('g'),
    textOutput('h')
)

server <- function(input, output) {

    d <- reactive({ input$a + input$b })
    e <- reactive({ input$b * 2 })
    f <- reactive({ input$c * 3 })

    output$g <- renderText({ paste0('A + B is ', d()) })
    output$h <- renderText({ paste0('(2*B) + (3*C) is ', e() + f()) })
}

shinyApp(ui, server)
         
         Copy
         
         

Copy the code and run the app yourself to get an idea of what is going on. There are three reactive values (A, B and C), three reactive expressions/ conductors (D, E and F), and two observers (G and H). If you are unsure on the difference between these three types of object, or what reactive() does exactly, take a look at the 'reactive() vs observe()' box in the 'Using reactivity' tab for more information.

We can plot the reactive dependencies of our objects, in order to better understand our app and how shiny monitors each of these. The reactlog package was designed to do exactly this (see the box 'reactlog').

The animation below shows what the reactive graph might look like when the graph is first initiated, and then the user changes the input 'B'. See this RStudio article, which heavily inspired the animation below.

This tab will introduce Shiny modules and discuss how and why you might use them. If you are developing a shiny app which you think could potentially grow in future or be anything more than a very simple app, you should absolutely, definitely use modules. There are two key benefits to using Shiny modules and they are outlined below:

• If you've ever built a complex shiny app you'll know it can easily reach thousands of lines of code if you aren't careful. This quickly becomes a nightmare to maintain and debug, as you can end up scrawling through lines of your server to find where that one reactive object is defined. Modules give you an easy way to split your app code into separate, manageable scripts.
• The '1000 button problem': usually if you find yourself repeating code, you should be writing a function and calling that instead. This isn't so straightforward with reactive Shiny code because of namespacing. Every instance of an object needs a unique ID to match the UI to the server and this can quickly cause major headaches when developing your app. Modules provide a neat way to resolve this.

From personal experience, the first of these is the more useful aspect of Shiny modules and the reason why I would recommend always using them unless you have a reason not to.

The box on the right shows an example of how modules can be used, so feel free to use this as a starting point for developing your own modules.

As mentioned above, one of the key benefits of using shiny modules is keeping different elements of your app separate and in short, manageable scripts. The workflow proposed here (in this tab and the previous tab) is inspired by various other frameworks and is opinionated. As always, feel free to pick and choose which of the below you use, but when it comes to using modules, the following points are recommended:

• Each page of your app should be a separate module. This is straightforward for dashboard-style apps where you can simply have a separate module for each tab, for other apps you may have to make judgement calls about the separate parts of your app.
• Each module (UI and server function) should be in its own script, named according to the conventions outlined on the previous tab (e.g. 'mod_01_homepage.R').
• Every page's module server function should have a mandatory data argument, to which can be passed your app_data reactiveValues (see the 'Sharing is caring' box on the previous tab).
• Each module function should be documented in the same way as a normal function with roxygen comments.

Following these steps, as well as following the advice to structure your app as a package, will also mean that you can reuse modules in different apps. You could even develop a package with a module in and then just use that package when you are developing other apps in future. For example, if you have a particular section of reactive UI that is used in multiple apps.

So in summary:

• Always use modules!
• Separate your app into sections and have each section in a separate
• Use modules as an alternative to functions which have reactive inputs and outputs
• Shiny modules are an implementation of the software development method of namespacing

Imagine you are building an app with several independent counter buttons. This can be achieved using functions for UI and server logic, but for a very large app you would run into problems due to the fact that input and output IDs share a global namespace in shiny. This means that all inputs and outputs must have unique IDs.

This issue is resolved in computer science by using namespacing whereby objects must only have unique IDs within their namespace and namespaces can be nested. In practice you can simply think of a namespace as a prefix to an ID. For example, if you have two outputs called 'my_button', you can put them into separate namespaces called 'first' and 'second'. This would be equivalent to (but far more elegant than) renaming the outputs 'first-my_button' and 'second-my_button'.

Shiny modules were introduced to allow developers to take advantage of namespacing. See below for how to use them, in the context of an app with multiple counters.

library(shiny)

counterButton <- function(id, label = 'Counter') {
  ns <- NS(id)
  tagList(
    actionButton(ns('button'), label = label),
    verbatimTextOutput(ns('out'))
  )
}

counterServer <- function(id) {
  moduleServer(
    id,
    function(input, output, session) {
      count <- reactiveVal(0)
      observeEvent(input$button, {
        count(count() + 1)
      })
      output$out <- renderText({
        count()
      })
      count
    }
  )
}

ui <- fluidPage(
  counterButton('counter1', 'Counter #1'),
  counterButton('counter2', 'Counter #2'),
  counterButton('counter3', 'Counter #3')
)

server <- function(input, output, session) {
  counterServer('counter1')
  counterServer('counter2')
  counterServer('counter3')
}

shinyApp(ui, server)
         
         Copy
         
         

There are 5 important points to note about the code above:

• The module is defined in two parts just like a shiny app, with a UI and a server function
• Both elements of a module take in an id argument. This is crucial as it allows shiny to match the module UI to module server for each instance of the module.
• The UI function uses a function from shiny to access the current namespace. NS(id) will return a function that adds the current (unique) namespace to whatever is supplied to it. So by applying the resulting ns() function to each output in the module, the outputs will be unique to that particular 'id'.
• Note the moduleServer call in the counterServer function. This allows you to write code as you would for a shiny server, using the input, output and session objects as required.
• Outputs in the server function do not need to be wrapped in a namespacing function because this is handled by moduleServer(), but if you use renderUI(), any outputs or inputs must be namespaced in the same way as in the UI function. This is done by adding the following line of code to the top of your moduleServer function call: ns <- session$ns

e.g. {leaflet}, {dygraphs}, {networkD3}, {echarts4r}, {highcharter}, {sigmajs}