The Taipan package is a tool that can be used to create Shiny surveys. The Shiny web applications can be run locally and allow users to export the survey results in a tidy data format, that is ready to use for analysis.
The app was initially created to streamline the process of manually tagging images to create a training set describing each image. It allows users to provide information regarding entire images, and smaller regions within. It is an interactive survey tool, with questions displayed for each image and area identified.
You can install the development version from Github using:
# install.packages("devtools")
devtools::install_github("srkobakian/taipan")
A folder has been provided in this package called sample_images
. It contains five sample of images provided by Tennis Australia, they depict scenes from the Channel 7 broadcast of the Australian Open 2016.
The following code creates the app that was used to tag images from the Australian Open in 2016.
First load taipan, and shiny.
# load packages
library(shiny)
library(taipan)
The survey creator provides questions using Shiny widgets, and the tutorials expand on the types of widgets you can use to collect information from users. Radio button widgets are used in the sample app, as all questions required one answer, and were limited to one answer.
Two sets of questions are required, one set that relates to the entire scene, and another set that relates to the smaller areas that can be selected in the images.
The following code shows the basic layout to include questions.
questions <- taipanQuestions(
scene = div(),
selection = div())
The following code creates the taipanQuestions
set for the tennis images:
# create questions to ask users
questions <- taipanQuestions(
scene = div(radioButtons("graphic", label = ("2D Graphic"),
choices = list("Live image", "2D Graphic")),
radioButtons("bg", label = ("Background"),
choices = list("Crowd",
"Court", "Logo wall", "Not applicable")),
radioButtons("person", label = ("Detectable Person"),
choices = list("Yes", "No"), selected = "Yes"),
radioButtons("shotangle", label = ("Shot angle"),
choices = list("Level with players",
"Birds eye",
"Upward angle")),
radioButtons("situation", label = ("Situation"),
choices = list("Court in play",
"Court player close-up",
"Court close-up not player",
"Crowd",
"Off court close up of player",
"Transition"))),
selection = div(radioButtons("detect", label = ("Detect Face"),
choices = list("Player" ,
"Other staff on court", "Fan", "None")),
radioButtons("obscured", label = ("Face obscured"),
choices = list("Yes", "No"), selected = "No"),
radioButtons("lighting", label = ("Lighting"),
choices = list("Direct sunlight", "Shaded", "Partially shaded")),
radioButtons("headangle", label = ("Head angle"),
choices = list("Front on", "Back of head",
"Profile", "Other")),
radioButtons("glasses", label = ("Glasses"),
choices = list("Yes", "No"), selected = "No"),
radioButtons("visorhat", label = ("Visor/hat"),
choices = list("Yes", "No")))
)
To build the shiny app the following buildTaipan
function is used. It requires the taipanQuestions, the location to the save the new app, and the location of images. Images can be accessed via a URL, or a file path is held locally. Note that these images will be downloaded or copied to the directory of the new app.
images <-
c("https://raw.githubusercontent.com/srkob1/taipan/master/sample_images/blue_player.png",
"https://raw.githubusercontent.com/srkob1/taipan/master/sample_images/pink_player.png",
"https://raw.githubusercontent.com/srkob1/taipan/master/sample_images/red_player.png",
"https://raw.githubusercontent.com/srkob1/taipan/master/sample_images/yellow_player.png",
"https://raw.githubusercontent.com/srkob1/taipan/master/sample_images/disgruntled_player.png")
buildTaipan(
questions = questions,
images = images,
file.path(tempdir(), "taipan")
)
The sample app is stored in a temporary directory. It will launch after it has been created, this default behaviour can be changed by setting launch = FALSE
.
The scene
questions need to be answered only once per image. These questions appear for the first image when the app begins, and will be displayed as long as there is no area selected.
The selection
questions are answered for each highlighted area. To make a selection, click on the image and drag the mouse to create a shaded box covering the desired area. When the area is selected, the selection
questions become available.
When a user is finished answering questions for ALL selections in an image, click the Next Image
button to continue to the next image available. A user can also choose the Export responses
button at any time, downloading and saving all current information as a text file.
When the questions for an area have been answered, the user should click the Save Selection
button, this will save the current answers as information for the currently shaded area of the image. Users can repeat the process of highlighting an area, answering and savings related questions as many times as needed.
While the answers provided are regularly saved, they are not exported unless the Export responses
button is used. This will save the comma separated values in the chosen directory.
The answers are provided in a tidy format. Each row contains one observation, we consider each selected area in an image an observation. The columns contain information regarding each selected area. This includes:
taipan_export_2018_09_04 <- read.csv("taipan-export-2018-09-04.csv")
taipan_export_2018_09_04
#> image_name graphic bg person shotangle
#> 1 blue_player.png Live image Logo wall Yes Level with players
#> 2 blue_player.png Live image Logo wall Yes Level with players
#> 3 disgruntled_player.png Live image Crowd Yes Level with players
#> 4 pink_player.png Live image Crowd Yes Level with players
#> 5 pink_player.png Live image Crowd Yes Level with players
#> situation xmin xmax ymin ymax
#> 1 Court player close-up 335.9307 440.9812 164.71154 285.8654
#> 2 Court player close-up 623.3766 711.1111 213.17308 292.7885
#> 3 Court in play 214.7186 346.3203 33.17308 184.3269
#> 4 Court in play 364.7908 468.6869 35.48077 179.7115
#> 5 Court in play 278.2107 349.7835 72.40385 154.3269
#> detect obscured lighting headangle glasses visorhat
#> 1 Player No Shaded Other No Yes
#> 2 Other staff on court No Shaded Front on Yes Yes
#> 3 Player No Shaded Other No No
#> 4 Player No Direct sunlight Front on No No
#> 5 Other staff on court No Shaded Profile No Yes
It is possible to view and edit answers for a selected area by double clicking within the box shown on the image.