Ways to create loop/analyse multiple images using Azure Computer Vision in R

Question

I am currently conducting analysis of >10k images using Microsoft Azure Computer Vision and operating it in R programming software. I wanted to see what tags are present across these 10k images and would then count the most frequents tags to least frequent tags. All of these images are currently on my computer. This is not for commercial purpose and for my research study only.

The thing is, I am newbie in R and mostly used it for data visualization using ggplot.

Currently, I only managed to run analysis of 1 image at at time, which I do using the following script:

library(AzureRMR) 
library(AzureCognitive)

# Create Computer Vision endpoint
endp <- cognitive_endpoint(
  url = "https://xxxx.cognitiveservices.azure.com/",
  service_type = "ComputerVision",
  key = "xxxx")
  
# Add image
img_raw <- readBin("C:/Users/xxxx/Documents/Sample/trial.jpg", "raw", file.info("C:/Users/xxxx/Documents/Sample/trial.jpg")$size)

# Call the computer vision endpoint
data <- call_cognitive_endpoint(
  endpoint = endp,
  operation = "analyze",
  body = img_raw,
  encode="raw",
  options = list(visualFeatures = "tags"),
  http_verb = "POST")

print(data)

datatable <- data.frame(x, stringsAsFactors = TRUE)

And the results shows as follow:

print(data) shows:

$tags
$tags[[1]]
$tags[[1]]$name
[1] "person"

$tags[[1]]$confidence
[1] 0.998097


$tags[[2]]
$tags[[2]]$name
[1] "indoor"

$tags[[2]]$confidence
[1] 0.9948725


$tags[[3]]
$tags[[3]]$name
[1] "toddler"

$tags[[3]]$confidence
[1] 0.9893367


$tags[[4]]
$tags[[4]]$name
[1] "human face"

$tags[[4]]$confidence
[1] 0.9281158


$tags[[5]]
$tags[[5]]$name
[1] "child"

$tags[[5]]$confidence
[1] 0.877107


$tags[[6]]
$tags[[6]]$name
[1] "boy"

$tags[[6]]$confidence
[1] 0.8087585


$tags[[7]]
$tags[[7]]$name
[1] "baby"

$tags[[7]]$confidence
[1] 0.7611696


$tags[[8]]
$tags[[8]]$name
[1] "clothing"

$tags[[8]]$confidence
[1] 0.7346113



$requestId
[1] "0deda3ab-a02c-4d81-9de7-78326eb5f593"

$metadata
$metadata$height
[1] 640

$metadata$width
[1] 640

$metadata$format
[1] "Jpeg"

And the datatable data frame looks like follow:

datatable

I wanted to be able to run this code for each of the 10k images inside one folder, which is the "Sample" folder and I want to be able to put the results into the dataframe table with results of each image into a row. I am wondering if anyone knows how to do this? Should I create loop? How will I be able to make this script to read each of the 10k images that I have?

I have tried importing the folder and list of images, and change img_raw to files as below but no luck...

enter code here
# Importing folder
folder <- "C:/Users/xxxx/Documents/Sample"

# Listing files inside the folder
files <- list.files(path = folder, recursive = TRUE, pattern = "*jpg", full.names = TRUE)

enter code here
# Call the computer vision endpoint
data <- call_cognitive_endpoint(
  endpoint = endp,
  operation = "analyze",
  body = files,
  options = list(visualFeatures = "tags"),
  http_verb = "POST")

print(data)

Any help would be much appreciated. Thank you Guru!

user_13434871 · Answer

This pattern of problem is a common one and easily solved in R using the lapply function or similar.

First, write a function that processes a single file. Ensure that the file to process is the first (or only) parameter to the function.

imageToDataFrame <-function(f) {
  x <- call_cognitive_endpoint(
    endpoint = endp,
    operation = "analyze",
    body = f,
    options = list(visualFeatures = "tags"),
    http_verb = "POST"
  )
  data.frame(x, stringsAsFactors = TRUE)
}

Now apply the function to a list of files.

folder <- "C:/Users/xxxx/Documents/Sample"
files <- list.files(path = folder, recursive = TRUE, pattern = "*jpg", full.names = TRUE)

listOfDataFrames <- lapply(files, imageToDataFrame)

This returns a list of data frames. To combine all the data frames into one you could

allInOneDataframe <- lapply(files, imageToDataFrame) %>% dplyr::bind_rows()

but this gives you no information about the file from which the image was obtained.

allInOneDataframe <- lapply(files, imageToDataFrame) %>%
                       dplyr::bind_rows(.id="FileID")

will give you an index number to identify the file. But better is probably to modify the processing function:

library(tidyverse)

imageToDataFrame1 <-function(f) {
  x <- call_cognitive_endpoint(
    endpoint = endp,
    operation = "analyze",
    body = f,
    options = list(visualFeatures = "tags"),
    http_verb = "POST"
  )
  data.frame(x, stringsAsFactors = TRUE) %>% 
    add_column(FileName=f, .before=1)
}

So that

allInOneDataframe <- lapply(files, imageToDataFrame1) %>% bind_rows()

gives you the file name itself as a column in the data frame.

Looking at your sample one line data frame, I think it would be helpful to tidy your data by pivoting it to long format.

All code untested as I don't have access to your C: drive nor knowledge of your API key.

Ways to create loop/analyse multiple images using Azure Computer Vision in R

1 Answers