Calculate errors from two data matrices using loops

user_16127011 0
Viewed 86

I have two data matrices of the same dimensions, let one matrix is denoted by A and the other is denoted by B having dimension 24*365 where 24 denotes the hours and 365 are the number of days (mean each matrix having hourly data of one year). Suppose I choose a single day, for example, Sunday, suppose it's in the 3rd column of both the matrices. similarly, I choose all the remaining columns of the Sunday from both the matrices so I obtain two submatrices from each matrix A and B of the Sunday. Then I take each submatrix as a vector and I calculate the mean square error and percentage mean square error from these two vectors. Similarly, the same process is repeated for the remaining days of the week. My question is that can any do this whole process using loops, that through loops for each day submatrix is selected from both matrices A and B. Then taking each submatrix as a vector and calculate mean square error and percentage mean square error separately for each day. I try to explain my question manually with an example of taking any two matrices denoted by C and D, but due to the large dimensions of my original data matrices, there are more submatrices which makes it's quite time-consuming when we do this manually.

C <- matrix(16:155, ncol=14, byrow=T)
D<- matrix(50:189,ncol=14, byrow=T)
sub_C1 <- C[,c(1+(0:6)*2)]
sub_D1 <- D[,c(1+(0:6)*2)]
sub_C2 <-  C[,c(2+(0:6)*2)]
sub_D2 <-  D[,c(2+(0:6)*2)]
sub_C3 <-  C[,c(1+(0:4)*3)]
sub_D3 <-  D[,c(1+(0:4)*3)]
################mean square error################
mse_1 <- mean(abs(as.vector(sub_C1)-as.vector(sub_D1)))
mse_2 <- mean(abs(as.vector(sub_C2)-as.vector(sub_D2)))
mse_3 <- mean(abs(as.vector(sub_C3)-as.vector(sub_D3)))
################## mean percentage absolute error############
mape_1 <- mean(abs(as.vector(sub_C1)-as.vector(sub_D1))/as.vector(sub_C1))
mape_2 <- mean(abs(as.vector(sub_C2)-as.vector(sub_D2))/as.vector(sub_C2))
mape_3 <- mean(abs(as.vector(sub_C3)-as.vector(sub_D3))/as.vector(sub_C3))
#############################################################

Can someone helps that through loops that the same submatrix is selected from each matrix C and D, and calculate errors from each submatrix separately.

partitioning iteration r loops
1 Answers

Since many proclaimed authors/developers often say as R is vectorised, loops in R are mostly avoidable. Here also no need to use loops. One more thing - binary operations such as + or - are already vectorised and therefore you need not convert matrix into vector as.vector them explicitly.

I think you may use purrr::map_df as detailed under, which will give you a dataframe like structure for all 7 days. First row will however correspond to the first day in the year(matrix).

Strategy explained -

  • store your desired n in a variable n. Here you desire 7 partitions so I have stored n <- 7
  • Generate a sequence 0 to n-1 So I have used seq(n) - 1 as first argument.
  • Now use modulo division on column numbers %% to get every nth row in every iteration. e.g. 8 %% 7 gives 1 and similarly 15 %% 7 also gives 1 so every 7th row will come in respective partition.
  • For partition we are using C[, seq(ncol(C)) %% n == .x] expression/formula for both C and D respectively. .x is first argument in map function. This will generate 7 partitions.
  • Check this
C[, seq(ncol(C)) %% 7 == 2]
     [,1] [,2]
 [1,]   17   24
 [2,]   31   38
 [3,]   45   52
 [4,]   59   66
 [5,]   73   80
 [6,]   87   94
 [7,]  101  108
 [8,]  115  122
 [9,]  129  136
[10,]  143  150
  • You'll notice that 2nd and 8th row has been partitioned by that expression.
  • Store these intermediate partitions in temp variables (inside {}) x1 and x2 (one for each matrix)
  • In two further intermediate variables say y1 and y2 calculate mse and mape as per given formula (using x1 and x2 here)
  • Since we are generating dataframe as an output, we need to setNames here, lastly within {}

I hope syntax is pretty clear now.

library(tidyverse)

C <- matrix(16:155, ncol=14, byrow=T)
D <- matrix(50:189,ncol=14, byrow=T)

#number of partitions you desire
n <- 7 # every 7th column

purrr::map_df(seq(n) - 1, ~{x1 <- C[, seq(ncol(C)) %% n == .x]; x2 <- D[, seq(ncol(D)) %% n == .x];
y1 <- mean(abs(x1 - x2)); y2 <- mean(abs(x1 - x2)/x1); 
setNames(c(.x+1, y1, y2), c('day', 'mse', 'mape'))})

#> # A tibble: 7 x 3
#>     day   mse  mape
#>   <dbl> <dbl> <dbl>
#> 1     1    34 0.520
#> 2     2    34 0.604
#> 3     3    34 0.588
#> 4     4    34 0.572
#> 5     5    34 0.558
#> 6     6    34 0.545
#> 7     7    34 0.532

If however, you want to use baseR only, you may the following list as output having seven elements, 1 each for a day. Syntax explained.

  • Strategy is similar to purrr here again. Here formula and input vector argument has switched places (according to design of these function).
  • Since we are generating list as an output, we will use `setNames() here outside the {} i.e. function/formula argument.
C <- matrix(16:155, ncol=14, byrow=T)
D <- matrix(50:189,ncol=14, byrow=T)

#number of partitions you desire
n <- 7 # every 7th column

Map(\(.x) {x1 <- C[, seq(ncol(C)) %% n == .x]; x2 <- D[, seq(ncol(D)) %% n == .x];
y1 <- mean(abs(x1 - x2)); y2 <- mean(abs(x1 - x2)/x1); 
setNames(c( y1, y2), c('mse', 'mape'))}, seq(n) - 1) |> setNames(paste0('Day', seq(n)))

#> $Day1
#>        mse       mape 
#> 34.0000000  0.5202581 
#> 
#> $Day2
#>        mse       mape 
#> 34.0000000  0.6044914 
#> 
#> $Day3
#>        mse       mape 
#> 34.0000000  0.5878023 
#> 
#> $Day4
#>      mse     mape 
#> 34.00000  0.57236 
#> 
#> $Day5
#>        mse       mape 
#> 34.0000000  0.5580051 
#> 
#> $Day6
#>        mse       mape 
#> 34.0000000  0.5446064 
#> 
#> $Day7
#>        mse       mape 
#> 34.0000000  0.5320546

A word of caution- As 1 %% 7 = 1 (and not 0) the second col will be considered as day-1 and upto seventh column as day-1 and first column will be day-7

user_28848590
Related