I made a function to calculate Population Weighted Average Densities (PWADs) of randomly generated (x,y) points on a graph in R. The function isn't perfect... I assign points to grid squares based not on the grid square they're in but instead based on which grid centre they're closest to, I then assume the area a centre pulls from is the size of the grid (either 1 or 0.25). The purpose of the function is to test sensitivity to grid location and size under varying "total populations". This is the function (the comments were written for a future version of me):
pwad.grid <- function(xval, yval, totalpop) {
values <- data.frame(xval, yval)
# I'll use the four grids that I've already got
# the original size grids
# I'm not cleaning up this code, i.e. it's as it was originally developed for a
# non-function use when I only wanted to do this once
# again, I'm using minimum Euclidean distance of (x, y) point to grid centre to
# fudge assignation to grids
lowbounds3 <- 0.25
highbounds3 <- 10.25
centres1 <- data.frame(x=seq(0.05, 1, .1) * 10, y=10 *
as.vector(matrix(rep(seq(0.05, 1, .1), each=10),
nrow=10)))
centres2 <- data.frame(x=seq(0, 10, 1), y=10 *
as.vector(matrix(rep(seq(0, 1, .1), each=11),
nrow=11)))
centres3 <- data.frame(x=seq(lowbounds3, highbounds3, 1), y=
as.vector(matrix(rep(seq(lowbounds3, highbounds3, 1),
each=11), nrow=11)))
#the quarter size integer aligned grid
lowbounds4 <- 0.25
highbounds4 <- 9.75
centres4 <- data.frame(x=seq(lowbounds4, highbounds4, .5), y=
as.vector(matrix(rep(seq(lowbounds4, highbounds4, .5),
each=20), nrow=20)))
# and now the stores, which I alter somewhat to allow for varying populations
# reminder: these are the calculations for the Euclidean distances
# the code inside the matrix calculates the distances on a repeated entry basis
# i.e. when there are 100 centres, each (x, y) is repeated 100 times, once for
# each centre #the matrix then arranges the results so that each (x, y) occupies
# only one row once again
stores1 <- matrix(sqrt(rowSums((values[rep(1:totalpop, each=100), ] -
centres1[rep(1:100, totalpop), ])^2)),
ncol=100, byrow=TRUE)
stores2 <- matrix(sqrt(rowSums((values[rep(1:totalpop, each=121), ] -
centres2[rep(1:121, totalpop), ])^2)),
ncol=121, byrow=TRUE)
stores3 <- matrix(sqrt(rowSums((values[rep(1:totalpop, each=121), ] -
centres3[rep(1:121, totalpop), ])^2)),
ncol=121, byrow=TRUE)
stores4 <- matrix(sqrt(rowSums((values[rep(1:totalpop, each=400), ] -
centres4[rep(1:400, totalpop), ])^2)),
ncol=400, byrow=TRUE)
# assigning points to groups based on the minimum Euclidean Distance
groups1 <- max.col(-stores1)
groups2 <- max.col(-stores2)
groups3 <- max.col(-stores3)
groups4 <- max.col(-stores4)
# calculating the PWADs
pwad1 <- sum(table(groups1) * table(groups1)/totalpop)
pwad2 <- sum(table(groups2) * table(groups2)/totalpop)
pwad3 <- sum(table(groups3) * table(groups3)/totalpop)
mill <- table(groups4) / 0.25
pwad4 <- sum(mill * table(groups4)/totalpop)
# outputs grouped together
data.frame(pwad1, pwad2, pwad3, pwad4)
}
In order to look at the effects of varying population size, I have been using for loops within R. Each loop is 1000 iterations and generates four groups of 1000 PWADs (one for each grid type). For a population bigger than 100, the loop takes more than a minute to complete on my machine. For a population of 1000 it takes about 12-13 minutes. Based on the various populations I've already done, I expected a population of 5000 to take about 66 minutes. That's an age, but I was going out so why not run it?
This is the loop and the antecedent code I ran for the population of 5000:
# I created sims earlier when I ran my very first population.
sims <- data.frame(baseline=1:1000, ptfive=1:1000, pt75=1:1000, qtrsize=1:1000)
# I did not run it again when I ran the below:
xvalues <- matrix(runif(5000 * 1000) * 10, ncol=1000)
yvalues <- matrix(runif(5000 * 1000) * 10, ncol=1000)
dim(xvalues)
start_time <- Sys.time()
for (i in 1:1000) {
xval <- xvalues[, i]
yval <- yvalues[, i]
sims[i, ] <- pwad.grid(xval, yval, 5000)
#commented out just in case I forget and run all chunks
}
end_time <- Sys.time()
#started at 5:51, expect to finish approx 6:51-
#write.csv(sims, "5000sim.csv")
end_time - start_time
And this is the console output running that (sims aside) generated:
xvalues <- matrix(runif(5000 * 1000) * 10, ncol=1000)
yvalues <- matrix(runif(5000 * 1000) * 10, ncol=1000)
dim(xvalues)
# [1] 5000 1000
start_time <- Sys.time()
for (i in 1:1000) {
xval <- xvalues[, i]
yval <- yvalues[, i]
sims[i, ] <- pwad.grid(xval, yval, 5000)
#commented out just in case I forget and run all chunks
}
# Warning: stack imbalance in 'for', 2 then -1
end_time <- Sys.time()
end_time - start_time
# Time difference of 1.251416 hours
As you can see, I got a warning (not an error!). Unfortunately, because I've been saving the outputs as .csv files, I haven't used set.seed() so it was the specific set of numbers I used that caused the warning...
My questions are these:
- What is "Warning: stack imbalance in 'for', 2 then -1"?
- Are my results for the population of 5000 compromised?
- Why did the warning happen?
- How might I avoid it if I ran the code for 10000 while watching a movie/overnight?
In searching Google, I see mostly descriptions of "stack imbalance" in the context of Rcpp or different languages. As you can see, I have used only base R functions to build my function, and a for loop, which is also from base R.
In case it's a memory thing:
But that's post-loop. I don't know what it was at before or during running it.
Not sure about tags, let me know if more details are needed. Many thanks!
Completion predictions from SLR:
cloudnumber <- c(10, 100, 250, 500, 500, 750, 900, 1000, 1250)
yseconds <- c(9.173949, 55.87789, 2.186122 * 60, 4.707054 * 60, 4.606928 * 60,
7.831578 * 60, 9.376838 * 60, 12.30255 * 60, 15.15093 * 60)
runtime <- lm(yseconds ~ cloudnumber)
predict(runtime, data.frame(cloudnumber=newdata, type="response")) / 60
Adding in the length for the 5000 population adjusts the predicted 10000 population from 120.62128 minutes to 150.58184 minutes.