I'm trying to get a table from a pdf. I'm so close, but the problem now lies in the fact that some row names (Region) span two lines in the pdf because they are too long (e.g. "Asia Pacific High Income"). It seems that str_replace_all checks left-to-right and does not recognise the row name as one continuous set of characters.
library(tidyverse)
library(pdftools)
file <- list.files("~/Documents/Coding",
pattern = "Dementia",
full.names = TRUE)
raw_text <- map(file, pdf_text)
# Scrape and clean table
clean_table <- function(raw) {
raw <- map(raw, ~ str_split(.x, "\\n") %>%
unlist())
raw <- reduce(raw, c)
# Define start and end points of table in pdf
table_start <- str_which(tolower(raw), "region ")
table_start <- table_start[1]
table_end <- str_which(raw, regex("world ", ignore_case = T))
table_end <- table_end[1]
# Build table and remove special characters
table <- raw[(table_start):(table_end)]
table <- str_replace_all(table, "\\s{2,}", "|")
print(table)
text_con <- textConnection(table)
data_table <- read.csv(text_con, sep = "|")
# Create a list of column names
colnames(data_table) <- c("Region, 2015, 2020, 2025, 2030, 2035, 2040, 2045, 2050")
data_table
}
results <- map_df(raw_text, clean_table)
head(results)
It outputs one very close but incorrect data frame as a result.
> head(results)
Region, 2015, 2020, 2025, 2030, 2035, 2040, 2045, 2050 ...2 ...3 ...4 ...5 ...6 ...7 ...8 ...9
1 Australasia 0.35 0.40 0.48 0.56 0.67 0.77 0.87 0.96
2 Asia Pacific High NA NA NA NA NA NA NA NA
3 3.57 4.23 4.86 5.48 6.10 6.67 7.04 7.40
4 Income NA NA NA NA NA NA NA NA
5 Oceania 0.02 0.03 0.03 0.04 0.05 0.06 0.08 0.09
6 Asia Central 0.33 0.36 0.41 0.47 0.57 0.71 0.84 0.95
>
> dput(raw_text)
list(c("Numbers of people with dementia around the world\nSummary\nThere are over 50 million people worldwide living with dementia in 2020. This number will almost\ndouble every 20 years, reaching 82 million in 2030 and 152 million in 2050. Much of the increase\nwill be in developing countries. Already 60% of people with dementia live in low and middle income\ncountries, but by 2050 this will rise to 71%. These figures are estimates based on the best currently\navailable evidence.\n\nThe 2015 estimates\nThe World Alzheimer Report 2015, ‘The global impact of dementia: An analysis of prevalence,\nincidence, cost and trends’, presented ADI’s global dementia data. By carrying out a full update of\nprevious systematic reviews, the report made key recommendations to provide a global framework\nfor action on dementia.\n\nThe report also included a systematic review of the evidence for and against recent trends in the\nprevalence and incidence of dementia over time, as well as an analysis of the broader societal\nimpact of dementia.\n\nLatest update\nThe global estimates of the number of people living with dementia were updated by the authors in\nDecember 2017, to be published for the launch of the World Health Organization’s Global Dementia\nObservatory. The updated estimates are presented below. Full details of the methodology, and\nfurther discussion, can be found in the World Alzheimer Report 2015.\n\nPrevious systematic reviews estimating global prevalence of dementia carried out for the World\nAlzheimer Report 2009 and 2015 were updated (up to April 2017). Numbers of people living with\ndementia were estimated by applying the age- and sex-specific prevalence of dementia for each\nregion to the UN population estimates for 2015 and the population projections until 2050.\n\nThe noticeable increase in the projected numbers of people living with dementia worldwide from the\n2015 estimates to the 2017 estimates is mainly due to the revised UN population estimates (2015\nUN estimates instead of the 2013 UN estimates used in the World Alzheimer Report 2015) and new\nevidence which impacted on the strategies used to estimate numbers for some of the Global Burden\nof Disease (GBD) regions, rather than secular trends in the prevalence of dementia.\n\nLimitations\nAs before, the projections assume that age-specific and age- and sex-specific prevalence of\ndementia in each region will remain constant over time, which is unlikely to be the case, particularly\nin regions undergoing rapid demographic, epidemiologic and social changes.\n\nAs highlighted in the World Alzheimer Report 2015, there is a need for more and better monitoring\nof the prevalence of dementia worldwide.\n",
"Estimated numbers of people with dementia by region (millions)\n\n Year\n Region 2015 2020 2025 2030 2035 2040 2045 2050\n Australasia 0.35 0.40 0.48 0.56 0.67 0.77 0.87 0.96\n Asia Pacific High\n 3.57 4.23 4.86 5.48 6.10 6.67 7.04 7.40\n Income\n Oceania 0.02 0.03 0.03 0.04 0.05 0.06 0.08 0.09\n Asia Central 0.33 0.36 0.41 0.47 0.57 0.71 0.84 0.95\n Asia East 9.49 11.50 14.14 17.68 21.85 26.35 31.21 36.12\n Asia South 6.96 8.41 10.12 12.15 14.56 17.47 20.94 24.86\n Asia Southeast 3.55 4.31 5.23 6.33 7.56 8.85 10.18 11.37\n Asia 24.28 29.23 35.28 42.71 51.37 60.90 71.16 81.75\n Europe Central 1.51 1.69 1.85 2.04 2.25 2.50 2.70 2.82\n Europe Eastern 2.03 2.10 2.20 2.23 2.45 2.68 2.77 2.77\n Europe Western 8.01 8.92 9.92 11.06 12.28 13.63 14.88 16.05\n Europe 11.55 12.71 13.97 15.33 16.98 18.82 20.35 21.64\n North America High\n 4.40 5.01 5.79 6.77 8.00 9.25 10.29 11.10\n Income\n Caribbean 0.41 0.48 0.56 0.66 0.77 0.88 1.01 1.14\n Latin America Andean 0.42 0.52 0.63 0.77 0.94 1.15 1.40 1.68\n Latin America Central 1.80 2.22 2.73 3.37 4.19 5.19 6.39 7.71\n Latin America\n 0.90 1.04 1.20 1.39 1.62 1.89 2.19 2.51\n Southern\n Latin America Tropical 1.69 2.14 2.70 3.38 4.19 5.18 6.30 7.48\n The Americas 9.62 11.42 13.60 16.33 19.70 23.54 27.59 31.63\n North Africa / Middle\n 2.49 2.99 3.62 4.44 5.52 6.85 8.40 10.10\n East\n Sub-Saharan Africa\n 0.21 0.25 0.30 0.36 0.44 0.53 0.65 0.79\n Central\n Sub-Saharan Africa\n 0.85 1.02 1.21 1.46 1.77 2.17 2.70 3.38\n East\n Sub-Saharan Africa\n 0.25 0.28 0.32 0.36 0.42 0.49 0.58 0.68\n Southern\n Sub-Saharan Africa\n 0.66 0.76 0.89 1.05 1.26 1.53 1.86 2.26\n West\n Africa 4.47 5.30 6.34 7.67 9.40 11.57 14.18 17.22\n\n World 49.92 58.66 69.20 82.05 97.45 114.83 133.28 152.24\n\n\n\nEstimated numbers of people with dementia by World Bank income group (millions)\n\n Year\n Income group 2015 2020 2025 2030 2035 2040 2045 2050\n Low 1.41 1.67 1.98 2.37 2.85 3.45 4.24 5.21\n Lower middle 11.81 14.06 16.82 20.10 24.05 28.74 34.13 40.00\n Upper middle 16.61 20.15 24.62 30.42 37.38 45.23 53.85 62.73\n High 20.09 22.78 25.77 29.17 33.16 37.41 41.06 44.30\n World 49.92 58.66 69.20 82.05 97.45 114.83 133.28 152.24\n"
))