wikidata sparql query timeout optimisation

Question

I want to retrieve all instances of musicians (Q639669) in a given city (P131) born after 1900. When I pass in the wikidata example city Rotterdam (Q34370) it works. However, replacing the city with a larger city (e.g., Paris, Q90) it will timeout.

Is there a way to optimise this or split into chunks to make repeated queries?

I'm actually only interested the number of cases it returns (i.e. a single value), without needing all the metadata about the artist name, etc.

Would be really helpful if someone can give me pointers to solving this. Thanks!

SELECT ?itemLabel ?itemDescription ?birth
WHERE {
  ?item wdt:P106/wdt:P279* wd:Q639669 .
  ?item wdt:P19/wdt:P131* wd:Q34370 .
  
  OPTIONAL {?item wdt:P569 ?birth}
  
  filter (?birth > "1900-01-01"^^xsd:dateTime)

  SERVICE wikibase:label {
    bd:serviceParam wikibase:language "en" .
  }
}

user_3101395 · Answer

The * (ZeroOrMore) property path operator used on the P131 ("located in the administrative territorial entity") is one of the culprits here. A simple approach to getting an answer would be to manually run queries that build up that property path one element at a time:

In query 1: ?item wdt:P19 wd:Q34370 .

In query 2: ?item wdt:P19/wdt:P131 wd:Q34370 .

In query 3: ?item wdt:P19/wdt:P131/wdt:P131 wd:Q34370 .

etc.

I found through experimentation that there is no data past 3 occurrences of P131. However, be aware that there are duplicates across these queries, because some people are listed as having birth places both "in Paris" and also in some sub-region of Paris (for example, Claude Arrieu (Q272886) listed as being born in both Paris and the 8th arrondissement).

You can also use UNION to put several of these property paths together into a single query, though be aware that this may increase the query time and move you back towards a timeout depending on the data:

SELECT ?item ?itemLabel ?itemDescription ?birth WHERE {
    ?item wdt:P106 / wdt:P279 * wd:Q639669 .
    {
        ?item wdt:P19 wd:Q90 .
    } UNION {
        ?item wdt:P19 / wdt:P131 wd:Q90 .
    } UNION {
        ?item wdt:P19 / wdt:P131 / wdt:P131 wd:Q90 .
    } UNION {
        ?item wdt:P19 / wdt:P131 / wdt:P131 / wdt:P131 wd:Q90 .
    } UNION {
        ?item wdt:P19 / wdt:P131 / wdt:P131 / wdt:P131 / wdt:P131 wd:Q90 .
    }
    OPTIONAL {
        ?item wdt:P569 ?birth
    }
    FILTER(?birth > "1900-01-01"^^xsd:dateTime)
    SERVICE wikibase:label
    {
        bd:serviceParam wikibase:language "en" .
    }
}

A couple of other comments:

If you only want the count of people, you can replace the SELECT variables with a count, which may improve the runtime a bit: SELECT (COUNT(DISTINCT ?item) AS ?count) WHERE { … } which returns the answer 1440.
The use of OPTIONAL to bind the ?birth variable combined with the FILTER outside of the OPTIONAL may not be what you want. The Filter will remove any results where ?birth is unbound, making the OPTIONAL really non-optional. Consider either removing the OPTIONAL and binding ?birth right next to the FILTER, or moving the FILTER inside the OPTIONAL to apply that date range filter only to people who have birth data (which changes the count from 1440 to 2456 – many musicians born in Paris missing birth dates, it seems!)

wikidata sparql query timeout optimisation

1 Answers