How to get all marklogic search records in single Rest API call?

user_4715785 0
Viewed 46

I am using Marklogic rest Api to search records in the marklogic server. I need to get all the results in a single call. As the default pagelength is set to 10, I am able to retrieve only 10 records if there is no specific mentione. Is there a way to retrieve all the records in a single marklogic rest api call?

marklogic search
2 Answers

To answer the specific question:

Please have a look at the options related to the endpoint.

v1/search

You will notice 2 of interest:

-start

-pageLength

This is probably what you are looking for.

However.... You want to get all of the results. Is that 10? 200? 2 million? It is likely that after a certain number of results, you will either (1) run out of memory or (2) time out. Pagination on RESTful services allows you to return your results in batches - allowing you t still get all results, but do it in a way that plays nicely with available resources.

user_32154850

Try to change the settings for the snippets. The default values might limit the number of responses which is misleading but in fact you are telling the search engine to return fewer responses. For that you need to use a structured search in which you define an xml or json search query and using it in the body of the post request. Here is a json structured query:

{
  "query": {
    "queries": [
      {
        "term-query": {
          "text": [
            "text to be searched"
          ]
        }
      }
    ]
  },
  "options": {
    "transform-results": {
      "apply": "snippet",
      "preferred-matches": "",
      "max-matches": 10,
      "max-snippet-chars": 10000,
      "per-match-tokens": 10
    }
  }
}

Play with the three attributes max-matches, max-snippet-chars, per-match-tokens to tune the number of responses.
max-matches indicates for each document how many snippets to be returned at most.
per-match-tokens indicates how many words in each snippet surround the main searched text at most.
max-snippet-chars is the overall number of characters in all returned snippets at most. Use a large number here.
More details in MarkLogic help page:
"Modifying Your Snippet Results" in chapter 8 (Search Customization Using Query Options) Link
"Example: Simple Structured Search" in Chapter 4 (Searching Using Structured Queries) Link

user_118452400
Related