Cannot index a document in a self-hosted Vespa instance

Viewed 95

We're trying to build our own Vespa cluster starting with a Kubernetes quick start, and indexing a document returns a "no space" error.

The problem is that we can't really understand where do we add that space. Any pointers on what to read/change?


The error we're getting:

{"pathId":"/document/v1/product/product/docid/1739707:651",
"id":"id:product:product::1739707:651",
"message":"[UNKNOWN(251009) @ tcp/vespa-0.vespa.staging.svc.cluster.local:19112/default]: ReturnCode(NO_SPACE, External feed is blocked due to resource exhaustion: disk on node 0 [vespa-0.vespa.staging.svc.cluster.local] (0.838 > 0.800)) "}

(are those numbers - 838 and 800 - indicating space necessary?)

Our application configuration looks like this:

> cat hosts.xml
<?xml version="1.0" encoding="utf-8" ?>
<!-- https://docs.vespa.ai/en/reference/hosts.html -->
<hosts>
  <host name="localhost">
    <alias>node1</alias>
  </host>
</hosts>

> cat services.xml
<?xml version="1.0" encoding="utf-8" ?>
<services version="1.0">
  <container version="1.0" id="default">
    <document-api/>
    <search/>
    <nodes>
      <node hostalias="node1" />
    </nodes>
  </container>

  <content id="product" version="1.0">
    <redundancy>1</redundancy>
    <documents>
      <document type="product" mode="index" />
    </documents>
    <nodes>
      <node hostalias="node1" distribution-key="0" />
    </nodes>
  </content>
</services>
1 Answers

Okay, properly asking question is a half way to an answer. Our disk usage looked like this:

overlay               98G   77G   17G  83% /

So Vespa wanted 80% of used space max and we provided storage with 83% used disk space. The solution is to add section like this side content tag:

    <tuning>
      <resource-limits>
        <disk>0.9</disk>
      </resource-limits>
    </tuning>
Related