Running Scibert on GPU

I built a docker image for Scibert for CPU according to instruction here: https://www.semi.technology/developers/weaviate/current/modules/text2vec-transformers.html#option-3-custom-build-with-a-private-or-local-model

Here the docker file:

FROM semitechnologies/transformers-inference:custom
RUN MODEL_NAME=allenai/scibert_scivocab_uncased ./download.py

Here is the docker-compose.yaml for CPU:

version: '3.4'
services:
  weaviate:
    image: semitechnologies/weaviate:1.7.2
    restart: on-failure:0
    ports:
     - "8080:8080"
    environment:
      QUERY_DEFAULTS_LIMIT: 20
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: "./data"
      DEFAULT_VECTORIZER_MODULE: text2vec-transformers
      ENABLE_MODULES: text2vec-transformers
      TRANSFORMERS_INFERENCE_API: http://t2v-transformers:8080
  t2v-transformers:
    image: scibert-inference
    environment:
      ENABLE_CUDA: 0

I can run it on CPU without error. NOW, I need to run it on the GPU with this docker-compose.yaml:

version: '3.4'
services:
  weaviate:
    image: semitechnologies/weaviate:1.10.1
    restart: on-failure:0
    ports:
     - "8080:8080"
    environment:
      QUERY_DEFAULTS_LIMIT: 20
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: "./data"
      DEFAULT_VECTORIZER_MODULE: text2vec-transformers
      ENABLE_MODULES: text2vec-transformers
      TRANSFORMERS_INFERENCE_API: http://t2v-transformers:8080
  t2v-transformers:
    image: scibert-inference
    environment:
      ENABLE_CUDA: '1'
      NVIDIA_VISIBLE_DEVICES: 'all'
    deploy:
      resources:
        reservations:
          devices:
          - capabilities: 
            - 'gpu'

When I ran it on GPU, I had the following error:

weaviate_1          | {"action":"transformer_remote_wait_for_startup","error":"send check ready request: Get \"http://t2v-transformers:8080/.well-known/ready\": dial tcp: lookup t2v-transformers on 127.0.0.11:53: no such host","level":"warning","msg":"transformer remote inference service not ready","time":"2022-02-08T03:47:54Z"}

My system specifications are as follows:

Cuda compilation tools, release 9.0, V9.0.176
Client: Docker Engine - Community
 Version:           20.10.7
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        f0df350
 Built:             Wed Jun  2 11:56:47 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.7
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       b0f5bc3
  Built:            Wed Jun  2 11:54:58 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.6
  GitCommit:        d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc:
  Version:          1.0.0-rc95
  GitCommit:        b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
docker-compose version 1.29.2, build 5becea4c