Rookio Ceph cluster : mon c is low on available space message

user_344669 0
Viewed 5354

I have setup RookIO 1.4 cluster in Kubernetes 1.18. with 3 nodes allocated 1TB storage on each of them. after creating cluster. when I run the ceph status cluster status shows as HEALTH_WARN with mon c is low on available space.

There is no data stored yet. why status how low on available space? How to clear this error?

[root@rook-ceph-tools-6bdcd78654-sfjvl /]#  ceph  status
  cluster:
    id:     ad42764d-aa28-4da5-a828-2d87205aff08
    health: HEALTH_WARN
            mon c is low on available space

  services:
    mon: 3 daemons, quorum a,b,c (age 37m)
    mgr: a(active, since 36m)
    osd: 3 osds: 3 up (since 37m), 3 in (since 37m)

  data:
    pools:   1 pools, 1 pgs
    objects: 0 objects, 0 B
    usage:   3.0 GiB used, 3.6 TiB / 3.6 TiB avail
    pgs:     1 active+clean

All three node has same size storage:

sdb                                                                                                    8:16   0  1.2T  0 disk
└─ceph--a6cd601d--7584--4b1f--bf82--48c95437f351-osd--data--ae1bc856--8ded--4b1e--8c87--30ca0f0959a3 253:3    0  1.2T  0 lvm
sdb                                                                                                    8:16   0  1.2T  0 disk
└─ceph--ccaf7144--d6a0--441c--bcd5--6a09d056bd7a-osd--data--36a9b28c--7207--400a--936b--edfb3255ce0b 253:3    0  1.2T  0 lvm
sdb                                                                                                    8:16   0  1.2T  0 disk
└─ceph--53e9b8a9--8925--4b21--a6ea--f8e17a322d5c-osd--data--6b1e779c--a18a--4e4d--960e--73ca9473d02f 253:3    0  1.2T  0 lvm

Thanks SR

ceph kubernetes
2 Answers

This alert is for your monitor disk space that is stored normally in /var/lib/ceph/mon. This path is stored in root fs that isn't related to your OSDs block device. This warn is raised when this path has less than 30% available space (see mon_data_avail_warn which is 30 by default). You can change it to ignore alert or resize that path to have more space for its RocksDB data.

user_50779810

As Seena explained, it was because the available space is less than 30%, in this case, you could compact the mon data by the command as follow.

ceph tell mon.`hostname -s` compact

There is another way to trigger the data compaction for mon, add the mon config to the ceph.conf, and then restart the mon.

[mon]
 mon compact on start = true
user_83802790
Related