Why am I seeing consistency issues with RocksDB (Java)?

Viewed 33

Relevant code

    public RocksCache(final RocksFactory RocksFactory, final boolean useSync) throws RocksDBException {
        db = RocksFactory.getDbInstance();
        rocksPath = RocksFactory.getDbPath();
        this.RocksFactory = RocksFactory;
        this.writeOptions = new WriteOptions().setSync(useSync);
    }
    
    public void put(String id, DataHolder holder) throws RocksCacheException {
        put(id.getBytes(StandardCharsets.UTF_8), holder);
    }
    
    void put(byte[] key, DataHolder DataHolder) throws RocksCacheException {
        try (WriteBatch writeBatch = new WriteBatch()) {
            writeBatch.put(key, DataHolder.toBytes());
        } catch (RocksDBException | JsonProcessingException e) {
            log.error("Unable to serialize {} due to {}", DataHolder, e.getMessage());
            throw new RocksCacheException(e);
        }
    }

Test Code

public void setup() throws Exception {
    String dbName = RandomStringUtils.randomAlphabetic(15);
    RocksOptions options = RocksOptions.builder()
            .createIfNeeded(true)
            .path("build/private/rocks")
            .dbName(dbName + ".db")
            .infoLogLevel(InfoLogLevel.WARN_LEVEL)
            .numLevels(3)
            .maxBackgroundJobs(4)
            .maxFileOpeningThreads(4)
            .build();
    RocksFactory factory = new RocksFactory(options);
    cache = new RocksCache(factory, true);
    cache.open();
}

@Test
public void testConsistency() throws Exception {
    for (int i = 0; i < 1000; i++) {
        try {
            testRocksConsistency();
        } catch (Exception e) {
            Assert.fail();
        }
    }
}

public void testRocksConsistency() throws Exception {
    Thread t2;
    String id = "documentId1";
    long initialTx = ThreadLocalRandom.current().nextLong();
    List<Long> transactions = IntStream.rangeClosed(1, 100).mapToObj(i -> initialTx + i)
            .collect(Collectors.toList());

    t2 = new Thread(() -> {
        List<String> states = List.of("state1", "state2");
        for (int i = 1; i < 1000; i++) {
            try {
                FutureTask<Optional<DataHolder>> f1 = new FutureTask<>(() -> cache.get(id));
                Document document = createDocument();
                document.setState(states.get(i % 2));
                DataHolder dataHolder = new DataHolder(document,
                        System.currentTimeMillis(), transactions.get(i), false, "");
                cache.put(id, dataHolder);
                Thread t1 = new Thread(f1);
                t1.start();
                Optional<DataHolder> docFromCache = f1.get();
                Assert.assertTrue(docFromCache.isPresent());
                Assert.assertEquals(DataHolder, docFromCache.get());
            } catch (Exception e) {
                throw new RuntimeException(e);
            }
        }
    });
    t2.start();
}

When I run the test, I get error messages like this:

Exception in thread "Thread-1839" java.lang.AssertionError: expected [true] but found [false]
    at org.testng.Assert.fail(Assert.java:99)
    at org.testng.Assert.failNotEquals(Assert.java:1037)
    at org.testng.Assert.assertTrue(Assert.java:45)
    at org.testng.Assert.assertTrue(Assert.java:55)
    at RocksCacheTest.lambda$testRocksConsistency$2(RocksCacheTest.java:181)
    at java.base/java.lang.Thread.run(Thread.java:832)

This Assertion fail is on the line Assert.assertTrue(docFromCache.isPresent());

So, the DataHolder is not correctly written to Rocks.

If I change the test to write and read from Rocks with the same thread, it all works correctly.

0 Answers
Related