Is it possible to use firebase query with Redux-toolkit or RTK Query in React?

Yes, instead of writing RTK Query endpoints with a query property, you would write them with a queryFn instead and use fakeBaseQuery instead of fetchBaseQuery.

import { createApi, fakeBaseQuery } from '@reduxjs/toolkit/query'

const api = createApi({
  baseQuery: fakeBaseQuery(),
  endpoints: (build) => ({
    getSomeDataFromFirebase: build.query({
      queryFn(arg) {
        try {
          const result = ... // call firebase here
          return { data: result.something }
        } catch (e) {
          return { error: e }
        }
      },
    }),
  }),
})