Algolia - get mass records & delete with filter

user_17461896 0
Viewed 112

I am using Algolia for search purposes and we got a huge pile of records. We want to delete some records and have decided to delete records that are older than X date.

First I was using this

const records = [];
const deleteRecordsBeforeDateAlgolia = (date) => {
    let client;

    *****

    const index = client.initIndex('function_message');
    //get the records before the given date
    try {
        index.search('',{
            filters: `time_stamp < ${date}`
        }).then(({hits}) => {
            if(hits.length > 0) {
                for (const hit of hits) {
                    index.deleteObject(hit.objectID);
                    records.push(hit.objectID);
                }
            }
            if(hits.length === 0) {
                console.log(`Deleted ${records.length} records`);
            } else {
                deleteRecordsBeforeDateAlgolia(date);
            }
        });
    } catch (err) {
        console.error(err);
    }
};

but I realized this isnt that optimized + will be very slow when deleting on prod. Can you tell me how I can get huge amounts of data with a filter (timestamp in this case) and then delete all of them?

EDIT

const records = [];
const deleteRecordsBeforeDateAlgolia = (date) => {
    let client;

    //creds stuff


    const index = client.initIndex('function_message');
    //get the records before the given date
    try {
        const search =  index.browseObjects({
            filters: `time_stamp < ${date}`
        }).then(res => {
            //IT SHOWS RESPONSE IS UNDEFINED
            res.forEach(record => {
                records.push(record);
            });
            console.log(`found ${records.length} records`);
        });
    } catch (err) {
        console.error(err);
    }
};
algolia node.js javascript
1 Answers

browseObjects takes a batch callback function that's called on every batch of hits where you can specify what to do with the batch. The optional parameter list can be found here

Something like this should work

const records = [];

const deleteFromIndex = (idArray,index) => {
    index.deleteObjects(idArray).then(({ objectIDs }) => {
        console.log(objectIDs);
    });
}


const deleteRecordsBeforeDateAlgolia = (date) => {
  let client;
  client = algoliasearch('algoliaApp', 'algoliaKey');

  const index = client.initIndex('function_message');
  try {
    index.browseObjects({
        filters: `time_stamp<${date}`,
        query: '',
        batch: (batch) => {       
          records.push(...batch);  //push each batch into records array
        }
      })
      .then(() => {
        const idArray = records.map(({objectID}) => objectID) //get an array of objectIDs
        deleteFromIndex(idArray, index)
      });
  } catch (err) {
    console.error(err);
  }
};

deleteRecordsBeforeDateAlgolia('some date')
user_135835100
Related