How to put all java ExecutorService's threads in sleep after having Google my business Exception :RATE_LIMIT_EXCEEDED

user_13860286 0
Viewed 69

In order to read all google updates for the whole locations of I have in MyBusinessBusinessInformation.Locations.GetGoogleUpdated, I am using ExecutorService, Supposing mybusinessUpdateLocations is of type MyBusinessBusinessInformation which takes care of authentication as well, and the variable list is of type java.util.List containing required info of the locations to do the request from Google. The problem is that sometimes my list includes more than 3000 locations and Google returns me Exception. my abstract code is:

 ExecutorService ex =Executors.newFixedThreadPool(20);
    for(int i=0;i<list.size();i++) {
    ex.execute(new Runnable() {
        @Override
    public void run() {
     try {
    String locationName ="The location name";
    String readMask="storeCode,regularHours,name,languageCode,title,phoneNumbers,categories,storefrontAddress,websiteUri,regularHours,specialHours,serviceArea,labels,adWordsLocationExtensions,latlng,openInfo,metadata,profile,relationshipData,moreHours";
                                
        MyBusinessBusinessInformation.Locations.GetGoogleUpdated 
     updateList=mybusinessUpdateLocations.locations()
        .getGoogleUpdated(locationName).setReadMask(readMask);
        GoogleUpdatedLocation response = updateList.execute();
      } catch (Exception e) {
        System.out.println( e);
      }

in order to make the program run faster I used ExecutorService , however sometimes the code falls into Exception and I lose the specific location info:

 {
  "code" : 429,
  "details" : [ {
    "@type" : "type.googleapis.com/google.rpc.ErrorInfo",
    "reason" : "RATE_LIMIT_EXCEEDED"
  } ],
  "errors" : [ {
    "domain" : "global",
    "message" : "Quota exceeded for quota metric 'Requests' and limit 'Requests per minute' of service 'mybusinessbusinessinformation.googleapis.com' for consumer 'project_number:XXXXXXXXX'.",
    "reason" : "rateLimitExceeded"
  } ],
  "message" : "Quota exceeded for quota metric 'Requests' and limit 'Requests per minute' of service 'mybusinessbusinessinformation.googleapis.com' for consumer 'project_number:XXXXXXXXXXX'.",
  "status" : "RESOURCE_EXHAUSTED"
}

The problem is that how I can put ALL the threads in sleep after a certain number of requests to Google API? for example after every 100 requests I want the executor to sleep for some seconds.

Thread.sleep(30000);

just put one thread on sleep and not all of them. Any help would be appreciated.

google-api-client google-api executorservice multithreading java
1 Answers

You can use different solutions , i have the idea of combining the usage of CyclicBarrier and Thread.sleep in a for loop, you can inspire from this example , run it an see the behaviour:

public class RequestSender {
    private void sendRequest() {
        System.out.println("Send request and wait 0,5 second");
    }

    public void performTask(CyclicBarrier c1) {
        try {
            sendRequest();
            c1.await();
        } catch (InterruptedException | BrokenBarrierException e) {
            e.printStackTrace();
        }
    }

    public static void main(String[] args) {
        ExecutorService service = null;
        try {
            service = Executors.newFixedThreadPool(4);
            RequestSender sender = new RequestSender();
            CyclicBarrier c1 = new CyclicBarrier(2, () -> System.out.println("***  2 Threads worked !"));

            for (int i = 0; i < 16; i++) {
                service.submit(() -> sender.performTask(c1));
                try {
                    // after 8 request , wait for 5 seconds
                    if(i==8)
                    Thread.sleep(5000);
                    else // normal case , wait just for 1 second to send a new request 
                        Thread.sleep(1000);
                    
                } catch (InterruptedException e) {
                    // TODO Auto-generated catch block
                    e.printStackTrace();
                }
            }
        } finally {
            if (service != null)
                service.shutdown();
        }
    }
}

CyclicBarrier

A synchronization aid that allows a set of threads to all wait for each other to reach a common barrier point. CyclicBarriers are useful in programs involving a fixed sized party of threads that must occasionally wait for each other. The barrier is called cyclic because it can be re-used after the waiting threads are released. A CyclicBarrier supports an optional Runnable command that is run once per barrier point, after the last thread in the party arrives, but before any threads are released. This barrier action is useful for updating shared-state before any of the parties continue.

user_170269000
Related