My bulk query has been taking hours in production today. Before this, it takes only a few minutes. Just wondering what might be the issue?
This takes 2 hours (This is a test query on production)
sf.bulk.__getattr__("Account").query("SELECT ID FROM Account limit 1", lazy_operation=True)
This takes 3 seconds
sf.query_all("SELECT ID FROM Account limit 1")
Any ideas? Thanks
The Bulk API is asynchronous by design. While two hours is an extremely long delay, in my experience, the Bulk API may take considerable time to execute a query and make results available. That time is dependent on many factors, including the overall load on your org and the instance where it is hosted. The Bulk API's saving grace is that it can do so even if the query returns enormous numbers of records.
If you continue to see extreme performance delays in the Bulk API, open a support case. There's not much else you can do.
For small data volumes, using the REST API (via query_all(), here) is much, much faster nearly 100% of the time. I've implemented multiple integrations that dynamically select the Bulk or REST APIs based on data volume to attempt to maximize performance.
Side note: you don't need the __getattr__() call, just do
sf.bulk.Account.query(...)