SAS PROC SQL: How to clear cache between testing

user_1276534 0
Viewed 837

I am reading this paper: "Need for Speed - Boost Performance in Data Processing with SAS/Access® Interface to Oracle". And I would like to know how to clear the cache / buffer in SAS, so my repeated query / test will be reflective of the changes accurately?

I noticed the same query running the first time takes 10 seconds, and (without) changes running it immediately after will take shorter time (say 1-2 seconds). Is there a command / instruction to clear the cache / buffer. So I can have a clean test for my new changes.

I am using SAS Enterprise Guide with data hosted on an Oracle server. Thanks!

proc-sql sas
2 Answers

In order to flush caches on the Oracle side, you need both DBA privileges (to run alter system flush buffer_cache; in Oracle) and OS-level access (to flush the OS' buffer cache - echo 3 > /proc/sys/vm/drop_caches on common filesystems under Linux).

If you're running against a production database, you probably don't have those permissions -- you wouldn't want to run those commands on a production database anyways, since it would degrade the performance for all users of the database, and other queries would affect the time it takes to run yours.

Instead of trying to accurately measure the time it takes to run your query, I would suggest paying attention to how the query is executed:

  • what part of it is 'pushed down' to the DB and how much data flows between SAS and Oracle
  • what is Oracle's explain plan for the query -- does it have obvious inefficiencies

When a query is executed in a clearly suboptimal way, you will find (more often than not) that the fixed version will run faster both with cold and hot caches.

To apply this to the case you mention (10 seconds vs 2 seconds) - before thinking how to measure this accurately, start by looking

  • if your query gets correctly pushed down to Oracle (it probably does),
  • and whether it requires a full table (partition) scan of a sufficiently large table (depending on how slow the IO in your DB is - on the order of 1-10 GB).

If you find that the query needs to read 1 GB of data and your typical (in-database) read speed is 100MB/s, then 10s with cold cache is the expected time to run it.

user_10260

I'm no Oracle expert but I doubt there's any way you can 'clear' the oracle cache (and if there were you would probably need to be a DBA to do so).

Typically what I do is I change the parameters of the query slightly so that the exact query no longer matches anything in the cache. For example, you could change the date range you are querying against.

It won't give you an exact performance comparison (because you're pulling different results) but it will give you a pretty good idea if one query performs significantly better than the other.

user_2149940
Related