Why SAS "proc sql" is way too slower than "data step"

Viewed 125

I was trying to calculate past average stock returns. I find using the following "data step" code is much better than using "proc sql" code.

The data step code:

%macro same(start = ,end = );
proc sql;drop view temp;quit;

proc sql;
    create table temp
    as select distinct a.*, mean(b.ret_dm) as same_&start._&end, count(b.ret_dm) as sc_&start._&end
    from msf1 as a left join msf1 as b
    on a.stkcd = b.stkcd and &start <= a.ym - b.ym <= &end and a.month = b.month
    group by a.stkcd,a.ym;
quit;

proc sql;
    create table same
    as select a.*, b.same_&start._&end, b.sc_&start._&end
    from same as a left join temp as b
    on a.stkcd = b.stkcd and a.ym = b.ym;
quit;

proc sql; drop table temp;quit;
%mend;

data same; set msf;run;
%same(start = 1,  end = 12);

The proc sql code:


%macro MA_1;
%do p = 2 %to 9; *;  
    %put p &p;
    proc printto log = junk ; run;      

    proc sql;
        create table price&p
        as select distinct a.*, b.count,b.ym
        from price&p as a left join tradingdate as b
        on a.date = b.date;
    quit;

    proc sort data = price&p; by stkcd ym date;quit;
    data msf;
        set price&p;
        by stkcd ym date;
        if last.ym;
    run;
    proc printto; run;  

    %do j = 1 %to %sysfunc(countw(&laglist));
        %let lag = %scan(&laglist,&j);
        %put lag &lag;

    /*********************************************/
        proc sql; drop table ma_&lag._&p ;quit;
        %do i = 1 %to 2018; *;
            proc printto log = junk ; run;      
            data getname;
                set stock;
                if _n_ = &i;
                call symput('stkcd',stkcd);
            run;
            proc printto; run;  
            %put &i &stkcd;
            proc printto log = junk ; run;      
            proc sql;
                create table temp
                as select distinct a.*, mean(b.prc) as ma_&lag._&p
                from msf (where = (stkcd = "&stkcd" )) as a left join price&p (where = (stkcd = "&stkcd" )) as b
                on a.stkcd = b.stkcd and 0 <= a.count - b.count <= &lag
                group by a.stkcd, a.date
                order by a.stkcd, a.date;
            quit;

            proc append base = ma_&lag._&p data = temp force; quit; 
            proc printto; run;  
        %end;
        dm "log; clear;";
        proc sql;
            create table ma_allprc
            as select a.*, b.ma_&lag._&p
            from ma_allprc as a left join ma_&lag._&p  as b
            on a.stkcd = b.stkcd and a.date = b.date;
        quit;

        proc sql; drop table ma_&lag._&p;quit;
    %end;
%end;
%mend;

%let laglist = 5 10 20 50 100 200 500 1000 2000; *  ;
data ma_allprc; set msf;run;
%ma_1;

"Proc sql" is much slower than I thought. "Data step" takes about 3 hours, but "Proc sql" takes about 2 days.

I even have to loop over each stock when using proc sql, cause it takes up too much of the memory space, I have to say that using proc sql to calculate past averages is dumb, but currently I have no better ideas. :(

Does anybody have a solution with that..

0 Answers
Related