I just started using the SSIS tool and I need quick help to load data quarterly
Here`s my scenario:
I came up with a query ( source Database: DB2 ) that will extract data from 2010-01-01 to 2021-12-31,(11 years of data) however the data volume is too high ( around 300 M), so I would like to split the data source query to load data into quarter wise. I tried year wise and still, I am getting more volume of data which my SSIS server is not able to handle. I have created a year loop to loop it through, in that created a script task into it followed by a data flow task.
For example,
select * from tab1 where start_date >= '2010-01-01' and end_Date <= '2010-12-31'
This I would like to loop it as, ( 4 times load, 1 for each quarter)
select from tab1 where start_date >= '2010-01-01' and end_Date <= '2010-03-31'
select from tab1 where start_date >= '2010-04-01' and end_Date <= '2010-06-30'
select from tab1 where start_date >= '2010-07-01' and end_Date <= '2010-09-30'
select from tab1 where start_date >= '2010-10-01' and end_Date <= '2010-12-31'
Year-wise perfectly works fine, however, I am not getting any idea how do I need to load the data into quarter-wise. I want to pass each quarter parameters to the source query as parameters, so overall I need to loop to 48 times ( 2010 to 2021 = 11 yrs * 4 quarters) Any help is greatly appreciated. I can send screenshots of what I have created for the year loop which is working perfectly fine.