I have a query that theoretically works but in practice doesn't have the performance to do the job. The idea is to get the week balance for column MTNSTT(quantity balance) within table INFORM3_M3FDB_MITTRA. However this is a transaction table, and sometimes there aren't any transactions for an entire month for certain articles. So there are gaps in the balance for which I came with the idea to use a CROSS JOIN on a Calendar Table, to find the LAST BALANCE closest to the End of the Week_ISO date, per Article (and other important fields regarding locations.
EDIT: https://pastebin.com/5V48sfb3 XML Execution Plan of Query, here only for a subset of the data, since the entire query never loads. It is somehow too heavy.
So, very oversimplified, just for the idea, instead of having This data:
| MTNSTT | MTTRGDT |
|--------|----------|
| 100 | 20220101 |
| 150 | 20220115 |
| 75 | 20220204 |
I have this data now available in current solution with the Calendar Table:
| MTNSTT | CALENDAR_ID |
|--------|-------------|
| 100 | 20220102 |
| 100 | 20220109 |
| 150 | 20220116 |
| 150 | 20220123 |
| 150 | 20220130 |
| 75 | 20220206 |
The following three tables are being used:
ODS.INFORM3_M3FDB_FCAAVP with ~11 million records
ODS.INFORM3_M3FDB_MITTRA with ~33 million records
ODS.INFORM3_M3FDB_MITBAL with ~16 million records
The query retrieves the following error while loading overnight:
A severe error occurred on the current command. The results, if any, should be discarded.
The query looks as follows, it does the job, but I assume it is a performance related issue.
I use the transaction table MITTRA twice, because I need two fields per week MTNSTT(balance of quantity) and MTTRPR (price). And I use a similar transaction table INFORM3_M3FDB_FCAAVP to get A7APPR (corrected price). All date columns in the query are YYYYMMDD INT data types.
SELECT
CALENDAR_ID
,WEEKDAY_ISO
,YYYYMM
,MBITNO --Article
,MBWHLO --Wharehouse
,MBFACI --Facility
,MBCONO --Company number
,MBWHSL --Wharehouse Location
,MBDIVI --Division
,(SELECT TOP 1 MTNSTT FROM ODS.INFORM3_M3FDB_MITTRA MITTRA
WHERE MITTRA.IsCurrent = 1
AND MTTRDT <= CALENDAR_ID
AND MTITNO = MBITNO
AND MTWHLO = MBWHLO
AND MTCONO = MBCONO
AND MTWHSL = MBWHSL
ORDER BY CONCAT(MTTRDT,MTTRTM,MTTMSX) DESC) AS [Quantity]
--The order by includes Date, Time and Timesuffix,
--necessary in the order by to get latest balance on specific date
,(SELECT TOP 1 MTTRPR FROM ODS.INFORM3_M3FDB_MITTRA MITTRA
WHERE MITTRA.IsCurrent = 1
AND MTTRDT <= CALENDAR_ID
AND MTITNO = MBITNO
AND MTWHLO = MBWHLO
AND MTCONO = MBCONO
AND MTWHSL = MBWHSL
ORDER BY CONCAT(MTTRDT,MTTRTM,MTTMSX) DESC) AS [Price]
,(SELECT TOP 1 A7APPR FROM ODS.INFORM3_M3FDB_FCAAVP
WHERE IsCurrent = 1
AND A7RGDT <= CALENDAR_ID
AND A7ITNO = MBITNO
AND A7FACI = MBFACI
AND A7WHLO = MBWHLO
AND A7CONO = MBCONO
ORDER BY A7LMTS DESC) AS [Corrected_Price]
--A7LMTS is the date including timestamp
FROM ODS.CALENDAR C
CROSS JOIN (SELECT DISTINCT MBITNO, MBWHLO, MBFACI, MBCONO, MBWHSL, MBDIVI FROM ODS.INFORM3_M3FDB_MITBAL) MITBAL
WHERE CALENDAR_ID <= YEAR(GETDATE())*10000
--WHERE LEFT(CALENDAR_ID,4) <= YEAR(GETDATE())
AND WEEKDAY_ISO = 7 --Here I drop everything except the Sundays, to only have Week balances
I use the following indexes, all fields I used in JOIN / ORDER BY exist in them:
CREATE NONCLUSTERED INDEX [NCL_IX_INFORM3_M3FDB_MITTRA_Balance] ON [ODS].[INFORM3_M3FDB_MITTRA]
(
[MTCONO] ASC,
[MTWHLO] ASC,
[MTWHSL] ASC,
[MTITNO] ASC,
[MTRGDT] ASC,
[MTRGTM] ASC,
[MTTRDT] ASC,
[MTTRTM] ASC,
[MTTMSX] ASC,
[IsCurrent] ASC
)
INCLUDE([MTRSCD],[MTTRPR],[MTTRQT]) WITH (STATISTICS_NORECOMPUTE = OFF, DROP_EXISTING = OFF, ONLINE = OFF, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX [NCL_IX_INFORM3_M3FDB_MITBAL_Balance] ON [ODS].[INFORM3_M3FDB_MITBAL]
(
[MBCONO] ASC,
[MBWHLO] ASC,
[MBWHSL] ASC,
[MBFACI] ASC,
[MBITNO] ASC,
[IsCurrent] ASC
)
INCLUDE([MBDIVI],[MBSUNO]) WITH (STATISTICS_NORECOMPUTE = OFF, DROP_EXISTING = OFF, ONLINE = OFF, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX [NCL_IX_INFORM3_M3FDB_FCAAVP_Balance] ON [ODS].[INFORM3_M3FDB_FCAAVP]
(
[A7CONO] ASC,
[A7FACI] ASC,
[A7ITNO] ASC,
[A7RGDT] ASC,
[A7WHLO] ASC,
[A7TMSX] ASC,
[IsCurrent] ASC
)WITH (STATISTICS_NORECOMPUTE = OFF, DROP_EXISTING = OFF, ONLINE = OFF, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
GO
