Microsoft R data ingestion is too slow

Viewed 397

Some time ago I wrote this question because I thought that the query was too slow but after much trial and error I found out that the problem is data ingestion in R. This query takes now 2 seconds to retrieve circa 175000 rows but when it's run in R, via input_data_1 it takes like 15 seconds. The execution plan tells it's the table scan that takes 100% but I can't understand why such a difference.

Any help is appreciated.

https://dba.stackexchange.com/questions/178242/queries-to-in-memory-table-are-slower-than-to-disk-based?noredirect=1#comment347134_178242

EDIT

Here is the code just in case. It's the same as in the query with no processing. I just return what's been read:

EXEC sp_execute_external_script
    @input_data_1 = N'SELECT DEPARTMENT, ACCOUNT, CATEGORY, YEAR, MONTH, CURRENCY, 
                             DIM1, DIM2, DIM3, DIM4, DIM5, VALUE
                      from   DATA 
                      where  YEAR = ''2013bdg''',
    @input_data_1_name=N't',
    @language = N'R',
    @script = N'OutputDataSet <- as.data.frame(t);'
    WITH RESULT SETS (([DEPARTMENT]   VARCHAR (MAX), [ACCOUNT]  VARCHAR (MAX), [CATEGORY] VARCHAR (MAX),
                      [YEAR] VARCHAR (MAX), [MONTH]     VARCHAR (MAX), [CURRENCY] VARCHAR (MAX),
                      [DIM1] VARCHAR (MAX), [DIM2] VARCHAR (MAX), [DIM3] VARCHAR (MAX), 
                      [DIM4] VARCHAR (MAX), [DIM5] VARCHAR (MAX), [VALUE]     FLOAT));
GO

EDIT #2

Through trial and error I have discovered that the slowness is due to the WITH RESULT SETS operator. The one I use (12 columns) is very expensive while if i set it like:

WITH RESULT SETS (( OK BIT not null))

it takes only 3 seconds.

Why this happens is the question.

0 Answers
Related