How Hash Group By and Sort Group By internally work in Oracle?

Oracle's SQL Tuning Guide says in the PLAN_TABLE reference:

HASH GROUP BY: Operation hashing a set of rows into groups for a query with a GROUP BY clause.

SORT GROUP BY: Operation sorting a set of rows into groups for a query with a GROUP BY clause.

Hmm. Not very enlightening. Sort merge join and hash join are documented in Doc Id 41954.1, and I believe that the sort and hash part of GROUP BY works similar to JOIN.

Algorithm

Given an example table:

CREATE TABLE t2 (id VARCHAR2(30), amount NUMBER);
INSERT INTO t2 VALUES ('A', 10);
INSERT INTO t2 VALUES ('C',  5);
INSERT INTO t2 VALUES ('B',  1);
INSERT INTO t2 VALUES ('B',  2);
INSERT INTO t2 VALUES ('A',  3);
INSERT INTO t2 VALUES ('C',  1);
INSERT INTO t2 VALUES ('A',  7);

SELECT id, sum(amount)
  FROM t2
 GROUP BY id;

My understanding is that a SORT GROUP BY would sort the whole table (in memory or on disk) and then run the aggregate function, for instance SUM(amount):

ID   Amount SUM
A    10
A     3
A     7     10+3+7=20

B     1
B     2     1+2=3

C     5
C     1     5+1=6

Whereas a HASH GROUP BY would scan the table once, compute a hash value for each row, and put the row into a bucket (in memory or on disk):

SELECT id, ora_hash(id, 4), amount from t2;

ID Bucket  Amount  Hash table
A     2      10    Bucket#2: A=10
C     4       5    Bucket#4: C=5
B     2       1    Bucket#2: A=10, B=1
B     2       2    Bucket#2: A=10, B=1+2
A     2       3    Bucket#2: A=10+3, B=1+2
C     4       1    Bucket#4: C=5+1
A     2       7    Bucket#2: A=10+3+7, B=1+2

After putting all the values into buckets, it needs to scan the hash table to calculate the aggregate:

Bucket#2: A=10+3+7, B=1+2
Bucket#4: C=5+1

Performance

We need a bigger table to measure the performance:

CREATE TABLE t AS 
SELECT RPAD(object_type, 3000, 'x') as gby, o.* 
  FROM all_objects o WHERE rownum <= 50000; COMMIT;
INSERT INTO t SELECT * FROM t; COMMIT;
EXEC dbms_stats.gather_table_stats(user, 't');

You can ask for a HASH GROUP BY with the hint USE_HASH_AGGREGATION:

SELECT /*+ USE_HASH_AGGREGATION */ gby, count(*)
  FROM t
 GROUP BY gby; 

Likewise for a SORT GROUP BY with the hint NO_USE_HASH_AGGREGATION:

SELECT /*+ NO_USE_HASH_AGGREGATION */ gby, count(*)
  FROM t
 GROUP BY gby; 

If you dig out the SQL_IDs, you can inspect the amount of memory needed by each operation:

SELECT * FROM v$sql WHERE sql_text LIKE '%USE_HASH_AGGREGATION%'; 

SELECT * FROM v$sql_workarea WHERE sql_id IN ('663t56n1tdr59','fp5z7z1fyz42p');

OPERATION_TYPE  EST_OPT_SIZE LAST_MEM_USED ACTIVE_TIME MAX_TEMP
GROUP BY (HASH)       697344       1519616      325145        -
GROUP BY (SORT)       145408        129024      460975        -

So, GROUP BY HASH needed 1519616 bytes memory and ran in 0.325145 seconds, while GROUP BY SORT used less than an tenth of cache, but ran slightly longer. Both ran completely in memory.

If it doesn't fit in memory and spills out to disk (which we can force here by lowering the memory limit artificially), the column max_tempseg_size is filled:

ALTER SESSION SET workarea_size_policy = MANUAL;
ALTER SESSION SET sort_area_size = 10000;

OPERATION_TYPE  EST_OPT_SIZE LAST_MEM_USED ACTIVE_TIME  MAX_TEMP
GROUP BY (HASH)       697344        623616    22756184 268435456
GROUP BY (SORT)       103424         43008     1064479   4194304

So, while spilling to disk, GROUP BY HASH needed 256 MB disk and ran in 22.7 seconds, while GROUP BY SORT needed only 4 MB disk and ran in 1.1 seconds.