If pandas uses xlsxwriter why do I have to install it when I want to use it directly?

Firstly, xlswriter is an optional dependency to pandas as in pandas documentation here. Therefore, it is not installed automatically with pandas because it is not a necessary dependency. Also, you should know that xlsxwriter module is installed separately to write worksheets in an Excel 2007+ XLSX file. If this module is not installed before, pandas could not work and arise an error as you can see in this problem here.

Finally, we do not need sometimes to install xlsxwriter manually. For instance, xlsxwriter is installed automatically with Anaconda as documented here.