I had observed the recompilation of angular projects using ng serve with Visual Studio Code. Vendor.js is not recompiled when you make changes to the project. It is only compiled during the first execution of ng serve.
If you wish to improve compile speed in development mode, you might want to consider implementing lazy loading. If lazy loading is implemented, whenever you make changes to a component, only that component's module is recompiled. This can greatly save compilation time in the long run.
This had happened to me in one of my first Angular projects (a school assignment). To cut the long story short, I had read about lazy-loading, implemented it to the admin dashboard module, but my friend refused to implement it to the user module because he would had to shift a lot of code around and restructure the project. As the project became bigger... it became evident how important lazy-loading can be...
First compilation: (I will explain a bit) manage-* modules are feature modules belonging to the admin module. Every component in the user-module: sidebar, navbar, filter, search, etc, all belongs to the user module, hence it's significantly larger, 3.46MB compared to admin and it's feature modules.

E.g. Commenting 3 lines of html in one of the component in user module recompiles the entire user-module and takes 5523ms. If I am running photoshop, or other memory intensive programs, it'd take much much longer!

E.g. Commenting 3 lines of html and recompiling a feature module in admin dashboard module takes < 1s:

I am still new to Angular but lesson learnt: do lazy-loading, it does saves user's bandwidth and your development time =]