A good reason to use dynamic libraries is for plug-in systems. The application can then dynamically find out what plugins are on the machine and run them. If a plugin isn't installed nothing breaks. It makes it possible to have a open API to a closed application. Say if you have a paint application and you have plugins for file loading, then users can write/install additional file formats as plugins, without the app changing. It can also be helpful in development since you can move dependencies to separate modules. If a file loader requires libpng, then you need that dependency to rebuild the plugin but not the core application.
In general I agree with Linus, an application should not be separated in to multiple parts unless it serves the user.
I read it as, modules that are loaded and unloaded dynamically, facilitating things like the ability to update a running application. On second reading you are probably right that plug-in interfaces even if plugins are only loaded at startup, would fall in to the category he describes.
In general I agree with Linus, an application should not be separated in to multiple parts unless it serves the user.