It's normally the opposite. Unit tests are too small.
If you unit test all of the objects(Because their all public) then refactor the organisation of those objects then all your tests break. Since you've changed the way objects talk to each other, all your mock assumptions go out the window.
If you define a small public api of just a couple of entry points, which you unit test, you can change the organisation below the public api quite easily without breaking tests.
Where to define those public apis is a matter of skill working out what objects work well together as a cohesive unit.
The notion of a public API is really more fluid in the context of internal codebases as well. It's important to maintain your contract for forwards/backwards compatibility when publishing a library for a world. When you can reliably rewrite every single caller of a piece of code, you don't have that problem.
I usually test whatever subset of code could be tested with less than about a dozen of test cases. If it's larger then test logical parts of it with mocks in the leaves. For small projects it could be usually a single controller with only some mocks on the edge of the system (database, external APIs etc.). Refactoring the code where there is one test suite per class could be a nightmare.
If you unit test all of the objects(Because their all public) then refactor the organisation of those objects then all your tests break. Since you've changed the way objects talk to each other, all your mock assumptions go out the window.
If you define a small public api of just a couple of entry points, which you unit test, you can change the organisation below the public api quite easily without breaking tests.
Where to define those public apis is a matter of skill working out what objects work well together as a cohesive unit.