WordPress inc and lib folders (directories) aren’t something that I would say belong solely in themes or only in plugins. Instead, I’ve used the inc directory and lib directory in both themes and plugins.
The inc Directory
As my general rule, I use the inc directory primarily to place collections of functions that are related to core functionality but aren’t necessarily meant to clutter up the primary core of the theme or plugin.
For example, if I’m working on a theme and I have a collection of functions for said theme that I use as helper functions, I’d much rather create inc/helpers.php than to drop them in functions.php. Over time, more helpers can be added.
In more specific cases, I also use the inc directory as a way to store core theme files such as theme-customizer.php or custom-header.php. This way, these files are focused solely on a single purpose and are easier to maintain over time. Plus, they are self-descriptive.
This keeps functions.php lean, and it keeps procedural programming files slightly more organized than having one giant “god-file” by the time of delivery.
In the case of plugins, I generally use object-oriented programming so the inc directory is normally used to hold additional classes that I write that are used as part of the core plugin file, but are dependencies.
This means that if I have the core plugin that depends on, say, a custom CSV parser or a serialization / de-serialization class, then these files would reside in inc.
The lib Directory
In short, the lib directory is used for third-party libraries. That is, these are used to make sure that I place code written by another author or team of developers in a place that I can easily retrieve (and attribute – don’t forget! :)) in my project.
Also, I know that an alternative to this is using a vendors directory. Though I’ve seen the convention and think it’s just as good, I’ve personally just stuck with lib.