I know from my limited work patching the kernel, this is the biggest barrier for me (and I'm an experienced C programmer). The code is clean. Individual parts are relatively self-documenting, but there's little documentation about subsystems. There's little documentation about why things are done certain ways. Many kernel systems (e.g. network drivers) are part of larger abstracted systems designed to reduce the amount of duplicate logic, but these abstract systems either aren't documented at all or, due to the rapid pace of kernel development, have out-of-date documentation. Furthermore, when people do have questions, they're directed to the kernel mailing lists, which are overwhelming and, dare I say, unfriendly to the new developer. The mailing list archives are littered with unanswered questions and reprimands from older developers to newer developers just trying to contribute.

Some on the kernel development team may like it that way because it keeps out the uneducated and let's them maintain their way of doing things. I think that's why we're also seeing the fragmentation of Linux development as larger corporations that count on Linux pull the source in house where they can introduce new staff to it in a more friendly way. Of course, when they do that, oftentimes, the work doesn't make it back into the mainline kernel, so that's really a detriment to the kernel itself.