Well for source code, there are plenty of open source kernels to dive into.
As for books, Robert Love's book on the Linux kernel is very informative.
There's no substitute for diving into the code. Try to find a driver or subsystem that you're interested in and poke around with it. With tools like VMware Workstation it's super easy to make whatever changes you want, snapshot the VM, and run your modified kernel. If the kernel panics on boot, who cares? Just jump back to the snapshot and fix the problem.
For books, I strongly recommend Linux Kernel Development by Robert Love. It's a wonderfully written book -- lots of information, organized sanely, and humorous... not dry reading at all.
I would like to suggest both the books by Robert Love. I've read a bit of the first one and its excellent. The latter was recommended by a friend.
Linux Kernel Development
Linux System Programming: Talking Directly to the Kernel and C Library