Tricks of the 3D Game Programming Gurus-Advanced 3D Graphics and Rasterization (Other Sams)
The book starts with little more than plotting a pixel on the screen. By the end you'll have a complete lit, shaded, 3D game engine that you've written entirely from scratch. And in the process you will learn absolutely everything that goes into it.
And one thing I'd add is that this might sound somewhat overwhelming but it's really not. The book is extremely well written and clearly was a labor of love. If you get some pleasure out of math and code, you'll have no technical troubles working your way through the book and in the end will be rewarded with an intimate, flexible understanding of graphics development that won't be hamstrung by dependence on a specific API.
 - https://www.amazon.com/Tricks-Programming-Gurus-Advanced-Gra...
It's basically a set of tutorials that lead you through the steps of building a software 3D graphics rasterizer. It covers rasterizing, lighting, shading, shadows, textures, etc, and the math behind each set of concepts. It's built on late-90s C and DirectX, but the capabilities used are covered by just about any game programming library. The author builds kind of an abstraction library on top of the DirectX code, and that's pretty easy to rewrite in whichever language and toolset you're comfortable with.