_Concrete Mathematics_ [1] is a good book that is fun to work through. The book can be considered an introduction/prerequisite to Knuth's _The Art of Computer Programming_ series [2], which contains much mathematics. If you would like a book that is not so difficult, _Introduction to Algorithms_ [3] (aka "CLRS") is also highly recommended.
https://www.amazon.ca/Computer-Programming-Volumes-1-4A-Boxe....
[1] http://www.amazon.com/Introduction-Algorithms-Edition-Thomas...
[2] http://www.amazon.com/Computer-Programming-Volumes-1-4A-Boxe...
http://www.amazon.com/Computer-Programming-Volumes-1-4A-Boxe...
[1] http://www.amazon.com/dp/0201558025/
[2] http://www.amazon.com/dp/0321751043/
[3] http://www.amazon.com/dp/0262033844/