Test Driven Development: By Example
1. [The Pragmatic Programmer](https://pragprog.com)
2. Martin Fowler's [Refactoring Book](https://martinfowler.com/books/refactoring.html)
3. Kent Beck's [Test Driven Development: By Example](https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/...)
4. [Thinking in Systems: A Primer](https://www.amazon.com/Thinking-Systems-Donella-H-Meadows/dp...)
5. [Zen Mind, Beginner's Mind: Informal Talks on Zen Meditation and Practice](https://www.amazon.com/Zen-Mind-Beginners-Informal-Meditatio...)
6. [Pragmatic Thinking and Learning: Refactor Your Wetware](https://www.amazon.com/Pragmatic-Thinking-Learning-Refactor....)
Robert C. Martin, Clean code: https://www.amazon.com/Clean-Code-Handbook-Software-Craftsma...
Vaughn Vernon, various: https://www.amazon.com/Code-Complete-Practical-Handbook-Cons... 2
Clean coder: https://www.amazon.com/Pragmatic-Programmer-Journeyman-Maste...
Hitchhiker's Guide to Python: https://www.amazon.com/Art-Readable-Code-Practical-Technique...
John Ousterhout, A Philosophy of Software Design: https://www.amazon.com/Philosophy-Software-Design-John-Ouste... This one looks particularly interesting, thanks AlexCoventry!
Kent Beck, Test Driven Development: https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/...
Dan Bader, Python Tricks: The Book: https://www.amazon.com/Software-Engineering-10th-Ian-Sommerv...
Svilen Dobrev, various: http://www.svilendobrev.com/rabota/
I think you have some misconceptions about TDD. For a good explanation and example of what it is and how to use it, I recommend reading Kent Beck's Test-Driven Development: By Example.
Here are a few further comments that may help you understand what TDD is and why some people swear by it:
"How do you combine test driven development with a design that has to change to reflect a growing understanding of the problem space?"
"How do you make the TDD practice work for you instead of against you?"
TDD is not "twice as much work" as not doing TDD. Yes, you'll write a lot of tests, but that doesn't really take much time, and the effort isn't wasted. You have to test your code somehow, right? Running automated tests are a lot quicker than manually testing whenever you change something.
A lot of TDD tutorials present highly detailed tests of every method of every class. In real life, people don't do this. It is silly to write a test for every setter, every getter, and so on. The Beck book does a good job of showing how to use TDD to quickly design and implement something, slowing down to "baby steps" only when things get tricky. See How Deep Are Your Unit Tests for more on this point.
TDD is not about regression testing. TDD is about thinking before you write code. But having regression tests is a nice side benefit. They don't guarantee that code will never break, but they help a lot.
When you make changes that cause tests to break, that's not a bad thing; it's valuable feedback. Designs do change, and your tests aren't written in stone. If your design has changed so much that some tests are no longer valid, then just throw them away. Write the new tests you need to be confident about the new design.
Personally I think TDD is at best overkill and at worst an impediment to a the creative process of programming. Time that is spent laboriously writing unit tests for each as yet unwritten methods/classes would be better spent solving the original problem. That being said I am a big fan of unit tests and believe wholeheartedly in them. If I have a particularly complex or troublesome piece of code I'm more than happy to write 20 unit tests for a single method but generally AFTER I have solved the problem. TDD, just like every other programming paradigm, is no silver bullet. If is suits you use it if not keep looking.
But take my opinion with a grain of salt. A much more interesting one comes from Kent Beck and How deep are your unit tests?.
Kent Beck's book gives some examples in Java and some in Python (to be honest, Kent doesn't strike me as a superstar in either language, judging from the example code in this book... but he definitely comes across as a superstar in TDD &c -- as well he should, given he's basically invented it as well as extreme programming, see his wikipedia entry).
I'd honestly recommend looking at Martin Fowlers Patterns of Enterprise Application Architecture. It discusses a lot of ways to make your application more organized and maintainable. In addition, I would recommend using unit testing to give you better comprehension of your code. Kent Beck's book on Test Driven Development is a great resource for learning how to address change to your code through unit tests.
I think that you are going in the right way. But I will send some suggestions;
To know more about TDD and tests you should read the book Test Driven Development: By Example
I'm not a purist in this matter (TDD involves more than just writing the tests first, it's also about initially writing very minimal, "hard coded" tests and refactoring them a lot -- see The Book by The Master himself).
I tend to test-first when I'm doing incremental development to add a feature to an existing module, and I insist on test-first when the incremental development I'm doing is to fix a bug (in the latter case I absolutely want a unit-test AND an integration-test that both reproduce the bug, before I fix the code that caused the bug).
I tend to be laxer when I'm doing "greenfield" development, especially if that's of an exploratory, "let's see what we can do here that's useful", nature -- which does happen, e.g. in data mining and the like -- you have a somewhat vague idea that there might be a useful signal buried in the data, some hypothesis about its possible nature and smart ways to [maybe] extract it -- the tests won't help until the exploration has progressed quite a bit.
And, once I start feeling happy with what I've got, and thus start writing tests, I don't necessarily have to redo the "exploratory" code from scratch (as I keep it clean and usable as I go, not too hard to do especially in Python, but also in R and other flexible languages).
Edit: For those that don't know Kent Beck is basically the godfather of TDD.