I think you have some misconceptions about TDD. For a good explanation and example of what it is and how to use it, I recommend reading Kent Beck's Test-Driven Development: By Example.
Here are a few further comments that may help you understand what TDD is and why some people swear by it:
"How do you combine test driven development with a design that has to change to reflect a growing understanding of the problem space?"
"How do you make the TDD practice work for you instead of against you?"
TDD is not "twice as much work" as not doing TDD. Yes, you'll write a lot of tests, but that doesn't really take much time, and the effort isn't wasted. You have to test your code somehow, right? Running automated tests are a lot quicker than manually testing whenever you change something.
A lot of TDD tutorials present highly detailed tests of every method of every class. In real life, people don't do this. It is silly to write a test for every setter, every getter, and so on. The Beck book does a good job of showing how to use TDD to quickly design and implement something, slowing down to "baby steps" only when things get tricky. See How Deep Are Your Unit Tests for more on this point.
TDD is not about regression testing. TDD is about thinking before you write code. But having regression tests is a nice side benefit. They don't guarantee that code will never break, but they help a lot.
When you make changes that cause tests to break, that's not a bad thing; it's valuable feedback. Designs do change, and your tests aren't written in stone. If your design has changed so much that some tests are no longer valid, then just throw them away. Write the new tests you need to be confident about the new design.
Personally I think TDD is at best overkill and at worst an impediment to a the creative process of programming. Time that is spent laboriously writing unit tests for each as yet unwritten methods/classes would be better spent solving the original problem. That being said I am a big fan of unit tests and believe wholeheartedly in them. If I have a particularly complex or troublesome piece of code I'm more than happy to write 20 unit tests for a single method but generally AFTER I have solved the problem. TDD, just like every other programming paradigm, is no silver bullet. If is suits you use it if not keep looking.
But take my opinion with a grain of salt. A much more interesting one comes from Kent Beck and How deep are your unit tests?.
Kent Beck's book gives some examples in Java and some in Python (to be honest, Kent doesn't strike me as a superstar in either language, judging from the example code in this book... but he definitely comes across as a superstar in TDD &c -- as well he should, given he's basically invented it as well as extreme programming, see his wikipedia entry).
I'd honestly recommend looking at Martin Fowlers Patterns of Enterprise Application Architecture. It discusses a lot of ways to make your application more organized and maintainable. In addition, I would recommend using unit testing to give you better comprehension of your code. Kent Beck's book on Test Driven Development is a great resource for learning how to address change to your code through unit tests.
I think that you are going in the right way. But I will send some suggestions;
To know more about TDD and tests you should read the book Test Driven Development: By Example
I'm not a purist in this matter (TDD involves more than just writing the tests first, it's also about initially writing very minimal, "hard coded" tests and refactoring them a lot -- see The Book by The Master himself).
I tend to test-first when I'm doing incremental development to add a feature to an existing module, and I insist on test-first when the incremental development I'm doing is to fix a bug (in the latter case I absolutely want a unit-test AND an integration-test that both reproduce the bug, before I fix the code that caused the bug).
I tend to be laxer when I'm doing "greenfield" development, especially if that's of an exploratory, "let's see what we can do here that's useful", nature -- which does happen, e.g. in data mining and the like -- you have a somewhat vague idea that there might be a useful signal buried in the data, some hypothesis about its possible nature and smart ways to [maybe] extract it -- the tests won't help until the exploration has progressed quite a bit.
And, once I start feeling happy with what I've got, and thus start writing tests, I don't necessarily have to redo the "exploratory" code from scratch (as I keep it clean and usable as I go, not too hard to do especially in Python, but also in R and other flexible languages).
Edit: For those that don't know Kent Beck is basically the godfather of TDD.
From "Test Driven Development" by Kent Beck, published 2002:
Here are some excerpts:
"What test do we need first? Looking at the list, the first test looks complicated. Start small or not at all. Multiplication, how hard could that be? We'll work on that one first."
The examples like this that were provided were of trivial methods. Trivial can be subjective, but, to me, methods whose bodies are almost always 2-5 lines long, not counting calls to other trivial methods, which may account for maybe another 2-3 additional lines, are basically trivial. It's one thing when you really don't need that much code, but it's another when you have classes upon classes upon classes upon classes, etc. to do something that could be OO and be in two classes with 1/2 as many lines and still be clear.
"Do these steps seem small to you? Remember TDD is not about taking teeny-tiny steps, it's about being able to take teeny-tiny steps. Would I code day-to-day with steps this small? No. But when things get the least bit wierd, I'm glad I can."
This is what Kent meant, but few of us picked up on it, per the first comment to his answer in your example from S.O. and per my experience.
I'm not blaming Kent, Ron, etc. or even saying that they are or were wrong. But, the commonly understood epitome of TDD used to be 100% test coverage and methods for just about everything. Those in the know said more like 60% was better overall but that was not really "true TDD". The implication of 100% "trivial" methods is more overhead for method calls (minor cost, depending, but can be cumulative, increase/decrease call stack size more quickly, which is inefficient), a large number of entry points (more stuff can be null/nil, cause NPE's or need checks if you don't know what is going to call it later, though that can be mitigated somewhat), and just generally too many LOC.
With a testing framework, you won't waste time rolling your own. For Java, you could start with JUnit.
TDD is a formalized approach to writing test functions as you work, which you already do, while simultaneously designing your code. There are many sources on it around the internet, but Kent Beck's book took the magic out of it (a good thing) for me:
Of course, there are many other specific techniques you can use to test that your algorithms do the right thing, all dependent on what you've made. Regardless, I try to make sure that the a real user gets their hands on an up to date build as often as possible, and they're sure to show me all of the ways that my software is technically excellent, yet in no way solves their actual problem.