Per Progamming Entity Framework: "You can either explicitly dispose the ObjectContext or wait for the garbage collector to do the job."
So in short, while the using statement isn't required, it's best practice if you know you're done using the ObjectContext since the resource is freed up immediately instead of waiting for garbage collection.
I'm affraid that it is so complex that you can start only here or buy this.
I'm playing with EF a lot. I'm using it in hobby project as well as in real project at work. I'm also playing with EF Code First which is currently in CTP (but will be released soon).
When I compare data access technologies available directly in .NET EFv4 is the best. There is no discussion about that - it is simply true. It's first tool from MS which can be at least called ORM. It allows you some basic mapping scenarions between realtion and object world and it simplifies data access via Linq-to-entities. If you have a problems with EF you can still fall back and call direct SQL via EF classes which again handle mapping for you. If you want to know more we can just discuss features you are interested in - I don't want to write a book here (more over THE BOOK already exists).
Yes EF is not the best tool on the market. It is still quite young and a lot of features are missing or not working as expected. Also extensibility is terrifying - current version of EF is mostly monolithic black box with no extensibility points. Dissassembling EF assemblies shows that it is full of internal, sealed, static classes. Another problem is that learning curve for EF is much longer especially if you have never used ORM before.
I started to learn NHibernate and I have to say the feature set is just amazing but I haven't use it in any project yet. Another problem is that for example at my current job we are not allowed to use it (I'm leaving soon).
The choice is up to you but it is always good to know at least basics of available tools.