I am currently using a release candidate of WordPress 2.3, which is the first version of WordPress to have built-in support for tags. (There were plug-ins available for previous versions.) I had to manually update my theme so that it actually displayed the tags, however, and I’m using a plugin to help organize and work with the tags a little bit.
Anyway, this was about as good a discussion as I’ve seen of the difference between tags and categories.
If you find there are categories you’ve set up using WordPress which are really specific enough to be tags, 2.3 has a very handy feature that lets you selectively convert whichever categories you like into tags, while leaving the rest as categories.