A Tag is a keyword which describes all or part of a Post. Think of it like a Category, but smaller in scope. A Post may have several tags, many of which relate to it only peripherally. Like Categories, Tags are usually linked to a page which shows all Posts having the same Tag. Tags can be created on-the-fly by simply typing them into the Tag field. By default, tags can be assigned only to the Post and custom post types.
Tags can also be displayed in clouds which show large numbers of Tags in various sizes, colors, etc. This allows for a sort of total perspective on the blog, allowing people to see the sort of things your blog is about most.
Many people confuse Tags and Categories, but the difference is easy: Categories generally don't change often, while your Tags usually change with every Post and are closer to the topic of the Post.