They're small amounts of data that's exchanged between your browser and the webserver, allowing information to be used across several pages (e.g. browsing through a catalogue, and selecting items to purchase as you move through different pages).
The data can be simply held in memory (session cookies), where it'll be lost once you complete the current browsing session (i.e. when you close the browser—completely quit using it, that is, not just closing the current window if you had several browser windows running). These sort of cookies can be used to maintain a logon through a website, facilitate shopping, etc.
Or the data can be stored on disc (persistent cookies), where it can be referred to the next time you visit the site again. These sort of cookies can be used to automatically log onto a website you've visited before, track your usage of a service (for statistical analysis, or spying on you), customise browsing a service, etc. They may have a short lifespan, or they could be set to expire 20 years in the future.
It's a bit like having your passport stamped as you travel the world. Someone looking at your passport can tell where you've been, and treat you differently based on that information.
You might not like the idea of one website sending you a cookie from some other website, thereby associating you with something, then you visit yet another website but also with cookies from one of those others, thereby associating you with something else, as well. After a while, a web of associations are being compiled on you, but you don't know what they're doing with the data.
Perhap's they'll do something annoying to you with the information that they've gathered about your browsing habits (like send you lots of junk mail), or influence the adverts that get served with some webpages (one accidental visit to something that you don't want to see could mean that you get targeted with more of the same sort of thing).
You also mightn't like the idea that your computer is steadily collecting more and more cookies that aren't going to be culled, filling your drives up with yet more junk.
Because of some of the annoying things done with cookies, some browsers allow the user to override the expiry time of persistent cookies, making them like session cookies (deleting them as you exit the browser). Likewise for selective blocking of cookies (cookies from certain places, or cookies that don't directly belong to the site that you're browsing).
They're also a bad web authoring idea for managing visitors' experiences with websites. Your logon or customisation details are stored on one computer, and for one particular browser on that computer. If you try to use the service on another computer, or with another browser, you've got to start again from scratch (assuming that you're able to remember things like your passwords, and don't mind wasting your time). Although cookies can be benign, it's annoyances like these that tend to outweigh their benefits (there's better ways to store user-information, like doing it on the server, so it's not dependent on all the things that can change at the user's side of things—which browser or computer they use).
By the way, I don't use cookies on my website, this information is only here because I'm forever explaining them to people.