Most cameras today take SD / SDHC cards. As a result, you have the most choice in card size, manufacturer and the likelihood of multicard deals increases. The HC addition (under the letters SD in the graphic) apply to cards 4GB and higher - these cards will likely not work in pre-2007(ish) cameras.
While they are the most common type - there are definitely exceptions:
Older Sony cameras take Memory Pro Stick Duo cards - same type as the PSP uses. They are more expensive than their SD counterparts and can also be harder to find. Newer Sony cameras take both SD-type and Memory Pro Stick Duo cards - still worth double checking though.
Older Olympus/Fuji cameras take the xD type card which - along with being more rare than either of the cards mentioned above - are also limited to 2GB. To get around this, some Olympus cameras come with an adaptor that takes a MicroSD card and puts it in a xD 'housing'. Newer Olympus/Fuji cameras are using SD though.
Some SLR cameras use the deceptively named Compact Flash card (it's actually the largest in physical size). Again rarity is an issue, though price is generally comparable to a similar capacity SD card. It can also have a speed edge on the SD type cards - but this only a factor if the camera itself isn't the bottleneck.
All of these cards have a rather long lifespan (10k-100k write cycles) and even the smallest commonly available (2GB) hold over 500 (jpg) pictures. As such, it might be tempting to dismiss the above information, get a few cards with the camera and be done with it. Of course, cards do get lost and not all external/internal card readers can accept every type and shape of storage. While memory cards are coming down in price, it's nice not to have to buy a new set of cards when/if you change/upgrade/replace cameras. Go with SDHC compatible.
A lot of devices today go quite a bit beyond simply taking a picture. Often these features have a varying degree of sophistication from model to model. A few of the things you might want to look out for:
It is what it sounds like. As covered in a previous post, the feature really helps with indoor shots. Be careful to read how the camera is accomplishing this though - if it mentions 'digital' don't bother.
This is a very common feature of most new point and shoot cameras. In some SLR cameras it is lens based and, as it absence reduces the price by a large margin, is still fairly common to find them without it.
At it's most basic, it determines if there is a person looking at the camera and focuses on them (as opposed to the background). It may also adjust the exposure/white balance and determine if a flash is necessary.
A rather cool, if somewhat creepy, feature of a camera to detect if someone is smiling - often to a variable threshold - and automatically take the picture.
Still somewhat more rare than the above, this detects if someone has their eyes closed and either notifies you, or automatically takes a second picture.
Black and White, Sepia, Fish Eye, Miniature Effect, Selective Colour, Red Eye Reduction and many, many others may be present on an increasing number of devices. Personally I prefer applying these effects after with software as it allows for greater flexibility - and undoing it if it doesn't work. On the other hand it does make picture taking more 'fun' and less workflow if it's on-camera. I wouldn't recommend getting a specific product for any of these features - but if you do have them, at least try them out.