For more information on where to get CDs, see Section 4.1, “Official Ubuntu CD-ROMs”.
Some installation methods require other images than CD images. Section 4.2.1, “Where to Find Installation Images” explains how to find images on Ubuntu mirrors.
The subsections below will give the details about which images you should get for each possible means of installation.
It's also possible to install from removable USB storage devices. For example a USB keychain can make a handy Ubuntu install medium that you can take with you anywhere.
The easiest way to prepare your USB memory stick is to download netboot/boot.img.gz, and use gunzip to extract the 8 MB image from that file. Write this image directly to your memory stick, which must be at least 8 MB in size. Of course this will destroy anything already on the memory stick.
There are other, more flexible ways to set up a memory stick to use the Ubuntu installer. For details, see Section 4.3, “Preparing Files for USB Memory Stick Booting”.