How to enable support for 4GB of RAM or more in Ubuntu ?

There are two choices to enable support for more than 3GB of RAM in Ubuntu.
One of them is installing a 64-bit version of Ubuntu. This may not be an option if you are upgrading an existing 32-bit install.

The other one, if you want to keep using a 32-bit version, is installing a Physical Address Extension (PAE) -enabled kernel.

The server kernel provides such functionality (package: linux-server). Ubuntu Desktop edition will run fine even when using a server kernel.
This will only work on PAE-capable CPUs. To verify if your CPU has PAE support, do:

$ grep --color=always pae /proc/cpuinfo

This is an example of output for a CPU that supports PAE:

flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt rdtscp lm 3dnowext 3dnow rep_good pni cx16 lahf_lm cmp_legacy svm extapic cr8_legacy 3dnowprefetch

Installing the server kernel will also make it part of your choices at boot in the GRUB menu. If you want to easily manage that on a Desktop edition install, you can also install the StartUp Manager utility (package: startupmanager).

See my another post
  http://linuximagination.blogspot.com/2010/02/setting-specific-kernel-or-alternate-os.html
for Setting a specific kernel or alternate OS to boot by default (GUI) for more information on how to do this.

PAE is not enabled by default in 32-bit Ubuntu kernels as it decreases performance. Also starting with Ubuntu 9.10 (Karmic) an i386-generic-pae kernel is available.