A Graphics Processing Unit or GPU (also occasionally called Visual Processing Unit or VPU) is a dedicated graphics rendering device for a personal computer, workstation, or game console.
Modern GPUs are very efficient at manipulating and displaying computer graphics, and their highly parallel structure makes them more effective than typical CPUs for a range of complex algorithms.
A GPU implements a number of graphics primitive operations in a way that makes running them much faster than drawing directly to the screen with the host CPU.
The most common operations for early 2D computer graphics include the BitBLT operation (combine two bitmap patterns using a RasterOp), usually in special hardware called a "blitter", and operations for drawing rectangles, triangles, circles, and arcs.
Modern GPUs also have support for 3D computer graphics, and typically include digital video-related functions as well.
A new concept application for GPU's is that of Stream processing and the General Purpose Graphics Processing Unit. This concept turns the massive floating-point computational power of a modern graphics accelerators' shader pipeline into general purpose computing power, as opposed to being dedicated solely to graphical operations.
In certain applications requiring massive vector operations, this can yield several orders of magnitude higher performance than a conventional CPU.
The two largest discrete GPU designers, ATi and nVidia, are beginning to aggressively pursue this new market with an array of applications. ATi has teamed with Stanford University to create a GPU-based client for its Folding@Home distributed computing project that in certain circumstances yields results forty times faster than the conventional CPU's traditionally used in such applications.
The Wikipedia entry for GPU can be found here
Although the GPU is hailed as the big thing in gaming, it is also used to do other things. The following video from Youtube gives you an idea how it can be used to generate physically based Simulation and visualization of Fire in Real-Time.