Intro to GPU programming

Check out CUDA by NVidia, IMO it's the easiest platform to do GPU programming. There are tons of cool materials to read. http://www.nvidia.com/object/cuda_home.html

Hello world would be to do any kind of calculation using GPU.

Hope that helps.

1. You get programmable vertex and pixel shaders that allow execution of code directly on the GPU to manipulate the buffers that are to be drawn. These languages (i.e. OpenGL's GL Shader Lang and High Level Shader Lang and DirectX's equivalents ), are C style syntax, and really easy to use. Some examples of HLSL can be found here for XNA game studio and Direct X. I don't have any decent GLSL references, but I'm sure there are a lot around. These shader languages give an immense amount of power to manipulate what gets drawn at a per-vertex or per-pixel level, directly on the graphics card, making things like shadows, lighting, and bloom really easy to implement.
2. The second thing that comes to mind is using openCL to code for the new lines of general purpose GPU's. I'm not sure how to use this, but my understanding is that openCL gives you the beginnings of being able to access processors on both the graphics card and normal cpu. This is not mainstream technology yet, and seems to be driven by Apple.
3. CUDA seems to be a hot topic. CUDA is nVidia's way of accessing the GPU power. Here are some intros

I think the others have answered your second question. As for the first, the "Hello World" of CUDA, I don't think there is a set standard, but personally, I'd recommend a parallel adder (i.e. a programme that sums N integers).

If you look the "reduction" example in the NVIDIA SDK, the superficially simple task can be extended to demonstrate numerous CUDA considerations such as coalesced reads, memory bank conflicts and loop unrolling.

See this presentation for more info:

http://www.gpgpu.org/sc2007/SC07_CUDA_5_Optimization_Harris.pdf

Take a look at the ATI Stream Computing SDK. It is based on BrookGPU developed at Stanford.

In the future all GPU work will be standardized using OpenCL. It's an Apple-sponsored initiative that will be graphics card vendor neutral.

CUDA is an excellent framework to start with. It lets you write GPGPU kernels in C. The compiler will produce GPU microcode from your code and send everything that runs on the CPU to your regular compiler. It is NVIDIA only though and only works on 8-series cards or better. You can check out CUDA zone to see what can be done with it. There are some great demos in the CUDA SDK. The documentation that comes with the SDK is a pretty good starting point for actually writing code. It will walk you through writing a matrix multiplication kernel, which is a great place to begin.

Try GPU++ and libSh

LibSh link has a good description of how they bound the programming language to the graphics primitives (and obviously, the primitives themselves), and GPU++ describes what its all about, both with code examples.