Latency is the term used to describe the amount of time it takes for the computer to generate a sound in response to a command. Practically speaking, it can be the delay between clicking Play and playback actually starting, or the time between pressing a key on a keyboard and hearing a sound from the computer. In the days before ASIO, latencies of half a second or more were the norm, which meant that playing anything live was impossible. These days, with the right soundcard or audio interface, latencies are commonly so low as to be unnoticeable, and playing sounds on a computer has become as responsive as any other instrument.
Latency is a natural function of computers, a built-in delay which allows the CPU to process information and get it out in an orderly fashion. The delay is caused by a buffer which is a small piece of memory where information to be processed is temporarily stored and sorted. As data comes in - in this case representing sound - it fills the buffer, while the CPU processes it and then sends it to the outputs in a constant stream. The larger the buffer, the larger the delay/latency, and the smoother the output. For real-time processing though, we want that buffer to be as small as possible, but this puts added pressure on the CPU as it has to work faster to maintain the same smooth output. If the data moves through the buffer too quickly then the CPU may not have time to process it all, and you end up with a glitch or crackle in the audio, so you can see that greater CPU power enables lower latencies, and happily, CPU power has kept on increasing!
When it comes to playing software instruments, a latency of 10ms or less is generally regarded as real-time. To benefit from such low latencies though, its essential that you have a soundcard or audio interface with drivers designed for the job.