How to Label Speech data For end to end speech to text training

I've been wondering that i have some audio data which contains recorded sentences so when i will use this data to train a neural network how will i label this audio sentence file do i have to just mention the work present in the audio file or should i label with character by character of a audio time interval. so if any one can give me an idea how to actually label the audio data to train a neural network and what will be the output of the neural network.