What would be the best method to code heading/title for <ul> or <ol>, Like we have <caption> in <table>?

Always use heading tags for headings. The clue is in the name :)

If you don’t want them to be bold, change their style with CSS. For example:

HTML:

<h3 class="list-heading">heading</h3>

<ul> 
    <li>list item </li>
    <li>list item </li>
    <li>list item </li>
</ul>

CSS

.list-heading {
    font-weight: normal;
}

You can associate the heading and the list more explicitly by using the <section> element, if they comprise a section of the document:

<section class=“list-with-heading”>
    <h3>heading</h3>

    <ul>
        <li>list item </li>
        <li>list item </li>
        <li>list item </li>
    </ul>
</section>

Then style thus:

.list-with-heading h3 {
    font-weight: normal;
}

how about making the heading a list-element with different styles like so

<ul>
 <li class="heading">heading</li>
 <li>list item</li>
 <li>list item</li>
 <li>list item</li>
 <li>list item</li>
</ul>

and the CSS

ul .heading {font-weight: normal; list-style: none;}

additionally, use a reset CSS to set margins and paddings right on the ul and li. here's a good reset CSS. once you've reset the margins and paddings, you can apply some margin on the list-elements other than the one's with the heading class, to indent them.

Would the use of <caption> be allowed?

<ul>
  <caption> Title of List </caption>
  <li> Item 1 </li>
  <li> Item 2 </li>
</ul>

h3 is absolutly a better solution than h2, h1 or h6 !

1. You have to use specific level : if you're in a h1, use h2, if you're in a h5, use h6 (if you're in a h6... hum, use strong or em for exemple). It not a obligation but a question of accessibility (Here, green part).

2. You don't have to give title to list... because this element it doesn't exist. So screen reader will not use something special.

Therefore, using Hn is probably one of the best solution, but surely not a specific level.