how to redirect http to https plus redirect `index.html` to another html page in .htaccess together?

RewriteEngine On
RewriteCond %{HTTP_HOST} ^example.com$
RewriteRule ^$ http://example.com/index.php [L,R=301]

Yes, this is an external redirect and redirects from http(s)://example.com/ to http://example.com/index.php. However, you are redirecting to HTTP, not HTTPS and you are redirecting to index.php, whereas you stated index.html in your question. There wouldn't seem to be a need to check the requested hostname, unless you have mutliple domains and specifically only want to redirect one instance.

Unless you plan to implement HSTS then include this redirect before your general HTTP to HTTPS redirect and include the canonical hostname in your HTTP to HTTPS redirect.

For example:

RewriteEngine On

# Redirect / or /index.html TO /dir/index.html (HTTPS)
RewriteRule ^(index\.html)?$ https://example.com/dir/index.html [R=302,L]

# Redirect HTTP to HTTPS
RewriteCond %{HTTP:X-Forwarded-Proto} =http
RewriteRule ^ https://example.com%{REQUEST_URI} [R=302,L]

However, having read at your linked question, an external "redirect" might not be the correct solution for your problem. It sounds like you are perhaps "incorrectly" linking to a file in the document root, when you should be linking to a file in a subdirectory? (Really, you should correct your internal links if this is the case...) However, a workable solution might be to internally rewrite the request to the desired URL-path or to change the DirectoryIndex if you are linking to the document root directory. This is entirely hidden from the user/client - the user sees URL A only, but you are in fact serving URL B. But this will depend on how your site is structured.

An external redirect, on the other hand, physically sends the user/browser to URL B. They see both URL A (the link they click on) and URL B (the URL they are being redirected to) - as does a search engine bot. The browser needs to make two requests. This is potentially slower for users and places more work on your server (not much more, but it is still more).

You want to do your http --> https first because that applies both to the original and the redirect for secure communication.

    RewriteEngine On
    RewriteCond %{HTTPS} !=on
    RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301,NE]
    Header always set Content-Security-Policy "upgrade-insecure-requests;"

Now you can apply your redirect to index.php

     <IfModule mod_rewrite.c>
     RewriteEngine On
     RewriteBase /
     RewriteRule ^index\.php$ - [L]
     RewriteCond %{REQUEST_FILENAME} -f
     RewriteCond %{REQUEST_FILENAME} -d
     RewriteRule . /index.php [L]
     </IfModule>

You can adjust the -f and -d to include ! and/or change the filename type to suit your specific needs.