In UML class diagram, should I draw a dependency from class A to B, if B is used by interface which is used by class A?

You describe a class A which uses an interface I which uses a class B. This could look somewhat like:

Should you show a dependency of A to B?

• First of all, you are not obliged to show all the potential dependencies in your model, but only those that you think are releveant to understand the problem.
• The usage dependency is not necessarily transitive: A can use I without using B, and even without knowing B. Your narrative does not establish that there is a direct usage dependency between A and B, so you should not show it (unless you know there is a direct dependency).

Now a definition:

A Usage is a Dependency in which one NamedElement requires another NamedElement (or set of NamedElements) for its full implementation or operation. The Usage does not specify how the client uses the supplier other than the fact that the supplier is used by the definition or implementation of the client.

It is not required by UML, nor by your design, but if you want to document the implementation, you may have a second thought at dependencies. Some languages such as C++ may require for compiling A, to have the definition of I which would require to include the definition of B. So to compile A you may need to know B. If you think it would help you to stay aware of such dependencies, you could show it. It is not wrong, but personally, I advise to show only direct dependencies, because you can easily deduce the transitive ones, without cluttering the diagram with a lot of extra arrows.