There are a several aspects to that question.

Cluster Security

Historically, the first one was security.

Prior Helm3, Helm did rely on some Tiller container, running in your cluster, some kind of cluster admin able to create resources on your behalf.

Back then, one of the main argument against Helm was Tiller, and its being a privileged target for any attacker trying to escalate its privileges.

As of Helm 3: Tiller is gone. This argument is no longer valid - though beware Helm 2 is not dead, I still encounter Tillers pods from time to time.

Compatibility

Helm Charts often won't work out of the box, depending on your cluster, and usually requires some patch.

A common issue would be charts trying to start privileged containers without any PodSecurityPolicy or SecurityContextConstraints configuration, resulting in containers not starting. Or more generally missing Pods or containers SecurityContexts, no way to customize them through existing Values, LoadBalancer Services when no cloud integration configured nor MetalLB available, ...

From experience, Charts tend to work great on Kubernetes with no RBAC, no security, ... Things could get "complicated", depending on which options your cluster enforces.

Quality

Which leads us to another common issue: quality. Which has two facets: the quality of the Kubernetes configuration provided, how relevant it is, how does it fit the application you are packaging. And the quality of the container images you are relying on, packaging that application.

Best case, your Chart comes from the editor of the software you are trying to install. Usually, the container image is relatively well built.

But the editor may not have a lot of Kubernetes expertise: they could be missing resources allocations, affinities/anti-affinities/topology spread constraints, there could be some deployment strategy or rolling params that doesn't make sense, very often trying to bind on privileged ports when this could be avoided, ... Sometime they don't have much time to work on those topics, focusing on their product itself.

Historically, the first Charts I could try used to be written by some GitHub user, with no affiliation with the editor, and variable understanding of what they were doing in both writing Kubernetes configurations and Dockerfiles.

Regardless of editors involvement, I won't blame anyone in particular, but there are a wide variety of container images and Helm Charts, binding on privileged ports and running as root, with no reason other than "that's how it works when you apt-get install". Chmod 777s. Stuff that may not work. Stuff that would, but I would question the security implications. Things that could be simplified, ...

Support

Last point to take under consideration, evaluating a Chart, would be how well maintained are its container images and Kubernetes configurations.

Can you find sources on a public CVS? How active is it, any outstanding bugs, are those fixed in a timely manner? Would you be able to contribute yourself, maybe, should anything bad show up on your setup?

Conclusion

Based on those aspects, and the requirements you have on your cluster (is this a test, a dev that should host CI, a production for a PCI-DSS bank, ...) and the time you can spend setting it up: you may trust a given Helm Chart, or prefer building your own solution.

Although Helm architecture used to be the main blocker, prior Helm 3, nowadays the rational not to adopt a Chart usually relates to its quality.

My first grudge against Helm is the variable - and sometimes, very poor - quality of images and configurations. Which is not to say there aren't good or great Charts. But that's for you to evaluate, on case by case, in relation to your context.

As pointed in comments: Helm isn't always the definitive answer. You could use Kustomize. Kustomize can in turn install Helm Charts. Or not. You could apply plain-text files. You could consider ArgoCD applying configurations out of a Git repository. OpenShift has "Templates" (though it does not allow for lots of templating: in that regard, Helm is better)

If we are to consider alternatives, it depends on what you want to do. Considering that most of the time, I would deploy several objects, with relations to each other, ... the "best" way to do this, arguably, is an operator. You could write yours in Go, Ansible (operator-sdk), Java, Python (kopf), ... there are tons of libraries available. While Helm/Templates/Kustomize would just apply a bunch of configurations, operators are state machines (wait for X to be up before creating Y).

And ansering to your last question: how do people actually configure, deploy and manage big K8s applications with layers upon layers of dependencies?

Depends. There's no definitive answer to this.

I could tell you the customer I'm working with nowadays uses Terraform, with a custom provider that applies Ansible Playbooks, generating Kustomization files from some templates, in turn applying plaintext files or sometimes calling Helm Charts hosted on Artifactories. Those would sometime deploy Operators (written in java/go/python), which would in turn create their own objects, ...

You can do this however you want. As long as it's OK with your security department if you have one, usable by your devs, manageable by your ops, ...