So, even though logs say "no such file or directory," the actual problem (at least in my case) was due to the difference in end-of-line (EOL) characters on Windows and Linux. Windows uses CRLF represent the end of a line and Unix/Linux uses LF.
I hadn't consider this as a potential problem since the files were freshly cloned from Github and were originally created on Linux. What I didn't know is that on Windows Git is set up to automatically convert EOL characters to CRLF.
Making Git retain original EOL characters (disabling autocrlf).
There are a few ways to go about doing this. autocrlf is the name of the attribute that decides whether git converts line endings. You'd only need to do one of the following options depending on what you need.
Disable autocrlf for one command
You can clone the files with the following to disable autocrlf as just a one time thing.
git clone https://github.com/someuser/somerepo --config core.autocrlf=false
Specify EOL type in .gitattributes
If you have a single repo that you know you want to always have autocrlf disabled, you can specify it in that repo's .gitattributes file. Just add the following line to your .gitattributes file.
* text eol=lf
Disable autocrlf in Git's config file
Navigate to the folder where Git is installed on your machine. For me it was installed at C:\ProgramData\Git. Open config in a text editor. Change autocrlf=true to autocrlf=false.

Changing EOL characters on existing files.
If you've got existing entrypoint scripts that you need to convert, or if you're writing your entrypoint scripts in Windows in the first place, you can easily set the EOL type with most popular text editors. I'll outline how to do it in Vim, Notepad++, and Sublime, but it should be easy enough to figure out by searching "change EOL" and the name of your text editor of choice.
Using Vim
To change the line endings to be compatible with Linux, do :set ff=unix. To change them so that they are compatible with Windows, do :set ff=dos.
Using Notepad++
On the menu bar click on Edit and then go to EOL Conversion and select the desired conversion. You'll want to select Unix (LF) to make it compatible with Linux.
Using Sublime
On the menu bar click 'View' and go to 'Line Endings' and from there select the desired conversion. You'll want to select Unix to make it compatible with Linux.
Converting EOL characters from your Dockerfile.
Alternatively, there's a useful tool called dos2unix that you can install in your image and use to convert your entrypoint script. Assuming an Ubuntu or Debian based image which uses apt-get , you can use it in the following way.
FROM php:7.2-fpm
RUN apt-get update && \
apt-get install -y dos2unix
COPY custom-docker-php-entrypoint /usr/local/bin/
RUN dos2unix /usr/local/bin/custom-docker-php-entrypoint
ENTRYPOINT ["custom-docker-php-entrypoint"]
If your Docker image is based on Alpine linux using apk for a package manager, you'll want to do something like this,
FROM alpine:latest
RUN apk --update add bash && \
apk add dos2unix
COPY entrypoint.sh /
RUN dos2unix /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
If your Docker image is based on Centos using yum as a package manager, you'll want to do something like this,
FROM centos:latest
RUN yum update -y && \
yum install dos2unix -y
COPY entrypoint.sh /
RUN dos2unix /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]