How to trap input to control the screenshot function?

Bash traps are not a way to handle keyboard input; they handle signals. It works for Ctrl+C because your terminal (not Bash) is configured to send a SIGINT signal to Bash when that key combination is pressed (run stty -a to see all such bindings). But signals aren't designed to be used in this way. trap ... SIGINT should almost always be used to interrupt and terminate the program.

As @MarkSetchell suggested, bind is the right tool for configuring key bindings like Ctrl+A (notation: C-a). The manual has more details about existing bindings you can configure (see the "Miscellaneous Commands" in particular, which provides some useful behavior like Esc Ctrl+A (M-C-e) which expands all variables in the current pending command). Notice that C-a is bound to beginning-of-line by default, and C-b is bound to backward-char. The Readline manual goes into more detail, since that's what's actually responsible for all this behavior. There's also plenty of articles and examples online about getting bind to do different things, so simply searching for "bind" instead of "trap" may be enough to get you going :)

Here's a couple examples you can try:

• bind "\C-a":display-shell-version will cause Ctrl+A to invoke the readline function display-shell-version which prints the current Bash version, instead of jumping the cursor to the start of the line.
• bind -x '"\C-b":"echo Hello World"' (note -x) will cause Ctrl+B to invoke the shell command echo Hello World, instead of moving the cursor back one character.

It sounds like you want to configure arbitrary keybindings, which you may find is a bit of an uphill battle due to the different layers involved. A dedicated GUI-aware program written in a "proper" programming language that monitors your keyboard would be better suited for something like what you're describing. Obviously that's more work than wrestling with Bash, so hopefully you can get something good-enough out of bind :)

As an alternative to using readline on a shell prompt, consider the following bash solution. Solution closely matching requirements. Basically:

Loop Forever:

• Wait for Ctrl/A
• Start screenshot in backgroun
• Wait for Ctrl/B
• Stop screen shot, Run external screenshot.sh

It uses bash to read one-character at a time line delimiter is needed.

Side note: Given program running in a window with dedicated terminal, the Ctrl/A and Ctrl/B can be replaced with something easier - just 'A' and 'B'.

#! /bin/bash

screenshot(){
    rm -f /tmp/out.mp4
    exec ffmpeg -video_size 1920x1080 -framerate 25 -f x11grab -i :0.0+0,0   /tmp/out.mp4
}

while true ; do
    a=""
    echo "Type CTRL/A to start"
    while [ "$a" != $'\x01' ] ; do
        read -N1 a
    done
    screenshot &
    sleep 3
    echo
    echo "Type CTRL/B to start"
    while [ "$a" != $'\x02' ] ; do
            read -N1 a
    done
    echo "Processing ..."
    kill $!
    bash myscreenshot.sh &
done

Ctrl+A doesn't cause a signal, Bash traps are not a way to handle keyboard input; they handle signals. It works for Ctrl+C because not the Bash but your terminal is configured to send a SIGINT signal to Bash when the key combination is pressed you can check it by stty -a. So in simplest way you can't trap it.

Also you can take a look of this examples:

bind "\C-a":display-shell-version will cause Ctrl+A to prints the current Bash version, instead of jumping the cursor to the start of the line. bind -x '"\C-b":"echo Hello World"' (note -x) will cause Ctrl+B to invoke the shell command echo Hello World.

Also consider running help bind